Cray lands $70m super deals

Cray has won two big supercomputer deals with a combined value of $70m just as it reported decent sales and a profit in its second quarter and raised its guidance for 2009.

The first deal is with Oak Ridge National Laboratory, upgrading its Jaguar XT5 massively parallel Opteron-Linux super to the latest six-core Istanbul Opteron processors from AMD. Oak Ridge is one of the many supercomputing research labs run by the US Department of Energy (DoE).

The Jaguar super is Cray's current flagship installation. It went into Oak Ridge in the summer of 2008 and debuted on the Top 500 supers list using Cray's XT4 interconnect and AMD's quad-core Shanghai Opteron processors. The initial Jaguar installation used the quad-core Budapest Opteron processors running at 2.1GHz, which plugged into single-socket server boards.

This machine had a total of 31,328 compute cores and delivered a peak performance on the Linpack Fortran benchmark test of 260.2 teraflops and a sustained performance of 205 teraflops. This used the SeaStar2 interconnect that was an upgraded version of the SeaStar interconnect deployed in the Red Storm super at Sandia National Laboratory. This initial machine had 62TB of main memory across its nodes and 600TB of disk.

By the end of 2008, when a new XT5 machine was plunked down next to the XT4, Jaguar got quad-core Barcelona Opterons running at 2.3GHz, with 149,504 compute cores, giving the machine a sustained performance of 1.06 petaflops. This portion of the Jaguar machine uses the SeaStar2+ interconnect - better, stronger, faster - and has 300TB of memory and 6PB of disk. It is important to note that with the XT5, Cray shifted to two-socket motherboards, after getting totally screwed over by AMD thanks to the lateness of the Budapest quad-core Opterons used in the XT4.

When you add it all up, the Jaguar boxes - which can be clustered to share workloads - are basically neck-and-neck with IBM's Roadrunner hybrid Opteron-Cell Linux cluster over at Los Alamos National Laboratory, yet another DoE lab. Both machines have broken through the petaflops barrier and both vendors are now pushing up to 10 petaflops and beyond.

Under the deal Cray has inked with Oak Ridge, the XT5 machine will be upgraded to the new six-core Istanbul chips, boosting the core count to over 224,256 on the compute nodes. The upgrade is expected to be completed on the XT5 partition on Jaguar by the end of the year and boost its peak performance to more than two petaflops. Neither Cray nor Oak Ridge said which Istanbul chips would be put into the XT5 partition of Jaguar, but it will probably be one of the standard Opteron parts announced in June, not one of the low-power or high-clock speed variants announced in mid-July.

The price difference between the 2.6GHz Opteron 2435 and the 2.4GHz Opteron 2431 is pretty high - $989 a pop versus $698 when buying in 1,000-unit quantities, or a 41.7 per cent price premium for an 8.3 per cent bump in performance - so you would guess Oak Ridge would go with the 2.4 GHz Istanbul chip. If it does, that would give the XT5 part of Jaguar a peak performance of 2.07 petaflops; boosting to the 2.6GHz Istanbul chip pushes the performance up to the 2.24 petaflops level. This upgrade is apparently worth just under $20m.

IBM beater

That was apparently the easier of the two announced deals for Cray to chase and win. The other, with Lawrence Berkeley National Laboratory's DoE-funded HPC center, called the National Energy Research Scientific Computing (NERSC) center, had pitted Cray against IBM for its next generation petaflops-scale supercomputer.

Cray won the deal with a multiyear contract that will see NERSC put in an XT5 and then upgrade it to a future Cray parallel super, presumably also based on Opteron processors. This deal is worth more than $50m, according to Cray, and the full machine will go into production by the end of 2010. NERSC currently has an XT4 box rated at 355 peak teraflops, nicknamed Franklin, which has 38,642 Budapest Opteron cores linked with the SeaStar2 interconnect.