To run, er, the entire internet, IBM looks to craft more flexible systems. To that end, the researchers have presented their case for splitting software jobs across a Blue Gene computer, putting in track-able administrative controls, adding sophisticated error checks and booting software over the network.

Interested parties can find the gory software details in the report (PDF), although we'll summarize by saying that IBM is making heavy use of Linux, a hypervisor microkernel, network-based management, software appliances and a quasi-stateless approach.

In sample jobs done with prototype systems, IBM thinks they perform pretty well.

We experimented with Web 2.0 applications that are typically constructed from a LAMP stack (that is Linux, Apache, MySQL and PHP). We package the PHP business logic and Apache webserver in a 20MB appliance. By separating the database from the rest of the application stack, the nodes remain stateless. It is interesting to note that once the cost has been paid to parallelize a workload, the performance of individual nodes becomes irrelevant compared to overall throughput and efficiency.

Since web programmers are implicitly forced to parallelize their programs through the use of stateless business and display logic, their workloads make a good fit for an efficient highly parallel machine like Blue Gene. It is also important for the survival of a web company that suddenly becomes popular to be able to quickly scale their capacity, something that is difficult to do with commodity hardware that can require weeks of integration effort to bring online an additional thousand nodes. In contrast, a Blue Gene rack of 1024 nodes is validated during manufacture as a single system.

Or, say, SPECjbb2005.

SPECjbb2005 is a Java benchmark that measures the number of business operations a machine can sustain. The benchmark has a multi-JVM mode to deal with non-scalable JVMs. We used this mode and were able to spread the load across 256 Blue Gene nodes by using a harness that transparently forwards the network and filesystem accesses made by each worker.

We were able to run the benchmark across the 256 nodes that were available to us with a per-node performance of 9565 Business Operations per second (BOPS), yielding a reported score of 2.4 million BOPS. It is important to note that the benchmark rules state a requirement of a single operating system image, so we are not able to submit our performance results at this point. However, our initial results show that Blue Gene/P provides a powerful generic platf orm to run complex workloads.

Is the Future Blue?

All of this sounds great, but the reality of IBM's approach is that it relies on PowerPC chips. Sure, you can run Linux on the processors, but how many folks doing open source web work will write code for the Power architecture?

IBM does offer tools now for moving Linux/x86 code over to Power, although there are some performance trade-offs. And performance trade-offs don't go well with screaming scale.

In addition, PowerPC simply fights the momentum of the x86 market.

That said, IBM has issued a novel approach to the utility computing, mega data center problem with this research. It has also given competitors a scare by flashing serious intentions to go after the utility business with systems that require a ton of investment and skill to match.

You can't help but get the feeling that IBM and others are on the right track by exploring these hybrid models which place an emphasis on low-power chips and tight, SMP-like design where needed. Maybe we'll all look back at clusters and laugh in a few years. ®

More comments…

Related stories

HP pulls memory Missing Link from bottle of beer (1 May 2008)
IBM backs the truth up out there with FilesX (10 April 2008)
Microsoft meths up data center with 220 containers (2 April 2008)
Buffett falls for free ride Google (15 March 2008)
Sun and Microsoft confirm data center lovechild (10 March 2008)
IBM gets back into PCs (5 March 2008)
Microsoft's data center offensive sounds offensive (3 March 2008)
IBM gives mainframe another push (26 February 2008)
Bungee Labs betas IDE to grid flowgasm (21 February 2008)
Sun turns to Neil Diamond for Amazon utility 'killer' (15 February 2008)
IBM updates oh-so-much storage gear (13 February 2008)
Geeks fight the smelter with embedded processor-based box (2 February 2008)
Half-petaflop IBM BlueGene supercomputer plan announced (1 November 2007)
Boffins go HPC crazy while America stands in the iQ (29 June 2007)
HPC bar goes lower and wider (27 June 2007)
IBM turbo charges world's fastest supercomputer (26 June 2007)
Sun's new supercomputer and switch are bigger than yours (26 June 2007)
Sun ships 52,000 Opteron cores to Texas (3 October 2006)
IBM and HP monopolize Top 397 supercomputers list (28 June 2006)
These supercomputers could be yours (22 November 2005)
IBM tops supercomputer list again (14 November 2005)
Blue Gene breaks supercomputing record, again (31 October 2005)
IBM puts Blue Genes up for sale (8 November 2004)