Fusion-io has put eight of its ioDrives on a single PCIe card to produce 800,000 IOPS and 6GB/sec bandwidth.
The ioDrive is a NAND flash solid state drive (SSD) that sits on a PCIe X4 card and comes with 80 or 120GB of single level cell (SLC) flash or 320GB of slower but cheaper multi-level cell (MLC) flash. These ioDrives form a tier of server memory between DRAM and hard disk drives and provide very much faster I/O performance than disk drives.
What Fusion has done is to build a special version of its product, called the ioDrive Octal, which has eight ioDrive memory modules fitted to a PCIe X16 generation 2 card with a double-wide slot. These Octal specials were built for two secret US government installations and were deployed in servers with total SSD capacities of hundreds of terabytes.
Fusion says each Octal card runs at 800,000 IOPS (4KB reads) with a 6GB/sec bandwidth and a maximum capacity of 5TB. These figures indicate that ioDRive Duo memory modules are being used, the Duo having 320GB of SLC capacity or 640GB of MLC capacity. An eighth of the Octal card's 5TB maximum is 625GB which is close to the Duo's 640GB capacity.
The Octal runs at 800,000 IOPS meaning 100,000 per memory module. The 640GB Duo runs at 121,008 IOPS with a 75/25 read/write mix and a 4KB packet size. Again that's pretty close.
Fusion aggregated 220 ioDrive Octals together in these deployments to reach 1TB/sec of bandwidth. They were fitted in Infiniband-attached I/O servers running the Lustre parallel file system.
Fusion's chief scientist, Apple co-founder Steve Wozniak, was quoted in Fusion's release, saying: “With this architecture, IOPS are easy. We achieved over a hundred million (100,000,000) IOPS, more than enough performance to meet our customer’s requirements. The real power in our architecture was the ability to also scale bandwidth... We look forward to productizing the ioDrive Octal in the future.”
It looks like the limiting factor here is the size of the NAND real estate on PCIe cards and the number of cards you can fit in a server. But Fusion's release said the Octal card "is capable of saturating the full performance of [the PCIe X16 gen 2 double-wide] slot".
We think Fusion is using 2-bit MLC, so the potential is there for further capacity increases with 3- and 4-bit MLC NAND as that becomes available. Expect a commercially-available product next year. ®