NetApp and TMS involved in Virgin Blue outage

Virgin Blue has fingered Texas Memory Systems as the cause of its 21-hour airline reservation system crash in Australia.

The airline's reservation system crashed at 8am on Monday in Australia. The cause was a hardware failure in the computer set-up running the New Skies Navitaire software, which was hosted by Navitaire, an Accenture subsidiary, at a Sydney data centre.

Virgin Blue released a statement saying: "At 0800 (AEST) yesterday the solid state disk server infrastructure used to host Virgin Blue failed resulting in the outage of our guest facing service technology systems."

Navitaire isolated the failure to the device in question quite quickly, but the decision to repair the device proved "less than fruitful and [it] also contributed to the delay in initiating a cutover to a contingency hardware platform." A failover process that should take around 90 minutes took the best part of a day.

The airline reverted to a manual reservation system and had to cope with a very large number of aggrieved customers and, in some cases, pay for their accommodation costs while they waited for a re-booked flight.

According to a Netapp blog, Navitaire uses a Texas Memory Systems RamSan solid-state storage array front-ended by a NetApp V-Series controller. The V-Series controller presents the RamSan storage as virtual NetApp storage. Such large solid-state arrays are built to hold entire databases in DRAM or NAND flash memory and provide the fastest possible response to database requests.

Accenture spokesman Eric Ulmer, an IT architect, presented on this system at SNW in Orlando last year. The NetApp blogger wrote: "I’ve seen a preview of his talk and I’m blown away by the professionalism and scope of this flash deployment."

NetApp announced support for the Texas Memory Systems (TMS) RamSan-500 SSD array in conjunction with the NetApp V-Series V3170 open storage controller in February last year. In terms of raw performance, it delivers incredibly low latencies for storage access (~1 millisecond) and very high I/O throughput. It does so at a much lower cost than a disk-based system capable of delivering the same IOPS. The RamSan-500 is a flash memory SSD device. Up to two RamSan-500 arrays were supported with a single or clustered V3170 controller in the first release.

NetApp positions Navitaire's use of the V-Series as a PaaS (Platform-as-a-service) private cloud service. Our understanding from a NetApp source is that the V-Series in such a high-IOPS application will normally be installed as a high-availability pair with RAID protection of the back-end storage when it is hard disk drive-based.

NetApp and TMS have been asked for comment but NetApp was not immediately able to respond. Levi Norman, TMS's director of marketing and OEM, said: "We have been working with [Navitaire] non-stop since we first heard of the issue to identify the root cause." ®