The data center at online brokerage house TD Ameritrade in Jersey City, New Jersey handled more drama yesterday than the usual ups and downs of the market, after a smoking power supply in a blade server caused fire suppression systems to kick in and the building to be evacuated.
This is not a situation that any IT vendor likes to get any press about, of course, but a spokesperson from TD Ameritrade's home office in Omaha, Nebraska confirmed that a server in the Jersey City facility overheated and started smoking around 3 PM on Thursday. The server, which according to people familiar with the data center (and not confirmed by the company), was an HP BladeSystem blade box.
The machine was unplugged, but not before the smoke had kicked off fire suppression systems in the data center. So IBM, Fujitsu, Sun Microsystems, and EMC, which all have their own gear in the TD data centers, are having a good laugh at HP's expense this morning. And IBM is particularly relieved, since the Jersey City data center has a mix of HP and IBM blades.
Because of fire codes in Jersey City, according to the TD Ameritrade, the company's offices and data center (which is on the 10th floor) had to be evacuated, and for 45 minutes, people milled around the Hyatt Hotel near the Hudson River until the fire department gave the all-clear.
The TD Ameritrade spokesperson said that no customer-facing applications were affected by the outage, which is why you have redundant data centers.
An HP spokesperson was unable to confirm the precise details of the event, but said that the company was "looking into it."
In January, as El Reg previously reported, HP issued an advisory that 2,250-watt power supplies for its c7000 BladeSystem chassis manufactured before March 20, 2008 can in "extremely rare" cases fail, resulting in an unplanned shutdown of the entire BladeSystem enclosure. The smokey power supply at TD Ameritrade might represent a different issue with the power supplies, or it may be related. So far, HP has not said.