Lloyds Group probes server crash behind ATM, cash card outage

Lloyds Banking Group is investigating the cause of a Hewlett-Packard server failure it blames for taking down thousands of its ATMs and crippling cash cards at the weekend.

The crash saw thousands of customers unable to withdraw money from their accounts or make payments using debit cards on Sunday afternoon.

The outage left shoppers stranded at supermarket checkouts, stuck at petrol pumps and staring hopelessly in the rain at cash-holes-in-the-wall.

Lloyds Banking Group consists of Lloyds, Halifax, Bank of Scotland and TSB and the problem struck between 3pm and 6pm GMT on Sunday.

Ironically, Lloyds introduced the world’s first computerised cash machine, developed with IBM. The machine was installed at Brentwood, Essex in December 1972.

The banking group has apologised for the problem, saying it had affected just a small number of its network of 7,000 ATMs. The problem was resolved by 7.30pm GMT.

We understand that online and telephone banking were not affected.

Technical details are sketchy and while bank said it’s still trying to find out the cause of the problem, it did tell The Reg it was a hardware failure. "There was no maintenance or update activity on the system at the time," a Lloyds spokesperson said.

TSB Bank CEO Paul Pester revealed that it was specifically a problem with an HP server which was responsible for the crash:

@dangerfield_gem Gemma, no truth in this. The issue was caused by a HP Server failing here in the UK. PDP

— Paul Pester | (@PaulPester) January 26, 2014

However, the group’s IT infrastructure is a known to be a patchwork of dated systems cobbled together through acquisitions over recent years of other banks.

The Lloyds core customer system that handles main payments is a Unisys system dating from the 1980s. Operations at Halifax, Bank of Scotland (HBOS) and TSB have now also been converged on this system.

The ATMs across the group are known to be provided by NCR, which is understood to also run the network.

However, IT chiefs are now breaking up the IT system as TSB spins out of Lloyds to become a “new” bank.

The group has also slashed IT jobs and sent them packing to India in waves of restructuring. It outsourced 593 positions in March 2012 and a further 200 in January 2013.

Chris Skinner, chairman of the Financial Services Club, who in the past has warned of an increased risk of IT-related outages striking banks and financial institutions, told The Reg the Lloyds crash was the latest example of dated systems being overstretched.

“They expose all the back end systems to the stresses and strains of payment in a mobile business age. These systems were built for overnight batch updates,” he said.

The Financial Conduct Authority (FCA), which supervises the conduct of the UK banks and financial institutions, told The Reg on Monday morning that it’s in talks with Lloyds to make sure the matter has been fixed and to ensure that customers are getting their money.

“They have assured us they solved the IT problems,” and FCA spokesperson said. “We will have future discussions today.”

The spokesperson said, though, it was unlikely to launch a formal investigation of the banking sector’s latest IT stuffup along the lines of the investigation underway into RBS.

The FCA is still investigating a crash that hit RBS, NatWest and Ulster Bank that blocked up to 16.7 million customers from their accounts.

RBS ATMs crashed in December, leaving millions of its customers unable to withdraw money for most of Cyber Monday. The ATM crash is understood to have been folded into the earlier investigation.

The FCA told The Reg that the Lloyds problem this weekend was unlikely to result into an investigation as it "appeared" to be a one-off and there’s no history of IT outages at the group - unlike at RBS. ®