India’s largest private bank, HDFC Bank, should have seen it coming. After all, its woes have been at least three years in the making. On Saturday, 21 November, at about 4 PM, the bank’s chief information officer (CIO) and group IT head Ramesh Lakshminarayanan got a call from the head of its data centre operations. Transactions were not going through.
By 5 PM, transactions across its entire payment stack—from internet banking and UPI UPI Unified Payments Interface UPI is a real-time digital payments system that’s owned by the central bank-promoted NPCI to mobile banking—began failing. “It’s then that we realised that something was wrong at the data centre,” said a senior HDFC executive. HDFC’s data centre is located at the Reliance Communication (RCom)-owned Dhirubhai Ambani Knowledge City (DAKC), a sprawling 133-acre campus in Navi Mumbai with five internet data centres.
While HDFC Bank and DAKC have not been forthcoming with the reason for this outage, The Ken learnt that there have been similar instances in the past when there have been outages because of unpaid power bills, said a senior HDFC executive. In addition to that, the senior executive added that the data centre’s power backups had also failed in the past. A second source who is associated with DAKC confirmed this. HDFC Bank and Reliance Group did not respond to specific questions about these issues.
Data centres need an uninterrupted power supply, so they have two levels of fail safes. First, a UPS system that will keep the lights on for 10-15 minutes. This is followed by power from a diesel-run AC generator that will ideally keep everything running for at least two days. But there have been problems with the backups in the past, said the senior HDFC executive. The Ken asked both HDFC Bank and Reliance Group about these claims via email, but received no response.
HDFC’s data centre issues are long-standing, according to at least three sources The Ken spoke to. India’s central bank, the Reserve Bank of India (RBI), even came down hard on it. It banned banned Business Standard Irked by outages, RBI asks HDFC Bank to halt fresh digital launches for now Read more HDFC from issuing new credit cards, one of its more profitable businesses. This sent HDFC, the largest credit card issuer in the country, running to fix its systems. Last week in a call call Livemint HDFC Bank submits blueprint to RBI on plugging e-banking outages Read more with analysts, it said it was laying out a plan to the RBI, which the regulator has to review.