Google Cloud blunder sinks Australian fund for a week

Australian superannuation fund UniSuper is lumbering back to life after an "unprecedented occurrence" at Google Cloud knocked its systems offline.

The May 9 estimate for the restoration of "some" member services means that the platform will have been down for a week after a bug and a misconfiguration at Google's end took UniSuper offline during the provisioning of its private cloud.

Google said: "The disruption of UniSuper services was caused by a combination of rare issues at Google Cloud that resulted in an inadvertent misconfiguration during the provisioning of UniSuper’s Private Cloud, which triggered a previously unknown software bug that impacted UniSuper’s systems.

"This was an unprecedented occurrence, and measures have been taken to ensure this issue does not happen again."

All very reassuring for members of the scheme unable to access its services. And, for that matter, other users of Google's cloud services wondering if another previously unknown bug and misconfiguration might be lying in wait for them.

Google said it was working "around the clock" with UniSuper to resolve the problems and stressed that the issue had nothing to do with malicious behavior or a cyber-attack. Instead, it looks very much like a candidate for a future edition of Who, Me?

Shoveling a company's workloads and data into the cloud is not necessarily a panacea for every outage.

Just ask OVH customers who suddenly realized that perhaps they ought to have paid a little more attention to their backups after one of the company's datacenters went up in flames in 2021.

It's all a bit awkward. UniSuper shifted its workloads to Google Cloud Platform (GCP) in 2023, having previously hosted them on a combination of Azure and its own datacenters. Less than a year later, the fund is suffering a lengthy outage at Google's hands.

According to UniSuper, it expects some services to be back online tomorrow "in a limited capacity" as it makes its way to a full restoration. The fund's CEO, Peter Chun, said earlier this week: "I would like to be clear that Google Cloud is not the only cloud service provider UniSuper utilizes, and this planning has ensured our ability to restore services and minimize data loss."

• Datacenter outages are on the decline, but when they hit, they hit hard
• Google says some AI call center agents took the morning off
• Britain enters period of mourning as Greggs unable to process payments
• From Gmail to Gfail: Google's G-Suite topples over for unlucky netizens, rights itself

However, initial balances will be shown as they were on Monday, April 29, 2024, and will be updated as the restoration progresses. Trading and investments have continued as normal, according to UniSuper.

The Register contacted Google for more information on what exactly happened.

In this case, it seems that using multiple cloud providers has saved the day, even if it has taken some time for UniSuper to post Google's explanation. The incident brings into focus the need to ensure regular backups are taken, tested, and stored outside any potential blast radius and that disaster recovery plans are verified regularly. You know, just in case that "previously unknown software bug" crops up. ®