A configuration change to Google's external proxy load balancer service early on Wednesday morning AEDT briefly caused a large number of popular websites, games and other online properties to 404.
Networking at Google Cloud covers connectivity between Google’s international data centre network, as well as multiple WANs that “connect all data centres together” or that are for “user-facing traffic entering the Google network”.
Problems were acknowledged by Google Cloud at 5.10am AEDT before the company indicated “partial” resolution seven minutes later.
The company said at about 6.26am AEDT that the root cause was a configuration change to its external proxy load balancer service that did "not take effect".
It later said that "Google Cloud Run, Google App Engine, Google Cloud Functions, and Apigee were also impacted."
Large web properties briefly went offline, including Snapchat, Spotify, Etsy, Discord, Shopify, Cloudflare and Google Cloud itself, according to DownDetector, although most seemed to recover in under 10 minutes.
The outages caught the attention of a number of internet observers, including watchdog NetBlocks, which tweeted: “Several online services including Spotify, Snapchat and Etsy have been experiencing international outages due to a Google Cloud Networking technical fault; incident not related to country-level internet disruptions or filtering.”
High-profile Google Cloud Platform principal engineer Kelsey Hightower said customers deserved “more than an apology” for the brief issue.
“When there is a GCP outage I tend to follow along in real-time,” Hightower tweeted.
“As someone who works really hard to help customers adopt our platforms, I take outages personally, and understand that my reputation is linked to the stability and quality of our platform.
“You deserve more than an apology, which is why we study these outages, and spend so much time on customer empathy as we work to prevent them in the future.”
.jpg&h=140&w=231&c=1&s=0)
