Anyone have any inside contacts at Google who have more information than their useless status page?
In particular, I have a high availability instance (HA!), with a bonus read replica. The main server and the failover server seem to be malfunctioning. The read replica appears to be fine, and I can access that to pull data.
Having been down this road perhaps a half-dozen or a dozen times, I take the same attitude towards exposing business processes to Google Cloud as I do to exposing myself to COVID19:
* Only when strictly necessary (there are no viable alternatives to Google login or YouTube for many purposes, and you need good search rankings)
* When absolutely necessary, use appropriate protection (don't rely on Google search for business, have fail overs for videos and auth, and do periodic Google Takeouts of my data)
If not for Google Takeout, I'd do a strict quarantine. It's kind of like the N95 mask for doing business with Google.
Note that this is on the B2B end. For personal use, I use many (free) Google services. It works great for things like maps, finding interesting videos on Youtube, using Android on my phone, or even sharing family photos. I am, apparently, a satisfied Google product (you know the old saying: if you're not a paying customer, you're a ....).
This has been going on for over two hours with no update from Google aside from the single sentence "We are experiencing an issue with Cloud SQL" in the status dashboard.
The errors have become less frequent since about 21:00 UTC. The most recent errors I've seen were at 22:45. The update of my instance has finished. As of now (23:03) there is still no new word on the incident page:
https://status.cloud.google.com/incident/cloud-sql/21001
My App Engine logs show Cloud SQL connection errors since 17:31 UTC. I use 2nd generation MySQL 5.6. My SQL console shows that the instance is being updated since 18:39 (more than an hour ago). Perhaps that's part of the attempted fix.
My instances have stayed up but I am having problems with API calls. If your instance is down, I wonder if it's isolated to CloudSQL Proxy connections. Seems like this could be an issue at the API level.
status.cloud.google.com now shows an actual incident thread (not just a note). The only new info is that it affects the us-central1 region, but it says we can expect an update by 21:00 UTC.
Thanks everyone for providing real-time information. It's unnerving when an entire SQL server disappears from your control panel:-). I'd rather not have to restore everything from backups, if it's all the same!
In particular, I have a high availability instance (HA!), with a bonus read replica. The main server and the failover server seem to be malfunctioning. The read replica appears to be fine, and I can access that to pull data.