
Google Cloud Console Incident - tonymet
https://status.cloud.google.com/incident/developers-console/19001
======
sethvargo
Hey everyone - Seth from Google here. We're aware of this issue and our teams
are working to resolve it as quickly as possible. We apologize for any
interruption.

~~~
ct0
Thanks Seth!

------
simook
I'm interested in the root cause.

Console operations and architecture for HA is one of my current
responsibilities. It's important to me that customers can depend on a
functional, responsive, and stable console.

I think the console experience heavily influences the trust and overall
customer experience of any service.

~~~
WestCoastJustin
100% agree. I wanted to record some screencasts of the interface today and it
is a little inconvenient. However, Google has an extremely robust post mortem
process and I am sure they will correct things so this specific event does not
happen again [1, 2]. They will typically post some type of public boiled down
version of the post mortem too. Hopefully that happens in this case!

[1] [https://landing.google.com/sre/sre-
book/chapters/postmortem-...](https://landing.google.com/sre/sre-
book/chapters/postmortem-culture/)

[2] [https://landing.google.com/sre/sre-
book/chapters/postmortem/](https://landing.google.com/sre/sre-
book/chapters/postmortem/)

~~~
sethvargo
Sorry for the interruption. I'm not sure how much information we'll be able to
share publicly, but there will definitely be an internal PM once the incident
is resolved.

~~~
WestCoastJustin
Yeah, I'm sure there is a healthy omg happening right now :)

------
gouggoug
This is the second time in 2 weeks this happens.

Last time it happened (Tue, Feb 26, around 12:41 PM), my Kubernetes cluster
became unresponsive (and my website). I was never able to figure out if that
was purely a coincidence or not, but I do not believe it was.

------
lfx
Also seems BitQuery having issues too.

Wondering if this end up being related with Summer Time change? Happened many
times in the past in companies where I worked.

~~~
WestCoastJustin
You might be able to use the older interface still (has been working for me)
[1]. I think they are testing a new beta interface that is right inside the
console. I still like the older interface myself.

[1] [https://bigquery.cloud.google.com](https://bigquery.cloud.google.com)

