5 ms·

Summary: Cooling related failure in one of our buildings that hosts zone europe-west2-a for region europe-west2. The other zones are not impacted

ec109685 4y ago
If it’s a single availability zone (which I presume west2-a is), then why are so many services down and recommending moving to other regions?
- longwave 4y ago
  I hope there is a public postmortem that explains this. I have Cloud SQL instances with high availability enabled that have been down for over 5 hours with no workaround, when (according to docs) they should have failed over to the unaffected zones within minutes.
- jadams3 4y ago
  GKE user in that region, we are down. I think it's deceptive to say it's a single zone. We have VMs across three zones, but anything stateful on zone a is stranded, we can't get the volumes back. It's not just that the zone is down, it's that services are not easy to fail out of the impacted zone or move that's the problem. Kind on implies regional isn't good enough to secure high availability. You need multi regional deployments. Good luck with that when the bill comes due.
  - justinholmes 4y ago
    No you just need regional PD
Hamuko 4y ago
I guess it's worth pointing out that europe-west2 is in London, which is currently at +35°C and was as high as +40°C at one point earlier today.
- justinholmes 4y ago
  Probably better to point out to move to Finland (europe-north1) as it's only 18 degrees.
  - mcs_ 4y ago
    If everyone here in the south follows this advice., Finland will sink
    - justinholmes 4y ago
      Belgium and Finland have better capacity than London/Zurich.
    - lesuorac 4y ago
      Context: https://www.bing.com/search?q=guam+capsize+youtube https://www.bing.com/search?q=guam+capsize+youtube
- matthewdrussell 4y ago
  Equinix LD4/5/6 (Slough, Greater London) is the datacentre affected
  - lol768 4y ago
    It's a bit of a stretch to call europe-west2 "London" when it's Slough
    - chatmasta 4y ago
      It's also a bit of a stretch to call it "Google" when it's an Equinix datacenter. I know this is standard across all major cloud providers, and call me old-fashioned, but personally I'd prefer a bit more accuracy in naming of each region. Instead there is obfuscation. It's weirdly difficult to find a list of which datacenters correspond to which cloud region.
    - 867-5309 4y ago

ec109685 4y ago

If it’s a single availability zone (which I presume west2-a is), then why are so many services down and recommending moving to other regions?

longwave 4y ago
I hope there is a public postmortem that explains this. I have Cloud SQL instances with high availability enabled that have been down for over 5 hours with no workaround, when (according to docs) they should have failed over to the unaffected zones within minutes.
jadams3 4y ago
GKE user in that region, we are down. I think it's deceptive to say it's a single zone. We have VMs across three zones, but anything stateful on zone a is stranded, we can't get the volumes back. It's not just that the zone is down, it's that services are not easy to fail out of the impacted zone or move that's the problem. Kind on implies regional isn't good enough to secure high availability. You need multi regional deployments. Good luck with that when the bill comes due.
- justinholmes 4y ago
  No you just need regional PD

Hamuko 4y ago

I guess it's worth pointing out that europe-west2 is in London, which is currently at +35°C and was as high as +40°C at one point earlier today.

justinholmes 4y ago
Probably better to point out to move to Finland (europe-north1) as it's only 18 degrees.
- mcs_ 4y ago
  If everyone here in the south follows this advice., Finland will sink
  - justinholmes 4y ago
    Belgium and Finland have better capacity than London/Zurich.
  - lesuorac 4y ago
    Context: https://www.bing.com/search?q=guam+capsize+youtube https://www.bing.com/search?q=guam+capsize+youtube
matthewdrussell 4y ago
Equinix LD4/5/6 (Slough, Greater London) is the datacentre affected
- lol768 4y ago
  It's a bit of a stretch to call europe-west2 "London" when it's Slough
  - chatmasta 4y ago
    It's also a bit of a stretch to call it "Google" when it's an Equinix datacenter. I know this is standard across all major cloud providers, and call me old-fashioned, but personally I'd prefer a bit more accuracy in naming of each region. Instead there is obfuscation. It's weirdly difficult to find a list of which datacenters correspond to which cloud region.
  - 867-5309 4y ago