8 ms·

Result of AWS outage https://news.ycombinator.com/item?id=45640754 https://news.ycombinator.com/item?id=45640754

• reader_1000 11mo ago
  > We have identified the underlying issue with one of our cloud service providers. Isn't it everyone using multiple cloud providers nowadays? Why are they affected by single cloud provider outage?
  • postexitus 11mo ago
    Not only they are not using multiple cloud providers, they are not using multiple cloud locations.
  • lvncelot 11mo ago
    I think more often than not, companies are using a single cloud provider, and even when multiple are used, it's either different projects with different legacy decisions or a conscious migration. True multi-tenancy is not only very rare, it's an absolute pain to manage as soon as people start using any vendor-specific functionality.
    • dijit 11mo ago
      > as soon as people start using any vendor-specific functionality It's also true in circumstances where things have the same name but act differently. You'd be forgiven for believing that AWS IAM and GCP IAM are the same thing for example, but in GCP an IAM Role is simply a list of permissions that you can attach to an identity. In AWS an IAM Role is the identity itself. Other examples; if you're coming from GCP, you'd be forgiven for thinking that Networks are regional in AWS, which will be annoying to fix later when you realise you need to create peering connections. Oh and while default firewall rules are stateful on both, if you dive into more advanced network security, the way rules are applied and processed can have subtle differences. The inherent global nature of the GCP VPC means firewall rules, by default, apply across all regions within that VPC, which requires a different mindset than AWS where rules are scoped more tightly to the region/subnet. There's like, hundreds of these little details.
      • DiggyJohnson 11mo ago
        Sounds like we’ve walked a similar path on this. Especially with IAM and network policies. > There’s like hundreds of these little issues Exactly. If it is a handful of things that is fine. It’s often as you describe.
    • OtherShrezzing 11mo ago
      I think there's some irony in Docker being impacted specifically, as they're one of the main tools to help achieve true multi-tenancy.
      • DiggyJohnson 11mo ago
        Depends on if you’re using Docker or Podman Desktop versus straight Docker/Podman and where you’re pulling your images from.
    • ikiris 11mo ago
      Multi cloud is just a way to have the outages of both.
    • brookst 11mo ago
      And even if you think it’s important enough to justify the expense and complexity, it’s times like this when you discover some minor utility service 1) is a critical dependency, and 2) is not multi-cloud. Complex systems are hard.
  • rcxdude 11mo ago
    Because it's hard enough to distribute a service across multiple machines in the same DC, let alone across multiple DCs and multiple providers.
  • nobleach 11mo ago
    Looking at the landscape around me, no. Everyone is in crisis cost-cutting, "gotta show that same growth the C-suite saw during Covid" mode. So being multi-provider, and even in some cases, being multi-regional, is now off the table. It's sad because the product really suffers. But hey, "growth".
  • madisp 11mo ago
    they are using multiple cloud providers, but judging by the cloudflare r2 outage affecting them earlier this year I guess all of them are on the critical path?
  • roywiggins 11mo ago
    You can be multi-cloud in the sense that you aren't dependent on any single provider, or in the sense that you are dependent on all of them.
    • hunter2_ 11mo ago
      A bit like the ambiguity of search facets: if I select one facet, I get results that match, but if I add a second facet, should the results expand (OR'ing my selections) or contract (AND'ing my selections)? Presumably they should be OR'd if they belong to the same category (like selecting multiple colors, if any given result has only one color) but AND'd otherwise (like selecting a color and a size). But then a category could consist of miscellaneous features, and I want results that have every feature I've selected, which goes against the general case.
  • oldpersonintx2 11mo ago
    [dead]
  • jelder 11mo ago
    No, that's pretty rare, and generally means you can't count on any features more sophisticated than VMs and object storage. On the other hand, it's pretty embarrassing at this point for something as fundamental as Docker to be in a single region. Most cloud providers make inter-region failover reasonably achievable.
    • richardwhiuk 11mo ago
      Almost all cloud providers help here by having inter-region failures as well. There are multiple AWS services which are "global" in the sense that they are entirely hosted out of AWS East 1
  • wredcoll 11mo ago
    > Isn't it everyone using multiple cloud providers nowadays? Why are they affected by single cloud provider outage? No? I very much doubt anyone is doing that.
  • DiggyJohnson 11mo ago
    Multi cloud is not nearly as trivial as often implied to implement for real world complex projects. Things get challenging the second your application steps off the happy path
  • walkabout 11mo ago
    > Isn't it everyone using multiple cloud providers nowadays? Oh yes. All of them, in fact, especially if you count what key vendors host on. > Why are they affected by single cloud provider outage? Every workload is only on one cloud. Nb this doesn’t mean every workflow is on only one cloud. Important distinction since that would be more stable.
  • pmontra 11mo ago
    Because even if service A is using multiple cloud providers not all the external services they use are doing the same thing, especially the smallest one or the cheapest ones. At least one of them is on AWS East-1, fails and degrades service A or takes it down. Being multi-cloud does not come for free: time, engineers, knowledge and ultimately money.