8 ms·

> It would be a good thing, if it would cause anything to change. It obviously won't. I agree wholeheartedly. The only change is internal to these organization

> It would be a good thing, if it would cause anything to change. It obviously won't. I agree wholeheartedly. The only change is internal to these organizations (eg: CloudFlare, AWS) Improvements will be made to the relevant systems, and some teams internally will also audit for similar behavior, add tests, and fix some bugs. However, nothing external will change. The cycle of pretending like you are going to implement multi-region fades after a week. And each company goes on continuing to leverage all these services to the Nth degree, waiting for the next outage. Not advocating that organizations should/could do much, it's all pros/cons. But the collective blast radius is still impressive.

chii 10mo ago
the root cause is customers refusing to punish these downtime. Checkout how hard customers punish blackouts from the grid - both via wallet, but also via voting/gov't. It's why they are now more reliable. So unless the backbone infrastructure gets the same flak, nothing is going to change. After all, any change is expensive, and the cost of that change needs to be worth it.
- whatevaa 10mo ago
  Grid reliability depends on where you live. In some places, UPS or even a generator is a must have. So it's a bad example, I would say.
- MikeNotThePope 10mo ago
  Is a little downtime such a bad thing? Trying to avoid some bumps and bruises in your business has diminishing returns.
  - aaron_m04 10mo ago
    Depends on the business.
  - krige 10mo ago
    What's "a little downtime" to you might be work ruined and day wasted for someone else.
    - fragmede 10mo ago
      It's 2025. That downtime could be be difference between my cat pics not loading fast enough, or someone's teleoperated robot surgeon glitching out.
    - bloppe 10mo ago
      I remember a Google cloud outage years ago that happened to coincide with one of our customers' massively expensive TV ads. All the people who normally would've gone straight to their website instead got 502. Probably a 1M+ loss for them all things considered. We got an extremely angry email about it.
    - cactusplant7374 10mo ago
      I have a lot of bad days every year. More than I can count. It's just part of living.
  - Xelbair 10mo ago
    Even more so when most of the internet is also down. What are customers going to do? Go to competitor that's also down? It is extremely annoying, will ruin your day, but as movie quote goes - if everyone is special, no one is.
    - immibis 10mo ago
      They could go to your competitor that's up. If you choose to be up, your competitor's customers could go to you.
      - dewey 10mo ago
        If it’s that easy to get the exact same service / product as another vendor the maybe your competitive advantage isn’t so high. If Amazon would be down I’d just wait a few hours as I don’t want to sign up on another site.
        MikeNotThePope 10mo ago
        I agree. These days it seems like everything is a micro-optimization to squeeze out a little extra revenue. Eventually most companies lose sight of the need to offer a compelling product that people would be willing to wait for.
        immibis 10mo ago
        Why can't we just take pride in doing a good job?
    - throwaway0352 10mo ago
      I think you’re viewing the issue from an office worker’s perspective. For us, downtime might just mean heading to the coffee machine and taking a break. But if a restaurant loses access to its POS system (which has happened), or you’re unable to purchase a train ticket, the consequences are very real. Outages like these have tangible impacts on everyday life. That’s why there’s definitely room for competitors who can offer reliable backup strategies to keep services running.
- mopsi 10mo ago
  Downtimes happen one way or another. The upside of using Cloudflare is that bringing things back online is their problem and not mine like when I self-host. :] Their infrastructure went down for a pretty good reason (let the one who has never caused that kind of error cast the first stone) and was brought back within a reasonable time.
- LoganDark 10mo ago
  > Checkout how hard customers punish blackouts from the grid - both via wallet, but also via voting/gov't. What? Since when has anyone ever been free to just up and stop paying for power from the grid? Are you going to pay $10,000 - $100,000 to have another power company install lines? Do you even have another power company in the area? State? Country? Do you even have permission for that to happen near your building? Any building? The same is true for internet service, although personally I'd gladly pay $10,000 - $100,000 to have literally anything else at my location, but there are no proper other wired providers and I'll die before I ever install any sort of cellular router. Also this is a rented apartment so I'm fucked even if there were competition, although I plan to buy a house in a year or two.
  - heartbreak 10mo ago
    The hyperscalers definitely vote with their wallets.
- tjwebbnorfolk 10mo ago
  > the root cause is customers refusing to punish these downtime. ok how do I punish cloudflare -- build my own globally-distributed content-delivery network just for myself so that I can be "decentralized"? Or should I go to one of their even-larger competitors like AWS or GCP? What exactly do you propose?
  - chii 10mo ago
    you are not a customer of cloudflare. You need to be punishing the services you "paid" to use, but had downtime. So did you terminate any of those services for downtime, or had any sort of punishment done to them as a result?
    - tjwebbnorfolk 10mo ago
      Ok but the price I am paying includes some % of downtime in the SLA, and I am ok with that. If I wanted 100.00000% uptime, I would have to pay much more, but I don't want to
  - niutech 10mo ago
    Why not just boycott CDNs like Cloudflare and instead host your website on a decentralized network like Bluesky (https://danielmangum.com/posts/this-website-is-hosted-on-bluesky/ https://danielmangum.com/posts/this-website-is-hosted-on-blu...) or IPFS (https://pinme.eth.limo/ https://pinme.eth.limo/) for free?
tracker1 10mo ago
And even in multi-region, you experience a DNS failure and it all goes up in flames anyway. There's always going to be something.

chii 10mo ago

the root cause is customers refusing to punish these downtime. Checkout how hard customers punish blackouts from the grid - both via wallet, but also via voting/gov't. It's why they are now more reliable. So unless the backbone infrastructure gets the same flak, nothing is going to change. After all, any change is expensive, and the cost of that change needs to be worth it.

whatevaa 10mo ago
Grid reliability depends on where you live. In some places, UPS or even a generator is a must have. So it's a bad example, I would say.
MikeNotThePope 10mo ago
Is a little downtime such a bad thing? Trying to avoid some bumps and bruises in your business has diminishing returns.
- aaron_m04 10mo ago
  Depends on the business.
- krige 10mo ago
  What's "a little downtime" to you might be work ruined and day wasted for someone else.
  - fragmede 10mo ago
    It's 2025. That downtime could be be difference between my cat pics not loading fast enough, or someone's teleoperated robot surgeon glitching out.
  - bloppe 10mo ago
    I remember a Google cloud outage years ago that happened to coincide with one of our customers' massively expensive TV ads. All the people who normally would've gone straight to their website instead got 502. Probably a 1M+ loss for them all things considered. We got an extremely angry email about it.
  - cactusplant7374 10mo ago
    I have a lot of bad days every year. More than I can count. It's just part of living.
- Xelbair 10mo ago
  Even more so when most of the internet is also down. What are customers going to do? Go to competitor that's also down? It is extremely annoying, will ruin your day, but as movie quote goes - if everyone is special, no one is.
  - immibis 10mo ago
    They could go to your competitor that's up. If you choose to be up, your competitor's customers could go to you.
    - dewey 10mo ago
      If it’s that easy to get the exact same service / product as another vendor the maybe your competitive advantage isn’t so high. If Amazon would be down I’d just wait a few hours as I don’t want to sign up on another site.
      - MikeNotThePope 10mo ago
        I agree. These days it seems like everything is a micro-optimization to squeeze out a little extra revenue. Eventually most companies lose sight of the need to offer a compelling product that people would be willing to wait for.
        immibis 10mo ago
        Why can't we just take pride in doing a good job?
  - throwaway0352 10mo ago
    I think you’re viewing the issue from an office worker’s perspective. For us, downtime might just mean heading to the coffee machine and taking a break. But if a restaurant loses access to its POS system (which has happened), or you’re unable to purchase a train ticket, the consequences are very real. Outages like these have tangible impacts on everyday life. That’s why there’s definitely room for competitors who can offer reliable backup strategies to keep services running.
mopsi 10mo ago
Downtimes happen one way or another. The upside of using Cloudflare is that bringing things back online is their problem and not mine like when I self-host. :] Their infrastructure went down for a pretty good reason (let the one who has never caused that kind of error cast the first stone) and was brought back within a reasonable time.
LoganDark 10mo ago
> Checkout how hard customers punish blackouts from the grid - both via wallet, but also via voting/gov't. What? Since when has anyone ever been free to just up and stop paying for power from the grid? Are you going to pay $10,000 - $100,000 to have another power company install lines? Do you even have another power company in the area? State? Country? Do you even have permission for that to happen near your building? Any building? The same is true for internet service, although personally I'd gladly pay $10,000 - $100,000 to have literally anything else at my location, but there are no proper other wired providers and I'll die before I ever install any sort of cellular router. Also this is a rented apartment so I'm fucked even if there were competition, although I plan to buy a house in a year or two.
- heartbreak 10mo ago
  The hyperscalers definitely vote with their wallets.
tjwebbnorfolk 10mo ago
> the root cause is customers refusing to punish these downtime. ok how do I punish cloudflare -- build my own globally-distributed content-delivery network just for myself so that I can be "decentralized"? Or should I go to one of their even-larger competitors like AWS or GCP? What exactly do you propose?
- chii 10mo ago
  you are not a customer of cloudflare. You need to be punishing the services you "paid" to use, but had downtime. So did you terminate any of those services for downtime, or had any sort of punishment done to them as a result?
  - tjwebbnorfolk 10mo ago
    Ok but the price I am paying includes some % of downtime in the SLA, and I am ok with that. If I wanted 100.00000% uptime, I would have to pay much more, but I don't want to
- niutech 10mo ago
  Why not just boycott CDNs like Cloudflare and instead host your website on a decentralized network like Bluesky (https://danielmangum.com/posts/this-website-is-hosted-on-bluesky/ https://danielmangum.com/posts/this-website-is-hosted-on-blu...) or IPFS (https://pinme.eth.limo/ https://pinme.eth.limo/) for free?

tracker1 10mo ago

And even in multi-region, you experience a DNS failure and it all goes up in flames anyway. There's always going to be something.