6 ms·

In general failures like this aren't so much because of a single point of failure but because they trigger cascading failures across the network. One piece of

In general failures like this aren't so much because of a single point of failure but because they trigger cascading failures across the network. One piece of infrastructure trips offline, causing an abnormal situation at another, which then trips etc.

ziofill 2y ago
Isn’t this the same as a single point of failure, just “spread out”?
- ffsm8 2y ago
  No, a single point of failure means that everything depends on a single thing. I.e. if you've got 10 gas engines all being fed by a single pipeline. Or a single powerline providing all electricity to the full grid. That pipeline/powerline is your single point of failure. It is still a single network, but you need multiple failures to manifest into a cascading issue like this. What you're probably thinking of is more akin to distributed system which fails when any one component fails, i.e. modern microservice architectures, aka distributed monoliths. But that's not the case here, because you constantly have minor issues on the grid. They're just continuously being handled. What becomes the issue is the cascade, with each failure increasing the likelihood of a following failure etc. In web developer terms, this is as if a production k8s cluster fails because a node went offline, which rebalanced too many containers to another node which ran out of memory, causing it to crash and starting the cascade, ultimately ending with adjacent clusters starting to crash because of the error quotas etc
  - ziofill 2y ago
    You are right, I was thinking of a distributed system, it’s not the same. But it’s still risky and brittle.
- tharkun__ 2y ago
  You'll want to read timelines of blackouts, e.g. this one: https://en.wikipedia.org/wiki/Northeast_blackout_of_2003#Timeline https://en.wikipedia.org/wiki/Northeast_blackout_of_2003#Tim... It starts with a small thing at 12:15 p.m. and takes a while for other things to happen. But then bam 4:05:57 p.m. to 4:13 p.m. results in: End of cascading failure. 256 power plants are off-line, 85% of which went offline after the grid separations occurred, most due to the action of automatic protective controls.
feldrim 2y ago
In finance, these are called systemic risks. Some nodes in the overall network of financial institutions that can trigger cascading failures, hence affecting the whole system.

ziofill 2y ago

Isn’t this the same as a single point of failure, just “spread out”?

ffsm8 2y ago
No, a single point of failure means that everything depends on a single thing. I.e. if you've got 10 gas engines all being fed by a single pipeline. Or a single powerline providing all electricity to the full grid. That pipeline/powerline is your single point of failure. It is still a single network, but you need multiple failures to manifest into a cascading issue like this. What you're probably thinking of is more akin to distributed system which fails when any one component fails, i.e. modern microservice architectures, aka distributed monoliths. But that's not the case here, because you constantly have minor issues on the grid. They're just continuously being handled. What becomes the issue is the cascade, with each failure increasing the likelihood of a following failure etc. In web developer terms, this is as if a production k8s cluster fails because a node went offline, which rebalanced too many containers to another node which ran out of memory, causing it to crash and starting the cascade, ultimately ending with adjacent clusters starting to crash because of the error quotas etc
- ziofill 2y ago
  You are right, I was thinking of a distributed system, it’s not the same. But it’s still risky and brittle.
tharkun__ 2y ago
You'll want to read timelines of blackouts, e.g. this one: https://en.wikipedia.org/wiki/Northeast_blackout_of_2003#Timeline https://en.wikipedia.org/wiki/Northeast_blackout_of_2003#Tim... It starts with a small thing at 12:15 p.m. and takes a while for other things to happen. But then bam 4:05:57 p.m. to 4:13 p.m. results in: End of cascading failure. 256 power plants are off-line, 85% of which went offline after the grid separations occurred, most due to the action of automatic protective controls.

feldrim 2y ago

In finance, these are called systemic risks. Some nodes in the overall network of financial institutions that can trigger cascading failures, hence affecting the whole system.