7 ms·
Is your argument that the application gracefully recovering from the scenario will somehow make the dev team accommodated? Hard crashes are not an acceptable s
by lobstrosity420 3y ago
Is your argument that the application gracefully recovering from the scenario will somehow make the dev team accommodated?
Hard crashes are not an acceptable substitute for observability, or continuous improvement.
- munchler 3y agoI'm not sure what you mean by "accommodated". This example doesn't even rise to the level of an "active incident" in the Erlang philosophy. In other words, it's not a bug, so there's no urgency to improve it.
- lobstrosity420 3y agoIf there is data loss it’s an incident, full stop. Your observability layer should be letting you know.
- munchler 3y agoI agree. However, the linked article that I was quoting from seems to see things differently. It describes a situation in which transactions are failing (i.e. data is being lost), but it's not an incident.
- throwawaymaths 3y agoIt's still in your logs and you're still tracking it with whatever o11y suite you're using.
- di4na 3y agoTransaction failing does not mean dataloss. If you think it is, you do not understand what graceful recovery means. Graceful recovery means that something handle that failure after these transactions failed. There is no data loss. They may have been slower, but i think we can agree that a slight temporary latency for no dataloss and graceful handling of unexpected stuff like your database machine being on fire is not so bad?
- lawn 3y agoYou're just misunderstanding the philosophy. It's still a bug that should be fixed, it's just that the effects are better contained thanks to the ability to self-heal.
- munchler 3y agoI'm just quoting the article. It's not an "active incident", whatever that means.
- bcrosby95 3y agoIt means resolving it can wait until work hours instead of waking someone up in the middle of the night on Saturday.
- pessimizer 3y agoIt means that it's not causing a service outage.