stephenorr

8 ms·

1.
by stephenorr 3y ago
I should probably also point out that the "improved" architecture turned out to be corrupting customer data for more than 6 months... which led to termination of that individual. I'd already left by that point, after a heated
2.
by stephenorr 3y ago
The attitude might have been something to do with the individual's SRE background. In that context, I understand the desire to ship small changesets that have a limited blast radius. It's just not really practical for actually bui
3.
by stephenorr 3y ago
I wish it had been satire. This was fundamentally one of the worst parts of my coding career, and it was disappointing because the company had been amazing to work for prior to this individual's arrival. Post-arrival, I was ordered to
4.
by stephenorr 3y ago
Well, things did get done, but not the right things. The rule wasn't hard and fast, but you had to have a damn good reason why you wanted to add bigger commits. Protobuf definitions didn't count, so you could get away with doing l
5.
by stephenorr 3y ago
An ex-manager (ex-Google, too) insisted we move from GitLab to Gerrit, and mandated that all commits be a maximum of 5 lines, ideally fewer. This led to a complete loss of all context; without that overview, you might very well be releasing
6.
by stephenorr 3y ago
It turned out that DO's base images for the worker nodes had automatic updates turned on; these were kicking in at around 6am each day, and causing the nodes to fail. I'm sure their official incident report will have more details,
7.
by stephenorr 3y ago
Still going on this morning. If you're experiencing difficulties, the fix for us was to create a new node pool, then force kill the individual nodes in the previous pool. That forces k8s to move the workloads to the new pool. You can&#
8.
by stephenorr 3y ago
There's been no real update from DO on this one. Absolutely shocking to have an entire production-grade product broken for more than 4 hours with bugger all in the way of information about what's going on. I have clients ready to