9 ms·

Nobody ever gets credit for fixing problems that never happened (2001) [pdf]

793 points by sam_bristow 3mo ago

matja 3mo ago
Apt comic: https://www.workchronicles.com/p/comic-prevention-vs-cure https://www.workchronicles.com/p/comic-prevention-vs-cure
lstodd 3mo ago
> The combined expenditure of U.S. companies on management consul- tants and training in 1997 was over $100 billion erhm, if this figure is close to true i can see what market ai companies is after.
- jacques_chester 3mo ago
  AI slots quite neatly into the capability trap model, actually. Which loop it belongs to in the model is left as an exercise for the reader.
Jtsummers 3mo ago
Two significant prior discussions: https://news.ycombinator.com/item?id=8940820 https://news.ycombinator.com/item?id=8940820 - 24 Jan 2015, 50 comments https://news.ycombinator.com/item?id=39472693 https://news.ycombinator.com/item?id=39472693 - 22 Feb 2024, 434 comments
jacques_chester 3mo ago
You'll see capability traps everywhere once you learn about them. Sterman, Repenning and other collaborators wrote several papers after this one. All fascinating and almost entirely depressing. Especially since MIT's Sloan school, where system dynamics first became a discipline, is just around the bend from Harvard Business school, where system dynamics first became ignored.
- macrocosmos 3mo ago
  One thing I don't get about the concept of capability traps is why is it expected that a company which is good at one thing would be capable at the new thing? What exactly makes a capability trap a trap?
  - jacques_chester 3mo ago
    The trap is that you can't get better without first getting worse. You can't get out of the destructive cycle of production pressure and decaying productivity without removing the pressure. Many managers expect, or at least behave as if they expect, improvement to be monotonic and costless.
    - thx67 3mo ago
      When you are at capacity and in a degraded state, you have no additional headroom to get out of that state. Why wounds won't heal, or the poor stay poor.
  - left-struck 3mo ago
    I think because it’s a negative feedback cycle. So once it starts it’s hard to go back, the deeper you are the harder. A trap is something that’s easy to get into and hard to get out of.
rmunn 3mo ago
Article published in the Summer 2001 edition of California Management Review, yet it never mentioned Y2K, the first thing I thought of when I read the line "fixing problems that never happened". Perhaps it was actually written in 1999 and took a while to get published, because otherwise that seems a very strange omission. The Y2K problem was very much over-hyped by the American news media at the time (no, at no point would airplanes have been falling out of the sky — I literally heard someone say that would happen once — even if no effort had been put into fixing the bug). But in recent years I have seen people (elsewhere, not on HN) claim that Y2K was a big nothingburger, and all the money spent on fixing the bug was wasted. No, that's not true either. All the money spent on fixing the bug was why it turned into a big nothingburger. Sure, some of that money was wasted, by executives who wanted an "official" Y2K-certified certificate, issued by a consulting firm that had nothing "official" about it except their own say-so. And so they spent $2 million learning what their own employees could have told them for $2,000. THAT money was wasted. But a lot of banks were running old COBOL code that used 2-digit years, and needed to be fixed. The fact that in January 2000, everyone's bank interest was still calculated correctly, and not calculated as if it was January 1900? THAT was entirely due to the vast amounts of money spent paying old COBOL coders to come out of retirement and fix the 2-digit years. The lesson I learned from that is that it's possible for a problem to be overhyped, even massively overhyped, and yet still be a serious problem. The other lesson I should have learned is that people rarely get credit (I won't go so far as the article authors and say "nobody ever gets credit") for fixing problems that never happened.
- armada651 3mo ago
  The problem is that a lot of people have a very binary view on life. Either something is a complete success or a complete waste of money, rarely do we accept that most projects fall somewhere in the middle.
  - cortesoft 3mo ago
    And even worse, they don't think probability is a thing. If something happens, it was certain to happen and we just failed to predict it correctly. So when someone predicts something will happen with a 90% probability, and then the 10% chances happens and the predicted event does not happen, people will talk about what a bad prediction that was and how they were clearly wrong. It's the same logic that causes people to say vaccines don't work because they don't stop a disease with 100% effectiveness, or that there is no point to wear a seatbelt because people still die while wearing one.
random3 3mo ago
Like nobody gets credit for avoiding problems or unnecessary things/complexity altogether. In fact the opposite may happen.
timmg 3mo ago
There are a lot of things like this. My favorite is how elegant solutions often look simple in retrospect. So if you noodle on a problem for a while and then come up with a clever solution: once you explain it to someone they'll be like, "yeah, of course." Meanwhile the guy next to you that overcomplicates the problem ends up getting kudos for building something so difficult :D
- johnthescott 3mo ago
  "elegance and speed go hand in hand" - d. mcilroy
- npunt 3mo ago
  I feel like AI coding is accelerating everyone's work toward greater solution complexity and I think it's pushing people to build defenses and be more averse to someone else's complexity rather than being impressed by it. Bigco's are probably well behind the curve on this and are still impressed by complexity, but for people on the receiving end of AI stuff either directly via your own hand or indirectly via others, it seems like complexity is not as impressive as it once was.
  - deleted 3mo ago
    [deleted]
  - nomel 3mo ago
    Definitely. Previously, intent and effort were required to increase complexity. Now intent and effort are required to prevent complexity*! But also, what a beautiful problem to have!
- nullhole 3mo ago
  The passage that comes to mind for me whenever this idea comes up, from the Brett version of the Holmes story "The Dancing Men": H: So, Watson. W: Hmm. H: You do not propose to invest in South African securities? W: How on earth do you know that? H: Now, confess, you are utterly taken aback. W: I am! H: I should make you sign a paper to that effect. W: Why? H: Because in a few minutes you will say it is all so absurdly simple. W: I should say nothing of the kind! H: You see, my dear Watson, it is not really difficult to construct a series of inferences, each dependent upon its predecessor and each simple in itself. If, after doing so, one simply knocks out the central inferences and presents one's audience with the starting point and the conclusion, one may produce a startling, though possibly a meretricious, effect. H: I can tell by an inspection of the groove between your left forefinger and thumb, that you have decided not to invest your small capital in the gold fields. W: I can see no connection. H: Very likely not; but I can quickly give you a close connection. H: Here are the missing links in the very simple chain: You had chalk between your forefinger and thumb when you returned from the club last night. You put chalk there when you play billiards, to ease the cue. You never play billiards except with Thurston. Now, Thurston, you told me, four weeks ago, had an option on some South African security which expired in a month, and which he desired you to share with him. Your checkbook is locked in my drawer, and you have not asked for the key. So, you do not propose to invest your money in that manner. W: How absurdly simple! H: Quite so. Every problem is absurdly simple when it is explained to you.
hedora 3mo ago
Two counter-examples: - Arnold bought a fleet of mobile hospitals that would have been perfect for covid response, but the next governor didn’t want to pay 1% the fleet cost per year to maintain it, so he scrapped it. - Under Obama, SARS v1 was stopped by US health workers that Trump fired because it was a “bad deal”. In the absence of that team, we got SARS v2, which was renamed to COVID 19. There’s also the related category of “never blamed for fixing problems poorly, creating even bigger problems”. Thanks to 9/11, plane cockpits can now be locked from the inside. Now, we have examples of commercial passenger airline pilots locking the doors and committing mass-murder-suicide by plane crash. For some reason, these stories don’t make the news.
- arcanemachiner 3mo ago
  You might want to double check your dates on that SARS claim. Are you talking about the swine flu outbreak?
senectus1 3mo ago
sounds like my day to day job experience.
mdmabatj 3mo ago
There is something I saw on a reddit post of all places, about how every manager who doesn't predict a baseline of "3 annoying problems every month, 1 awful problem every 3 months" is essentially a bad manager. The reasoning being that, if your number of problems is under that threshold, then someone is doing a 'good job'.
keyle 3mo ago
I've been in those companies where "struggling departments" ended up getting all the praises and raise in budgets the following quarter because of the heroic saves they did, and raising awareness on how important they are... For stuff they totally caused on themselves. Meanwhile, my perfectly purring department was struggling to keep the lights on. It's a serious problem in this industry due to the disconnect between non-technical management (who understands how to double click) and engineering (who holds the company standing). <insert IBM story about IT department cost cuts> I'm not sure how we solve this, other than having management come from engineering.
- kshacker 3mo ago
  lol. I hate presentations. I like to run a tight ship. But that does not shine, so they made me do presentations every quarter. If you do some work, you must "take" credit. It is kinda a need when you manage people since you need to build their careers. I finally moved on to be an IC. Same story, same pressure :) You need to present to directors not because they need to know, but because your managers have a quota of N presentations per quarter, and if you back out, someone else needs to step up. Needless to say my productivity reduces by half and sometimes to almost zero during the week or fortnight of presentations every quarter.
  - bruce511 3mo ago
    You define "productivity" as coding. The business defines it as "meetings, presentations, support, coding, whatever". Your productivity remains at 100% when you are doing what they want. I get that you thought you were hired as a coder, and thus measure your productivity by that. That's what I thought too. I ended up doing a lot of support (which is good, but that's another thread). Until I recalibrated my definition of productivity that frustrated me. When I realized that support was productivity I got much less frustrated.
    - kshacker 3mo ago
      When did I say I code? I have been on the industry for 35 years. I have seen my share of technology evolutions and o have seen the work from a dozen different dimensions. If after all that time, I find the process painful, just trust me -- they can't change me, and I can't change them. You take the warts with the wins and move on. 2-3 bad weeks, 10 good weeks. Life moves on to next quarter. Complete CEO mindset :)
erelong 3mo ago
So let's create moments or days of observance to make people aware of preventative measures taken
- al_borland 3mo ago
  A moment of silence to appreciate the silence the proactive measures gave us.
- alok-g 3mo ago
  I align as such. Then we soon see non-technical people start leading the same, pushing for some people to be recognized for this every sprint. Meaningless recognition starts coming in. The process fades out in just a couple weeks. The problem is there are these people in the mix, often leading, who do not understand.
Guestmodinfo 3mo ago
Human civilization runs on personal sacrifices but money bags will never care about that.
- N_Lens 3mo ago
  Good thing we’re converting human civilization into money bags at maximum speed, then. Solves all problems elegantly.
nxy 3mo ago
Very true! Along with it comes with peace/quietness at work so it’s not too bad.
sublinear 3mo ago
Making critical decisions without oversight is just as bad, or maybe worse. If you frame it this way in a meeting, you will get the attention you want. Don't say I didn't warn you because that comes with a lot of scrutiny you might not want.
ChicknNuggt 3mo ago
This is exactly the problem with the nature prevention. When it's well done, it seems like nothing was done.
- HerbManic 3mo ago
  When Covid started, our local government was very clear from the start in saying "If people think we have over reacted, then that means we have done a good job." Alas, that doesn't always fly with the populace.
  - teiferer 3mo ago
    Trouble is that it is not always true either. You can legitimately overreact and in hindsight it can be hard to distinguish between these two things. Plus, even if you did overreact, that can still be the better side to have erred on, in moderation.
dang 3mo ago
Related. Others? Nobody ever gets credit for fixing problems that never happened (2001) [pdf] - https://news.ycombinator.com/item?id=39472693 https://news.ycombinator.com/item?id=39472693 - Feb 2024 (424 comments) Nobody Ever Gets Credit for Fixing Problems That Never Happened (2001) [pdf] - https://news.ycombinator.com/item?id=8940820 https://news.ycombinator.com/item?id=8940820 - Jan 2015 (50 comments)
tjmc 3mo ago
This is why I'll never be a fire protection engineer
smath 3mo ago
I'm looking for some data -- if anyone has it -- on the fraction of companies that are led (CEO) by a technical person, over the years/decades. I have the (anecdotal) impression that this fraction has been falling (stories like Boeing), but it would be cool to support or refute this with hard data. Anyone know where to find/assemble something like this? Also, if this trend is true, then why?
- fsagx 3mo ago
  If this has been true, perhaps at some point the pendulum will swing back the other way. BTW the current CEO of Boeing is a mechanical engineer. https://www.boeing.com/company/bios/kelly-ortberg https://www.boeing.com/company/bios/kelly-ortberg
  - falsemyrmidon 3mo ago
    After decades of infamous quality drops that have impacted their reputation and finally bottom line under non technical CEOs.
- doctorwho42 3mo ago
  I don't have the data, but I think a good case study is the company MITRE. Originally it was engineers from the top down, but over the last 15-20 years those leaders with engineering backgrounds have retired and been replaced by non-engineer MBA's. And the more I look around, the more I see that as a common trope is the US.
didgetmaster 3mo ago
We all learned this back in first grade. The kids that behaved in class and did their homework did not command most of the teacher's time and effort. It was the problem children who refused to follow the rules and needed constant praise for every bit of actual effort that they put into their studies; that got the teacher's attention.
- hattmall 3mo ago
  The squeaky wheel gets the grease, is how I always heard it phrased.
jmyeet 3mo ago
This is the real problem with performance reviews in companies, which then feeds into opportunities, promotions and compensation. It's just a popularity contest. And this is particularly harmful to people who are neurodivergent, particularly if they're on the autism spectrum, because neurotypical people, who end up making all these decisions, view such people negatively for literally no reason. You could spin up a team of 6 engineers and have them go away and try some greenfield project. They could come up back in 6 months having shipped nothing. Which of these descriptions fits the facts? 1. The team learned a lot and ultimately decided there was no product-market fit and decided it was best to reallocate resources elsewhere. The learnings from that project will help a whole bunch of other projects across the division; and 2. They failed to ship and get subpar performance ratings for having no impact. The answer is... both. Or either. How you are treated will depend on how you are viewed by your management chain and that's a social function. We've all encountered people who never shut up about how hard their job is. Often they end up solving problems that they created, often by not listening to anyone that those problems would occur. And they get credit for it. You could say to people who anticipate problems to stop because it gets you nowhere. Let people fail. If only it worked that way. Instead you'll get blamed for not seeing a problem someone else created because you're viewed as competent but you aren't liked through no fault of your own. Google seems to be the posterchild for a company that briefly solved this problem and then forgot what made them successful. I am referring to Project aristotle [1], which ultimately determined that psychological safety was the key ingredient in a team's success. Now amplify all of this with constant rounds of layoffs where the environment isn't just for pay bumps and opportunities but where the cost of failing is losing your income. What you've created is an environment where office politics is everything. [1]: https://psychsafety.com/googles-project-aristotle/ https://psychsafety.com/googles-project-aristotle/
sandeepkd 3mo ago
Its really hard to measure effectiveness, problem becomes even harder when a non-engineering person has the job to measure effectiveness of a engineering person. On other hand. for software engineering some of the signals that can be used to measure such a management itself can be 1. On call requirement, outages and team burnout - A well written software should not require on-calls from the dev team 2. Ask them about the "concrete" roadmap for next 6 months to a year - Absence of concrete items is a bad sign
whatever1 3mo ago
I never gave credit to my electricity company for delivering electricity to me. I only get mad when there is an outage.
- HerbManic 3mo ago
  Just remember, the power grid fails in theory but works in practice.
- deleted 3mo ago
  [deleted]
- m463 3mo ago
  I think of electric vehicle fires and jet airliner crashes. Also, telsa self-driving. yes, we know about the greatly publicized accidents, or the tweets of the founder, but the avoided incidents not so much.
  - thx67 3mo ago
    This is where the industry made and continues to make mistakes wrt autonomous driving. They should be able to quantitatively say how many crashes were reduced, avoided and spotted. The autonomous safety system should be running all the time and it should detect not only issues with primary vehicle but it should also catalog issues it sees in other vehicles in its vicinity. We shouldn't have gotten AD before we got automated crash avoidance.
    - projektfu 3mo ago
      My car (Honda Prologue) is a little bit of a nervous Nellie, warning me to brake when someone is turning out of my lane, but it has definitely noticed developing situations many times before I have.
      - oenton 3mo ago
        I’m curious, what’s an example of a developing situation it detected for you? I ask because after car pooling with a friend in their Tesla a few times I can confidently say I would never trust what Tesla calls FSD. Forget developing situations, there were some close calls with what’s immediately in front of it. Think- about to miss a left turn at 40mph or drive into construction cones that are blocking a lane. I think I’d prefer basic driver assistance over that.
everyone 3mo ago
The jokes around Y2K being a nothing burger always annoy me. Nothing happened because a lot of talented people worked their asses off fixing it.
throw0101c 3mo ago
Aka: * https://en.wikipedia.org/wiki/Preparedness_paradox https://en.wikipedia.org/wiki/Preparedness_paradox
- teiferer 3mo ago
  My favorite example is the introduction of speed limit on some accident-ridden stretch of the Autobahn north of Berlin. After introducing the speed limit, the accident numbers went down dramatically. What did the local administration decide? Remove the speed limit again -- cause there were no accidents anymore!
Viacol 3mo ago
[dead]
cm11 3mo ago
I'd guess a lot of people here consider this "reality" at this point. Has anyone come up with a response—not a fix for the company or leaders behaving this way, but a response for their own path? Did you change from a quiet diligent one to manipulating and playing the game (now that you know the game)? Did you go from quiet and diligent to quiet and not diligent (why do good work when meh work does the trick)? Another path?
- staplers 3mo ago
  https://ribbonfarm.com/2009/10/07/the-gervais-principle-or-the-office-according-to-the-office/ https://ribbonfarm.com/2009/10/07/the-gervais-principle-or-t...
- sfn42 3mo ago
  I just do my job to the best of my ability. I have to change jobs every couple years anyway to get proper pay bumps, so I don't really care what the higher ups think of me. The people near me are who I'll use as references and they generally know I'm great at what I do.
tinyhouse 3mo ago
This is very true and applies for everything in life.
throwawa1 3mo ago
Its called doing your job.
- Steve16384 3mo ago
  That depends. It might be going above-and-beyond your job.
protocolture 3mo ago
Depends on the business. When everyone is technical to some degree, I find that credit for technical rescue is forthcoming.
thelastgallon 3mo ago
This is how people get promoted at work. They break something, it escalates and gets visibility, emails sent to executives. Now, they 'fix' it, many thank yous from everyone for a job well done. Another version of this is delay the work you are supposed to do a long time back and let it gain visibility. Executives are blind, they can't see work being done by people who take ownership and get shit done before it becomes a problem. However, executive will remember the name of the person who breaks shit and 'saves' the day.
- doublerabbit 3mo ago
  The sociopathic backhand. Someone breaks it, blames you, drags your name through the mud exacerbating the issue, goes to "fix it" yet makes it worse because of their incompetence. They continue to cuss you out for it all while licking the arse of the executive. "No, we better not let doublerabbit touch that", "I don't think they're a team player". All while it was my infrastructure. And people ask why I hate humans.
CalChris 3mo ago
People don’t get credit for fixing problems that do happen. Maybe possibly in a sales scenario where your fix unblocked the sale. Otherwise nada.
insumanth 3mo ago
Boring is Better
ananthrk 3mo ago
"Titanic effect" - No captain gets credit for preventing a disaster!
inejge 3mo ago
Couldn't help noticing: In other words, it’s not just a tool problem, any more than it’s a human resources problem or a leadership problem. Instead it is a systemic problem [...] Shades of an LLMism, a bit padded, a quarter of a century ago. These days someone could easily give it a stink-eye. I'm sure that training has ingested this along with countless similar examples.
- teiferer 3mo ago
  That should really be a cautionary tale for everybody accusing everyone of LLM manufacturing texts. Many people write like that. The self-censoring nowadays to try to avoid sounding like an LLM is really sth we need to grow out of.
SteveGerencser 3mo ago
I began migrating from network/hardware/IT work and into marketing after nearly 2 years of heavy lifting getting ready for Y2K. In the end, "nothing happened," so all that time and money was wasted, according to nearly every company I worked with. Even had one demand a full refund. I agreed as long as I could revert all the work that I had done. They agreed, and the next day after that their entire system collapsed. I couldn't even get my own dad to pay for network support for his company since he would never pay my rate for anyone no matter what. After 2 other people failed to solve his problem I fixed it in 15 minutes and then he "really" didn't want to pay because it only took 15 minutes. I was very good at what I did but got no appreciation for keeping things from breaking, only for fixing things after they broke. Marketing paid better, and I could point at real world numbers daily and justify my pay. I don't like it anywhere near as much, but at least it gets more respect than any other IT work I did.
- pulse7 3mo ago
  I'm sorry your dad didn't respect your IT work...
  - menaerus 3mo ago
    If I move out a bit of my circle, where people do all kinds of work, I'd say that there's generally a stigma on the "IT" workers. Moreover the stigma is there even within the IT company/industry itself, where sales, marketing, non-engineering parts of management, and other similar types of supporting roles also look down on the engineering. And this unfortunately includes family members too. I learned that people are mostly envy but when you're surrounded by many it can become overwhelming - numerous times I heard phrases like "oh, you're bricklayers of a modern age" ... like wtf
    - forshaper 3mo ago
      I do think of all my computer work as predominantly janitorial.
      - 3mo ago
Tsarp 3mo ago
Ian Rush said it best: "It's best being a striker. Miss five, score the winner, you're a hero. The goalkeeper plays a blinder, lets one in, and he's a villain." Every place I've worked rewards the firefighter over the person who made sure nothing ever caught fire. And the worst part is the math is obvious to everyone except the people who set the incentives.
- joelthelion 3mo ago
  How would you set the incentives, though? Almost by definition, it's hard to reward things that aren't visible. Note that there is also the flip side of the coin, people who spend all their time worrying about things that never happen, so it's not like you can just reward a defensive attitude things are more complicated than that.
  - Tsarp 3mo ago
    I dont have an answer and you are mostly correct. I received some advice based on this that made sense which was to pick the roles in your career that naturally made it easier. Sales, PM, Dev etc and not support, Devops, escalation management, CSM etc.
  - an0malous 3mo ago
    One basic example is not counting bugs as points in your ticket tracker. At my last job I had coworkers whose velocity was almost double everyone else’s but it was because they kept deploying and then fixing their own bugs.
  - forshaper 3mo ago
    The equivalent of 'days since last injury' bonuses is the first method that comes to mind, until you consider that this would mean people would be more likely to hide things going wrong. So then many things are to rely on executive culture, and an executive who will walk the line and get their info from people at the bottom is like a unicorn. That won't scale, but it does work if you do have such an executive. Naturally they would need a basic understanding of how supply is created in their firm. Yet there is something. Toyota Hiluxes and Honda Super Cubs got popular due to maintenance ease. AK-47s. Miele vacuums. Older Thinkpads. What measures would make the human equivalent visible?
latentframe 3mo ago
Prevention is hard to sustain because the success is invisible : nobody notice the defects delays or crises that never happened
WalterBright 3mo ago
People do get credit for making things that "just work".
agumonkey 3mo ago
And people often get credit for fixing issues they partially created. Human groups work on shallow signalling and distributed confusion.
doener 3mo ago
Since Covid nearly everyone in Germany knows this saying: "There is no glory in prevention."
mihaaly 3mo ago
But the same person can get paid for it. So there is an incentive to create, or at least pretend problems.
coldtea 3mo ago
Pendatically speaking, people do get credit for fixing problems that never happened. E.g. if the problems are quantifiable and there's a record, like dropping homicides from 100 per year to 20 per year in a city. Those extra homicides "didn't happen", but the improvement is understood. For an one-off problem, it depends on how clear the path to the problem is. An electrician doing an inspection and noticing and fixing big electrical issues in the installation, would be appreciated, even if the accidents didn't happen.
- dormento 3mo ago
  > Those extra homicides "didn't happen", but the improvement is understood. People are gonna criticize by saying "see? it was an overreaction to the problem, since there's not been many homicides at all!", when in fact the homicides were prevented by fixing the original problem. Same way with the electrician: "how much are you gonna charge again? And you're charging for a fix to a problem that didn't happen yet? Nah, I'll call you when the problem happens". Its maddening.
- Steve16384 3mo ago
  > An electrician doing an inspection and noticing and fixing big electrical issues in the installation, would be appreciated, even if the accidents didn't happen. Not if nobody knew he'd fixed it.
  - coldtea 3mo ago
    Electricians are no celebrities for people in general to know what they fixed or didn't fix. The person / building owner who comissioned them to do the check, will know what they fixed or what they didn't fix, from the report they give.
afisxisto 3mo ago
I remember finding a comment in the first codebase I ever worked on professionally in my first ever job. It read "This fixes a bug that hasn't happened yet". It seemed really smart at first, but later I learned that the developer that added that code also had a pattern of appending spaces to the start and end of user input and comparing the length to 2 to determine whether the value was empty or not... So I'm fairly sure "that hasn't happened yet" was probably more a case of "that I personally haven't introduced unnecessarily yet" :)
alkonaut 3mo ago
This is especially nice in the age of AI. I (the graybeard senior developer) does all the risky refactoring. I can take a performance issue and turn it into six regressions in half a day ($100). Then everyone is impressed when I let Opus fix these regressions in 20 minutes and $2 worth of AI. No one notices when you cut 20% of some expensive process but cause no regressions.
deleted 3mo ago
[deleted]
awesome_dude 3mo ago
We know that the squeaky wheel is the one that gets oiled, and we don't want the wheels to come off, so we need the wheel to squeak loud enough to be heard ;)
jdw64 3mo ago
Avengers get the glory, preventers get no story.
arkensaw 3mo ago
I feel obliged to point out Stanslav Petrov, who absolutely got credit for fixing a problem that never happened. Granted it's a very extreme case.
- ekjhgkejhgk 3mo ago
  Credit only in fame. https://en.wikipedia.org/wiki/Stanislav_Petrov#Aftermath https://en.wikipedia.org/wiki/Stanislav_Petrov#Aftermath > Petrov underwent intense questioning by his superiors about his judgment. Initially, he was praised for his decision.[2] Colonel-general Yuri Votintsev, the then-commander of the Soviet Air Defense's Missile Defense Units, who was the first to hear Petrov's report of the incident (and the first to reveal it to the public in the 1990s), states that Petrov's "correct actions" were "duly noted".[2] Petrov himself states he was initially praised by Votintsev and promised a reward,[2][22] but recalls that he was also reprimanded for improper filing of paperwork because he had not described the incident in the war diary.[22][23] > Petrov has said that he was neither rewarded nor punished for his actions.[24] According to Petrov, he received no reward because the incident and other bugs found in the missile detection system embarrassed his superiors and the scientists who were responsible for it, so that if he had been officially rewarded, they would have had to be punished.[2][24][22][23] He was reassigned to a less sensitive post,[23] took early retirement (although he emphasized that he was not "forced out" of the army),[22] and suffered a nervous breakdown.[23]
  - pedroza_alex 3mo ago
    The same article points out that he received at least £26k in awards. It could be argued that the reward isn't proportional to the magnitude of his actions, but it exists.
  - thx67 3mo ago
    His page links to https://en.wikipedia.org/wiki/Nuclear_close_calls https://en.wikipedia.org/wiki/Nuclear_close_calls which is a harrowing thing to read. We keep rolling the dice with no changes to the game.
t43562 3mo ago
I recognise almost every aspect of this document - it's exactly what's so intractable about the software business. This is why I think you do need to do some programming every now and again no matter what your level is because otherwise you cannot see what's happening and you'll be tempted into the "lazy developer" attribution.
markus_zhang 3mo ago
The title reminds me of an interesting ancient Chinese anecdote. And it is also a bit ironic that Toyota has gotten itself into some scandals recently (https://www.bbc.com/news/articles/c1wwj1p2wdyo https://www.bbc.com/news/articles/c1wwj1p2wdyo). King Wen of Wei asked Bian Que: “Of you three brothers, all physicians, who is the finest in the healing art?” Bian Que replied: “My eldest brother is the finest; my second brother comes next; I, Bian Que, am the least of the three.” King Wen said: “May I hear why?” Bian Que answered: “My eldest brother sees illness in the spirit, before it has taken shape, and removes it unseen; therefore his name is known only within our household. My second brother treats illness when it is but a hair’s breadth from appearing; therefore his name does not travel beyond our village lane. As for me, Bian Que: I pierce the blood vessels, administer strong medicines, and cut open the flesh. Thus, by such visible acts, my name has spread among the lords.”
- tom2026hn 3mo ago
  An ounce of prevention is worth a pound of cure.
  - ashirviskas 3mo ago
    How many euros or kilograms is it?
    - projektfu 3mo ago
      Three Troy ergs.
      - deleted 3mo ago
        [deleted]
    - jimnotgym 3mo ago
      I would say in nice round numbers 50 grams prevention for a kilo of cure. That is my only criticism of the metric system, it doesn't often coincide with quantities handy for every day use, so makes poor 'sayings'.
cameronh90 3mo ago
As Futurama said, when you do things right, people won't be sure you've done anything at all.
- teddyh 3mo ago
  They stole that from the Tao Te Ching, chapter 17: When the Master governs, the people are hardly aware that he exists. Next best is a leader who is loved. Next, one who is feared. The worst is one who is despised. If you don’t trust the people, you make them untrustworthy. The Master doesn’t talk, he acts. When his work is done, the people say, “Amazing: we did it, all by ourselves!” — <https://ttc.tasuki.org/section:17 https://ttc.tasuki.org/section:17>
j3th9n 3mo ago
The best hackers are the ones who never get caught.
-warren 3mo ago
My time in IT has really oscillated between these two extremes: - "Everything around is working just fine. What are we paying IT for?" - "Everything is broken. What are we even paying IT for?" Personally, I strive for the former rather than the latter; I like to say "If I do my job right, you never know I'm here." But that's what got me let go. (and for karma's sake, I keep in touch with folks at the old company; it's an absolute crapshow. So I got that going for me; which is nice)
frictasolver 3mo ago
[flagged]
sharmi 3mo ago
[dead]
harimau777 3mo ago
I had this problem at a previous job. I spent almost all of my time taking care of the behind the scenes administrative work (scheduling meetings, making sure that people had the information they needed to come into the meetings prepared, etc.). However, when performance review came around I was told that the only thing that they cared about was that I hadn't completed many story points because I was too busy keeping things from falling apart. So I stopped doing all the administrative work and focused on just completing story points. A week or two later my manager asks the team "how come all of our meetings are falling apart now? We get into a meeting and no one knows what's going on."
- projektfu 3mo ago
  Like what would have happened to the 4077th MASH if Radar O'Reilly had been reassigned.
Sam6late 3mo ago
Here is another take in s different context.I had a very bad manger who was credited earlier with causing several companies to go out of business while he was taking advantage of having worked for a FAANG company. He worked there for less than 6 months as a business development dude when the that big shot company was new in that market.After he joined us and when I was processing some payments I noticed that he was paying some people for 2 months ahead of their starting dates, it was some $20k.I notified him, and he corrected the date,then I was told by others in the department that my problem was that I should not 'interrupt the enemy when he is making a mistake'. Eventually, the company got rid of him but it was after a very serious damage.
api 3mo ago
Chernobyl instantly comes to mind. The giant problem with that particular RBMK design was known to a few people, but nobody fixed it, and if they had... we wouldn't even know, since it would be lost to history.
pugworthy 3mo ago
This brings to mind Neil Rickert's "The Parable of the Two Programmers", which was published in the ACM SIG Software Engineering newsletter, January 1985. https://dl.acm.org/action/showFmPdf?doi=10.1145%2F1012443 https://dl.acm.org/action/showFmPdf?doi=10.1145%2F1012443 for the original, or https://realmensch.org/2017/08/25/the-parable-of-the-two-programmers/ https://realmensch.org/2017/08/25/the-parable-of-the-two-pro... for a reprint.
amelius 3mo ago
This is why Apple never gets any credits.
nostrademons 3mo ago
The headline is only half true. A more accurate rendition might be "Transactions don't happen because you fixed a problem that never occurred." That illustrates the converse. People will absolutely try to avoid future problems if they are the ones that bear the consequences for them. Use birth control (or put your kid on it) so you don't have to raise another child for 20+ years. Don't hang out with that volatile "friend" who always seems to be having another crisis. Fix your roof so you don't lose the house. Don't go into debt because you're the one who will be paying interest on it. But almost by definition, bearing the consequences of your own decisions implies that there's no transaction. It's interesting and fitting that the article begins with a discussion of Toyota. People buy Toyotas because they want to avoid problems. The biggest selling point is reliability; a Toyota's value prop is that you can keep it for 20 years and you won't have unexpected things go wrong with it. Toyota has managed to turn this into a sales driver because they appeal to the self-interest of a buyer who will be living with the car for 10-20 years. There are thousands of individual decisions that Toyota makes to avoid problems with their cars, starting with a very conservative aversion to new technology or anything that is engineeringly risky. Each individual one is invisible to the customer, and often comes with significant costs in isolation. But because their sales driver is "be reliable at all costs" and they've ingrained that into the culture, they've built an organization that is willing to make these feature trade-offs for reliability. Also, an interesting corollary is "Oftentimes, a good life is lived with few transactions."
Root_Access 3mo ago
This is true but if you remove the need for credit then you can just get back to work and not have to create a category about it.
kouru225 3mo ago
Hikaru Nakamura had a good comment on famous chess moves like this: All the famous chess moves are famous sacrifices, genius saves, and last minute checkmates, but when you plug the games into stockfish you find out that these scenarios only happened cause the players made a series of awful moves early on in the game
abustamam 3mo ago
This is interesting because you can't prove you prevented something from happening. It never happened after all. Hank Green has an interesting video on preventative measures are often unsung, because how do you know? https://youtu.be/ndeB_BpsRGk?is=03lR2DMyhfwkT3L- https://youtu.be/ndeB_BpsRGk?is=03lR2DMyhfwkT3L- Kids in Africa who would have otherwise died from measles, but didn't get measles in the first place because some people decided to fund and make vaccines and thousands of unnamed and unrecognized volunteers helped distribute and administer these vaccines... These kids are alive and they don't even know that they wouldn't be alive if it weren't for some people they may never meet, and whose names they'll never know. Also tangentially, reminds me of an excerpt of a book called The Trigger Effect I read as a kid, which said something along the lines of, when you use your car's brakes to prevent an accident, your life depends on whoever assembled the brakes and quality controlled them. You'll never know their name, and they'll never know that they saved your life.
AndrewChamp 3mo ago
"When you things right, no one will ever know you did anything at all" - Futurama
smolder 3mo ago
Yeah, I have fixed issues that could have potentially destroyed the company I was working at. There was an instance of a lazily written script which just took user input for a file open. It could read sensitive files on the system because of lazy development. The other devs praised me for fixing it, but I didn't get any recognition comparable to the losses our company could have had if it got exploited.
thelastgallon 3mo ago
Doing nothing at work: https://news.ycombinator.com/item?id=48442880 https://news.ycombinator.com/item?id=48442880 The first comment is spot on. https://news.ycombinator.com/item?id=48496420 https://news.ycombinator.com/item?id=48496420: Time and time again at many companies, including well-reputed ones, I have seen that preventing issues gets you no recognition, but building a giant pile of kindling and then putting out the inevitable fire will get you recognition twice. Even in "good" orgs.
eskimobloood 3mo ago
I wonder why the whole topic of system thinking comes (self-enforcing/balancing feedback loops etc.) only come up in older literature from the 80s til late 90s. Why did it become out of fashion with the start up culture.

793 points by sam_bristow 3mo ago

matja 3mo ago

Apt comic: https://www.workchronicles.com/p/comic-prevention-vs-cure https://www.workchronicles.com/p/comic-prevention-vs-cure

lstodd 3mo ago

> The combined expenditure of U.S. companies on management consul- tants and training in 1997 was over $100 billion erhm, if this figure is close to true i can see what market ai companies is after.

jacques_chester 3mo ago
AI slots quite neatly into the capability trap model, actually. Which loop it belongs to in the model is left as an exercise for the reader.

Jtsummers 3mo ago

Two significant prior discussions: https://news.ycombinator.com/item?id=8940820 https://news.ycombinator.com/item?id=8940820 - 24 Jan 2015, 50 comments https://news.ycombinator.com/item?id=39472693 https://news.ycombinator.com/item?id=39472693 - 22 Feb 2024, 434 comments

jacques_chester 3mo ago

You'll see capability traps everywhere once you learn about them. Sterman, Repenning and other collaborators wrote several papers after this one. All fascinating and almost entirely depressing. Especially since MIT's Sloan school, where system dynamics first became a discipline, is just around the bend from Harvard Business school, where system dynamics first became ignored.

macrocosmos 3mo ago
One thing I don't get about the concept of capability traps is why is it expected that a company which is good at one thing would be capable at the new thing? What exactly makes a capability trap a trap?
- jacques_chester 3mo ago
  The trap is that you can't get better without first getting worse. You can't get out of the destructive cycle of production pressure and decaying productivity without removing the pressure. Many managers expect, or at least behave as if they expect, improvement to be monotonic and costless.
  - thx67 3mo ago
    When you are at capacity and in a degraded state, you have no additional headroom to get out of that state. Why wounds won't heal, or the poor stay poor.
- left-struck 3mo ago
  I think because it’s a negative feedback cycle. So once it starts it’s hard to go back, the deeper you are the harder. A trap is something that’s easy to get into and hard to get out of.

rmunn 3mo ago

Article published in the Summer 2001 edition of California Management Review, yet it never mentioned Y2K, the first thing I thought of when I read the line "fixing problems that never happened". Perhaps it was actually written in 1999 and took a while to get published, because otherwise that seems a very strange omission. The Y2K problem was very much over-hyped by the American news media at the time (no, at no point would airplanes have been falling out of the sky — I literally heard someone say that would happen once — even if no effort had been put into fixing the bug). But in recent years I have seen people (elsewhere, not on HN) claim that Y2K was a big nothingburger, and all the money spent on fixing the bug was wasted. No, that's not true either. All the money spent on fixing the bug was why it turned into a big nothingburger. Sure, some of that money was wasted, by executives who wanted an "official" Y2K-certified certificate, issued by a consulting firm that had nothing "official" about it except their own say-so. And so they spent $2 million learning what their own employees could have told them for $2,000. THAT money was wasted. But a lot of banks were running old COBOL code that used 2-digit years, and needed to be fixed. The fact that in January 2000, everyone's bank interest was still calculated correctly, and not calculated as if it was January 1900? THAT was entirely due to the vast amounts of money spent paying old COBOL coders to come out of retirement and fix the 2-digit years. The lesson I learned from that is that it's possible for a problem to be overhyped, even massively overhyped, and yet still be a serious problem. The other lesson I should have learned is that people rarely get credit (I won't go so far as the article authors and say "nobody ever gets credit") for fixing problems that never happened.

armada651 3mo ago
The problem is that a lot of people have a very binary view on life. Either something is a complete success or a complete waste of money, rarely do we accept that most projects fall somewhere in the middle.
- cortesoft 3mo ago
  And even worse, they don't think probability is a thing. If something happens, it was certain to happen and we just failed to predict it correctly. So when someone predicts something will happen with a 90% probability, and then the 10% chances happens and the predicted event does not happen, people will talk about what a bad prediction that was and how they were clearly wrong. It's the same logic that causes people to say vaccines don't work because they don't stop a disease with 100% effectiveness, or that there is no point to wear a seatbelt because people still die while wearing one.

random3 3mo ago

Like nobody gets credit for avoiding problems or unnecessary things/complexity altogether. In fact the opposite may happen.

timmg 3mo ago

There are a lot of things like this. My favorite is how elegant solutions often look simple in retrospect. So if you noodle on a problem for a while and then come up with a clever solution: once you explain it to someone they'll be like, "yeah, of course." Meanwhile the guy next to you that overcomplicates the problem ends up getting kudos for building something so difficult :D

johnthescott 3mo ago
"elegance and speed go hand in hand" - d. mcilroy
npunt 3mo ago
I feel like AI coding is accelerating everyone's work toward greater solution complexity and I think it's pushing people to build defenses and be more averse to someone else's complexity rather than being impressed by it. Bigco's are probably well behind the curve on this and are still impressed by complexity, but for people on the receiving end of AI stuff either directly via your own hand or indirectly via others, it seems like complexity is not as impressive as it once was.
- deleted 3mo ago
  [deleted]
- nomel 3mo ago
  Definitely. Previously, intent and effort were required to increase complexity. Now intent and effort are required to prevent complexity*! But also, what a beautiful problem to have!
nullhole 3mo ago
The passage that comes to mind for me whenever this idea comes up, from the Brett version of the Holmes story "The Dancing Men": H: So, Watson. W: Hmm. H: You do not propose to invest in South African securities? W: How on earth do you know that? H: Now, confess, you are utterly taken aback. W: I am! H: I should make you sign a paper to that effect. W: Why? H: Because in a few minutes you will say it is all so absurdly simple. W: I should say nothing of the kind! H: You see, my dear Watson, it is not really difficult to construct a series of inferences, each dependent upon its predecessor and each simple in itself. If, after doing so, one simply knocks out the central inferences and presents one's audience with the starting point and the conclusion, one may produce a startling, though possibly a meretricious, effect. H: I can tell by an inspection of the groove between your left forefinger and thumb, that you have decided not to invest your small capital in the gold fields. W: I can see no connection. H: Very likely not; but I can quickly give you a close connection. H: Here are the missing links in the very simple chain: You had chalk between your forefinger and thumb when you returned from the club last night. You put chalk there when you play billiards, to ease the cue. You never play billiards except with Thurston. Now, Thurston, you told me, four weeks ago, had an option on some South African security which expired in a month, and which he desired you to share with him. Your checkbook is locked in my drawer, and you have not asked for the key. So, you do not propose to invest your money in that manner. W: How absurdly simple! H: Quite so. Every problem is absurdly simple when it is explained to you.

hedora 3mo ago

Two counter-examples: - Arnold bought a fleet of mobile hospitals that would have been perfect for covid response, but the next governor didn’t want to pay 1% the fleet cost per year to maintain it, so he scrapped it. - Under Obama, SARS v1 was stopped by US health workers that Trump fired because it was a “bad deal”. In the absence of that team, we got SARS v2, which was renamed to COVID 19. There’s also the related category of “never blamed for fixing problems poorly, creating even bigger problems”. Thanks to 9/11, plane cockpits can now be locked from the inside. Now, we have examples of commercial passenger airline pilots locking the doors and committing mass-murder-suicide by plane crash. For some reason, these stories don’t make the news.

arcanemachiner 3mo ago
You might want to double check your dates on that SARS claim. Are you talking about the swine flu outbreak?

senectus1 3mo ago

sounds like my day to day job experience.

mdmabatj 3mo ago

There is something I saw on a reddit post of all places, about how every manager who doesn't predict a baseline of "3 annoying problems every month, 1 awful problem every 3 months" is essentially a bad manager. The reasoning being that, if your number of problems is under that threshold, then someone is doing a 'good job'.

keyle 3mo ago

I've been in those companies where "struggling departments" ended up getting all the praises and raise in budgets the following quarter because of the heroic saves they did, and raising awareness on how important they are... For stuff they totally caused on themselves. Meanwhile, my perfectly purring department was struggling to keep the lights on. It's a serious problem in this industry due to the disconnect between non-technical management (who understands how to double click) and engineering (who holds the company standing). <insert IBM story about IT department cost cuts> I'm not sure how we solve this, other than having management come from engineering.

kshacker 3mo ago
lol. I hate presentations. I like to run a tight ship. But that does not shine, so they made me do presentations every quarter. If you do some work, you must "take" credit. It is kinda a need when you manage people since you need to build their careers. I finally moved on to be an IC. Same story, same pressure :) You need to present to directors not because they need to know, but because your managers have a quota of N presentations per quarter, and if you back out, someone else needs to step up. Needless to say my productivity reduces by half and sometimes to almost zero during the week or fortnight of presentations every quarter.
- bruce511 3mo ago
  You define "productivity" as coding. The business defines it as "meetings, presentations, support, coding, whatever". Your productivity remains at 100% when you are doing what they want. I get that you thought you were hired as a coder, and thus measure your productivity by that. That's what I thought too. I ended up doing a lot of support (which is good, but that's another thread). Until I recalibrated my definition of productivity that frustrated me. When I realized that support was productivity I got much less frustrated.
  - kshacker 3mo ago
    When did I say I code? I have been on the industry for 35 years. I have seen my share of technology evolutions and o have seen the work from a dozen different dimensions. If after all that time, I find the process painful, just trust me -- they can't change me, and I can't change them. You take the warts with the wins and move on. 2-3 bad weeks, 10 good weeks. Life moves on to next quarter. Complete CEO mindset :)

erelong 3mo ago

So let's create moments or days of observance to make people aware of preventative measures taken

al_borland 3mo ago
A moment of silence to appreciate the silence the proactive measures gave us.
alok-g 3mo ago
I align as such. Then we soon see non-technical people start leading the same, pushing for some people to be recognized for this every sprint. Meaningless recognition starts coming in. The process fades out in just a couple weeks. The problem is there are these people in the mix, often leading, who do not understand.

Guestmodinfo 3mo ago

Human civilization runs on personal sacrifices but money bags will never care about that.

N_Lens 3mo ago
Good thing we’re converting human civilization into money bags at maximum speed, then. Solves all problems elegantly.

nxy 3mo ago

Very true! Along with it comes with peace/quietness at work so it’s not too bad.

sublinear 3mo ago

Making critical decisions without oversight is just as bad, or maybe worse. If you frame it this way in a meeting, you will get the attention you want. Don't say I didn't warn you because that comes with a lot of scrutiny you might not want.

ChicknNuggt 3mo ago

This is exactly the problem with the nature prevention. When it's well done, it seems like nothing was done.

HerbManic 3mo ago
When Covid started, our local government was very clear from the start in saying "If people think we have over reacted, then that means we have done a good job." Alas, that doesn't always fly with the populace.
- teiferer 3mo ago
  Trouble is that it is not always true either. You can legitimately overreact and in hindsight it can be hard to distinguish between these two things. Plus, even if you did overreact, that can still be the better side to have erred on, in moderation.

dang 3mo ago

Related. Others? Nobody ever gets credit for fixing problems that never happened (2001) [pdf] - https://news.ycombinator.com/item?id=39472693 https://news.ycombinator.com/item?id=39472693 - Feb 2024 (424 comments) Nobody Ever Gets Credit for Fixing Problems That Never Happened (2001) [pdf] - https://news.ycombinator.com/item?id=8940820 https://news.ycombinator.com/item?id=8940820 - Jan 2015 (50 comments)

tjmc 3mo ago

This is why I'll never be a fire protection engineer

smath 3mo ago

I'm looking for some data -- if anyone has it -- on the fraction of companies that are led (CEO) by a technical person, over the years/decades. I have the (anecdotal) impression that this fraction has been falling (stories like Boeing), but it would be cool to support or refute this with hard data. Anyone know where to find/assemble something like this? Also, if this trend is true, then why?

fsagx 3mo ago
If this has been true, perhaps at some point the pendulum will swing back the other way. BTW the current CEO of Boeing is a mechanical engineer. https://www.boeing.com/company/bios/kelly-ortberg https://www.boeing.com/company/bios/kelly-ortberg
- falsemyrmidon 3mo ago
  After decades of infamous quality drops that have impacted their reputation and finally bottom line under non technical CEOs.
doctorwho42 3mo ago
I don't have the data, but I think a good case study is the company MITRE. Originally it was engineers from the top down, but over the last 15-20 years those leaders with engineering backgrounds have retired and been replaced by non-engineer MBA's. And the more I look around, the more I see that as a common trope is the US.

didgetmaster 3mo ago

We all learned this back in first grade. The kids that behaved in class and did their homework did not command most of the teacher's time and effort. It was the problem children who refused to follow the rules and needed constant praise for every bit of actual effort that they put into their studies; that got the teacher's attention.

hattmall 3mo ago
The squeaky wheel gets the grease, is how I always heard it phrased.

jmyeet 3mo ago

This is the real problem with performance reviews in companies, which then feeds into opportunities, promotions and compensation. It's just a popularity contest. And this is particularly harmful to people who are neurodivergent, particularly if they're on the autism spectrum, because neurotypical people, who end up making all these decisions, view such people negatively for literally no reason. You could spin up a team of 6 engineers and have them go away and try some greenfield project. They could come up back in 6 months having shipped nothing. Which of these descriptions fits the facts? 1. The team learned a lot and ultimately decided there was no product-market fit and decided it was best to reallocate resources elsewhere. The learnings from that project will help a whole bunch of other projects across the division; and 2. They failed to ship and get subpar performance ratings for having no impact. The answer is... both. Or either. How you are treated will depend on how you are viewed by your management chain and that's a social function. We've all encountered people who never shut up about how hard their job is. Often they end up solving problems that they created, often by not listening to anyone that those problems would occur. And they get credit for it. You could say to people who anticipate problems to stop because it gets you nowhere. Let people fail. If only it worked that way. Instead you'll get blamed for not seeing a problem someone else created because you're viewed as competent but you aren't liked through no fault of your own. Google seems to be the posterchild for a company that briefly solved this problem and then forgot what made them successful. I am referring to Project aristotle [1], which ultimately determined that psychological safety was the key ingredient in a team's success. Now amplify all of this with constant rounds of layoffs where the environment isn't just for pay bumps and opportunities but where the cost of failing is losing your income. What you've created is an environment where office politics is everything. [1]: https://psychsafety.com/googles-project-aristotle/ https://psychsafety.com/googles-project-aristotle/

sandeepkd 3mo ago

Its really hard to measure effectiveness, problem becomes even harder when a non-engineering person has the job to measure effectiveness of a engineering person. On other hand. for software engineering some of the signals that can be used to measure such a management itself can be 1. On call requirement, outages and team burnout - A well written software should not require on-calls from the dev team 2. Ask them about the "concrete" roadmap for next 6 months to a year - Absence of concrete items is a bad sign

whatever1 3mo ago

I never gave credit to my electricity company for delivering electricity to me. I only get mad when there is an outage.

HerbManic 3mo ago
Just remember, the power grid fails in theory but works in practice.
deleted 3mo ago
[deleted]
m463 3mo ago
I think of electric vehicle fires and jet airliner crashes. Also, telsa self-driving. yes, we know about the greatly publicized accidents, or the tweets of the founder, but the avoided incidents not so much.
- thx67 3mo ago
  This is where the industry made and continues to make mistakes wrt autonomous driving. They should be able to quantitatively say how many crashes were reduced, avoided and spotted. The autonomous safety system should be running all the time and it should detect not only issues with primary vehicle but it should also catalog issues it sees in other vehicles in its vicinity. We shouldn't have gotten AD before we got automated crash avoidance.
  - projektfu 3mo ago
    My car (Honda Prologue) is a little bit of a nervous Nellie, warning me to brake when someone is turning out of my lane, but it has definitely noticed developing situations many times before I have.
    - oenton 3mo ago
      I’m curious, what’s an example of a developing situation it detected for you? I ask because after car pooling with a friend in their Tesla a few times I can confidently say I would never trust what Tesla calls FSD. Forget developing situations, there were some close calls with what’s immediately in front of it. Think- about to miss a left turn at 40mph or drive into construction cones that are blocking a lane. I think I’d prefer basic driver assistance over that.

everyone 3mo ago

The jokes around Y2K being a nothing burger always annoy me. Nothing happened because a lot of talented people worked their asses off fixing it.

throw0101c 3mo ago

Aka: * https://en.wikipedia.org/wiki/Preparedness_paradox https://en.wikipedia.org/wiki/Preparedness_paradox

teiferer 3mo ago
My favorite example is the introduction of speed limit on some accident-ridden stretch of the Autobahn north of Berlin. After introducing the speed limit, the accident numbers went down dramatically. What did the local administration decide? Remove the speed limit again -- cause there were no accidents anymore!

Viacol 3mo ago

[dead]

cm11 3mo ago

I'd guess a lot of people here consider this "reality" at this point. Has anyone come up with a response—not a fix for the company or leaders behaving this way, but a response for their own path? Did you change from a quiet diligent one to manipulating and playing the game (now that you know the game)? Did you go from quiet and diligent to quiet and not diligent (why do good work when meh work does the trick)? Another path?

staplers 3mo ago
https://ribbonfarm.com/2009/10/07/the-gervais-principle-or-the-office-according-to-the-office/ https://ribbonfarm.com/2009/10/07/the-gervais-principle-or-t...
sfn42 3mo ago
I just do my job to the best of my ability. I have to change jobs every couple years anyway to get proper pay bumps, so I don't really care what the higher ups think of me. The people near me are who I'll use as references and they generally know I'm great at what I do.

tinyhouse 3mo ago

This is very true and applies for everything in life.

throwawa1 3mo ago

Its called doing your job.

Steve16384 3mo ago
That depends. It might be going above-and-beyond your job.

protocolture 3mo ago

Depends on the business. When everyone is technical to some degree, I find that credit for technical rescue is forthcoming.

thelastgallon 3mo ago

This is how people get promoted at work. They break something, it escalates and gets visibility, emails sent to executives. Now, they 'fix' it, many thank yous from everyone for a job well done. Another version of this is delay the work you are supposed to do a long time back and let it gain visibility. Executives are blind, they can't see work being done by people who take ownership and get shit done before it becomes a problem. However, executive will remember the name of the person who breaks shit and 'saves' the day.

doublerabbit 3mo ago
The sociopathic backhand. Someone breaks it, blames you, drags your name through the mud exacerbating the issue, goes to "fix it" yet makes it worse because of their incompetence. They continue to cuss you out for it all while licking the arse of the executive. "No, we better not let doublerabbit touch that", "I don't think they're a team player". All while it was my infrastructure. And people ask why I hate humans.

CalChris 3mo ago

People don’t get credit for fixing problems that do happen. Maybe possibly in a sales scenario where your fix unblocked the sale. Otherwise nada.

insumanth 3mo ago

Boring is Better

ananthrk 3mo ago

"Titanic effect" - No captain gets credit for preventing a disaster!

inejge 3mo ago

Couldn't help noticing: In other words, it’s not just a tool problem, any more than it’s a human resources problem or a leadership problem. Instead it is a systemic problem [...] Shades of an LLMism, a bit padded, a quarter of a century ago. These days someone could easily give it a stink-eye. I'm sure that training has ingested this along with countless similar examples.

teiferer 3mo ago
That should really be a cautionary tale for everybody accusing everyone of LLM manufacturing texts. Many people write like that. The self-censoring nowadays to try to avoid sounding like an LLM is really sth we need to grow out of.

SteveGerencser 3mo ago

I began migrating from network/hardware/IT work and into marketing after nearly 2 years of heavy lifting getting ready for Y2K. In the end, "nothing happened," so all that time and money was wasted, according to nearly every company I worked with. Even had one demand a full refund. I agreed as long as I could revert all the work that I had done. They agreed, and the next day after that their entire system collapsed. I couldn't even get my own dad to pay for network support for his company since he would never pay my rate for anyone no matter what. After 2 other people failed to solve his problem I fixed it in 15 minutes and then he "really" didn't want to pay because it only took 15 minutes. I was very good at what I did but got no appreciation for keeping things from breaking, only for fixing things after they broke. Marketing paid better, and I could point at real world numbers daily and justify my pay. I don't like it anywhere near as much, but at least it gets more respect than any other IT work I did.

pulse7 3mo ago
I'm sorry your dad didn't respect your IT work...
- menaerus 3mo ago
  If I move out a bit of my circle, where people do all kinds of work, I'd say that there's generally a stigma on the "IT" workers. Moreover the stigma is there even within the IT company/industry itself, where sales, marketing, non-engineering parts of management, and other similar types of supporting roles also look down on the engineering. And this unfortunately includes family members too. I learned that people are mostly envy but when you're surrounded by many it can become overwhelming - numerous times I heard phrases like "oh, you're bricklayers of a modern age" ... like wtf
  - forshaper 3mo ago
    I do think of all my computer work as predominantly janitorial.
    - 3mo ago

Tsarp 3mo ago

Ian Rush said it best: "It's best being a striker. Miss five, score the winner, you're a hero. The goalkeeper plays a blinder, lets one in, and he's a villain." Every place I've worked rewards the firefighter over the person who made sure nothing ever caught fire. And the worst part is the math is obvious to everyone except the people who set the incentives.

joelthelion 3mo ago
How would you set the incentives, though? Almost by definition, it's hard to reward things that aren't visible. Note that there is also the flip side of the coin, people who spend all their time worrying about things that never happen, so it's not like you can just reward a defensive attitude things are more complicated than that.
- Tsarp 3mo ago
  I dont have an answer and you are mostly correct. I received some advice based on this that made sense which was to pick the roles in your career that naturally made it easier. Sales, PM, Dev etc and not support, Devops, escalation management, CSM etc.
- an0malous 3mo ago
  One basic example is not counting bugs as points in your ticket tracker. At my last job I had coworkers whose velocity was almost double everyone else’s but it was because they kept deploying and then fixing their own bugs.
- forshaper 3mo ago
  The equivalent of 'days since last injury' bonuses is the first method that comes to mind, until you consider that this would mean people would be more likely to hide things going wrong. So then many things are to rely on executive culture, and an executive who will walk the line and get their info from people at the bottom is like a unicorn. That won't scale, but it does work if you do have such an executive. Naturally they would need a basic understanding of how supply is created in their firm. Yet there is something. Toyota Hiluxes and Honda Super Cubs got popular due to maintenance ease. AK-47s. Miele vacuums. Older Thinkpads. What measures would make the human equivalent visible?

latentframe 3mo ago

Prevention is hard to sustain because the success is invisible : nobody notice the defects delays or crises that never happened

WalterBright 3mo ago

People do get credit for making things that "just work".

agumonkey 3mo ago

And people often get credit for fixing issues they partially created. Human groups work on shallow signalling and distributed confusion.

doener 3mo ago

Since Covid nearly everyone in Germany knows this saying: "There is no glory in prevention."

mihaaly 3mo ago

But the same person can get paid for it. So there is an incentive to create, or at least pretend problems.

coldtea 3mo ago

Pendatically speaking, people do get credit for fixing problems that never happened. E.g. if the problems are quantifiable and there's a record, like dropping homicides from 100 per year to 20 per year in a city. Those extra homicides "didn't happen", but the improvement is understood. For an one-off problem, it depends on how clear the path to the problem is. An electrician doing an inspection and noticing and fixing big electrical issues in the installation, would be appreciated, even if the accidents didn't happen.

dormento 3mo ago
> Those extra homicides "didn't happen", but the improvement is understood. People are gonna criticize by saying "see? it was an overreaction to the problem, since there's not been many homicides at all!", when in fact the homicides were prevented by fixing the original problem. Same way with the electrician: "how much are you gonna charge again? And you're charging for a fix to a problem that didn't happen yet? Nah, I'll call you when the problem happens". Its maddening.
Steve16384 3mo ago
> An electrician doing an inspection and noticing and fixing big electrical issues in the installation, would be appreciated, even if the accidents didn't happen. Not if nobody knew he'd fixed it.
- coldtea 3mo ago
  Electricians are no celebrities for people in general to know what they fixed or didn't fix. The person / building owner who comissioned them to do the check, will know what they fixed or what they didn't fix, from the report they give.

afisxisto 3mo ago

I remember finding a comment in the first codebase I ever worked on professionally in my first ever job. It read "This fixes a bug that hasn't happened yet". It seemed really smart at first, but later I learned that the developer that added that code also had a pattern of appending spaces to the start and end of user input and comparing the length to 2 to determine whether the value was empty or not... So I'm fairly sure "that hasn't happened yet" was probably more a case of "that I personally haven't introduced unnecessarily yet" :)

alkonaut 3mo ago

This is especially nice in the age of AI. I (the graybeard senior developer) does all the risky refactoring. I can take a performance issue and turn it into six regressions in half a day ($100). Then everyone is impressed when I let Opus fix these regressions in 20 minutes and $2 worth of AI. No one notices when you cut 20% of some expensive process but cause no regressions.

deleted 3mo ago

[deleted]

awesome_dude 3mo ago

We know that the squeaky wheel is the one that gets oiled, and we don't want the wheels to come off, so we need the wheel to squeak loud enough to be heard ;)

jdw64 3mo ago

Avengers get the glory, preventers get no story.

arkensaw 3mo ago

I feel obliged to point out Stanslav Petrov, who absolutely got credit for fixing a problem that never happened. Granted it's a very extreme case.

ekjhgkejhgk 3mo ago
Credit only in fame. https://en.wikipedia.org/wiki/Stanislav_Petrov#Aftermath https://en.wikipedia.org/wiki/Stanislav_Petrov#Aftermath > Petrov underwent intense questioning by his superiors about his judgment. Initially, he was praised for his decision.[2] Colonel-general Yuri Votintsev, the then-commander of the Soviet Air Defense's Missile Defense Units, who was the first to hear Petrov's report of the incident (and the first to reveal it to the public in the 1990s), states that Petrov's "correct actions" were "duly noted".[2] Petrov himself states he was initially praised by Votintsev and promised a reward,[2][22] but recalls that he was also reprimanded for improper filing of paperwork because he had not described the incident in the war diary.[22][23] > Petrov has said that he was neither rewarded nor punished for his actions.[24] According to Petrov, he received no reward because the incident and other bugs found in the missile detection system embarrassed his superiors and the scientists who were responsible for it, so that if he had been officially rewarded, they would have had to be punished.[2][24][22][23] He was reassigned to a less sensitive post,[23] took early retirement (although he emphasized that he was not "forced out" of the army),[22] and suffered a nervous breakdown.[23]
- pedroza_alex 3mo ago
  The same article points out that he received at least £26k in awards. It could be argued that the reward isn't proportional to the magnitude of his actions, but it exists.
- thx67 3mo ago
  His page links to https://en.wikipedia.org/wiki/Nuclear_close_calls https://en.wikipedia.org/wiki/Nuclear_close_calls which is a harrowing thing to read. We keep rolling the dice with no changes to the game.

t43562 3mo ago

I recognise almost every aspect of this document - it's exactly what's so intractable about the software business. This is why I think you do need to do some programming every now and again no matter what your level is because otherwise you cannot see what's happening and you'll be tempted into the "lazy developer" attribution.

markus_zhang 3mo ago

The title reminds me of an interesting ancient Chinese anecdote. And it is also a bit ironic that Toyota has gotten itself into some scandals recently (https://www.bbc.com/news/articles/c1wwj1p2wdyo https://www.bbc.com/news/articles/c1wwj1p2wdyo). King Wen of Wei asked Bian Que: “Of you three brothers, all physicians, who is the finest in the healing art?” Bian Que replied: “My eldest brother is the finest; my second brother comes next; I, Bian Que, am the least of the three.” King Wen said: “May I hear why?” Bian Que answered: “My eldest brother sees illness in the spirit, before it has taken shape, and removes it unseen; therefore his name is known only within our household. My second brother treats illness when it is but a hair’s breadth from appearing; therefore his name does not travel beyond our village lane. As for me, Bian Que: I pierce the blood vessels, administer strong medicines, and cut open the flesh. Thus, by such visible acts, my name has spread among the lords.”

tom2026hn 3mo ago
An ounce of prevention is worth a pound of cure.
- ashirviskas 3mo ago
  How many euros or kilograms is it?
  - projektfu 3mo ago
    Three Troy ergs.
    - deleted 3mo ago
      [deleted]
  - jimnotgym 3mo ago
    I would say in nice round numbers 50 grams prevention for a kilo of cure. That is my only criticism of the metric system, it doesn't often coincide with quantities handy for every day use, so makes poor 'sayings'.

cameronh90 3mo ago

As Futurama said, when you do things right, people won't be sure you've done anything at all.

teddyh 3mo ago
They stole that from the Tao Te Ching, chapter 17: When the Master governs, the people are hardly aware that he exists. Next best is a leader who is loved. Next, one who is feared. The worst is one who is despised. If you don’t trust the people, you make them untrustworthy. The Master doesn’t talk, he acts. When his work is done, the people say, “Amazing: we did it, all by ourselves!” — <https://ttc.tasuki.org/section:17 https://ttc.tasuki.org/section:17>

j3th9n 3mo ago

The best hackers are the ones who never get caught.

-warren 3mo ago

My time in IT has really oscillated between these two extremes: - "Everything around is working just fine. What are we paying IT for?" - "Everything is broken. What are we even paying IT for?" Personally, I strive for the former rather than the latter; I like to say "If I do my job right, you never know I'm here." But that's what got me let go. (and for karma's sake, I keep in touch with folks at the old company; it's an absolute crapshow. So I got that going for me; which is nice)

frictasolver 3mo ago

[flagged]

sharmi 3mo ago

[dead]

harimau777 3mo ago

I had this problem at a previous job. I spent almost all of my time taking care of the behind the scenes administrative work (scheduling meetings, making sure that people had the information they needed to come into the meetings prepared, etc.). However, when performance review came around I was told that the only thing that they cared about was that I hadn't completed many story points because I was too busy keeping things from falling apart. So I stopped doing all the administrative work and focused on just completing story points. A week or two later my manager asks the team "how come all of our meetings are falling apart now? We get into a meeting and no one knows what's going on."

projektfu 3mo ago
Like what would have happened to the 4077th MASH if Radar O'Reilly had been reassigned.

Sam6late 3mo ago

Here is another take in s different context.I had a very bad manger who was credited earlier with causing several companies to go out of business while he was taking advantage of having worked for a FAANG company. He worked there for less than 6 months as a business development dude when the that big shot company was new in that market.After he joined us and when I was processing some payments I noticed that he was paying some people for 2 months ahead of their starting dates, it was some $20k.I notified him, and he corrected the date,then I was told by others in the department that my problem was that I should not 'interrupt the enemy when he is making a mistake'. Eventually, the company got rid of him but it was after a very serious damage.

api 3mo ago

Chernobyl instantly comes to mind. The giant problem with that particular RBMK design was known to a few people, but nobody fixed it, and if they had... we wouldn't even know, since it would be lost to history.

pugworthy 3mo ago

This brings to mind Neil Rickert's "The Parable of the Two Programmers", which was published in the ACM SIG Software Engineering newsletter, January 1985. https://dl.acm.org/action/showFmPdf?doi=10.1145%2F1012443 https://dl.acm.org/action/showFmPdf?doi=10.1145%2F1012443 for the original, or https://realmensch.org/2017/08/25/the-parable-of-the-two-programmers/ https://realmensch.org/2017/08/25/the-parable-of-the-two-pro... for a reprint.

amelius 3mo ago

This is why Apple never gets any credits.

nostrademons 3mo ago

The headline is only half true. A more accurate rendition might be "Transactions don't happen because you fixed a problem that never occurred." That illustrates the converse. People will absolutely try to avoid future problems if they are the ones that bear the consequences for them. Use birth control (or put your kid on it) so you don't have to raise another child for 20+ years. Don't hang out with that volatile "friend" who always seems to be having another crisis. Fix your roof so you don't lose the house. Don't go into debt because you're the one who will be paying interest on it. But almost by definition, bearing the consequences of your own decisions implies that there's no transaction. It's interesting and fitting that the article begins with a discussion of Toyota. People buy Toyotas because they want to avoid problems. The biggest selling point is reliability; a Toyota's value prop is that you can keep it for 20 years and you won't have unexpected things go wrong with it. Toyota has managed to turn this into a sales driver because they appeal to the self-interest of a buyer who will be living with the car for 10-20 years. There are thousands of individual decisions that Toyota makes to avoid problems with their cars, starting with a very conservative aversion to new technology or anything that is engineeringly risky. Each individual one is invisible to the customer, and often comes with significant costs in isolation. But because their sales driver is "be reliable at all costs" and they've ingrained that into the culture, they've built an organization that is willing to make these feature trade-offs for reliability. Also, an interesting corollary is "Oftentimes, a good life is lived with few transactions."

Root_Access 3mo ago

This is true but if you remove the need for credit then you can just get back to work and not have to create a category about it.

kouru225 3mo ago

Hikaru Nakamura had a good comment on famous chess moves like this: All the famous chess moves are famous sacrifices, genius saves, and last minute checkmates, but when you plug the games into stockfish you find out that these scenarios only happened cause the players made a series of awful moves early on in the game

abustamam 3mo ago

This is interesting because you can't prove you prevented something from happening. It never happened after all. Hank Green has an interesting video on preventative measures are often unsung, because how do you know? https://youtu.be/ndeB_BpsRGk?is=03lR2DMyhfwkT3L- https://youtu.be/ndeB_BpsRGk?is=03lR2DMyhfwkT3L- Kids in Africa who would have otherwise died from measles, but didn't get measles in the first place because some people decided to fund and make vaccines and thousands of unnamed and unrecognized volunteers helped distribute and administer these vaccines... These kids are alive and they don't even know that they wouldn't be alive if it weren't for some people they may never meet, and whose names they'll never know. Also tangentially, reminds me of an excerpt of a book called The Trigger Effect I read as a kid, which said something along the lines of, when you use your car's brakes to prevent an accident, your life depends on whoever assembled the brakes and quality controlled them. You'll never know their name, and they'll never know that they saved your life.

AndrewChamp 3mo ago

"When you things right, no one will ever know you did anything at all" - Futurama

smolder 3mo ago

Yeah, I have fixed issues that could have potentially destroyed the company I was working at. There was an instance of a lazily written script which just took user input for a file open. It could read sensitive files on the system because of lazy development. The other devs praised me for fixing it, but I didn't get any recognition comparable to the losses our company could have had if it got exploited.

thelastgallon 3mo ago

Doing nothing at work: https://news.ycombinator.com/item?id=48442880 https://news.ycombinator.com/item?id=48442880 The first comment is spot on. https://news.ycombinator.com/item?id=48496420 https://news.ycombinator.com/item?id=48496420: Time and time again at many companies, including well-reputed ones, I have seen that preventing issues gets you no recognition, but building a giant pile of kindling and then putting out the inevitable fire will get you recognition twice. Even in "good" orgs.

eskimobloood 3mo ago

I wonder why the whole topic of system thinking comes (self-enforcing/balancing feedback loops etc.) only come up in older literature from the 80s til late 90s. Why did it become out of fashion with the start up culture.