8 ms·

100% and I’m a software developer and have been for ~30 years. Good QA people know how to find regression and bugs _that you didn’t think about_ which is the wh

100% and I’m a software developer and have been for ~30 years. Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. One of the QA people I work with currently is one of my favorite people. They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software, but in the end it makes a better, more resilient product.

LatencyKills 6mo ago
Exactly. I spent 20 years split between MS and Apple. Some of the best people I ever worked with were in QA. One guy in particular was an extremely talented engineer who simply didn't enjoy the canonical "coding" role; what he did enjoy was finding bugs and breaking things. ;-)
- MoreQARespect 6mo ago
  Really? The best people I worked with were never QA. Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field. I wish it werent so (hence my username) but there is a definite class divide between devs and QA and it shows up not just in terms of the pay packets but also who gets the boot in down times and who gets listened to. This definitely affects the quality of people. I think it's overdue an overhaul much like the sysadmin->devops transition.
  - LatencyKills 6mo ago
    We have differing experiences, which shouldn't be surprising. My example explicitly referred to someone who was a good engineer who enjoyed the QA role. This might have been an Apple/MS thing, but we always had very technical QA people on the dev tools team. For example, the QA lead for the C++ compiler had written their own compiler from scratch and was an amazing contributor.
    - stiglitz 6mo ago
      In the Windows team (back before the test org was decimated) I saw the described "class divide". Anybody who was good enough would switch from SDET to SDE [disclaimer: obviously there were some isolated exceptions]. The test team produced reams of crappy test frameworks, each of which seemed like a "proving project" for its creators to show they could be competent SDEs. After the Great Decimation my dev team took ownership of many such frameworks and it was a total boondoggle; we wasted years trying (and mostly failing) to sort through the crappy test code. This was all unfortunate, and I agree in principle with having a separate test org, but in Windows the culture unfortunately seemed to be built around testers as second-class software developers.
      - LatencyKills 6mo ago
        I spent most of my time working on Visual Studio (in the Boston time frame) so we got to interact with pretty much every team. I absolutely hated interacting with the Windows team. Everything was a fight for no reason. As I said above, everyone has their own experiences but the QA folks I worked with at MS were fantastic. Not sure if you're aware but Dave Plumber now has a really good YT channel [0] where he talks about MS back in those days. It's a fun walk down memory lane. [0]: https://www.youtube.com/@DavesGarage https://www.youtube.com/@DavesGarage
  - pixl97 6mo ago
    I mean the people that come up thru QA may be the best while getting enough time in the company to go to a position that pays. But yea, so many companies cheap their QA and then wonders why their QA sucks.
  - saghm 6mo ago
    > Really? The best people I worked with were never QA. > Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field. That sort of seems circular. If they're not respected or paid well, of course most of the talented people would not want to remain in QA, and eventually you'd just have mediocre QA. That doesn't really give you any insight into whether high quality QA would be useful though. (edit: I see now that's basically the point you're trying to make, so I guess we're in agreement)
stingraycharles 6mo ago
> Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” I don’t understand the reasoning here why QA shouldn’t be engineering.
- 9wzYQbTYsAIc 6mo ago
  > I don’t understand the reasoning here why QA shouldn’t be engineering. Who watches the watcher, right? That aside, the core idea is the same as the principles of independent audit, peer review, or even simply just specialization. Red team / Blue team?
  - stingraycharles 6mo ago
    Yes but both the red team and blue team would still be engineering.
    - 9wzYQbTYsAIc 6mo ago
      Yes, but police and military are both law enforcement, on one level, but each are very different from the other. Even the military have police, right? edit: ultimately, it comes down to the importance of independent audit, the builders and the breaker/fixers are very different groups in engineering.
    - IAmBroom 6mo ago
      The red team and blue team should not share supervisors. Nor, in the case of QA, should the audit team be engineers trained to act and think like the ones who wrote the software. A fresh perspective is useful. But in the long run, supervisory independence is the real deal. I know of a QA manager who shut down an entire factory's output until a major safety issue (that had been kicked down the road several times) was addressed. It took chutzpah, and serious power, to do that. The Dir. of Engrg. would NEVER have allowed it.
- liampulles 6mo ago
  Frankly, calling software development engineering is quite debatable. We should be calling less things engineering that aren't actually engineering qualifications.
canpan 6mo ago
Yes, QA is important. My code will always "work" in that everything I tested is bug free. But having someone other test, especially someone who knows the service is gold. But there is also bad QA: The most worthless QA I was forced to work with, was an external company, where I, as developer, had to write the test sheet and they just tested that. Obviously they could not find bugs as I tested everything on the sheet. My most impressive QA experience where when I helped out a famous Japanese gaming company. They tested things like press multiple buttons in the same frame and see my code crash.
- Ekaros 6mo ago
  I do think the type of testing where QA just follows pre-generated script has place. But it is about long term regression. The first round absolutely should not find anything. But with complex system it also should find nothing in a year or three or five years... Offloading this to dedicate resource could be useful in certain industries.
  - canpan 6mo ago
    I did not think of that. Maybe for some industries, it might make sense. But if I want a regression test, I would probably set it up as automated test. In the case I mentioned above it was the only test beside my own for a new service.
- darth_aardvark 6mo ago
  > But there is also bad QA: The most worthless QA I was forced to work with, was an external company, where I, as developer, had to write the test sheet and they just tested that. Obviously they could not find bugs as I tested everything on the sheet. This was my sole experience at the one place I worked with an internal QA team. They absolutely could never find bugs that devs missed, often mis-marked ones that didn't exist, and failed to find obvious edge cases that did exist. Multiple devs fired because the CEO believed the QA over the engineering team; if they marked a bug as present, it was the engineer's fault for writing it. If they didn't catch a bug that made it to prod, it was the engineer's fault for not including it in the test plan. They represented nothing but red tape and provided no value. Good QA sounds great! I'd love to know what that's like someday! It'd be great to have someone testing my code and finding breakages I missed! I'm only slightly (incredibly) bitter about my bad experience with its implementation.
jeremyjh 6mo ago
Agreed. QA specialists are there to think about what the engineer didn't think about. Unless the engineer is incompetent or the organization is broken, the engineer has already written tests for everything they could think of, but they can't think of everything. More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe to work than to ship a feature on time. Like everything else though, its contextual. Complexity of domain, surface area and age of product, depth of experience on team and consequences of failure are all so variable that there cannot be only one answer. I have done it both ways for many years. I have worked on teams where QA is a frustrating nuisance, and teams where they were critical to success. I have worked on teams that did pretty good without them, and probably those were the highest throughput, most productive teams because the engineers were forced to own all the consequences - every bug they shipped was a production issue they were immediately forced to track down and resolve. But those were very small teams, and eventually I was the only founding engineer left on the team and far too many mistakes by other people made it to my desk because I was the only person who could find them in review or track them down quickly in production. That was when I started hiring QA people.
- 9wzYQbTYsAIc 6mo ago
  > More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe to work than to ship a feature on time. Personal liability and professional insurance works for all the actual “professions” in the US, to some extent, right? It might be time to start the considerations for professional licensing for platform scale or commercially published software.
  - ivan_gammel 6mo ago
    More like certified products. New ISO standard may require professional liability for software products, which will be adopted as requirement by big consumers and will pull the industry into certification loop, because insurers will ask for it. This will obviously put a high entry barrier to many product categories, slowing down innovation.
jghn 6mo ago
The tension is that QA is important. But most QA practitioners are not good. The world is filled with QA people who couldn't make it as an SWE and now are button pushers. But high quality QA people are amazing. These are the ones who understand how to break apart a system, push it to its limits, and engineer the quality plan. This is an area where I expect AI to create a bimodal future. The smaller group of high quality QA people will now be able to offload the activity to agents instead of the QA drones. They'll still be worth their weight in gold, whereas the drones will be redundant.
- malfist 6mo ago
  I've worked with a lot of QA folks that just repackage up the unit tests the dev already writes. And I've met a few that strikes out on their own and comes up with real tests. The latter is much more high touch, but they're often worth their weight in gold. The former is kinda pointless.
  - jghn 6mo ago
    Exactly. I think AI tooling will make that good group even more effective. And it will make the bad group even more pointless.
_doctor_love 6mo ago
Couldn't agree more strongly. The best QA people have the old-school “test to break” mentality. They do weird things, like pulling the network cable out of their machine mid-transaction, or kicking off a series of performance scripts and then powering off servers in a distributed system just to see what happens. When I got started in software, QA was already in a heavy decline. A mentor who had been a QA manager at Apple told me that being a QA person (in the industry) was once seen as a high-trust position. He was sad at what it became.
- cindyllm 6mo ago
  [dead]
- ffsm8 6mo ago
  When I got into software that employer was pretty small (50 people overall I think). Their approach to QA was to have this be a optional thing the service people could do. It worked surprisingly well, with the caveat that they never created regression tests. The employer eventually switched someone from that team to establish these regression tests full time, but they had no programming experience and by the time I left no real progress was done in around 6 month. No idea what came of that, and a few years later they fired a lot of the team
z500 6mo ago
We don't have dedicated QA at my job, but personally I'm thrilled when a coworker finds a bug in my code. It's much more preferable to the customer finding it
- D-Coder 6mo ago
  Yes! I had a QA person apologize once for finding a bug in my code. I thanked him for not letting it make me look like an idiot.
efsavage 6mo ago
Great QA people are rarer than great developers, and potentially even more valuable.
- pphysch 6mo ago
  Yes, QA and test engineers tend to have a better ability to specify the correct behavior of a system than anyone else. It's literally their job. This is a big asset in the current paradigm where a LLM agent can effectively implement behavior only when given a robust test suite to iterate against.
mememememememo 6mo ago
It should be under engineering but as a seperate skill/role. To ship fast you want QA aligned with milestones, sprints etc. You want QA to feel the deadlines, and then like an Eng push up if things aren't realistic. What I have seen before with QA is a queue like system. Hand to QA. get back in 3 days. Not the QA persons fault. It is the setup if the org that is wrong. Imagine if you had a code review role and code review team and you had to wait to get your code OKed. We wouldn't put up with it. QA teams seperate from Eng worked when the software was burned to CD rather than uploaded to s3.
dotancohen 6mo ago
> They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software, but in the end it makes a better, more resilient product. Did you try with 2 QA people? 65536 QA people? 0 QA people? NULL QA people?
Perepiska 6mo ago
> with their bugs This is funny.
locknitpicker 6mo ago
> Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. I think the core of the issue is the weasel word "Good QA" instead of just "QA". You're underlining the importance of having someone in a team who has good product understanding and has good ownership skills. How many QAs fit that description? Not many, unfortunately. Some FANGs outright eliminated the role and replaced it with a mix of product owners and team ownership, and some QAs are just doing their 9-to-5 job going through their manual test scripts to verify it something meets a definition of done. What happens when you can have a LLM agent doing the same instantly as a step in a random CICD pipeline?

LatencyKills 6mo ago

Exactly. I spent 20 years split between MS and Apple. Some of the best people I ever worked with were in QA. One guy in particular was an extremely talented engineer who simply didn't enjoy the canonical "coding" role; what he did enjoy was finding bugs and breaking things. ;-)

MoreQARespect 6mo ago
Really? The best people I worked with were never QA. Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field. I wish it werent so (hence my username) but there is a definite class divide between devs and QA and it shows up not just in terms of the pay packets but also who gets the boot in down times and who gets listened to. This definitely affects the quality of people. I think it's overdue an overhaul much like the sysadmin->devops transition.
- LatencyKills 6mo ago
  We have differing experiences, which shouldn't be surprising. My example explicitly referred to someone who was a good engineer who enjoyed the QA role. This might have been an Apple/MS thing, but we always had very technical QA people on the dev tools team. For example, the QA lead for the C++ compiler had written their own compiler from scratch and was an amazing contributor.
  - stiglitz 6mo ago
    In the Windows team (back before the test org was decimated) I saw the described "class divide". Anybody who was good enough would switch from SDET to SDE [disclaimer: obviously there were some isolated exceptions]. The test team produced reams of crappy test frameworks, each of which seemed like a "proving project" for its creators to show they could be competent SDEs. After the Great Decimation my dev team took ownership of many such frameworks and it was a total boondoggle; we wasted years trying (and mostly failing) to sort through the crappy test code. This was all unfortunate, and I agree in principle with having a separate test org, but in Windows the culture unfortunately seemed to be built around testers as second-class software developers.
    - LatencyKills 6mo ago
      I spent most of my time working on Visual Studio (in the Boston time frame) so we got to interact with pretty much every team. I absolutely hated interacting with the Windows team. Everything was a fight for no reason. As I said above, everyone has their own experiences but the QA folks I worked with at MS were fantastic. Not sure if you're aware but Dave Plumber now has a really good YT channel [0] where he talks about MS back in those days. It's a fun walk down memory lane. [0]: https://www.youtube.com/@DavesGarage https://www.youtube.com/@DavesGarage
- pixl97 6mo ago
  I mean the people that come up thru QA may be the best while getting enough time in the company to go to a position that pays. But yea, so many companies cheap their QA and then wonders why their QA sucks.
- saghm 6mo ago
  > Really? The best people I worked with were never QA. > Moreover, the best QAs would almost always try to be not QA - to shift into a better respected and better paid field. That sort of seems circular. If they're not respected or paid well, of course most of the talented people would not want to remain in QA, and eventually you'd just have mediocre QA. That doesn't really give you any insight into whether high quality QA would be useful though. (edit: I see now that's basically the point you're trying to make, so I guess we're in agreement)

stingraycharles 6mo ago

> Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” I don’t understand the reasoning here why QA shouldn’t be engineering.

9wzYQbTYsAIc 6mo ago
> I don’t understand the reasoning here why QA shouldn’t be engineering. Who watches the watcher, right? That aside, the core idea is the same as the principles of independent audit, peer review, or even simply just specialization. Red team / Blue team?
- stingraycharles 6mo ago
  Yes but both the red team and blue team would still be engineering.
  - 9wzYQbTYsAIc 6mo ago
    Yes, but police and military are both law enforcement, on one level, but each are very different from the other. Even the military have police, right? edit: ultimately, it comes down to the importance of independent audit, the builders and the breaker/fixers are very different groups in engineering.
  - IAmBroom 6mo ago
    The red team and blue team should not share supervisors. Nor, in the case of QA, should the audit team be engineers trained to act and think like the ones who wrote the software. A fresh perspective is useful. But in the long run, supervisory independence is the real deal. I know of a QA manager who shut down an entire factory's output until a major safety issue (that had been kicked down the road several times) was addressed. It took chutzpah, and serious power, to do that. The Dir. of Engrg. would NEVER have allowed it.
liampulles 6mo ago
Frankly, calling software development engineering is quite debatable. We should be calling less things engineering that aren't actually engineering qualifications.

canpan 6mo ago

Yes, QA is important. My code will always "work" in that everything I tested is bug free. But having someone other test, especially someone who knows the service is gold. But there is also bad QA: The most worthless QA I was forced to work with, was an external company, where I, as developer, had to write the test sheet and they just tested that. Obviously they could not find bugs as I tested everything on the sheet. My most impressive QA experience where when I helped out a famous Japanese gaming company. They tested things like press multiple buttons in the same frame and see my code crash.

Ekaros 6mo ago
I do think the type of testing where QA just follows pre-generated script has place. But it is about long term regression. The first round absolutely should not find anything. But with complex system it also should find nothing in a year or three or five years... Offloading this to dedicate resource could be useful in certain industries.
- canpan 6mo ago
  I did not think of that. Maybe for some industries, it might make sense. But if I want a regression test, I would probably set it up as automated test. In the case I mentioned above it was the only test beside my own for a new service.
darth_aardvark 6mo ago
> But there is also bad QA: The most worthless QA I was forced to work with, was an external company, where I, as developer, had to write the test sheet and they just tested that. Obviously they could not find bugs as I tested everything on the sheet. This was my sole experience at the one place I worked with an internal QA team. They absolutely could never find bugs that devs missed, often mis-marked ones that didn't exist, and failed to find obvious edge cases that did exist. Multiple devs fired because the CEO believed the QA over the engineering team; if they marked a bug as present, it was the engineer's fault for writing it. If they didn't catch a bug that made it to prod, it was the engineer's fault for not including it in the test plan. They represented nothing but red tape and provided no value. Good QA sounds great! I'd love to know what that's like someday! It'd be great to have someone testing my code and finding breakages I missed! I'm only slightly (incredibly) bitter about my bad experience with its implementation.

jeremyjh 6mo ago

Agreed. QA specialists are there to think about what the engineer didn't think about. Unless the engineer is incompetent or the organization is broken, the engineer has already written tests for everything they could think of, but they can't think of everything. More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe to work than to ship a feature on time. Like everything else though, its contextual. Complexity of domain, surface area and age of product, depth of experience on team and consequences of failure are all so variable that there cannot be only one answer. I have done it both ways for many years. I have worked on teams where QA is a frustrating nuisance, and teams where they were critical to success. I have worked on teams that did pretty good without them, and probably those were the highest throughput, most productive teams because the engineers were forced to own all the consequences - every bug they shipped was a production issue they were immediately forced to track down and resolve. But those were very small teams, and eventually I was the only founding engineer left on the team and far too many mistakes by other people made it to my desk because I was the only person who could find them in review or track them down quickly in production. That was when I started hiring QA people.

9wzYQbTYsAIc 6mo ago
> More importantly, it is almost impossible for engineers to be as well incentivized to spend extra time exploring edge cases in something they already believe to work than to ship a feature on time. Personal liability and professional insurance works for all the actual “professions” in the US, to some extent, right? It might be time to start the considerations for professional licensing for platform scale or commercially published software.
- ivan_gammel 6mo ago
  More like certified products. New ISO standard may require professional liability for software products, which will be adopted as requirement by big consumers and will pull the industry into certification loop, because insurers will ask for it. This will obviously put a high entry barrier to many product categories, slowing down innovation.

jghn 6mo ago

The tension is that QA is important. But most QA practitioners are not good. The world is filled with QA people who couldn't make it as an SWE and now are button pushers. But high quality QA people are amazing. These are the ones who understand how to break apart a system, push it to its limits, and engineer the quality plan. This is an area where I expect AI to create a bimodal future. The smaller group of high quality QA people will now be able to offload the activity to agents instead of the QA drones. They'll still be worth their weight in gold, whereas the drones will be redundant.

malfist 6mo ago
I've worked with a lot of QA folks that just repackage up the unit tests the dev already writes. And I've met a few that strikes out on their own and comes up with real tests. The latter is much more high touch, but they're often worth their weight in gold. The former is kinda pointless.
- jghn 6mo ago
  Exactly. I think AI tooling will make that good group even more effective. And it will make the bad group even more pointless.

_doctor_love 6mo ago

Couldn't agree more strongly. The best QA people have the old-school “test to break” mentality. They do weird things, like pulling the network cable out of their machine mid-transaction, or kicking off a series of performance scripts and then powering off servers in a distributed system just to see what happens. When I got started in software, QA was already in a heavy decline. A mentor who had been a QA manager at Apple told me that being a QA person (in the industry) was once seen as a high-trust position. He was sad at what it became.

cindyllm 6mo ago
[dead]
ffsm8 6mo ago
When I got into software that employer was pretty small (50 people overall I think). Their approach to QA was to have this be a optional thing the service people could do. It worked surprisingly well, with the caveat that they never created regression tests. The employer eventually switched someone from that team to establish these regression tests full time, but they had no programming experience and by the time I left no real progress was done in around 6 month. No idea what came of that, and a few years later they fired a lot of the team

z500 6mo ago

We don't have dedicated QA at my job, but personally I'm thrilled when a coworker finds a bug in my code. It's much more preferable to the customer finding it

D-Coder 6mo ago
Yes! I had a QA person apologize once for finding a bug in my code. I thanked him for not letting it make me look like an idiot.

efsavage 6mo ago

Great QA people are rarer than great developers, and potentially even more valuable.

pphysch 6mo ago
Yes, QA and test engineers tend to have a better ability to specify the correct behavior of a system than anyone else. It's literally their job. This is a big asset in the current paradigm where a LLM agent can effectively implement behavior only when given a robust test suite to iterate against.

mememememememo 6mo ago

It should be under engineering but as a seperate skill/role. To ship fast you want QA aligned with milestones, sprints etc. You want QA to feel the deadlines, and then like an Eng push up if things aren't realistic. What I have seen before with QA is a queue like system. Hand to QA. get back in 3 days. Not the QA persons fault. It is the setup if the org that is wrong. Imagine if you had a code review role and code review team and you had to wait to get your code OKed. We wouldn't put up with it. QA teams seperate from Eng worked when the software was burned to CD rather than uploaded to s3.

dotancohen 6mo ago

> They don’t always make me happy (in the moment) with their bugs or with how they decide to break the software, but in the end it makes a better, more resilient product. Did you try with 2 QA people? 65536 QA people? 0 QA people? NULL QA people?

Perepiska 6mo ago

> with their bugs This is funny.

locknitpicker 6mo ago

> Good QA people know how to find regression and bugs _that you didn’t think about_ which is the whole reason why it shouldn’t be under “engineering” and that it should exist. I think the core of the issue is the weasel word "Good QA" instead of just "QA". You're underlining the importance of having someone in a team who has good product understanding and has good ownership skills. How many QAs fit that description? Not many, unfortunately. Some FANGs outright eliminated the role and replaced it with a mix of product owners and team ownership, and some QAs are just doing their 9-to-5 job going through their manual test scripts to verify it something meets a definition of done. What happens when you can have a LLM agent doing the same instantly as a step in a random CICD pipeline?