8 ms·

Why unit testing is a waste of time

34 points by ine8181 17y ago

swombat 17y ago
Reading such a long article with such a moot point is a waste of time... More seriously, the point of the article, from a skim, appears to be that "writing too many unit tests is a waste of time". Well, duh. Save 15 minutes of your life and pass on this one, methinks.
- jerryji 17y ago
  Thank you very much!
- tmtmtm 17y ago
  Hi, I'm the original author. That wasn't the point I was trying to make at all - despite the attention grabbing headline it's really talking about the value of functional and integration tests. In retrospect I wish I'd used a different title and skipped the first few paragraphs - I'd intended it to be a bit of humorous indirection making it sound like the post was going to be against automated testing whereas it is really very much in favour of it, but unfortunately people have taken it the wrong way.
  - scott_s 17y ago
    The internet is filled with authors who do that trick. It's a bait-and-switch, and I know I don't like reading it. People didn't "take it the wrong way." You did not clearly express yourself. If you want to talk about the value of functional and integration tests, then talk about that.
    - tmtmtm 17y ago
      Hi Scott - I admit it was a cheap trick. This is the first time I've had something like this posted to an audience so I've had one or two things to learn! I'd appreciate it if anyone interested in the post would skip the first few paragraphs and start at "Benefits of automated functional tests" since that was what I was intending to convey.
      - scott_s 17y ago
        I think a better idea is to just edit your essay directly. Maybe even re-submit the edited version to HN - you may find people are more receptive to discuss the content you wanted to convey.
      - swombat 17y ago
        I wouldn't necessarily discard the trick altogether. However, I'd say that based on my experiences/intuitions, it only works for short posts, and you have to clarify the point of the post by the time you get to the second paragraph. The trick was simply not the right thing for such a long article. Packaging really matters where blog posts are concerned... the hook needs to be crafted well for each article, or people won't bite. I'd take scott's advice and just repost... many people will probably read it then, since the subject is relevant.
        tmtmtm 17y ago
        I've changed the title on the post and added an Update to the top of it. I'd rather not resubmit to hacker news since I think it's taken up enough attention already. Thanks to you both for the suggestions.
        10ren 17y ago
        I would appreciate it if you would resubmit, as all the top comments here are complaints about a different (version of the) essay. Generally, people like having their attention taken up, provided you follow their advice. :-)
        sutro 17y ago
        Too bad you didn't have the stones to stand by your original claim. Most unit testing is a waste of time. A crazy orthodoxy has arisen around it: never question the value of massive unit testing, just do it! Test-driven development is cumbersome and frequently useless. If you're really doing trapeze acrobatics you need to work with a net, but most of us are just walking down the street, in which case rigging up a bunch of elaborate safety netting is just overkill. Commence downmodding in 3, 2, 1...
  - nomoresecrets 17y ago
    Indirection? Misdirection.
- mixmax 17y ago
  This is a pretty good quote though: "I think sometimes we take for granted how fortunate we are as developers to be able to make such a direct impact on problems instead of being wholly dependent on others."
- zacharypinter 17y ago
  I thought it was an interesting article. If you want to skip the long read, a better summary would be "you can save a lot of code and hassle by writing a few smart functional tests instead of a ton of unit tests." I continue to assert that most unit tests are the same type of drudge work that good programmers try to avoid in other scenarios (data entry, text editing, email templates, deploy scripts, etc).
- mahmud 17y ago
  No, unit tests are the new API documentation. If you're too lazy to write good docs, please, at least leave me some tests.
- roder 17y ago
  long article... moot point... unit & functional testing is important. My friend made an awesome analogy to the real world application here: http://app.arat.us/blog/?p=159 http://app.arat.us/blog/?p=159
fb 17y ago
Is there anyone who have really invested time to learn proper unit testing and still thinks it is not one of the best programming inventions ever? Seriously, there are lots of naive posts lately on HN...
- maurycy 17y ago
  What are good resources to learn proper unit testing, by the way? (no offensive; curious)
  - steveklabnik 17y ago
    I personally just sort of figured it out by doing BDD on a few small projects to get a feel for how things go. I'm not 100% "driven" now, but overall, it's pretty high. I tend to do Cucumber tests more than straight up unit testing, as I find them more valuble personally... Doing is the best way to learn most things in the software world.
  - makecheck 17y ago
    It definitely helps to start with an environment where tests are easy to write, so that you can focus on the principles of testing without having to learn a lot of mechanics. Python's built-in "unittest" module is an example of that. While it's not perfect, you can quickly form good habits by using it. And it is not limited to Python. For instance, I've written unittest-compatible classes that basically run other programs as test cases. Either way, you're forced to think about things like "how do I automatically detect that this failed?" and "is the purpose of this test clear?".
  - danprager 17y ago
    Kent Beck's book: Test-Driven Development Don't just read it: Work your way through it. You need to follow the recipe to develop the discipline/knack. Fairly quick and reasonably easy.
  - silentbicycle 17y ago
    I like _Working Effectively with Legacy Code_ by Michael Feathers. It's a collection of techniques for bootstrapping tests into an existing code base - carefully, incrementally restructuring the code to add enough tests to safely perform deeper structural changes. At the same time, it's ultimately a book about improving legacy codebases (testing is just a major technique), so it focuses on using testing when it's most useful. (Also, maybe it's just me, but a book that starts from, "Help! I just inherited a C++ project with ten years of rot and half the comments are in Norwegian! How do I even start adding tests to this without breaking it further?" seems a bit more practical than one that starts by applying unit tests to example code designed for convenient testing. It's seldom that easy.)
philjackson 17y ago
I simply can't imagine developing without unit tests now. I very rarely (never) get things spot on the first time and the unit tests help me maintain pride and position before most of my big merges. Sometimes I do wonder if I rely on them too much, but they aren't a waste of time.
geebee 17y ago
One of the really great things about software development is that reality, rather than a committee, gets the final word. I think that unit testing is a great idea, in theory and in practice, but I do think that the unit testing may get to the point where some practitioners who are not careful will spend far longer to complete a project with no discernible improvement in stability or maintainability than someone who uses unit testing more judiciously. I have used the end-to-end automated functional test approach before, and it did enable me to find problems pretty quickly. However, I would have died a slow and painful mental death without unit tests in some of the tricker sections of my code. That said, yeah, "unit testing is a waste of time" is link-bait-ish.
makecheck 17y ago
I do agree with the point that it is possible to spend a lot of time fixing "broken" tests. The lesson that should often be learned is that certain tests are not adding enough value to warrant the constant maintenance. (Code alone requires plenty of maintenance as it is.) One thing that works extremely well is to have a test environment. In other words, spend the time to replicate your entire production flow 100% (but in isolation), so that you can "deploy" changes and see if anything blows up. This not only catches stupid deployment bugs, but it also allows you to avoid some unit testing by simply relying on customers themselves: have them log into the isolated environment and do exactly what they would have done in production, and you'll know if it works.
danprager 17y ago
Good, but overly long article. I have reached the similar conclusions in favor of functional tests over the last decade or so. Personally, I have worked seriously used DBC, TDD, and automated scenario testing, and hybrids in that order of learning. My conclusions about the sweet-spot (and this may vary depending on your area): 1. Automated functional testing in the large, and DBC in the small is a winning combo. 2. Special techniques for graphics intensive programming: I get my scenario tests to show what's happening on the screen, and I can slow 'em down and step through 'em 3. I never got a lot of dependency injection / mocks (like the author) -- not sure if I don't get 'em or application area is wrong 4. You need special techniques for intensive algorithmic code where there's a combinatorial explosion of cases and line coverage won't do; e.g. randomly generated test cases + compute intensive sanity checks Bottom-line: Automated scenario tests exercise the code; DBC pin-points illegal states. DBC = Design by Contract = pre-conditions + post-conditions + invariants. In practice understanding the method (this takes work) and writing explicit pre-conditions gives ~70-80% of the benefit. To do invariants well needs language support.

34 points by ine8181 17y ago

Reading such a long article with such a moot point is a waste of time... More seriously, the point of the article, from a skim, appears to be that "writing too many unit tests is a waste of time". Well, duh. Save 15 minutes of your life and pass on this one, methinks.

jerryji 17y ago
Thank you very much!
tmtmtm 17y ago
Hi, I'm the original author. That wasn't the point I was trying to make at all - despite the attention grabbing headline it's really talking about the value of functional and integration tests. In retrospect I wish I'd used a different title and skipped the first few paragraphs - I'd intended it to be a bit of humorous indirection making it sound like the post was going to be against automated testing whereas it is really very much in favour of it, but unfortunately people have taken it the wrong way.
- scott_s 17y ago
  The internet is filled with authors who do that trick. It's a bait-and-switch, and I know I don't like reading it. People didn't "take it the wrong way." You did not clearly express yourself. If you want to talk about the value of functional and integration tests, then talk about that.
  - tmtmtm 17y ago
    Hi Scott - I admit it was a cheap trick. This is the first time I've had something like this posted to an audience so I've had one or two things to learn! I'd appreciate it if anyone interested in the post would skip the first few paragraphs and start at "Benefits of automated functional tests" since that was what I was intending to convey.
    - scott_s 17y ago
      I think a better idea is to just edit your essay directly. Maybe even re-submit the edited version to HN - you may find people are more receptive to discuss the content you wanted to convey.
    - swombat 17y ago
      I wouldn't necessarily discard the trick altogether. However, I'd say that based on my experiences/intuitions, it only works for short posts, and you have to clarify the point of the post by the time you get to the second paragraph. The trick was simply not the right thing for such a long article. Packaging really matters where blog posts are concerned... the hook needs to be crafted well for each article, or people won't bite. I'd take scott's advice and just repost... many people will probably read it then, since the subject is relevant.
      - tmtmtm 17y ago
        I've changed the title on the post and added an Update to the top of it. I'd rather not resubmit to hacker news since I think it's taken up enough attention already. Thanks to you both for the suggestions.
        10ren 17y ago
        I would appreciate it if you would resubmit, as all the top comments here are complaints about a different (version of the) essay. Generally, people like having their attention taken up, provided you follow their advice. :-)
        sutro 17y ago
        Too bad you didn't have the stones to stand by your original claim. Most unit testing is a waste of time. A crazy orthodoxy has arisen around it: never question the value of massive unit testing, just do it! Test-driven development is cumbersome and frequently useless. If you're really doing trapeze acrobatics you need to work with a net, but most of us are just walking down the street, in which case rigging up a bunch of elaborate safety netting is just overkill. Commence downmodding in 3, 2, 1...
- nomoresecrets 17y ago
  Indirection? Misdirection.
mixmax 17y ago
This is a pretty good quote though: "I think sometimes we take for granted how fortunate we are as developers to be able to make such a direct impact on problems instead of being wholly dependent on others."
zacharypinter 17y ago
I thought it was an interesting article. If you want to skip the long read, a better summary would be "you can save a lot of code and hassle by writing a few smart functional tests instead of a ton of unit tests." I continue to assert that most unit tests are the same type of drudge work that good programmers try to avoid in other scenarios (data entry, text editing, email templates, deploy scripts, etc).
mahmud 17y ago
No, unit tests are the new API documentation. If you're too lazy to write good docs, please, at least leave me some tests.
roder 17y ago
long article... moot point... unit & functional testing is important. My friend made an awesome analogy to the real world application here: http://app.arat.us/blog/?p=159 http://app.arat.us/blog/?p=159

fb 17y ago

Is there anyone who have really invested time to learn proper unit testing and still thinks it is not one of the best programming inventions ever? Seriously, there are lots of naive posts lately on HN...

maurycy 17y ago
What are good resources to learn proper unit testing, by the way? (no offensive; curious)
- steveklabnik 17y ago
  I personally just sort of figured it out by doing BDD on a few small projects to get a feel for how things go. I'm not 100% "driven" now, but overall, it's pretty high. I tend to do Cucumber tests more than straight up unit testing, as I find them more valuble personally... Doing is the best way to learn most things in the software world.
- makecheck 17y ago
  It definitely helps to start with an environment where tests are easy to write, so that you can focus on the principles of testing without having to learn a lot of mechanics. Python's built-in "unittest" module is an example of that. While it's not perfect, you can quickly form good habits by using it. And it is not limited to Python. For instance, I've written unittest-compatible classes that basically run other programs as test cases. Either way, you're forced to think about things like "how do I automatically detect that this failed?" and "is the purpose of this test clear?".
- danprager 17y ago
  Kent Beck's book: Test-Driven Development Don't just read it: Work your way through it. You need to follow the recipe to develop the discipline/knack. Fairly quick and reasonably easy.
- silentbicycle 17y ago
  I like _Working Effectively with Legacy Code_ by Michael Feathers. It's a collection of techniques for bootstrapping tests into an existing code base - carefully, incrementally restructuring the code to add enough tests to safely perform deeper structural changes. At the same time, it's ultimately a book about improving legacy codebases (testing is just a major technique), so it focuses on using testing when it's most useful. (Also, maybe it's just me, but a book that starts from, "Help! I just inherited a C++ project with ten years of rot and half the comments are in Norwegian! How do I even start adding tests to this without breaking it further?" seems a bit more practical than one that starts by applying unit tests to example code designed for convenient testing. It's seldom that easy.)

philjackson 17y ago

I simply can't imagine developing without unit tests now. I very rarely (never) get things spot on the first time and the unit tests help me maintain pride and position before most of my big merges. Sometimes I do wonder if I rely on them too much, but they aren't a waste of time.

geebee 17y ago

One of the really great things about software development is that reality, rather than a committee, gets the final word. I think that unit testing is a great idea, in theory and in practice, but I do think that the unit testing may get to the point where some practitioners who are not careful will spend far longer to complete a project with no discernible improvement in stability or maintainability than someone who uses unit testing more judiciously. I have used the end-to-end automated functional test approach before, and it did enable me to find problems pretty quickly. However, I would have died a slow and painful mental death without unit tests in some of the tricker sections of my code. That said, yeah, "unit testing is a waste of time" is link-bait-ish.

makecheck 17y ago

I do agree with the point that it is possible to spend a lot of time fixing "broken" tests. The lesson that should often be learned is that certain tests are not adding enough value to warrant the constant maintenance. (Code alone requires plenty of maintenance as it is.) One thing that works extremely well is to have a test environment. In other words, spend the time to replicate your entire production flow 100% (but in isolation), so that you can "deploy" changes and see if anything blows up. This not only catches stupid deployment bugs, but it also allows you to avoid some unit testing by simply relying on customers themselves: have them log into the isolated environment and do exactly what they would have done in production, and you'll know if it works.

danprager 17y ago

Good, but overly long article. I have reached the similar conclusions in favor of functional tests over the last decade or so. Personally, I have worked seriously used DBC, TDD, and automated scenario testing, and hybrids in that order of learning. My conclusions about the sweet-spot (and this may vary depending on your area): 1. Automated functional testing in the large, and DBC in the small is a winning combo. 2. Special techniques for graphics intensive programming: I get my scenario tests to show what's happening on the screen, and I can slow 'em down and step through 'em 3. I never got a lot of dependency injection / mocks (like the author) -- not sure if I don't get 'em or application area is wrong 4. You need special techniques for intensive algorithmic code where there's a combinatorial explosion of cases and line coverage won't do; e.g. randomly generated test cases + compute intensive sanity checks Bottom-line: Automated scenario tests exercise the code; DBC pin-points illegal states. DBC = Design by Contract = pre-conditions + post-conditions + invariants. In practice understanding the method (this takes work) and writing explicit pre-conditions gives ~70-80% of the benefit. To do invariants well needs language support.