10 ms·

The Start-Up Trap

149 points by filpen 14y ago

lubujackson 14y ago
In my mind, TDD is a defense against complexity. Complexity is always the enemy. If you can simplify your code through structure or clean design you can minimize or remove testing. The moment you can't hold the whole thing in your head with ease is the moment you should have done TDD a while ago.
readme 14y ago
Great, more TDD evangelism. I would love to see an empirical study proving the claims made about TDD in this article and numerous others. It seems to me TDD is a huge waste of time when prototyping a minimum viable product. You want me to spend 2 hours writing tests for a feature that someone's going to tell me to rip out 15 minutes later? No thanks. It's really easy to tell others to use TDD, and even admonish them for not using it. But unless you are the one in their shoes, you will not know the whole of their reality. In a perfect world I'd write tests for everything. In the real world, I write tests for almost nothing. Most of the work I do is on stuff that might be gone tomorrow.
- hartator 14y ago
  I agree. Prototype are not very well defined and prone to rapid changes. Besides that, when you have an app that interact a lot with third parties APIs, it's almost impossible to make useful tests. One guy working with us has basically mocked the whole world wide web to allow him to do TDD, sometimes TDD is just plain stupid.
  - hack_edu 14y ago
    I don't understand your point. If your third-party APIs are volatile and out of your control, causing a liability in maintaing tests... wouldn't that cause an even bigger liability in your application code? Such volatility is even more dangerous to your application. This is why we have integration tests. This is classic case of why you need tests and integration policy. When your APIs change or break, the tests break and you are quickly made aware. You just saved yourself a huge embarrassment and likely have the competitive advantage to those we don't look forward like you should do. Need to make a release quick and need to update your handlers? Go for it. If you're on a deadline and secure with the release, go ahead and push it with a couple tests breaking with false-negatives then swing back for an hour and fix them. You'll thank yourself when they release v2 of that API and sunset your methods.
    - hartator 14y ago
      I agree in parts. In real life, you can't always do integration tests against third parties for various reasons, the main one is that some websites/apps don't like to be taken as a sandbox to try out every CRUD operations we can have in mind. If you isolate too much, your code will be running against only itself and won't be again that useful. The way to assure software quality for our case is to carefully check on logs from data from real users, pretty regular actually. Not as sexy as TTD, but customers appreciate you take care about their real account or not another Foo Bar foo@bar.com profile.
- tomgallard 14y ago
  Although TDD does give you more confidence to make changes to your code in future, I think you're overlooking a major benefit, which is allowing swifter code/test/debug cycles. So rather than having to make a change, run up the app in my browser, and manually test that the latest bit of code is working, I can test individual functions just by running my unit tests. Of course this doesn't obviate the need for browser based testing too - but it can reduce the amount of it you need. TDD allows me to produce working code faster by helping me notice and fix my errors sooner. If you're an excellent coder, maybe you don't need this- but I certainly do!
- donw 14y ago
  Some workflows don't make sense with TDD. For example, if your company builds proof-of-concept security exploits as part of a security audit or penetration test, then TDD offers negative value, because in many ways, your code is the test -- if it runs and exploits the vulnerability, then it works. Likewise, when I'm figuring out how something might need to work, like learning a new API or trying to figure out how to build one, I don't do TDD. But I also don't do this as part of my main project -- it goes into a special '~/playground' where I can experiment with ideas. Other than that, I do everything TDD. Here's why. It doesn't take two hours to write tests, because by and large, you're writing the same code you'd throw into the REPL, or performing the same actions (in code) that you would need to repeat over-and-over again in a browser to see if the feature works. It's the same amount of work, you just need to write it down so that it can be repeated. I've found that this saves me tons of time, because I can run the same test over-and-over while I make stuff work. This assumes you are familiar with your testing tools -- if not, then yes, getting going requires learning the tools, which is the same overhead as learning a new library, algorithm, or language. If you're working on stuff that might be gone tomorrow, you're doing your customer discovery wrong. Sure, you're going to implement features that it turns out were needless from time to time, but if you're regularly implementing work that gets thrown away fifteen minutes later, you're wasting a huge amount of time. There are better ways to solve the question of "what does the customer want" other than building it and showing it to them -- check out all the work that Janice Fraser has done over at Luxr.
- shadowfiend 14y ago
  “I would love to see an empirical study proving the claims made about TDD in this article and numerous others.” This point is so important that I think it should have been the only one you made. Tests look good on paper. And they are great intuitively. We can make great arguments for them. But last time I asked for a clear study indicating that TDD led to better results than a non-TDD development, everyone seemed to come up blank. What it sounds like to me is religion. Doesn't mean I won't test. And it certainly doesn't mean I'll eschew testing on a team that does testing. But it still smells suspiciously like religion, and that's very worrisome to me.
  - dasil003 14y ago
    TDD teaches you how to write tests at all costs. Once you know how to test in every conceivable scenario than you can discard TDD and replace it with the wisdom of what tests are useful and understand the real ROI of writing this test or that test. If you haven't forced yourself to get good at writing tests at some point then you will fail to write a valuable test simply because the cost of writing the test seemed higher than the return even though it was only because of your lack of skill in that department.
- InvisibleCities 14y ago
  >You want me to spend 2 hours writing tests for a feature that someone's going to tell me to rip out 15 minutes later? No thanks. If you are going to rip out the feature 15 minutes later, why even waste the two hours writing it? Also, as someone who typically practices TDD, writing the feature + tests doesn't tend to take any more time than just writing the feature (and I have the cycle time metrics from my current project to prove it).
- unclebobmartin 14y ago
  There have been several posts on this thread in which research papers in support of TDD have been referenced. You can find them if you simply google it. I suggest you do that.
Xcelerate 14y ago
TDD is but one method of writing code; one that I've never found particularly useful. There's plenty of other things you can do to perform your job well.
kgo 14y ago
"Of course one of the disciplines I'm talking about is TDD. Anybody who thinks they can go faster by not writing tests is smoking some pretty serious shit." Why do so many TDD people think that people who don't use TDD don't write tests?
- theoj 14y ago
  Also, why do so many TDD people think that no automated tests = no testing at all?
kyleashipley 14y ago
Although I've been a hardcore TDD advocate from time to time, I find myself writing fewer and fewer tests in the early stages these days. In the very earliest days of a startup, you can easily hold the entirety of the problem space in your head. Refactoring is simple, tests or not. I can destroy and rewrite a feature from scratch in hours or days. A non-trivial number of the features I'm writing will not exist in any form a week or two later. This is not a problem with process or planning; this is the nature of a startup. I think that TDD is immensely valuable as a team and product grows, but claiming that your startup will fail because you aren't applying engineering "best practices" from day 1 is counter to most successful startups I've worked with.
vellum 14y ago
>Of course one of the disciplines I'm talking about is TDD. Anybody who thinks they can go faster by not writing tests is smoking some pretty serious shit. That's some serious strawman. In between 0 tests and TDD, there's a lot of room to maneuver.
- j_baker 14y ago
  Is it a strawman? Because I have heard plenty of developers espouse this attitude.
  - vellum 14y ago
    Some people don't write tests. But Bob was only comparing TDD and 0 tests. A lot of people sprinkle a few unit tests for sanity, instead of letting tests drive development. There’s a wide spectrum of successful coding practices that get products shipped. Bob’s article only compares the outliers.
thetrumanshow 14y ago
TDD is like those best-practices books that everyone pretends to read, but hardly anyone ever really reads them (I'm looking at you Code Complete). The Folks that do read these books know the dirty secret that no one else really reads them either, but they feel that endorsing the books somehow gives them credibility. TDD has its place, as most all main-stream methodologies do. But, lets just admit that the people that use this methodology are in the minority. The rest of us are working on smallish projects that are struggling to be worthy of the time-budget that they've been granted, and we're more worried about shipping than caring about how not having TDD in place will slow us down in phase III. Assuming phase III ever happens. We're using all the best-practices we can, but TDD doesn't rise above the bar most of the time.
- j_baker 14y ago
  You're strawmanning. Uncle Bob isn't saying "If you aren't using TDD at the startup phase, you suck!" What he's saying is "Just because you're in the startup phase and think you're invincible doesn't mean you should throw best practices out the window." There are good excuses for not using TDD, and I tend to agree that "We're in a startup phase. We don't have time for this crap." isn't one of them.
  - thetrumanshow 14y ago
    Had I opened with "Interesting article, but tangentially..." then we'd still disagree, but maybe we could have avoided the straw-man ding. :)
    - j_baker 14y ago
      You're right. Then we'd have a passive-aggressive ding. The only thing you would have to add to make it perfect would be "...but I'd never claim the author is trying to do that."
      - derefr 14y ago
        I wouldn't even call it passive-aggressive; it's soapboxing, plain and simple. > This article is about X. But I don't care what the article says about X. I have something to say about X, and by God, I'm going to say it. --But I don't know that this is a particularly problematic thing to be happening in a threaded comment system, since people can get sidetracked by "soapbox issues" in a subthread while letting the "parent conversation" continue around them. It's just kind of confusing for people who treat this place like a linear-chronological message-board that discusses one topic at a time.
JoeAltmaier 14y ago
Skipping testing amounts to risk. The winners in a brutal race take every risk to win. So yes, its foolish to skip testing. And yes, the race goes to the foolish.
- unclebobmartin 14y ago
  The winners in a brutal race have learned how to avoid as much risk as possible. The prize does not go to the biggest risker. The prize goes to those who do everything they can to minimize the risk of going slow. What slows a programmer down? Debugging. Messy code. Fear of change. How do you minimize those risks? A comprehensive suite of tests. How do you get that suite? TDD.
  - JoeAltmaier 14y ago
    And you come in 2nd or 3rd in the race, and lose.
    - unclebobmartin 14y ago
      That is certainly your fear. It is a fear I don't share. Firstly, I think you'll get there faster if you work well. Secondly, as MySpace showed, being first isn't really all it's cracked up to be.
      - JoeAltmaier 14y ago
        Im not making myself clear. You might beat me, personally, once. But you won't beat that one lucky guy that didn't do the testing, got his code to work the 1st time, and got to market before either one of us. That's how the game's played. Conservative, careful, correct coders don't win that game. And if your business plan is to be 2nd to market, well, good luck with that.
gearoidoc 14y ago
Whilst not exclusive to TDD, I think DHH said it best: "The road to programming hell is paved with “best practices” applied too early."
- jaytaylor 14y ago
  But if you don't start with TDD, then when are you going to put it in? How do you transition into it? Are you going to go back over everything you've written and write tests for all of it?
  - gearoidoc 14y ago
    I think you build it in when you know what you're actually building. In the initial dev cycle, everything and anything could be ripped out so I'd suggest holding off til perhaps halfway through your MVP cycle until you start TDDing.
- freshhawk 14y ago
  And the road to failure is ignoring best practices because "I'm special" and "I'll do it later because I'll magically have time then". It's a balancing act that pithy aphorisms don't help with. There's a reason people use the metaphor "Design Debt". Debt is a tool with tradeoffs, financial or design, use appropriately.
  - gearoidoc 14y ago
    But what if the debt outweighs the benefits? What if you can't measure the debt?
    - freshhawk 14y ago
      If the debt outweighs the benefits you don't take it, I have a project right now that I "know" will be fragile and a maintenance headache later. So even though it's urgent I'm writing the appropriate tests and doing all those best practice things that you know will pay off 10x later but often skip because you don't want to take the time now. You could measure how much debt your team tends to take on, by measuring how much time is spent on new features and how much on the type of bugs and the refactoring that pays off design debt. This still misses what I think is the most important part of design debt, how much longer it takes to implement new ideas because you are paying the "the pieces this depends on were rushed and don't work/integrate/extend well" tax (I guess I should call this "interest payments" to not mix metaphors) . You can't measure debt as you take it on though, the best you can do is estimate how much work it will be to fix it later (and that's only if it needs to be fixed later, maybe you get lucky and your hacked up code just stays good enough). We all know more than enough about the pitfalls of estimating software projects and this adds in more uncertainty about future need. This is all part of the craft side of software development, experience helps, but it's not something easily measured. Too many people take that as an excuse to just do the quick and easy thing and say "move fast and break things!" or fall back on over designing and never get any work done. HN talks about the latter more often and pretty much ignores the former. I find this strange, I've read plenty of accounts of failure where the reasons boiled down to "we got to a point where we couldn't adapt our codebase to changes needed to face a new competitor, change in the landscape, business model pivot, etc. because it was too crufty". Enough design debt means some smaller and more agile competitor will eat your lunch. Well that's definitely a long enough answer to two simple short questions.
zwieback 14y ago
Hacking up a quick demo the non-TDD way definitely has its upsides but if you don't force yourself to review and rewrite your early code you would have been better off doing the right thing from the start. Just preaching TDD is not helpful, we need a coding process that allows clean separation of early and production code.
jmduke 14y ago
I like to talk about TDD using a finance analogy. One of the first concepts you learn in introductory finance is that, in a mature market (ie the stock market -- basically something with a bunch of people and a bunch of information) you have to be compensated for risk. This is why a riskier stock -- like tech stocks or penny stocks -- can boast incredible amounts of risk but generally will give you higher returns over the aggregate than something like pork commodities or a CD. When you choose to purchase a tech stock -- or any stock at all -- you're saying "okay, I recognize that this is riskier, but I think I'm being fairly compensated for the risk, too." Choosing to eschew TDD is like purchasing stocks. By definition, going through TDD is going to be a safe route, but it's rare that TDD (at least in my and my friends'/peers' experiences) is actually going to make you get from Point A to Point B any faster. TDD isn't, by default, a superior or inferior approach to anything: it's a tradeoff -- do you want risk or do you want return? Sometimes, you want to minimize risk, and that's probably smart. Sometimes, you just want to produce an MVP -- and that's okay, too.
- cbracco 14y ago
  Couldn't agree more, it depends on your situation and goals as a start-up.
- donw 14y ago
  I like your analogy, but it needs a little something extra. TDD (and BDD) is an assurance that the promises that your code made yesterday will be kept tomorrow. If you break those promises, you do so consciously. Not doing TDD is like investing in the pink sheets -- the company you invest in might be a fly-by-night operation, or it might be a legitimate business. It could be both, but you never know, and assuming you're not part of a pump-and-dump scheme, the rewards aren't really higher than listed stocks. But the risk is a lot higher than listed stocks. Lots and lots of risk, but very little upside, which is the same as writing code without tests. You save a little time getting started, but spend a lot more time doing the repetitive work that testing automates, and in doing so, eat a lot of long-term technical risk that no company should consider acceptable.
- d2vid 14y ago
  To take your analogy further, an early-stage startup has a high cost of capital. You are expected to produce massive returns (or fail). You thus are explicitly required by investors to take risks. You have a duty to not invest in the software equivalent of CDs.
spartango 14y ago
My feeling is that the point of this article isn't strictly to promote TDD or any single engineering tool that startups tend to eschew. Instead, the author seeks to make the broader point that cutting corners in engineering is not a good idea even in a startup. While it can make startup engineers feel like they are moving quickly relative to the competition, the engineering debt incurred is non-negligible. I strongly agree with this point, and don't buy the blanket rationale that speed trumps everything in a startup. Yes, speed is quite important when you are bringing something new to market; it lets you get your foot in the door and rapidly find the right fit for your product. But if you cut corners then your product will suffer, and you'll end up treating your customers poorly by it. There's a fine balance to all these things, of course. We all have to find the right balance between keeping engineering standards high and heavy-handed structures.
- freshhawk 14y ago
  Boy, did I have to scroll down far to find someone who read the article. I was hoping for tips about how to find a balance when other team members have bought the blanket rationale that speed trumps everything and then complain about why we spend so much time fixing bugs and fighting fires instead of working on the next thing. Instead a bunch of reading comprehension challenged commenters are bitching about TDD again.
tptacek 14y ago
I thought this was going to be an article about overwork, or about charging for what you build as a way of proving your market, or about how employee stock options are often a bad bet. Instead, it's about TDD. What a disappointment.
- orangethirty 14y ago
  Which is ironic, because the real startup trap is the one where people build things but are told to not charge for them (or charge very little for it).
- wglb 14y ago
  Its Uncle Bob!
moocow01 14y ago
Largely agree with everything here but in my opinion many times there can be a very strong current that forces you to do it the fast, stupid way. In my opinion if you dont have any of these problems, you work in a unique place... - Agile - Agile IMHO largely screws you in making sound technical decisions. Its not necessarily because agile is flawed - its usually because business/management uses agile as an excuse to randomly take a hard left turn every other week making it much harder to make long term architectural choices that are beneficial. - No agreed upon standards or unification amongst software professionals - The accountant in this situation has a set of standards and expectations that allow/force him or her to do things in a moderately set way. This allows for the accountant to usually fend off management from pressuring things to be done in a shoddy or just get it done way. On the other hand its much more difficult for software professionals to say "we as a group do not condone writing shitty software" (because ironically a large number of us do ... "move fast, break things" has done more good than bad from my perspective) - Ageism - Also known as experience doesn't really matter. Some will say thats because technology changes so much - but it actually doesn't. Just because you've been grappling with software problems and design patterns in Java doesn't mean that when you switch over to Python that really anything changes. Same shit, same problems, different words - but in all honesty we seem to be pretty bad at building on the experience of our elders because they are over the hill at 35... what could they possibly teach us. For me it at times has been frustrating because coming out of school I really enjoyed the fun of designing systems and code that are sustainable, performant, etc etc but there seemingly is typically more reward for just throwing quality out the window in startups. Just my personal experience.
MattRogish 14y ago
(tl;dr: it depends) It's a hallmark of "experienced" non-dogmatic product people (UI/UX/Dev) that can use their intuition to know what are the happy paths that they need to test, what interfaces are likely to not change (e.g. a user is probabilistically always going to be a member of an organization, so unit tests around that are likely not waste), and what level of quality to introduce for a given feature relative to the probability the feature is going to exist in perpetuity. You can concretize this by calling it "spike driven development" if you want (that's what we do) but the point isn't that TDD is faster or slower but that high coverage might be inappropriate (TDD isn't binary - degree of test coverage goes from 0-100%) at different phases of the build/learn/refine cycle. For example, if we're building a speculative feature that we think has value but hasn't been proven yet, we want to spend the least amount of effort (code/time/money/pick some) possible to prove that it's viable. Small bugs aren't really that interesting (provided the thing "works") and so we're explicitly willing to live with degraded quality to prove the feature is worth further investment. We're also explicitly not building a "large" feature (so it's very likely to get coded in a day or two) so the surface area for major showstopper bugs is minimized. Often the feature will be thrown away or majorly refactored into something that is completely different. In this case, full-bore 9x% coverage TDD is probably waste as the feature existed for a short period of time, the interface changed dramatically, or otherwise was proven invalid. High test coverage makes major interface refactors really expensive and you really don't need that level of quality for speculative features. After you prove this feature is a "thing" you want to keep around (and you've nailed down the interface), then it's a perfect time to re-write it with a much higher degree of test coverage.
- jfarmer 14y ago
  Stop being sensible, will you? This is exactly how we did things at Everlane (http://everlane.com http://everlane.com) and Dev Bootcamp (http://devbootcamp.com http://devbootcamp.com).
- pifflesnort 14y ago
  > After you prove this feature is a "thing" you want to keep around (and you've nailed down the interface), then it's a perfect time to re-write it with a much higher degree of test coverage. I find that tests help me write things that are complete and "nailed down" the very first time.
jasim 14y ago
It is easy to hate TDD. In a Rails project, the Rails boot-up time makes TDD a painful experience if you are not using tools like Zeus or Spork. Even in the presence of such tools, you need a powerful machine to not hate the slowness of the whole thing and worse still, break your flow. My recommendation for someone who hates TDD for the wrong reason, aka: breaking the flow, would be to get a fast machine, fix the code to make TDD as painless as possible and to use the right tools. But once you start using TDD as a tool to organize your thoughts and model your domain, you might end up becoming too dependant on it and find it hard to work any other way. This is anecdotal experience. Also, learning how to wield TDD properly takes a lot of time, error and practice. Good things don't come easy. There are obviously places where TDD isn't a good fit - a spike, requirement that is known in advance to change soon, and exploratory programming are all candidates. However, good practice dictates that you refactor your code once a spike calcifies into production code. At this point, TDD becomes just unit-testing. Most of the arguments against TDD in this thread seems to be against unit testing in general. But we know unit tests are important. Doing it before the fact increases the value of the unit tests manifold and also ensures that you do have coverage (though that is not at a primary objective).
lifeisstillgood 14y ago
Know the rules before you break the rules. The accountant analogy is dead on - that and the best quote of the day: You'd fire his ass! You'd fire it so fast that the rest of his worthless carcass would be left outside the door wondering where his ass went! Brilliant.
petenixey 14y ago
Man, I've been back and forth on tests from one extreme to the other. I have to say that I do agree with this but with one caveat - Test Driven Development is a bit miserable. Test "Driven" Development is a real invitation to write too many tests. A small, good set of tests gives you freedom to work fast, to refactor and to have multiple developers pushing simultaneously. Not having them really isn't sustainable past a certain point but equally to many tests will drag you to the ground. Tests are overhead, they cost time to write, maintain and run and TDD tends to drag you into the deep end. Avoiding them altogether ends up being a false economy (if only in that it makes you too nervous to push often). To those who fear the slippery slope, a nice self-annealing approach to get into testing is to only write tests for something that has failed. That way you waste no time writing tests for things that are actually pretty robust but equally you avoid addressing (and fearing) the same issue twice.
- agilord 14y ago
  I'd like to emphasize this approach: "a nice self-annealing approach to get into testing is to only write tests for something that has failed." It is great for old-timer enterprises without any test, and balances the time pressures of startups. However, you need to write those test not only for prod failures, but also if that something failed in your dev environment.
stickbranch 14y ago
I want to vomit when I see titles like Master Craftsman.
- unclebobmartin 14y ago
  Please use the air-sickness bag located in the seat back in front of you.
rcconf 14y ago
The way I do my TDD is by writing the unit tests to specifically test the functionality of the code. For example, if I'm writing a module that does validation for a specific type (for example, in JS) I tend to test the functionality while writing the tests. I am unable to speak amongst others, but you have to test your code eventually, and it's either going to be in the classes that are using the code, the REPL, or it's going to be in your unit test/integration framework. I usually pick the test framework first because it serves 2 benefits: 1) I can prove my code works. (I must prove this to my self, or the people using the code at some point.) 2) I can reliably change the code later. Complexity is also mentioned in a couple of comments here. TDD helps simplify complexity by writing code that is testable. It also makes you think about your API as you're writing the different possibilities the module/code can be used in. Further, I think there are multiple issues with tests, and depending on the type of test your doing, different problems can arrise. You have your unit tests, and then you have your integration tests. Unit tests are relatively easy to write (if your code is split into individual chunks, that do a specific task.) and they should always be written regardless of the time you have. I think the major issues arrise in terms of time are the integration tests. When you're testing the functionality between complicated modules that require databases, i/o writes, network communication, it is sometimes hard to write the tests, and it may not even be worth it. The majority of people who disregard TDD, usually disregard a specific sub-problem of TDD. I think TDD has its benefits depending on what type of tests you're doing, and how easily the problems can be solved with those types of tests. You can always pick to write unit tests, and disregard integration tests, etc. I think there are a lot more issues to expand upon, like the language you're using, the platform you're using and how often your code is required to change, but TDD has large benefits, and code rot is very real and TDD can help mitigate it.
johnrob 14y ago
So the key to meeting your short term goals is... blow them off and focus on the long term goals. Doesn't make sense to me.
gfodor 14y ago
Over the years, I've grown less and less likely to write lots of tests, after being a very large test zealot during the peak of the TDD wave. There are a few reasons. First, yes, TDD slows you down. The reason this matters is because a lot of our time as developers is spent exploring ideas, and it's pretty well understood that the faster you can get feedback on your ideas the easier it is to creatively develop them. In fact, the final result of your creative process can be completely different depending on the feedback cycle you have. From the micro to the macro perspective, religious TDD introduces an constant factor slowdown for small projects that yes, ends up being a net win in the long run if you live with the code for long, but is a net loss in the short run and also can prevent you from finding a solution to a problem since your creativity is stifled by the slow speed of idea development. Second, when building web applications, building tests turns out to be fairly overrated. First, people will tolerate most bugs. (Write good tests around the parts they won't.) Second, if you have a lot of traffic, bugs will be surfaced nearly instantly in logs and very quickly via support tickets/forum posts. (What if there are bugs that don't make it into the logs and don't affect people? If a tree falls in a forest...) Much more important than mean time between failures is mean time to recovery. I'd rather have ten bugs that I fix within 5 minutes of them affecting someone than one bug that festers for a month. Not only because this is healthier for the code base in terms of building robustness, but also because human behavior is such that many fast bug fixes make everyone feel good but few lingering bugs make everyone miserable. People want to feel like you care, and are much more likely to feel cared for when you fix their problems quickly, and are not often interested in just how few bugs you ship if the one that you do has been affecting them for a month. This isn't theoretical nonsense, it's a very real phenomenon where you use tests and manual testing to get up to a basic working prototype and then just throw it over the fence to flush out the bugs. It's the only way to do it anyway. (This only really works if you have traffic and can deploy changes quickly. To paraphrase a colleague, production traffic is the blood of continuous deployment.) Obsession over deploying bug-free software (an oxymoron anyway) is usually coming from people who haven't gotten over the fact that we don't need to ship software once a month or once a year but can do it every 10 minutes in certain domains. Instead of focusing on not shipping bugs, focus on shipping faster.
- randomdata 14y ago
ardit33 14y ago
Wait, since when TDD become a best practice? I know this author is advocating it, but has 0 (zero) solid data or evidence to back this assertion. He is just trying to sell another religion. What I have noticed, TDD is being pedled heavy by RoR contracting shops that just care about billable hours and not vested if the startup or company is going to make it on the long run. I see often that it is the usually youngish and guidable engineers that fall in the trap of eating it up. TDD is something that makes younger and less experienced developers feel good (more locs) and a false sense that the code is correct while there is solid data that even 100% unit testing coverage finds only 20% of defects, at best. Having shipped many world class products, and none of the processes that build those products had any resemble of TDD in them. So, to all the pragmatists out there feel free to ignore the advice on the article. I'd yet to hear a super successful startup that used TDD. Having Tested Code (which is a good goal) != TDD. TDD to me is putting the carriage in front of the horse. Anybody that have created large systems from scratch, TDD is a major slowdown on the refining process. Your job is to SHIP a working product. Testing is only a small part of it, and its positives should be considered alongside its negatives and side effects on the shipping timelines. Having worked for over 10 years in the industry I have to meet any great programmer that preferred TDD. I hate to attack the author, but from his bio he seems a bit like a process peddler. I don't see him have worked on any successful startups, or anything that had a good exit. So take his claims with a grain of salt. http://www.8thlight.com/our-team/robert-martin http://www.8thlight.com/our-team/robert-martin He is trying to sell a religion, so buyer beware.
- nthj 14y ago
  I'm not a huge fan of TDD either. Tests are helpful when refactoring or adding new features because you avoid regressions. Regressions are the worst kinds of bugs because clients get really, really annoyed with "I thought we fixed this the last 3 times" and "we already paid to fix this" type scenarios. That said, I think the article has a solid thesis. Just because you don't practice TDD doesn't that mean building things correctly won't help you ship working software faster. If your code is brittle, write some tests around it, refactor a bit, and keep running fast. We can debate over whether you should always write tests first or only write tests to cover brittle and important features, but that's a "how much" argument. The real problem is when you run into methodologies that always involve work-arounds, and you let the technical debt continually pile up for weeks or months because "you're a startup." This will cause you to run very slowly, and sooner rather than later. I feel that's Uncle Bob's real point, and it's a valid one.
matt2000 14y ago
I disagree with this almost entirely. Startups _are_ different: 1. What you are doing starts at 0 value. 2. You can (usually) break it and it's ok. 3. You need to change stuff a LOT to figure out something that someone wants to use, rewriting tests makes that slower. Every test case you write has a cost: It verifies that something of unknown value works. What if the value of that code is 0? Well, you just doubled down on something useless. There are no absolutes in software development, but how we did it was have uncomfortably too few test cases when something is new, change the product for a while until someone actually wants to use it, then add a bunch of tests until we know it works from there on out. And I hesitate to add: Also we work in Java so a bunch of testing comes for free, so there's that.
lucian303 14y ago
Great article. Not just about TDD but about all of startup culture. The discussion here is about TDD but how about the long hours. They are not necessary either. No process needs to change. If your company requires long hours, management has made huge mistakes. Period.
DaniFong 14y ago
Imagine test driven development for visual art. How would one write a test for visual art, beyond viewing and comprehending the partially completed work? If one could write a test in advance, one would have made a significant advance: distilled to a verbal form a description of what good art is. Instead, many artists -- or software authors, or chefs, find a way to repeatedly sample and appraise their work as it develops over time. I think this is far more important than TDD, because the truly important problems of software engineering are not in making software that can achieve simple correctness, but in making something that people want.
- mattvanhorn 14y ago
  Actually, that is a great metaphor. But you should know, TDD is the way I repeatedly sample and appraise my work as I go. Multiple times per hour, in fact. Red -> Green is the sampling and Green -> Refactor is the appraising.
barrkel 14y ago
"as if any bug is acceptable!" This is the point at which I stopped reading. Yes, some bugs are acceptable. The point of writing software is not to create bug-free code; it's to create value. The marginal returns on eliminating the last bug are much lower than implementing new functionality. As for the gist of the post, from my experience, TDD is OK; but it is unit tests that are essential. They are far more important in dynamic languages, because you otherwise have very little feedback when you make mistakes as simple as a typo. They're also important in static languages, but fairly large programs can successfully be written with without anything near the volume of testing needed with a dynamic language.
PasserBy2 14y ago
"Oh, I know you are a warrior-god." Well, at least someone noticed!
alexrson 14y ago
A useful reminder
Mahn 14y ago
I'm not sure if the author has any idea what is it like to bootstrap a web product and watch the money go down the drain day after day, with zero income to compensate, and with the certainty that you won't make it for another 6 months if the product is not out and making revenue. I'll deal with the technical debt later, thanks. I'm pretty sure Facebook and Google didn't do anything remotely close to TDD when they first shipped, and yet they survived.
icedchai 14y ago
TDD is often a waste of time. I worked at some place that had taken the TDD philosophy to the extreme: every getter and setter had a test. On the opposite side of the spectrum, I worked on software that processed millions of dollars in daily transactions and there were maybe 5 "tests" in the whole system. This was about 500,000 lines of C and C++ code in the early 2000's. My personal philosophy is to write unit tests where I think it's important, not test everything.
- mattvanhorn 14y ago
  I won't write tests for getters and setters, but then again, I try not to write getters and setters very much in the first place.
michaelochurch 14y ago
This "trap" is Deadline Culture and it doesn't pertain only to code quality, but to other matters like product direction and personnel decisions, which are made quickly and often badly. It's sloppy and destructive. Companies with Deadline Culture often make it easy for a total asshole to gain power just by citing existential risks that either do not exist or are of extremely minor importance. Deadline Culture companies are obsessed with competitors, even though it's their own internal corrosion that does them in. Sometimes, having deadlines is unavoidable. They might be set externally and exist for regulatory reasons. Deadline Culture is when the company starts encouraging the formation of unreasonable deadlines that triumph over professional integrity and intellectual honesty. VC-istan seems to encourage it with the whole "if we get bored with you, we'll fund your competitors" arrangement. Deadline Culture is, however, great for the true fascist. Fascists love (real or imaginary) existential threats, especially vague ones they can continually adapt to their needs, but that come with time pressure.
unclebobmartin 14y ago
To those of you who asked for "links" to studies that prove that TDD works; google around, you'll find plenty. Some are positive, some are negative -- what else is new. Now, please show me the studies that show that _your_ discipline works. To those of you who consider TDD a religion; you are being silly. TDD is a discipline, like double-entry bookkeeping, or sterile procedure. Like those disciplines it has aspects that are dogmatic; but that aim at a purpose. That purpose is to help you write software fast and well. To those of you who think I'm a process peddler. You're right; I am. I make a very good living teaching people how to do TDD. I have been doing so for over ten years now. I hope to continue for some years to come. My goal is to help raise our industry to a higher level of professional behavior; because at the moment our behavior is pretty poor. To those of you who wonder whether I've ever worked at a real start-up. I've worked at several over the years. My first was in 1976; and it was very successful. Another was in 1989, and it didn't do so well. I've recently founded cleancoders.com a startup in video training for software developers. The website is done entirely with TDD. And this doesn't count the rather large number of startups I have consulted for in the last 10 years. So I've got a _lot_ of experience with startups. Folks, I am 60 years old. I got my first job as a programmer when I was 18. I wrote my first program when I was 12 (on a plastic 3-bit computer). I started using TDD in 1999, after I'd been programming for thirty years. I continue to write code using TDD today. I've seen the startup trap. I've lived the startup trap. I strongly advise you to avoid that trap.
spullara 14y ago
I find that making sure the foundation of an application is solid (I'm an API first kind of person) let's you get away with a lot less testing in the upper layers. This lets you move faster and experiment far more quickly than if you were built on quicksand.
digitalWestie 14y ago
I see this as an argument for testing, but not necessarily TDD. Why have the two become so intertwined?
pardner 14y ago
Statistically speaking, a web startup can safely not invest in TDD. For precisely same reason that a daily Russian Roulette player can safely not invest in health insurance. But for the minority who miraculously dodge the bullet, the extra investment will eventually become a lifesaver.

149 points by filpen 14y ago

In my mind, TDD is a defense against complexity. Complexity is always the enemy. If you can simplify your code through structure or clean design you can minimize or remove testing. The moment you can't hold the whole thing in your head with ease is the moment you should have done TDD a while ago.

readme 14y ago

Great, more TDD evangelism. I would love to see an empirical study proving the claims made about TDD in this article and numerous others. It seems to me TDD is a huge waste of time when prototyping a minimum viable product. You want me to spend 2 hours writing tests for a feature that someone's going to tell me to rip out 15 minutes later? No thanks. It's really easy to tell others to use TDD, and even admonish them for not using it. But unless you are the one in their shoes, you will not know the whole of their reality. In a perfect world I'd write tests for everything. In the real world, I write tests for almost nothing. Most of the work I do is on stuff that might be gone tomorrow.

hartator 14y ago
I agree. Prototype are not very well defined and prone to rapid changes. Besides that, when you have an app that interact a lot with third parties APIs, it's almost impossible to make useful tests. One guy working with us has basically mocked the whole world wide web to allow him to do TDD, sometimes TDD is just plain stupid.
- hack_edu 14y ago
  I don't understand your point. If your third-party APIs are volatile and out of your control, causing a liability in maintaing tests... wouldn't that cause an even bigger liability in your application code? Such volatility is even more dangerous to your application. This is why we have integration tests. This is classic case of why you need tests and integration policy. When your APIs change or break, the tests break and you are quickly made aware. You just saved yourself a huge embarrassment and likely have the competitive advantage to those we don't look forward like you should do. Need to make a release quick and need to update your handlers? Go for it. If you're on a deadline and secure with the release, go ahead and push it with a couple tests breaking with false-negatives then swing back for an hour and fix them. You'll thank yourself when they release v2 of that API and sunset your methods.
  - hartator 14y ago
    I agree in parts. In real life, you can't always do integration tests against third parties for various reasons, the main one is that some websites/apps don't like to be taken as a sandbox to try out every CRUD operations we can have in mind. If you isolate too much, your code will be running against only itself and won't be again that useful. The way to assure software quality for our case is to carefully check on logs from data from real users, pretty regular actually. Not as sexy as TTD, but customers appreciate you take care about their real account or not another Foo Bar foo@bar.com profile.
tomgallard 14y ago
Although TDD does give you more confidence to make changes to your code in future, I think you're overlooking a major benefit, which is allowing swifter code/test/debug cycles. So rather than having to make a change, run up the app in my browser, and manually test that the latest bit of code is working, I can test individual functions just by running my unit tests. Of course this doesn't obviate the need for browser based testing too - but it can reduce the amount of it you need. TDD allows me to produce working code faster by helping me notice and fix my errors sooner. If you're an excellent coder, maybe you don't need this- but I certainly do!
donw 14y ago
Some workflows don't make sense with TDD. For example, if your company builds proof-of-concept security exploits as part of a security audit or penetration test, then TDD offers negative value, because in many ways, your code is the test -- if it runs and exploits the vulnerability, then it works. Likewise, when I'm figuring out how something might need to work, like learning a new API or trying to figure out how to build one, I don't do TDD. But I also don't do this as part of my main project -- it goes into a special '~/playground' where I can experiment with ideas. Other than that, I do everything TDD. Here's why. It doesn't take two hours to write tests, because by and large, you're writing the same code you'd throw into the REPL, or performing the same actions (in code) that you would need to repeat over-and-over again in a browser to see if the feature works. It's the same amount of work, you just need to write it down so that it can be repeated. I've found that this saves me tons of time, because I can run the same test over-and-over while I make stuff work. This assumes you are familiar with your testing tools -- if not, then yes, getting going requires learning the tools, which is the same overhead as learning a new library, algorithm, or language. If you're working on stuff that might be gone tomorrow, you're doing your customer discovery wrong. Sure, you're going to implement features that it turns out were needless from time to time, but if you're regularly implementing work that gets thrown away fifteen minutes later, you're wasting a huge amount of time. There are better ways to solve the question of "what does the customer want" other than building it and showing it to them -- check out all the work that Janice Fraser has done over at Luxr.
shadowfiend 14y ago
“I would love to see an empirical study proving the claims made about TDD in this article and numerous others.” This point is so important that I think it should have been the only one you made. Tests look good on paper. And they are great intuitively. We can make great arguments for them. But last time I asked for a clear study indicating that TDD led to better results than a non-TDD development, everyone seemed to come up blank. What it sounds like to me is religion. Doesn't mean I won't test. And it certainly doesn't mean I'll eschew testing on a team that does testing. But it still smells suspiciously like religion, and that's very worrisome to me.
- dasil003 14y ago
  TDD teaches you how to write tests at all costs. Once you know how to test in every conceivable scenario than you can discard TDD and replace it with the wisdom of what tests are useful and understand the real ROI of writing this test or that test. If you haven't forced yourself to get good at writing tests at some point then you will fail to write a valuable test simply because the cost of writing the test seemed higher than the return even though it was only because of your lack of skill in that department.
InvisibleCities 14y ago
>You want me to spend 2 hours writing tests for a feature that someone's going to tell me to rip out 15 minutes later? No thanks. If you are going to rip out the feature 15 minutes later, why even waste the two hours writing it? Also, as someone who typically practices TDD, writing the feature + tests doesn't tend to take any more time than just writing the feature (and I have the cycle time metrics from my current project to prove it).
unclebobmartin 14y ago
There have been several posts on this thread in which research papers in support of TDD have been referenced. You can find them if you simply google it. I suggest you do that.

Xcelerate 14y ago

TDD is but one method of writing code; one that I've never found particularly useful. There's plenty of other things you can do to perform your job well.

kgo 14y ago

"Of course one of the disciplines I'm talking about is TDD. Anybody who thinks they can go faster by not writing tests is smoking some pretty serious shit." Why do so many TDD people think that people who don't use TDD don't write tests?

theoj 14y ago
Also, why do so many TDD people think that no automated tests = no testing at all?

kyleashipley 14y ago

Although I've been a hardcore TDD advocate from time to time, I find myself writing fewer and fewer tests in the early stages these days. In the very earliest days of a startup, you can easily hold the entirety of the problem space in your head. Refactoring is simple, tests or not. I can destroy and rewrite a feature from scratch in hours or days. A non-trivial number of the features I'm writing will not exist in any form a week or two later. This is not a problem with process or planning; this is the nature of a startup. I think that TDD is immensely valuable as a team and product grows, but claiming that your startup will fail because you aren't applying engineering "best practices" from day 1 is counter to most successful startups I've worked with.

vellum 14y ago

>Of course one of the disciplines I'm talking about is TDD. Anybody who thinks they can go faster by not writing tests is smoking some pretty serious shit. That's some serious strawman. In between 0 tests and TDD, there's a lot of room to maneuver.

j_baker 14y ago
Is it a strawman? Because I have heard plenty of developers espouse this attitude.
- vellum 14y ago
  Some people don't write tests. But Bob was only comparing TDD and 0 tests. A lot of people sprinkle a few unit tests for sanity, instead of letting tests drive development. There’s a wide spectrum of successful coding practices that get products shipped. Bob’s article only compares the outliers.

thetrumanshow 14y ago

TDD is like those best-practices books that everyone pretends to read, but hardly anyone ever really reads them (I'm looking at you Code Complete). The Folks that do read these books know the dirty secret that no one else really reads them either, but they feel that endorsing the books somehow gives them credibility. TDD has its place, as most all main-stream methodologies do. But, lets just admit that the people that use this methodology are in the minority. The rest of us are working on smallish projects that are struggling to be worthy of the time-budget that they've been granted, and we're more worried about shipping than caring about how not having TDD in place will slow us down in phase III. Assuming phase III ever happens. We're using all the best-practices we can, but TDD doesn't rise above the bar most of the time.

j_baker 14y ago
You're strawmanning. Uncle Bob isn't saying "If you aren't using TDD at the startup phase, you suck!" What he's saying is "Just because you're in the startup phase and think you're invincible doesn't mean you should throw best practices out the window." There are good excuses for not using TDD, and I tend to agree that "We're in a startup phase. We don't have time for this crap." isn't one of them.
- thetrumanshow 14y ago
  Had I opened with "Interesting article, but tangentially..." then we'd still disagree, but maybe we could have avoided the straw-man ding. :)
  - j_baker 14y ago
    You're right. Then we'd have a passive-aggressive ding. The only thing you would have to add to make it perfect would be "...but I'd never claim the author is trying to do that."
    - derefr 14y ago
      I wouldn't even call it passive-aggressive; it's soapboxing, plain and simple. > This article is about X. But I don't care what the article says about X. I have something to say about X, and by God, I'm going to say it. --But I don't know that this is a particularly problematic thing to be happening in a threaded comment system, since people can get sidetracked by "soapbox issues" in a subthread while letting the "parent conversation" continue around them. It's just kind of confusing for people who treat this place like a linear-chronological message-board that discusses one topic at a time.

JoeAltmaier 14y ago

Skipping testing amounts to risk. The winners in a brutal race take every risk to win. So yes, its foolish to skip testing. And yes, the race goes to the foolish.

unclebobmartin 14y ago
The winners in a brutal race have learned how to avoid as much risk as possible. The prize does not go to the biggest risker. The prize goes to those who do everything they can to minimize the risk of going slow. What slows a programmer down? Debugging. Messy code. Fear of change. How do you minimize those risks? A comprehensive suite of tests. How do you get that suite? TDD.
- JoeAltmaier 14y ago
  And you come in 2nd or 3rd in the race, and lose.
  - unclebobmartin 14y ago
    That is certainly your fear. It is a fear I don't share. Firstly, I think you'll get there faster if you work well. Secondly, as MySpace showed, being first isn't really all it's cracked up to be.
    - JoeAltmaier 14y ago
      Im not making myself clear. You might beat me, personally, once. But you won't beat that one lucky guy that didn't do the testing, got his code to work the 1st time, and got to market before either one of us. That's how the game's played. Conservative, careful, correct coders don't win that game. And if your business plan is to be 2nd to market, well, good luck with that.

gearoidoc 14y ago

Whilst not exclusive to TDD, I think DHH said it best: "The road to programming hell is paved with “best practices” applied too early."

jaytaylor 14y ago
But if you don't start with TDD, then when are you going to put it in? How do you transition into it? Are you going to go back over everything you've written and write tests for all of it?
- gearoidoc 14y ago
  I think you build it in when you know what you're actually building. In the initial dev cycle, everything and anything could be ripped out so I'd suggest holding off til perhaps halfway through your MVP cycle until you start TDDing.
freshhawk 14y ago
And the road to failure is ignoring best practices because "I'm special" and "I'll do it later because I'll magically have time then". It's a balancing act that pithy aphorisms don't help with. There's a reason people use the metaphor "Design Debt". Debt is a tool with tradeoffs, financial or design, use appropriately.
- gearoidoc 14y ago
  But what if the debt outweighs the benefits? What if you can't measure the debt?
  - freshhawk 14y ago
    If the debt outweighs the benefits you don't take it, I have a project right now that I "know" will be fragile and a maintenance headache later. So even though it's urgent I'm writing the appropriate tests and doing all those best practice things that you know will pay off 10x later but often skip because you don't want to take the time now. You could measure how much debt your team tends to take on, by measuring how much time is spent on new features and how much on the type of bugs and the refactoring that pays off design debt. This still misses what I think is the most important part of design debt, how much longer it takes to implement new ideas because you are paying the "the pieces this depends on were rushed and don't work/integrate/extend well" tax (I guess I should call this "interest payments" to not mix metaphors) . You can't measure debt as you take it on though, the best you can do is estimate how much work it will be to fix it later (and that's only if it needs to be fixed later, maybe you get lucky and your hacked up code just stays good enough). We all know more than enough about the pitfalls of estimating software projects and this adds in more uncertainty about future need. This is all part of the craft side of software development, experience helps, but it's not something easily measured. Too many people take that as an excuse to just do the quick and easy thing and say "move fast and break things!" or fall back on over designing and never get any work done. HN talks about the latter more often and pretty much ignores the former. I find this strange, I've read plenty of accounts of failure where the reasons boiled down to "we got to a point where we couldn't adapt our codebase to changes needed to face a new competitor, change in the landscape, business model pivot, etc. because it was too crufty". Enough design debt means some smaller and more agile competitor will eat your lunch. Well that's definitely a long enough answer to two simple short questions.

zwieback 14y ago

Hacking up a quick demo the non-TDD way definitely has its upsides but if you don't force yourself to review and rewrite your early code you would have been better off doing the right thing from the start. Just preaching TDD is not helpful, we need a coding process that allows clean separation of early and production code.

jmduke 14y ago

I like to talk about TDD using a finance analogy. One of the first concepts you learn in introductory finance is that, in a mature market (ie the stock market -- basically something with a bunch of people and a bunch of information) you have to be compensated for risk. This is why a riskier stock -- like tech stocks or penny stocks -- can boast incredible amounts of risk but generally will give you higher returns over the aggregate than something like pork commodities or a CD. When you choose to purchase a tech stock -- or any stock at all -- you're saying "okay, I recognize that this is riskier, but I think I'm being fairly compensated for the risk, too." Choosing to eschew TDD is like purchasing stocks. By definition, going through TDD is going to be a safe route, but it's rare that TDD (at least in my and my friends'/peers' experiences) is actually going to make you get from Point A to Point B any faster. TDD isn't, by default, a superior or inferior approach to anything: it's a tradeoff -- do you want risk or do you want return? Sometimes, you want to minimize risk, and that's probably smart. Sometimes, you just want to produce an MVP -- and that's okay, too.

cbracco 14y ago
Couldn't agree more, it depends on your situation and goals as a start-up.
donw 14y ago
I like your analogy, but it needs a little something extra. TDD (and BDD) is an assurance that the promises that your code made yesterday will be kept tomorrow. If you break those promises, you do so consciously. Not doing TDD is like investing in the pink sheets -- the company you invest in might be a fly-by-night operation, or it might be a legitimate business. It could be both, but you never know, and assuming you're not part of a pump-and-dump scheme, the rewards aren't really higher than listed stocks. But the risk is a lot higher than listed stocks. Lots and lots of risk, but very little upside, which is the same as writing code without tests. You save a little time getting started, but spend a lot more time doing the repetitive work that testing automates, and in doing so, eat a lot of long-term technical risk that no company should consider acceptable.
d2vid 14y ago
To take your analogy further, an early-stage startup has a high cost of capital. You are expected to produce massive returns (or fail). You thus are explicitly required by investors to take risks. You have a duty to not invest in the software equivalent of CDs.

spartango 14y ago

My feeling is that the point of this article isn't strictly to promote TDD or any single engineering tool that startups tend to eschew. Instead, the author seeks to make the broader point that cutting corners in engineering is not a good idea even in a startup. While it can make startup engineers feel like they are moving quickly relative to the competition, the engineering debt incurred is non-negligible. I strongly agree with this point, and don't buy the blanket rationale that speed trumps everything in a startup. Yes, speed is quite important when you are bringing something new to market; it lets you get your foot in the door and rapidly find the right fit for your product. But if you cut corners then your product will suffer, and you'll end up treating your customers poorly by it. There's a fine balance to all these things, of course. We all have to find the right balance between keeping engineering standards high and heavy-handed structures.

freshhawk 14y ago
Boy, did I have to scroll down far to find someone who read the article. I was hoping for tips about how to find a balance when other team members have bought the blanket rationale that speed trumps everything and then complain about why we spend so much time fixing bugs and fighting fires instead of working on the next thing. Instead a bunch of reading comprehension challenged commenters are bitching about TDD again.

tptacek 14y ago

I thought this was going to be an article about overwork, or about charging for what you build as a way of proving your market, or about how employee stock options are often a bad bet. Instead, it's about TDD. What a disappointment.

orangethirty 14y ago
Which is ironic, because the real startup trap is the one where people build things but are told to not charge for them (or charge very little for it).
wglb 14y ago
Its Uncle Bob!

moocow01 14y ago

Largely agree with everything here but in my opinion many times there can be a very strong current that forces you to do it the fast, stupid way. In my opinion if you dont have any of these problems, you work in a unique place... - Agile - Agile IMHO largely screws you in making sound technical decisions. Its not necessarily because agile is flawed - its usually because business/management uses agile as an excuse to randomly take a hard left turn every other week making it much harder to make long term architectural choices that are beneficial. - No agreed upon standards or unification amongst software professionals - The accountant in this situation has a set of standards and expectations that allow/force him or her to do things in a moderately set way. This allows for the accountant to usually fend off management from pressuring things to be done in a shoddy or just get it done way. On the other hand its much more difficult for software professionals to say "we as a group do not condone writing shitty software" (because ironically a large number of us do ... "move fast, break things" has done more good than bad from my perspective) - Ageism - Also known as experience doesn't really matter. Some will say thats because technology changes so much - but it actually doesn't. Just because you've been grappling with software problems and design patterns in Java doesn't mean that when you switch over to Python that really anything changes. Same shit, same problems, different words - but in all honesty we seem to be pretty bad at building on the experience of our elders because they are over the hill at 35... what could they possibly teach us. For me it at times has been frustrating because coming out of school I really enjoyed the fun of designing systems and code that are sustainable, performant, etc etc but there seemingly is typically more reward for just throwing quality out the window in startups. Just my personal experience.

MattRogish 14y ago

(tl;dr: it depends) It's a hallmark of "experienced" non-dogmatic product people (UI/UX/Dev) that can use their intuition to know what are the happy paths that they need to test, what interfaces are likely to not change (e.g. a user is probabilistically always going to be a member of an organization, so unit tests around that are likely not waste), and what level of quality to introduce for a given feature relative to the probability the feature is going to exist in perpetuity. You can concretize this by calling it "spike driven development" if you want (that's what we do) but the point isn't that TDD is faster or slower but that high coverage might be inappropriate (TDD isn't binary - degree of test coverage goes from 0-100%) at different phases of the build/learn/refine cycle. For example, if we're building a speculative feature that we think has value but hasn't been proven yet, we want to spend the least amount of effort (code/time/money/pick some) possible to prove that it's viable. Small bugs aren't really that interesting (provided the thing "works") and so we're explicitly willing to live with degraded quality to prove the feature is worth further investment. We're also explicitly not building a "large" feature (so it's very likely to get coded in a day or two) so the surface area for major showstopper bugs is minimized. Often the feature will be thrown away or majorly refactored into something that is completely different. In this case, full-bore 9x% coverage TDD is probably waste as the feature existed for a short period of time, the interface changed dramatically, or otherwise was proven invalid. High test coverage makes major interface refactors really expensive and you really don't need that level of quality for speculative features. After you prove this feature is a "thing" you want to keep around (and you've nailed down the interface), then it's a perfect time to re-write it with a much higher degree of test coverage.

jfarmer 14y ago
Stop being sensible, will you? This is exactly how we did things at Everlane (http://everlane.com http://everlane.com) and Dev Bootcamp (http://devbootcamp.com http://devbootcamp.com).
pifflesnort 14y ago
> After you prove this feature is a "thing" you want to keep around (and you've nailed down the interface), then it's a perfect time to re-write it with a much higher degree of test coverage. I find that tests help me write things that are complete and "nailed down" the very first time.

jasim 14y ago

It is easy to hate TDD. In a Rails project, the Rails boot-up time makes TDD a painful experience if you are not using tools like Zeus or Spork. Even in the presence of such tools, you need a powerful machine to not hate the slowness of the whole thing and worse still, break your flow. My recommendation for someone who hates TDD for the wrong reason, aka: breaking the flow, would be to get a fast machine, fix the code to make TDD as painless as possible and to use the right tools. But once you start using TDD as a tool to organize your thoughts and model your domain, you might end up becoming too dependant on it and find it hard to work any other way. This is anecdotal experience. Also, learning how to wield TDD properly takes a lot of time, error and practice. Good things don't come easy. There are obviously places where TDD isn't a good fit - a spike, requirement that is known in advance to change soon, and exploratory programming are all candidates. However, good practice dictates that you refactor your code once a spike calcifies into production code. At this point, TDD becomes just unit-testing. Most of the arguments against TDD in this thread seems to be against unit testing in general. But we know unit tests are important. Doing it before the fact increases the value of the unit tests manifold and also ensures that you do have coverage (though that is not at a primary objective).

lifeisstillgood 14y ago

Know the rules before you break the rules. The accountant analogy is dead on - that and the best quote of the day: You'd fire his ass! You'd fire it so fast that the rest of his worthless carcass would be left outside the door wondering where his ass went! Brilliant.

petenixey 14y ago

Man, I've been back and forth on tests from one extreme to the other. I have to say that I do agree with this but with one caveat - Test Driven Development is a bit miserable. Test "Driven" Development is a real invitation to write too many tests. A small, good set of tests gives you freedom to work fast, to refactor and to have multiple developers pushing simultaneously. Not having them really isn't sustainable past a certain point but equally to many tests will drag you to the ground. Tests are overhead, they cost time to write, maintain and run and TDD tends to drag you into the deep end. Avoiding them altogether ends up being a false economy (if only in that it makes you too nervous to push often). To those who fear the slippery slope, a nice self-annealing approach to get into testing is to only write tests for something that has failed. That way you waste no time writing tests for things that are actually pretty robust but equally you avoid addressing (and fearing) the same issue twice.

agilord 14y ago
I'd like to emphasize this approach: "a nice self-annealing approach to get into testing is to only write tests for something that has failed." It is great for old-timer enterprises without any test, and balances the time pressures of startups. However, you need to write those test not only for prod failures, but also if that something failed in your dev environment.

stickbranch 14y ago

I want to vomit when I see titles like Master Craftsman.

unclebobmartin 14y ago
Please use the air-sickness bag located in the seat back in front of you.

rcconf 14y ago

The way I do my TDD is by writing the unit tests to specifically test the functionality of the code. For example, if I'm writing a module that does validation for a specific type (for example, in JS) I tend to test the functionality while writing the tests. I am unable to speak amongst others, but you have to test your code eventually, and it's either going to be in the classes that are using the code, the REPL, or it's going to be in your unit test/integration framework. I usually pick the test framework first because it serves 2 benefits: 1) I can prove my code works. (I must prove this to my self, or the people using the code at some point.) 2) I can reliably change the code later. Complexity is also mentioned in a couple of comments here. TDD helps simplify complexity by writing code that is testable. It also makes you think about your API as you're writing the different possibilities the module/code can be used in. Further, I think there are multiple issues with tests, and depending on the type of test your doing, different problems can arrise. You have your unit tests, and then you have your integration tests. Unit tests are relatively easy to write (if your code is split into individual chunks, that do a specific task.) and they should always be written regardless of the time you have. I think the major issues arrise in terms of time are the integration tests. When you're testing the functionality between complicated modules that require databases, i/o writes, network communication, it is sometimes hard to write the tests, and it may not even be worth it. The majority of people who disregard TDD, usually disregard a specific sub-problem of TDD. I think TDD has its benefits depending on what type of tests you're doing, and how easily the problems can be solved with those types of tests. You can always pick to write unit tests, and disregard integration tests, etc. I think there are a lot more issues to expand upon, like the language you're using, the platform you're using and how often your code is required to change, but TDD has large benefits, and code rot is very real and TDD can help mitigate it.

johnrob 14y ago

So the key to meeting your short term goals is... blow them off and focus on the long term goals. Doesn't make sense to me.

gfodor 14y ago

Over the years, I've grown less and less likely to write lots of tests, after being a very large test zealot during the peak of the TDD wave. There are a few reasons. First, yes, TDD slows you down. The reason this matters is because a lot of our time as developers is spent exploring ideas, and it's pretty well understood that the faster you can get feedback on your ideas the easier it is to creatively develop them. In fact, the final result of your creative process can be completely different depending on the feedback cycle you have. From the micro to the macro perspective, religious TDD introduces an constant factor slowdown for small projects that yes, ends up being a net win in the long run if you live with the code for long, but is a net loss in the short run and also can prevent you from finding a solution to a problem since your creativity is stifled by the slow speed of idea development. Second, when building web applications, building tests turns out to be fairly overrated. First, people will tolerate most bugs. (Write good tests around the parts they won't.) Second, if you have a lot of traffic, bugs will be surfaced nearly instantly in logs and very quickly via support tickets/forum posts. (What if there are bugs that don't make it into the logs and don't affect people? If a tree falls in a forest...) Much more important than mean time between failures is mean time to recovery. I'd rather have ten bugs that I fix within 5 minutes of them affecting someone than one bug that festers for a month. Not only because this is healthier for the code base in terms of building robustness, but also because human behavior is such that many fast bug fixes make everyone feel good but few lingering bugs make everyone miserable. People want to feel like you care, and are much more likely to feel cared for when you fix their problems quickly, and are not often interested in just how few bugs you ship if the one that you do has been affecting them for a month. This isn't theoretical nonsense, it's a very real phenomenon where you use tests and manual testing to get up to a basic working prototype and then just throw it over the fence to flush out the bugs. It's the only way to do it anyway. (This only really works if you have traffic and can deploy changes quickly. To paraphrase a colleague, production traffic is the blood of continuous deployment.) Obsession over deploying bug-free software (an oxymoron anyway) is usually coming from people who haven't gotten over the fact that we don't need to ship software once a month or once a year but can do it every 10 minutes in certain domains. Instead of focusing on not shipping bugs, focus on shipping faster.

randomdata 14y ago

ardit33 14y ago

Wait, since when TDD become a best practice? I know this author is advocating it, but has 0 (zero) solid data or evidence to back this assertion. He is just trying to sell another religion. What I have noticed, TDD is being pedled heavy by RoR contracting shops that just care about billable hours and not vested if the startup or company is going to make it on the long run. I see often that it is the usually youngish and guidable engineers that fall in the trap of eating it up. TDD is something that makes younger and less experienced developers feel good (more locs) and a false sense that the code is correct while there is solid data that even 100% unit testing coverage finds only 20% of defects, at best. Having shipped many world class products, and none of the processes that build those products had any resemble of TDD in them. So, to all the pragmatists out there feel free to ignore the advice on the article. I'd yet to hear a super successful startup that used TDD. Having Tested Code (which is a good goal) != TDD. TDD to me is putting the carriage in front of the horse. Anybody that have created large systems from scratch, TDD is a major slowdown on the refining process. Your job is to SHIP a working product. Testing is only a small part of it, and its positives should be considered alongside its negatives and side effects on the shipping timelines. Having worked for over 10 years in the industry I have to meet any great programmer that preferred TDD. I hate to attack the author, but from his bio he seems a bit like a process peddler. I don't see him have worked on any successful startups, or anything that had a good exit. So take his claims with a grain of salt. http://www.8thlight.com/our-team/robert-martin http://www.8thlight.com/our-team/robert-martin He is trying to sell a religion, so buyer beware.

nthj 14y ago
I'm not a huge fan of TDD either. Tests are helpful when refactoring or adding new features because you avoid regressions. Regressions are the worst kinds of bugs because clients get really, really annoyed with "I thought we fixed this the last 3 times" and "we already paid to fix this" type scenarios. That said, I think the article has a solid thesis. Just because you don't practice TDD doesn't that mean building things correctly won't help you ship working software faster. If your code is brittle, write some tests around it, refactor a bit, and keep running fast. We can debate over whether you should always write tests first or only write tests to cover brittle and important features, but that's a "how much" argument. The real problem is when you run into methodologies that always involve work-arounds, and you let the technical debt continually pile up for weeks or months because "you're a startup." This will cause you to run very slowly, and sooner rather than later. I feel that's Uncle Bob's real point, and it's a valid one.

matt2000 14y ago

I disagree with this almost entirely. Startups _are_ different: 1. What you are doing starts at 0 value. 2. You can (usually) break it and it's ok. 3. You need to change stuff a LOT to figure out something that someone wants to use, rewriting tests makes that slower. Every test case you write has a cost: It verifies that something of unknown value works. What if the value of that code is 0? Well, you just doubled down on something useless. There are no absolutes in software development, but how we did it was have uncomfortably too few test cases when something is new, change the product for a while until someone actually wants to use it, then add a bunch of tests until we know it works from there on out. And I hesitate to add: Also we work in Java so a bunch of testing comes for free, so there's that.

lucian303 14y ago

Great article. Not just about TDD but about all of startup culture. The discussion here is about TDD but how about the long hours. They are not necessary either. No process needs to change. If your company requires long hours, management has made huge mistakes. Period.

DaniFong 14y ago

Imagine test driven development for visual art. How would one write a test for visual art, beyond viewing and comprehending the partially completed work? If one could write a test in advance, one would have made a significant advance: distilled to a verbal form a description of what good art is. Instead, many artists -- or software authors, or chefs, find a way to repeatedly sample and appraise their work as it develops over time. I think this is far more important than TDD, because the truly important problems of software engineering are not in making software that can achieve simple correctness, but in making something that people want.

mattvanhorn 14y ago
Actually, that is a great metaphor. But you should know, TDD is the way I repeatedly sample and appraise my work as I go. Multiple times per hour, in fact. Red -> Green is the sampling and Green -> Refactor is the appraising.

barrkel 14y ago

"as if any bug is acceptable!" This is the point at which I stopped reading. Yes, some bugs are acceptable. The point of writing software is not to create bug-free code; it's to create value. The marginal returns on eliminating the last bug are much lower than implementing new functionality. As for the gist of the post, from my experience, TDD is OK; but it is unit tests that are essential. They are far more important in dynamic languages, because you otherwise have very little feedback when you make mistakes as simple as a typo. They're also important in static languages, but fairly large programs can successfully be written with without anything near the volume of testing needed with a dynamic language.

PasserBy2 14y ago

"Oh, I know you are a warrior-god." Well, at least someone noticed!

alexrson 14y ago

A useful reminder

Mahn 14y ago

I'm not sure if the author has any idea what is it like to bootstrap a web product and watch the money go down the drain day after day, with zero income to compensate, and with the certainty that you won't make it for another 6 months if the product is not out and making revenue. I'll deal with the technical debt later, thanks. I'm pretty sure Facebook and Google didn't do anything remotely close to TDD when they first shipped, and yet they survived.

icedchai 14y ago

TDD is often a waste of time. I worked at some place that had taken the TDD philosophy to the extreme: every getter and setter had a test. On the opposite side of the spectrum, I worked on software that processed millions of dollars in daily transactions and there were maybe 5 "tests" in the whole system. This was about 500,000 lines of C and C++ code in the early 2000's. My personal philosophy is to write unit tests where I think it's important, not test everything.

mattvanhorn 14y ago
I won't write tests for getters and setters, but then again, I try not to write getters and setters very much in the first place.

michaelochurch 14y ago

This "trap" is Deadline Culture and it doesn't pertain only to code quality, but to other matters like product direction and personnel decisions, which are made quickly and often badly. It's sloppy and destructive. Companies with Deadline Culture often make it easy for a total asshole to gain power just by citing existential risks that either do not exist or are of extremely minor importance. Deadline Culture companies are obsessed with competitors, even though it's their own internal corrosion that does them in. Sometimes, having deadlines is unavoidable. They might be set externally and exist for regulatory reasons. Deadline Culture is when the company starts encouraging the formation of unreasonable deadlines that triumph over professional integrity and intellectual honesty. VC-istan seems to encourage it with the whole "if we get bored with you, we'll fund your competitors" arrangement. Deadline Culture is, however, great for the true fascist. Fascists love (real or imaginary) existential threats, especially vague ones they can continually adapt to their needs, but that come with time pressure.

unclebobmartin 14y ago

To those of you who asked for "links" to studies that prove that TDD works; google around, you'll find plenty. Some are positive, some are negative -- what else is new. Now, please show me the studies that show that _your_ discipline works. To those of you who consider TDD a religion; you are being silly. TDD is a discipline, like double-entry bookkeeping, or sterile procedure. Like those disciplines it has aspects that are dogmatic; but that aim at a purpose. That purpose is to help you write software fast and well. To those of you who think I'm a process peddler. You're right; I am. I make a very good living teaching people how to do TDD. I have been doing so for over ten years now. I hope to continue for some years to come. My goal is to help raise our industry to a higher level of professional behavior; because at the moment our behavior is pretty poor. To those of you who wonder whether I've ever worked at a real start-up. I've worked at several over the years. My first was in 1976; and it was very successful. Another was in 1989, and it didn't do so well. I've recently founded cleancoders.com a startup in video training for software developers. The website is done entirely with TDD. And this doesn't count the rather large number of startups I have consulted for in the last 10 years. So I've got a _lot_ of experience with startups. Folks, I am 60 years old. I got my first job as a programmer when I was 18. I wrote my first program when I was 12 (on a plastic 3-bit computer). I started using TDD in 1999, after I'd been programming for thirty years. I continue to write code using TDD today. I've seen the startup trap. I've lived the startup trap. I strongly advise you to avoid that trap.

spullara 14y ago

I find that making sure the foundation of an application is solid (I'm an API first kind of person) let's you get away with a lot less testing in the upper layers. This lets you move faster and experiment far more quickly than if you were built on quicksand.

digitalWestie 14y ago

I see this as an argument for testing, but not necessarily TDD. Why have the two become so intertwined?

pardner 14y ago

Statistically speaking, a web startup can safely not invest in TDD. For precisely same reason that a daily Russian Roulette player can safely not invest in health insurance. But for the minority who miraculously dodge the bullet, the extra investment will eventually become a lifesaver.