From time to time I notice people saying this like “TDD can’t lead to good design”, and, like a dope, I can’t ignore them. Quite ironically, they make these statements because of the very act that practising TDD helped me understand how to do well: abstraction. I’m getting ahead of myself.

When this happened recently, it took several exchanges to reach to the heart of the difference of understanding. Along the way I ran into a handful of assumptions, preconceptions, and hasty conclusions that tend to come in discussions of the merit and power of TDD. At its core, I came to face (again) one unavoidable fact.

Many people have the impression that TDD advocates believe that TDD magically always leads to amazing results.

I don’t even like resorting to writing “many people” here, because on its own, that sets up a “weasely” claim that one could easily defend by shifting the goal posts. (How many is “many”?) Maybe I can improve the situation by recasting this as a common observation.

I routinely see people expressing annoyance at the apparent belief of TDD advocates that TDD magically always leads to amazing results.

This would annoy me, too, if I didn’t know better. Sadly, we have a vicious cycle: a person who believes that TDD advocates drastically oversell TDD will tend not to try TDD and will therefore not as readily discover what TDD can help them do nor that thoughtful advocates don’t adopt such easily-refuted positions. Such a person, for example, might look at a novice practising TDD, see the novice produce a hideously complicated design on their first or second try, not understand what that programmers has learned in the process, and conclude that TDD didn’t deliver on its promises. I see so many logical problems with this line of reasoning that it makes my head spin, but I accept that a significant number of people have had this experience.

To you who has felt this way, I can only apologise. I have, in the past, carelessly made the claim that TDD magically leads to amazing results. I have even believed it. I have since refined my position (and the way I convey it), and I hope others will feel inspired or pressured or guilted into doing the same.

TDD Doesn’t “Do” Anything

First things first: TDD is an idea and ideas don’t do things. Ideas don’t have literal agency. We sometimes imbue ideas with agency, but only as a metaphor and typically only as a convenient shortcut. We say, “TDD generates good design” when we really mean “people who practise TDD tend to generate good design”. Well… when we really mean “people who practise TDD tend to generate better designs on average”. Well… when we really, really mean “experienced practitioners practising TDD mindfully tend to generate designs more suited to their needs on average, all other things being equal”.

You can see why we stick to “TDD generates good design”. You can also see why this breaks down.

I implore you not to take statements like this too literally. When someone says to you, “TDD makes my design better”, you could easily interpret this in the narrowest, least generous way, as a claim that a trained monkey could perform the steps of TDD and turn out masterpieces. Please don’t. Please instead look for ways to interpret the statement that make obvious sense to you. That doesn’t mean that you have to believe the statement, but I beg you to try to interpret the statement in a way that you would have at least non-trivial difficulty refuting. Aha!, you might think, this person treats TDD as an integral part of the process that helps them design software well. I wonder how that works? and could they have got that benefit without TDD at all? This would lead to a much more enjoyable and productive discussion, don’t you think?

At a minimum, when someone says “TDD does X” or “Agile does X” or “Scrum doesn’t do X”, please remember that ideas don’t do things, but people do. Therefore, Scrum can’t guarantee regular delivery of shippable features, Agile can’t fix communication problems, and TDD can’t generate good design. We say these things as shorthands, people outside our immediate audience read or hear them, and this causes confusion. This confusion will probably not go away any time soon, so please remember to look for ways to interpret these shorthand statements as plausible before you jump down the writer’s or speaker’s throat.

Even If It Didn’t Work For You This Time…

On the other hand, also don’t look to blame ideas for your failed implementation—of anything. If you must blame anything, blame people. This, by the way, does not mean that all ideas are valid and helpful and that all failures are the fault of people trying to implement them. It means that the success or failure of your implementation of an idea depends on thousands of variables, many of which you couldn’t control, and that you can’t conclude “good idea” or “bad idea” from the result of your implementation. If an overwhelming proportion of implementations fail or succeed, this tells you how easily one can implement the idea, but not whether the idea “works”. Not a lot of people can build rocketships, but space flight still works.

“It works” ranks as one of the most confusing sentences one can utter in a language. Its lack of precision often leads to wild differences in understanding and interpretation. Without the appropriate context, saying “it works” or “it doesn’t work” has the effect of throwing a hand grenade in the middle of a group of people and walking away. For that reason, I try very hard not to say “it works” except where I have specified very carefully and precisely which “it” and how I judge “works”.

For example, when discussing TDD—in particular the testing aspect—I tell people that I’ll use “it works” as shorthand for “it passes all the tests we’ve written so far”. I do this to make the point that when I practise TDD, as long as the tests pass 100%, I’ve built something that “works”, and the set of tests articulates what the thing does. I like the feeling of building upon a solid foundation, and thinking about my code “it works” gives me more confidence to add a steady stream of new behavior. I like how this feels. I get this good feeling as long as I remember that by “it works” I mean specifically “this code base passes the set of tests I’ve written so far”, and that “it works” says nothing at all about whether “it does what I wanted”.

Accordingly, I find it really unhelpful to engage in any discussion centered around the question “Does TDD work?” I’d much rather explore what you expect to change when you practise TDD, which problems you expect to better be able to solve, which effects you expect to notice, and which obstacles you expect it to guide you towards overcoming. Usually, when someone asks, “Does TDD work?” they mean at least one of these things. I like to get past the shorthand early in a discussion in order to test my own assumptions about what we’re “really” talking about.

If you tried to practise TDD and didn’t get the results you expected, then you need to consider several possibilities.

  • Maybe you did the steps wrong.
  • Maybe you expected to see results that very few other people have ever got.
  • Maybe you didn’t understand how TDD guides programmers towards improvement.
  • Maybe you didn’t reflect on what started changing after you started practising TDD.
  • Maybe you didn’t ask enough questions of experienced practitioners when you got into strange territory, and that led to developing counterproductive habits or assumptions.
  • Maybe you didn’t stick with it long enough.
  • Maybe you stuck with it too long without challenging yourself to try new things.
  • Maybe the problems that TDD usually helps one solve weren’t anywhere near the bottleneck of the system around you.
  • Maybe unseen forces in the system around you conspired to thwart your efforts. Maybe you need to try to look for those forces and understand them.
  • Maybe TDD is absurd and nobody should do it.

Please note that this list does not imply “If you failed with TDD, then you did it wrong.” Interpret more generously! If you failed to get the results you expected when you practised TDD, then I imagine some important ingredient of learning was missing, and the fault might or might not lie with you.

  • I’ve seen people write 10 tests at a time, then make them all pass. It might help, but TDD practitioners recommend strongly against it.
  • I’ve watched programmers practise TDD who haven’t yet developed enough sensitivity to various code smells, and don’t notice what appear to me as obvious potential improvements. Hell, I was that programmer when I first started.
  • People routinely expect TDD to result in an 80% reduction in the cost of shipping features within a month. I don’t know where they get this idea.1
  • The rules themselves only give the programmer specific kinds of feedback, but the programmer has to decide how to use the feedback. I teach TDD as a mechanism for practising how to notice and react to that feedback.
  • Practising TDD challenges some fundamental assumptions about “good design”. TDD encourages a more pluggable design, which competes with many programmers’ conception of encapsulation. It took me years to resolve this apparent contradiction to my own satisfaction, but I might still “have it wrong”.
  • Practising TDD challenges some fundamental habits we have about how to write code. We literally start by working backwards. This leads to an adjustment period—even ignoring the learning curve. This slows us down. Still, even after nearly 20 years of warning them, some people don’t see this coming.

How Do I Succeed With TDD, Then?

I don’t know. You have to try it and see. I can share with you what worked for me.

  1. Practise the steps of TDD diligently and seriously. Find a place and a task where you can tolerate going at a glacial pace while you learn.
  2. Find a place to ask questions and get answers. Most of my questions amounted to, “Is this really better than that?” and “The rules tell me to do X, but my intuition says I should do Y. What does that mean?” (I admit, I found this easier in 1999 than today. Back then, you had a few places to go for answers and all the “smart folks” hung out there.)
  3. After you’ve become comfortable with the red-green-refactor cycle, start pushing the boundaries of refactoring. Refactor way past what feels sensible. Push hard. Share your code and beg for feedback from experienced practitioners. Prepare yourself to fall off the cliff a bunch of times—you’ll learn how to find the edge that way.
  4. Don’t just push the boundaries of refactoring in production code, but also in tests. Design tests like you’d design production code.
  5. Give yourself time to sit and think about what you’ve done. In some cases you’re challenging long-held beliefs about how to design software. You won’t resolve that uncertainty overnight.

Maybe most importantly, articulate why you want to practise TDD in the first place. I did it because I wanted to go home at night and not worry about my code. Early pioneers Kent Beck, Ron Jeffries, Chet Hendrickson, and Martin Fowler promised me that TDD would help me discover my own mistakes (“bugs”) sooner and even start eliminating them just before I injected them. (So-called “pre-bugging”.) I thought that if I could catch most of my own mistakes, then I could avoid embarrassing confrontations with testers and managers about the quality of my work. Little did I know at the time that with TDD I can do more than merely reduce the number of mistakes I publish in my code… but at the time, achieving that goal benefited me more than enough. That strange pain in my chest went away.

TDD is not magic. The rules themselves do not guarantee success. You have to keep the brain switched on. You have to pay attention to what’s happening. You need a place to ask questions and get answers. You need to know that when you practise TDD, your code starts to speak to you, but in a langauge you don’t yet understand, and that when you write tests and run them, this act equates to learning a language by hearing it, trying to speak it, and by native speakers correcting you. If you do these things, then I expect TDD to “work” for you.

Epilogue

So where should we hang out today to get good advice from thoughtful people about TDD, good design, and all that stuff?

Sadly, I don’t have a magic answer here. I don’t hang out in the Yahoo! and Google groups much any more, because I see too much noise. (Good news: nobody goes there any more, so we’re swinging back around to mostly thoughtful people going there, which I like.) You can use Stack Overflow and Quora and its cousins, but you really have to wade through a lot of stuff to find good advice. People (rightly so) use it as a springboard for their careers, and so you have a lot of people trying to get your attention with strong opinions and I can’t really tell whom to trust.

Look everywhere. Find people that you think you can trust. Follow them everywhere: Twitter, Facebook—even Google Plus. (No, really.) I just think that today’s netizen looking for good advice has to resign themself to kissing a lot more frogs before finding a prince. Unfortunately, I know of no secret meeting space where all the cool kids go.

Maybe that simply means that I’m not a cool kid, in which case, don’t ask me where they are.

References

J. B. Rainsberger, “Don’t Let Miscommunication Spiral Out of Control”. I describe the Satir Interaction Model, which helps me navigate differences of perception, understanding, and interpretation, all of which help me keep communication problems from erupting into seething, burning hatred.

Ron Jeffries, Chet Hendrickson, and Ann Anderson. Extreme Programming Installed. I first learned TDD by reading about it here, so it really is Chet’s fault.

J. B. Rainsberger, “Becoming an Accomplished Software Designer”. This article briefly describes the interplay of mechanics (following the rules) and intuition (internalising the principles) in learning “better” software design.

  1. Matt Duffy tells me that the IEEE Software magazine special issue on TDD made a claim similar to this somewhere. I really hope not.↩︎