I was interrupted today by a post debunking some person’s claim about TDD. I had planned to do some boring administrative work, but I felt a strong reaction to what I read, so I dropped everything to write this.

You don’t need to read that post before reading the rest of this article, but you might prefer to. I’m reacting to a post that disagrees strongly with the following statement. In it, the author easily debunks the claims and dismisses it as an ordinary complaint of “I tried TDD and it didn’t work”. It reminded me of Ron Jeffries’s classic “We Tried Baseball and It Didn’t Work”.

I’d like to try to engage this claim differently. Let’s see what happens.

TDD goes against agility. It forces you to have a plan before coding. The purpose of testing is validating that something behaves as expected, so using tests as the driver reduces flexibility.

Are you yelling at your screens yet? Have you rolled your eyes? It’s OK. Take a breath. It’ll be fine.

What do I agree with in this statement? Which feelings do I infer lie behind it?

Indeed, TDD demands that you have a plan before writing code, although I suspect everyone “has a plan” before writing code—at least some kind of plan. I doubt that this person dismisses entirely the need for at least some kind of plan. What kind of “plan” is their “plan”? How detailed is it? How much detail do they need in their plan before they feel confident that they understand what to do next? What do they think will happen when their plan meets reality and things start to go wrong?

Some programmers, early in their TDD experience, have very uneven experiences. Sometimes they know very clearly how they want to design something, so then they build the pieces test-first (to suffer less from defects), put them together, and they mostly just work. They feel successful, because they’ve found all the simple mistakes themselves that testers (or customers!) used to find. This works well when they have a clear idea how to decompose a solution into modules or classes or functions or whatever they use to organize their code.

But what happens when they don’t have that clear idea? They feel like they’re missing a plan. They feel like they’re wandering aimlessly, making it up as they go along. They feel unprepared for what happens when they start writing code. This is just hacking!

If that sounds like you—or someone you work with—then I have good news for you: no, you don’t need such a detailed design plan to start driving code with your first few tests. It will be OK. Take a breath.

What’s Happening Here?

Many people worry like this when they are leaving “Stage 1” of TDD and entering “Stage 2”. (What are these stages?! Queue up this video in the background.) It feels uncertain and worrying. When they don’t have a clear picture in their head of how to design things, they feel like they might know maybe where to start—at least a little—but after that, they worry that they’ll get lost along the way. They don’t have an extensive test list or a 5-box diagram of the components they’ll probably need. It feels like a jumbled mess in their minds. This can’t be right!

I could absolutely imagine such a person feeling disorganized, even incompetent!¹ They might feel like they’d be “doing it wrong” if they tried to proceed without a clear and detailed plan of what to build next. After all, that’s how they’ve experienced success in the past! They also see other people succeeding with TDD who have those things, so they must be critical! In a mood of frustration, I can imagine a person seeing that kind of planning as making commitments up front, as reducing flexibility, and ultimately as going against agility.

Good! Maybe we understand them enough to move forward.

Evolutionary Design Means Exactly That: Evolving

You don’t need to have all those details in mind in order to practise TDD. You can, and it feels quite pleasant when you do, but you don’t absolutely and always need them. Indeed, I’ve noticed this pattern: as programmers stop drowning in defects, writing code that works becomes routine—even boring. When this happens, the cost of planning all those things out up front—extensive test lists, seven classes, three packages, whatever—seems more and more onerous as well as less and less easy to justify. Excellent! That’s a clear sign that they’re leaving Stage 1 and entering Stage 2. They probably feel ready to experiment with not making all those plans in such detail before they start writing code. They probably feel ready to turn their attention away from “TDD as a way to reduce defects” towards “TDD as a way to guide the design to evolve over time”.

I’m here to tell you that this is normal. Yes, I’ve chosen the word “normal” intentionally here. I mean to say that it’s ordinary, to be expected, common, sensible, and a typical part of the learning process. It happens often and feeling uncertain about it also happens often. If there’s anything “wrong” with you, it’s not this.

There’s a reason I refer to these as “Stage 1” and “Stage 2”: I’ve seen this pattern so often that it has become a model I use to help people expect, confront, and get past the frustration that leads to absurd-looking-but-perfectly-reasonable reactions such as “TDD goes against agility!” Not everyone goes through the stages exactly like this, but I’ve seen it enough that I have found it helpful to describe the pattern. If you want to know more about the model and how it might help you, watch this video about how you’ll probably learn TDD. Warning: it’s a little over 20 minutes. (You can watch it at 2x speed if you need to. I won’t tell anyone.)

?Redo From Start

So now let’s try to understand this statement: “TDD goes against agility. It forces you to have a plan before coding. The purpose of testing is validating that something behaves as expected, so using tests as the driver reduces flexibility.”

I consider the first sentence the thesis, so I’ll just set it to the side. I don’t agree with it, even though I can understand why someone might form that impression. Everything I’ve written here so far, I hope, makes that clear.

“TDD forces you to have a plan before coding.” It does, but different people need different plans before writing code. Someone firmly in Stage 1 needs a more-detailed plan to feel comfortable starting. Someone in Stage 2 is actively experimenting with this question in mind: How much do I need to change my mind while I’m writing new code? How do I refactor away from choices that don’t work out? They gradually become more comfortable with refactoring incrementally over rewriting in larger batches, which helps them feel more comfortable changing their mind. Eventually they realize that if they can change their mind more freely, then maybe they don’t need to agonize so much over the details in their plan! Someone in Stage 3 is actively pushing this envelope, trying to find their own limits and the limits of the technique: what’s the smallest test that could possibly fail next? and what’s the least production code that could possibly pass? What are the fewest commitments I can make before starting? What happens if I try to discover everything as I go? How detailed a plan do I truly need before I write the first test?

Again… all normal, typical, common, sensible, reasonable.

A breakthrough of sorts happens when the programmer starts to notice that they can start writing production code with merely a single failing test at any scope: unit, integrated, integration, system, end-to-end, micro, whatever!

What do these programmers have that others don’t? They trust their refactoring skill enough to feel confident that they’ll figure out the details when they absolutely need to. They trust their habits in reading code, detecting design risks, assessing their impact, and dealing with them before they spiral out of control. They trust the intuition they’ve built through pragmatic deliberate practice.

Someone with that trust in themselves and their habits can confidently start writing code with a vague “plan” that would seem laughably rash to someone who hasn’t built that trust yet. Everyone gets there in their own time. Or not. You don’t need to practise TDD to write profitable code that you feel comfortable changing.

It’s all permitted. It’s fine.

How Do I Get There From Here?

I encourage programmers leaving Stage 1 to push themselves—firmly but gently—to allow themselves to write their first test as code before they figure out a detailed design plan. I encourage them to write down everything that comes immediately to their mind, because that settles the mind, then pick a reasonable place to start and just write a test that they expect to fail. Here are some things that are likely to happen to them:

They’ll regret some choices, but then they practise refactoring away from them.
They’ll notice that they discovered a surprising idea along the way, so then they feel good about not having committed up front (or anchored themselves) to a particular choice.
They’ll notice that the picture becomes clearer as they make more tests pass, and that seems oddly satisfying and even encouraging. It happens a little at first, but it feels easier with repetition.
They’ll design themselves into a corner, but then instead of throwing it away and starting again, they practise refactoring away from the corner. It seems to happen less often over time and they feel less bad about it when it still happens.
They might discover that they got the overall boundary or scope of their design wrong: they missed something or they started rebuilding something that already exists. When they adjust their sights, they start again with increased confidence. “Now I get it!” After a few instances of this, they might begin to feel that this kind of mistake is merely annoying, not disastrous. They recover relatively quickly. It feels odd, but not bad.

There’s more, but that’s enough for now.

What’s happening here? The hypothetical programmer experiencing these things is leaving behind the need to design so much up front and embracing both the power and uncertainty of Evolutionary Design. They are experiencing what it feels like to Program with Options: to defer commitment, to reduce the cost of recovering from mistakes, to reduce the cost of changing their mind (even when they’re not recovering from a mistake).

And is this better than what they did before? Maybe. Often. They need to decide for themselves. What matters right here for me today is to help a frustrated programmer see their feelings differently, in case it helps them to know that their current judgments about TDD being “against agility” might be signaling something different than what they’ve understood so far. They’ve made sense of their experience one way and I’m asking them to reconsider. They might still decide that TDD “isn’t for them”, but if this helps them give TDD another look, I’ve done my job: I’ve helped them see more options available to them, not fewer, in the hopes that one of those options becomes a pleasant surprise for them.

What about the claim that “The purpose of testing is validating that something behaves as expected, so using tests as the driver reduces flexibility.”? That sounds like textbook Stage 1 thinking to me. That makes me feel even more confident in what I’ve written here. If we changed “The purpose of testing is” to “I use testing for”, then the resulting opinion would seem perfectly sensible to me. Someone focusing on tests as a way to reduce defects, to increase confidence in the runtime behavior of their code, would usually think of each test as yet another potentially-harmful constraint on their design choices. That would obviously reduce flexibility and clearly go against agility. If they practised refactoring more, then they might see that differently. If they wanted to practise refactoring more, I’d be happy to help them learn.

I do have this training course and this mentoring and support group, but if they’d prefer to save their money, they can find what they need on the web and learn what they need by practising.

Some people would call this “unprofessional”. I don’t. It’s a whole thing. Please don’t use “unprofessional” as code for “irresponsible” or “incapable” or “uncaring” or whatever. If you mean those things, then please just say those things. We already suffer from implied value judgments enough in a life; let’s not volunteer to pile on.↩︎