When details leak into our scenarios (any tests), they become brittle. If you have brittle tests, then you probably have leaky abstractions in tests and/or in production code. It’s so easy to let abstractions leak. The best of us do it. We often don’t notice that we’re doing it, so we need to be on guard for it.

It’s not always easy to plug the leak. We might not understand the context enough to do it well. When we don’t understand the context enough, then a suitable abstraction doesn’t come easily, and it can feel artificial to try. We don’t want this to become dogmatic. Let’s remember why we raise the level of abstraction: to allow us to change the details that we need to change when we need to change them. This is “flexibility”.

Let me take an example from the Cucumber Java book.

I’m not disparaging the authors of the book. Remember: the best of us do it and we often don’t notice that we’re doing it. Moreover, they wrote this as an example in a book, and often authors include intentionally problematic examples in order to explain how to improve it. (Later in the book, they indeed did something quite similar to what I’m about to do here.)

This is a scenario from a banking system.

Scenario: Change PIN successfully

  • Given I have been issued a new card
  • And I insert the card, entering the correct PIN
  • When I choose “Change PIN” from the menu
  • And I change the PIN to 9876
  • Then the system should remember my PIN is now 9876

Whenever I write an example (scenario, code, whatever), I want to hide as much detail as possible. This makes it easier to grasp the purpose of the example. This makes it easier to understand what the example signifies. This makes it easier to see what variations of the example we should consider. In particular, I want to hide irrelevant details, meaning details that we provide that don’t factor in to what we check. We provide some irrelevant details because it feels natural to do it. We provide some irrelevant details because the system design forces us to do it. We provide some irrelevant details because we don’t yet realise that we don’t need them. The checks/assertions need details, otherwise they don’t check anything. I start here and link the details in the checks to details in the setup and action. I get rid of all the other details.

In the scenario in question:

  • “9876” in the assertion matches “9876” in the action.
  • “my PIN” in the assertion matches “my PIN” in the action.
  • “…is now…” in the assertion matches (conceptually) “…I change…” in the action.

I don’t need any more detail to run the scenario. I call the remaining details irrelevant. “Accidents of the current design of the system”, perhaps.

There is no such thing as a harmless irrelevant detail. Irrelevant details have latent cost. They represent risks, if not yet problems. We have to manage them like any other risk. Ruthlessly driving them out is one risk management strategy, and not necessarily the best one at all times.

I find these details irrelevant:

  • “I have been issued a new card”
  • “I insert the card”
  • “…entering the correct PIN”
  • I choose “Change PIN” from the menu

I notice a few ways that irrelevant details leak into our designs:

  1. Everyone does it this way (now), so there’s no benefit to hiding the details.
  2. Humans fundamentally need to bounce back and forth between abstract and concrete thinking in order to solve problems, so we find it difficult to hide the details.
  3. This isn’t likely to change any time soon, so there’s no benefit for hiding the details.

Examples from the Example

The detail “I have been issued a new card” probably motivated writing the scenario, but that doesn’t make it part of the steps of the scenario. (I’d document the motivation, however, which Cucumber/Gherkin directly supports.) When my bank issues a new card to me, it gives me a temporary PIN and I have to change it before I can perform any other transactions on that card. This makes sense to anyone in the domain, and the ubiquity of this convention explains why this detail has leaked into the example. Since so many of us have experienced this convention (and that’s all it is—a convention, or a common safeguard, and not necessarily the best way to do it), we assume it without thinking about whether we need it, so we put it in the example. I notice, however, that I could remove “I have been issued a new card” from the scenario, and it loses absolutely none of its meaning. Although “Joe gets a new card” provides a natural trigger for “Joe changes his PIN”, Joe could change his PIN any time he wants, and so his doing so in response to receiving a new card has no bearing on this example. It does, however, remind us to write another scenario, which I put in the inbox: “Force customer to change PIN when using a card for the first time.” We really have to watch out for this kind of irrelevant detail, because we find it so natural to mention.

The detail “I choose ‘Change PIN’ from the menu” represents another common way that examples leak details: in spite of our best efforts, UI details creep in. I have watched projects kill themselves for over a decade by letting these details leak all through their tests. The day that they implement “Login”, everyone cries. When a software designer needs to change an HTML element ID, 184 examples fail and it wastes hours of hand-wringing about whether to refactor, followed by hours of fixing the problem. Worse, the system shows signs of viscosity: it actively encourages software designers not to refactor the system, because the tests know too many details of the current design. It encourages people to do the wrong thing. Stop it! What does “I choose ‘Change PIN’ from the menu” tell us that “I change the PIN to 9876” doesn’t? I see nothing. Even if we eliminate “I choose ‘Change PIN’ from the menu”, the example retains all its meaning. This corresponds to the design principle separate triggering an action from performing it, so that we can perform the same action the same way in response to as many different triggers as we need. In this case, we could use the same back-office module to handle changing PIN by ATM, by phone banking, by web browser… hell, even by Chris sending a letter by post to the bank branch. (Chris is 92. Let him have his fun.) We really have to watch out for this kind of irrelevant detail, because in spite of ourselves, we need to envision a system through its current UI; it simply helps us think about how the system works. I don’t want you to stop thinking, but you don’t have to put all that detail into the scenario.

You can start making a similar argument for “I insert the card” and “…entering the correct PIN”, which both represent UI details, but when I try to eliminate these details, I think the scenario loses something. In order to hide the details, I ask “Why do we care?” (Asking and answering “Why do we care?” is the act of abstracting!) So, “why do we care that the customer inserts the card, entering the correct PIN?” The ATM needs a way to trust that it should allow whoever’s standing at the ATM to change the PIN. The customer uses the card and PIN to authorise the bank to change the PIN associated with that card. For the purposes of this scenario, the authorisation matters, but not how the customer does it. A retinal scan would do equally well (and probably will, sooner than I’d like). So we can replace these two details with a corresponding, more abstract, condition. We raise the level of abstraction, and in the process, protect against changes to the UI. You might not think that we’ll ever get away from using PINs and bank cards, but you can already use your fingerprint to unlock your phone. One big things that stops us from changing the way we authorise transactions with an ATM is the sheer mountain of crap that depends specifically (and unnecessarily!!) on the fact that ATMs happen to authorise customers by card and PIN! By flippantly saying, “Oh, it’ll never change”, you’re making the problem worse. Stop it.

Refactoring the Scenario

Let’s replace the irrelevant details with relevant ones.

Scenario: Change PIN successfully

  • Given I have authorised the ATM to make changes to my bank card
  • When I change my PIN to 9876
  • Then the system should remember my PIN is now 9876

I’m not a big fan of “…the system should remember…”, because it’s another detail, but I don’t prefer the alternative ways to check the outcome of this scenario that leap to mind, such as authorising a transaction using the new PIN. If someone comes up with a better way to check the outcome of this scenario, then I’ll happily change the “Then” clause. Giddily.

References

Seb Rose and others. The Cucumber Java Book. A wonderful guide to help you get Cucumber up and running on the JVM, even if you write your system in Clojure.

Robert Martin. Agile Software Development: Principles, Patterns, and Practices Viscosity is but one of the smells that Bob highlighted in this classic book.

J. B. Rainsberger, “What Your Tests Don’t Need To Know Will Hurt You”. Another example of how irrelevant details make testing difficult and reveal production code design smells.

Need Help?

If you’re not using techniques like these, then you’re missing out on most of the value of agile software development. Really! One CEO told me that learning techniques like these saved his company multiples of $100,000 in investment in software development. This is no joke. (If you don’t care about a few hundred thousand dollars either way, that’s fine, but it’s still a lot of money to me.)

Don’t waste your agile transition building the wrong product better. Learn more about Value-Driven Product Development training.

If you’re not ready for training, but you’d like me as your trusted adviser, get fractional consulting, coaching, and mentoring. It costs less than you’d think!