On June 3, 2020, TechTalk hosted a meetup at which I spoke about managing the various kinds of uncertainty that we routinely encounter on projects that involve legacy code. I presented a handful of ideas for how we might improve our practices related to testing, design, planning, and collaboration. These ideas and practices help us with general software project work, but they help us even more when working with legacy code, since legacy code tends to add significant uncertainty and pressure to every bit of our work. Fortunately, we can build our skill while doing everyday work away from legacy code, then exploit that extra skill when we work with legacy code. Here are some questions that came up during this session and some answers to those questions.

You’ll find the remaining articles in this series here as they are released.

Some Questions About Test Frameworks

If the code base is too old even for any available test frameworks, how you handle it?

Testing does not need frameworks. Testing never needed frameworks. You can always start by just writing tests and refactoring them. If you do this long enough, you will extract a testing framework. If you’ve never tried it, then I recommend it! Kent Beck’s Test-Driven Development: By Example included this exercise.

Every test framework began life with if (!condition) { throw Error("Test failure.") }. If you can write this, then you can build a testing framework; if this suffices, then you don’t need a testing framework. Start there!

If you can execute one part of the system in isolation from the rest, then you can write unit tests. In the early days of web browsers, we could only execute Javascript in the browser, because even so, we could (and did!) write unit tests without frameworks. We merely had to run those tests in a browser window. Eventually, someone decided to run Javascript outside the browser, which made it easier to write microtests for Javascript code. This made it easier to write tests, but we were writing tests long before NodeJS existed.

If you can invoke a function (or procedure or division or block of code) and you can signal failure (such as by raising an error), then you can write tests without waiting for someone else to build a framework.

In addition, you don’t need to write your tests in the same language or environment as the running system. Golden Master technique helps us write tests for any system that offers a text-based interface. Any protocol could help us here: for example, think of HTTP as “merely” a special way of formatting requests and responses with text. If you have (or can easily add) this kind of interface or protocol to your system, then you can write tests in any language that might offer a convenient test framework. Use Python to test your COBOL code. Why not?

Finally, not all testing must be automated. As I wrote earlier, programmers have a strong habit of forgetting alternatives to techniques that they’ve found helpful. If you don’t know how to automate your tests easily, then don’t automate them yet. Instead, make them repeatable and document them. One day, someone will have a good idea about how to automate them.

You may have to write your own test framework but it can prove a daunting task.

In addition to what I wrote in the previous answer, I encourage you to follow the general advice about building any software with a Lightweight (Agile, Lean, …) approach: build the first feature that you need, then start using it, then add more features one at a time. You don’t need to build a fully-featured testing framework before you start to benefit from it. Start with if (!assertion) throw Error() and then use it! The testing framework SUnit was built incrementally. All the testing frameworks you know began from there. You can do it, too. Merely start, then take one step at a time.

There are testing frameworks for COBOL and NATURAL. What could be older?

Indeed, the “framework” portion of testing relates to identifying tests, collecting test results, and reporting them in a unified way, as well as adding standard mechanisms for “set up” and “tear down”. We don’t need those things to start writing tests, although eventually we will probably want to have them. Simply start writing tests, then remove duplication in any way that your programing language allows. I don’t know what might be older than COBOL or NATURAL.

—