Unit Tests and Asserts in Legacy Code
Thursday, March 22nd, 2007Because I’m currently learning about unit testing, TDD, and other agile practices, I admit that I recently considered the possibility of doing some greenfield development. While a part of me would love the experience, the challenge of using these techniques in legacy code to improve the product is very motivating to me. Along those lines, I recently followed an interesting debate about the relationship between unit tests and asserts in production code on an internal discussion list. Microsoft is big enough that you hear from people using all types of technologies and at lots of different stages in the transition to unit testing and TDD.
Assertions
Years ago, the state of the art practices for verifying code included a liberal dose of assertions scattered through the code that would run in a debug build. Generally, an assertion would notify the user and abort the program. When an assertion failed, you could debug to find out what was wrong. Generally the assertions are closer to the problem in the code than any other bad behavior you might see if the assertion hadn’t fired. Often assertions were used to verify contracts along an interface. Unfortunately, in some cases assertions were used in place of proper error handling. At other times assertions were used to verify global state assumptions. Assertions generally lead to two sets of binaries for your program - an official set and a “debug” set that developers and testers can use to find bugs.
Good Guy, Bad Guy
Assertions can often be categorized as either error handling or as there-is-no-way-this-could-happen. Assertions that are used for handling errors are bad. Instead errors should be handled either via a proper return value, or via exceptions. I won’t get into the debate here about which should be used. Assertions that fall into the there-is-no-way-this-could-happen category are probably better handled via code correctness tools that analyze code paths to verify certain properties of the code, though there is probably little harm in using these assertions.
Often assertions were used to do in-place unit testing. External unit testing is a natural next step from such assertions. Before discussing unit tests, however, it’s helpful to note that these assertions were a good thing. They tested the code every time it was run (at least in the debug version) and was often easy to understand with the surrounding code as context.
Unit Tests
Unit tests are a different way of validating code correctness from within the code. Instead of creating two sets of binaries, you essentially create two sets of source code: one that is the program, and one that verifies the program (Note: two sets of binaries may still be created). Unit tests, if written well, can verify the code in ways that are not easy to do with asserts alone. Unit tests are run regularly while writing code, while asserts fire only if a given code path is exercised in some way. Unit tests can specifically test code paths that might not be exercised normally. Unit tests can be written with the code to shape the code itself, if you are doing TDD.
Unit tests can do a lot more … Oh wait, like I said, I’m still new to all this agile stuff. So it was interesting to listen to the agile believers argue against any assertions while others made the case that assertions were useful in some situations. I must admit that the idea of having no assertions at all was shocking at first. But it immediately became appealing as an ideal once I understood how clean code without assertions might be. However, in a large legacy code base with lots of asserts you’re not going to escape assertions easily. And honestly, after some thought and experience, I believe that the two methods are complementary.
Asserts in existing legacy code are an excellent starting point when trying to get the code under test. Using the techniques in Working Effectively With Legacy Code, write tests for your code that exercise it well. Initially, if you’ve got a good set of asserts, positive unit tests can be written that don’t even verify much, as long as your framework interprets asserts as unit test failures. Of course you’ll want to write tests that cover more than just that, but it is a start and will provide some level of safety when refactoring. If your unit test does not interpret asserts as unit test failures you can use a seam (link seam, preprocesser seam) to redefine assert in the binary you compile for unit testing.
Do unit tests make assertions redundant?
In writing new code within the legacy code base I definitely find myself using less asserts because I am writing unit tests as I go. However, certain things are much easier to test with a well placed assert in the code, rather than the unit test. In talking with a colleague the other day I was reminded that making code unit-testable can make it more complex than it has to be. Because of this, I believe that assertions in code can have the beneficial effect of simplifying the code you write by making it somewhat easier to write unit tests around it. This can lead to better factored code that doesn’t make it so obvious that it’s been engineered in order to accomodate unit testing.
Resources
_ASSERT macro , assert , Java’s guide to Programming With Assertions