01 · Definition
What is a characterization test?
A characterization test documents the current behaviour of a piece of code, including its bugs.
- 01
The assertion matches what the code does now. A normal test starts from a specification; this one starts from the running system.
- 02
Michael Feathers introduced the term in Working Effectively with Legacy Code (2004). His working definition of legacy code is code without tests, and his answer to it is to add tests that describe reality before anyone refactors. Without them nobody can show a change kept behaviour intact.
02 · Comparison
How is a characterization test different from a unit test?
| Unit test (new code) | Characterization test (legacy code) | |
|---|---|---|
| Expected value comes from | The requirement | Running the code today |
| Answers | Is it correct? | Did it change? |
| A failure means | The code is wrong | Behaviour moved; decide if that was intended |
| Written | Before or with the code | Before the first modification |
| Lifetime | Permanent | Often temporary, until proper tests replace it |
03 · How to
How do you write characterization tests?
Work on one unit at a time, with the part you are about to change first.
Get the code into a test harness
Instantiate the class or call the function from a test. If it will not construct because of hard-wired dependencies, find a seam, a place where behaviour can be altered without editing the code at that place, and use it.
Call it with realistic inputs
Use real or recorded input values, including the awkward ones: nulls, empty collections, boundary dates, odd currencies.
Assert something you know is wrong
Write an assertion that you expect to fail. The failure message prints the actual value.
Paste the actual value in as expected
Replace the wrong value with what the code returned. The test now passes and records current behaviour.
Repeat until the area you will touch is covered
Aim at the paths your change can reach, not at coverage of the whole system.
04 · Example
What does a characterization test look like in Java?
Take an illustrative legacy method InvoiceCalculator.total(Order order) that nobody fully understands.
- 01
In a JUnit 5 test you build an order with three lines and a discount code, call
total(order), and assertassertEquals("0.00", result.toPlainString()). It fails and the report says the actual result was127.35. - 02
You change the expected value to
127.35and the test passes. You repeat with an empty order, a null discount code and a refund line. You now have a net: if a later refactor makestotalreturn127.34, a test fails and you must decide whether that is a fix or a regression. This example is illustrative; the numbers are not from a real system. - 03
For output too large to paste, such as a generated report, use approval testing: save the full output to a file the first time, and compare each later run against that file.
05 · Limits
When should you stop, and what do these tests not tell you?
Stop when the code you plan to change is covered along the paths that matter.
- 01
Characterization tests do not tell you the behaviour is correct, only that it is unchanged, so a recorded bug stays a bug until someone decides to fix it.
- 02
They can also be brittle where output includes timestamps, random ids or ordering that is not guaranteed. Pin or strip those values before you assert. Replace characterization tests with intent-revealing unit tests as you learn what the code is for.
06 · In practice
Where do characterization tests fit in a modernization?
They are the first step of any refactor or upgrade, because they define what "same" means.
- 01
Our legacy application modernization work starts with tests around the code before AI-assisted changes are made, and AceMQ engineers review every change.
- 02
If the test gap is large, it is also a measure of technical debt and the first thing the modernization assessment sizes. For the wider option set, see legacy system modernization approaches.