Skip to content

Characterization Test Harness

A test harness — instantiates Behavior-Preserving Refactoring

Pins down what legacy code currently does — bugs and all — by capturing its outputs on a batch of inputs as the golden baseline, so any later change that alters that behavior shows up immediately.

You cannot safely restructure code whose behavior you cannot pin down, and much legacy code has no tests and no one who fully understands it. Characterization Test Harness manufactures the missing safety net. It runs the current code over a batch of inputs, captures whatever it produces as the expected output, and freezes those captures as tests — whether or not the outputs are correct. Its defining stance is descriptive, not prescriptive: it asserts what the system does, never what it should do, because the goal is to lock present behavior in place so that any change during refactoring shows up as a diff. This is the opposite of a specification test, and it is precisely what makes it usable on code too poorly understood to spec. Where a Behavioral Diff Gate enforces equivalence in the pipeline, the harness is what creates the reference equivalence is measured against.

Example

A team must modify a 1990s insurance premium calculator — thousands of lines, no tests, several people gone who wrote it, and a couple of outputs everyone quietly suspects are wrong. Before touching it, they wrap it: they run roughly 5,000 representative historical policy records through the calculator and snapshot each computed premium as the expected value — including the suspect ones. The code is now "characterized." When they begin restructuring, any deviation from those 5,000 captured outputs lights up red, so they can pull the tangle apart fearlessly. And because the net is descriptive, the two premiums they believe are wrong are handled separately and deliberately: each is changed as an explicit, reviewed behavior change that re-blesses its snapshot — never quietly, and never mixed into the refactoring.

How it works

The harness selects a set of inputs (recorded production traffic, a representative sample, or generated cases), runs them through the unchanged current system, and records the outputs verbatim as the reference. Those input/output pairs are frozen into assertions: run the code again after any edit and a mismatch fails. It makes no correctness judgment — a known bug is captured as faithfully as correct behavior — because the property it protects is unchanged, not right. What distinguishes it from an ordinary test suite is exactly this: it is authored by observing the system, not by encoding a specification of it.

Tuning parameters

  • Input coverage — how many and which cases are captured. Behavior on inputs you never fed in is unprotected, so coverage sets the true size of the net; recorded production traffic covers the real distribution better than hand-picked cases.
  • Capture granularity — final outputs only, versus intermediate states and side effects too. Deeper capture catches subtler regressions but is more work and more brittle.
  • Snapshot fidelity — exact bytes versus normalized output. Normalizing away timestamps, ordering, and volatile IDs prevents floods of false diffs; over-normalizing hides real ones.
  • Re-bless policy — how an intended behavior change updates its snapshots. A deliberate, reviewed re-bless step is what keeps a real regression from hiding behind a routine snapshot update.
  • Volume — how large the captured corpus is. More cases mean a stronger net but slower runs and heavier review when snapshots legitimately change.

When it helps, and when it misleads

Its strength is standing up a safety net fast where none existed, which is what makes untested legacy refactorable at all: it converts "I'm afraid to touch this" into "I can restructure and the net will catch me," without first having to understand or specify the code.[1]

It misleads in two matched ways. Because it captures current behavior as "expected," it locks in existing bugs as correct, so it must be paired with a deliberate way to distinguish an intended fix from a regression — otherwise a genuine correction reads as a failure and a real regression can be re-blessed away by reflex. And "all green" means only "green on the cases I captured": behavior on un-captured inputs is entirely unprotected, which lends the suite false confidence proportional to the gaps in its coverage. The classic misuse is treating characterization tests as a correctness specification. The discipline is to capture broadly, normalize noise deliberately, and route every intended change through an explicit re-bless review.

How it implements the components

  • legacy_behavior_baseline — it creates the baseline by recording the current system's actual outputs as the reference of record, rather than assuming a documented one exists.
  • golden_case_library — the captured input/output snapshots become the corpus of golden cases every downstream check compares against.
  • behavior_preservation_test — each captured case is turned into a runnable assertion, so "did behavior change?" becomes a concrete test rather than a manual eyeball.

The harness produces the corpus but does not itself *gate a merge on it (that is Behavioral Diff Gate) or compare a live parallel system against real production traffic (that is Shadow Run or Parallel Run); those consume the baseline it builds.*

Notes

The harness is usually the first mechanism stood up on legacy code: the cleanup sprint, the small-step workflow, and the diff gate all lean on the corpus it produces, so its input coverage silently caps how safe every one of them can be. A thin characterization net makes everything downstream feel safer than it is — the honest question about any green result is always "green on which cases?"

References

[1] Michael Feathers' characterization test (from Working Effectively with Legacy Code): a test written not to specify what code should do but to document what it currently does, so it can be changed safely. It is closely related to golden-master / approval testing, where a captured "approved" output becomes the reference for later comparison. ↩