Equivalence Test Suite¶

Behavioral verification suite — instantiates Equivalence Class Consolidation

A battery of comparison tests that runs variants through the consolidation rule and checks they still produce the same required output, flagging where they diverge.

An Equivalence Test Suite is the mechanism that falsifies a consolidation before you trust it. Everything else in the archetype asserts that certain variants are the same for a purpose; this suite goes and checks whether that claim actually holds by running the variants — or the pre- and post-consolidation paths — against the same inputs and comparing the results that are supposed to be identical. Its distinguishing job is behavioral: it does not care whether two things look alike, only whether they produce the same required output, eligibility, or decision under the rule. Where variants pass, the equivalence is evidenced rather than assumed; where they diverge, the suite is what turns a hidden false-merge into a visible, located failure.

Example¶

A benefits agency is retiring two legacy eligibility engines — one for each of two merged programs — in favor of a single consolidated rules engine, on the claim that the programs are equivalent for determination purposes. Before the switch, the team builds an equivalence test suite. It replays tens of thousands of historical cases plus generated edge cases through both the old engines and the new one, and asserts that the determination (eligible / not) and the benefit amount match for each. Most cases agree. A cluster of disagreements turns out to be a genuine bug in the new engine's income rounding; another small cluster is a legitimate difference the law requires for one program and must be preserved as an exception, not "fixed." The agency reports not "the engines are equivalent" but "equivalent across the case space we covered, with three declared exceptions and thin coverage on self-employment income" — and holds the cutover until that gap is tested.

How it works¶

Its distinguishing traits are that it compares outputs and treats coverage as part of the result:

It pins down the required-equal outputs first — the specific determinations, amounts, or behaviors that must agree — and asserts only on those, ignoring incidental differences.
It runs a corpus of inputs (real cases, generated cases, adversarial edges) through each variant and the consolidated path, then compares. Divergences are localized, not just counted.
It separates two kinds of divergence: defects (the consolidation broke something) from exceptions (a difference the rule intends to keep), and reports coverage as the boundary of what the pass actually proves.

Tuning parameters¶

Assertion strictness — exact-match on outputs versus a tolerance band. Tight assertions catch subtle breaks but flag harmless noise; loose ones risk waving through real differences.
Input corpus — replayed real cases, synthetically generated cases, or deliberately adversarial edges. The mix decides which failures the suite is even capable of finding.
Oracle choice — differential comparison (run both paths, compare) versus a fixed golden master. Differential needs no known-correct answer; a golden master anchors to one but can enshrine an old mistake.
Coverage target — how much of the input space must be exercised before a pass counts. Higher targets raise confidence and cost.
Exception handling — whether declared exceptions are excluded from the pass criterion or tracked as first-class expected differences.

When it helps, and when it misleads¶

Its strength is converting "these should be equivalent" into evidence you can act on: it catches behavior-breaking merges before they ship, distinguishes a bug from an intended difference, and pins each divergence to a specific input so it can be fixed or blessed.

Its fundamental limit is that a passing suite proves equivalence only over the inputs it tried — the untested region of the input space is exactly where a false equivalence hides, so "all green" is a statement about coverage, not about the universe.^[1] The oracle problem sharpens this: when there is no independent known-correct answer, the suite can confirm that two paths agree but not that either is right — two implementations can be equivalently wrong. The misuse to watch is running the suite after a merge has already shipped to manufacture reassurance, or loosening tolerances until it goes green. The discipline is to report coverage and confidence honestly rather than a bare pass, seed the corpus with adversarial edges, and keep every exception explicit so it cannot masquerade as a passing case.

How it implements the components¶

relevant_property_set — the suite defines and asserts on exactly the outputs that must match for the variants to count as equivalent; naming those is its first act.
exception_rule — divergences that the rule intends to preserve are captured as declared exceptions rather than failures, keeping false equivalence from being papered over.
confidence_level — coverage of the input space is surfaced as an explicit confidence bound on whether the equivalence really holds beyond the tested cases.

It does not define which records refer to the same entity — that inference is the Identity Resolution Model — nor perform the consolidation itself (Deduplication Workflow), nor author the treatment policy it checks against (Policy Equivalence Rule). It verifies; it does not decide or execute.

Instantiates: Equivalence Class Consolidation — the suite is the verification that a claimed consolidation preserves the behavior it was supposed to.
Consumes: Policy Equivalence Rule (or another stated consolidation rule) supplies the equivalence claim the suite is built to test.
Sibling mechanisms: Policy Equivalence Rule · Identity Resolution Model · Alias Resolution Table · Canonicalization Pipeline · Crosswalk Table · Deduplication Workflow · Master Record Consolidation · Synonym Merge Review · Taxonomy Merge Workshop · Unit Normalization Table

References¶

[1] In software testing the oracle problem is the difficulty of knowing the correct expected output; differential testing sidesteps it by checking that two implementations agree rather than that either is correct — which is why an equivalence suite must report coverage and cannot, on its own, certify correctness. ↩