Regression Correspondence Harness¶
Software or tool — instantiates Correspondence Violation Detection and Theory Refinement
Automatically re-runs the full set of already-passing correspondence cases on every change and fails the build if a refinement regresses one.
A Regression Correspondence Harness is an automated tool that, on every change to a formulation, re-runs the full set of already-passing correspondence cases and fails the build if any previously-valid case now diverges. Its defining quality is that it guards the rear: its job is to preserve cases that already worked, not to discover new violations. A green run means "this refinement broke nothing we knew to be right"; a red run means "your change regressed a canonical case." Where a limiting-case suite pushes forward into new limits looking for fresh failures, the harness stands behind every commit as a ratchet, enforcing the archetype's hardest-won invariant — that a repair must not make the theory locally better and globally less reliable.
Example¶
A numerical linear-algebra library ships a new blocked eigensolver meant to be faster on large matrices while still matching the reference solver's eigenvalues on a canonical suite — Hilbert matrices, known-spectrum test matrices, symmetric and Hermitian cases — to within 1e-10. The Regression Correspondence Harness runs in continuous integration: every commit re-executes the entire canonical set against stored expected values. A refactor that improves large-matrix throughput but shifts a 12×12 symmetric case's smallest eigenvalue past the 1e-10 tolerance turns the build red before the change can merge. The harness caught, automatically and immediately, a "fix" that would have traded a real correctness guarantee for speed — the exact silent-regression the archetype warns about, blocked at the gate.
How it works¶
- Hold the canonical corpus. The harness stores the set of known-valid correspondence cases with their expected results and per-case tolerances.
- Re-run on every change. Each commit or refinement triggers a full re-execution of that corpus, not a spot check.
- Compare and gate. Results are compared to stored expectations; any case now outside tolerance fails the build and blocks the merge.
- Report the regressions, not interpret them. The harness names which cases regressed and by how much; classifying and fixing them is left to other mechanisms.
Tuning parameters¶
- Corpus coverage — how much of the known-valid space the suite holds. Broader coverage catches more regressions but lengthens every run.
- Per-case tolerance — how tight each expected-result comparison is. Tight tolerances catch subtle drift but flag benign numerical noise; loose ones are quiet but let real regressions through.
- Run cadence — every commit versus nightly. Per-commit gating is safest but slows the loop; nightly is cheaper but lets a regression live for a day.
- Block versus warn — whether a failure hard-blocks the merge or only warns. Blocking enforces the invariant; warning keeps velocity at the cost of discipline.
When it helps, and when it misleads¶
Its strength is that it makes the regression-guarded-update invariant automatic and non-negotiable: refinements can no longer quietly break old cases, because the corpus is re-checked before every change lands, which is the structural cure for ad hoc patch accumulation.
Its failure mode is brittleness — the characterization-test[n1] trap: a corpus so large or so tightly-toleranced that it flags harmless floating-point jitter turns red so often that engineers learn to ignore or bypass it, and the guard rots into noise. The mirror-image misuse is tuning a refinement specifically to make the harness pass — overfitting to the frozen corpus rather than genuinely preserving correspondence. The guarding discipline is to tie tolerances to meaningful thresholds rather than exact bit-reproduction, quarantine and fix flaky cases promptly, and periodically add fresh cases so the corpus tracks real correspondence rather than a stale snapshot.
How it implements the components¶
The harness fills the preserve-known-cases part of the machinery:
regression_guarded_update— it is the automated regression guard: re-running the known-valid corpus on every change and failing the build on any regression.canonical_case_library— it holds and executes the corpus of known-valid correspondence cases, operating that library as a runnable guard rather than a static catalog.
It does not implement violation_detection_protocol or overlap_or_limit_regime_set — finding new violations and defining fresh limit regimes to probe is the job of Limiting-Case Test Suite. The suite probes the frontier for new reduction failures; the harness re-runs the known-good corpus to catch regressions a change introduces.
Related¶
- Instantiates: Correspondence Violation Detection and Theory Refinement — the harness is the automated guard that keeps refinements from breaking known-valid correspondence.
- Consumes: Overlap-Regime Benchmark Table — the canonical cases and tolerances it re-runs on every change.
- Sibling mechanisms: Anomaly Triage Board · Assumption Audit Worksheet · Boundary Condition Memo · Expert Adjudication Panel · Failure Mode Annotation Card · Limiting-Case Test Suite · Overlap-Regime Benchmark Table · Parameter Sweep Matrix · Residual Divergence Map
Editorial Notes¶
Form Classification¶
Form family: Experiment, Test & Rehearsal
Rationale: Regression Correspondence Harness operates as an active test, trial, simulation, drill, or rehearsal that generates evidence through a deliberate attempt or perturbation because it automatically re-runs the full set of already-passing correspondence cases on every change and fails the build if a refinement regresses one.
Independent corroboration: The frozen evidence defines Regression Correspondence Harness as 'Automatically re-runs the full set of already-passing correspondence cases on every change and fails the build if a refinement regresses one', so its operative form is Experiment, Test & Rehearsal.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Automated regression harnesses failing a build are canonical software testing infrastructure.
Review resolution: Both blind reviewers agree that computer_science is the primary origin. Explicit reconciliation of alternate origin disagreement, origin mode disagreement adopts reviewer_a's classification because automated regression harnesses failing a build are canonical software testing infrastructure. The resulting lineage records alternates=none, origin_mode=single_lineage, and domain_reach=specialized; these describe formative provenance separately from later applicability.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] A characterization test (a term from Michael Feathers' work on legacy code) pins down a system's current behavior so that later changes cannot alter it unnoticed — the software analogue of a regression guard. Its known hazard is brittleness: tests that assert too exactly flag benign changes and train people to ignore them, which is why tolerances must track meaningful, not incidental, differences. ↩