Overlap-Regime Benchmark Table¶
Artifact — instantiates Correspondence Violation Detection and Theory Refinement
The reference table that catalogues, row by row, each case where two formulations are expected to agree — with inputs, conditions, tolerance, and observed divergence.
An Overlap-Regime Benchmark Table is the reference artifact that catalogues, one row per case, every situation where the new and old formulations are expected to agree — recording for each its inputs, conditions, the tolerance for acceptable divergence, and the observed result. Its defining quality is that it is a static map-plus-ledger of expected correspondence: it declares what should agree with what, under which conditions, and to what tolerance, and it curates that set of cases as the shared corpus of record. It does not execute reductions and it does not search a parameter space for a breakpoint — it is the written expectation and the case library that other, active mechanisms run against. A benchmark table is what makes the phrase "this is a violation" meaningful, because the row already says what agreement was expected and by how much.
Example¶
A utility migrates from a legacy billing-and-reporting system to a new one and runs both in parallel for three months; during the overlap they must produce the same figures. The Overlap-Regime Benchmark Table lists the cases that must match: total booked revenue for the month (tolerance ±0.1%), residential customer count (exact), commercial energy billed in kWh (±0.05%), delinquency dollars (±0.2%) — each row carrying the input month, the conditions (same rate schedule, same cutoff date), the tolerance, the observed value from each system, and the divergence. Most rows sit inside tolerance and are marked agreeing. Row 7, commercial kWh, comes in 0.4% apart — outside its budget — and is flagged for triage. The table is the single source of truth for what we expected to match and by how much, and the ledger where each real mismatch is registered before anyone argues about it.
How it works¶
- Enumerate expected-agreement cases. Each row names a case where correspondence should hold and why — the shared regime, the overlap period, the canonical example.
- Attach inputs, conditions, and a tolerance. Every row carries the inputs and conditions that make the comparison well-posed and the error budget that defines acceptable divergence.
- Record the observed result. The observed values and their divergence are logged against the expectation, turning the table into a live ledger, not a static wish list.
- Serve as the canonical corpus. The rows are the reference set that executable mechanisms rerun and that reviewers cite.
Tuning parameters¶
- Case coverage — how many and how representative the rows are. Broad coverage catches more failure classes but costs curation and can dilute focus on the cases that matter.
- Per-row tolerance — how tight each error budget is set. Tight budgets catch subtle divergence but raise false alarms; loose budgets are quiet but hide boundary failures.
- Canonical versus exhaustive — a small curated set of load-bearing cases versus an attempt at completeness. Curated is maintainable; exhaustive is thorough but rots.
- Tolerance basis — whether budgets are set statistically (measurement uncertainty) or by policy (what the business will accept), which changes what a flagged row means.
When it helps, and when it misleads¶
Its strength is that it forces the correspondence expectation to be explicit and commensurable before anyone cries violation, which is the single best defense against the archetype's false-alarm-from-translation failure: the row already fixed the units, the conditions, and the acceptable gap.
Its failure mode is Goodhart's law[n1]: once the table's rows become the target, a model can be tuned to pass exactly those cases while quietly breaking the ones nobody listed — the benchmark ceases to measure correspondence and starts measuring conformity to the benchmark. Aggregate rows can also mask a subgroup failure folded inside a passing total. The guarding discipline is to keep the table a regression guard, not a goal — continue hunting for new violation cases beyond it, refresh the rows, and prefer disaggregated rows over comfortable totals.
How it implements the components¶
The table fills the declare-and-curate part of the machinery:
expected_correspondence_map— each row states what should agree with what and why, which is the correspondence map made concrete.canonical_case_library— the collected rows are the curated corpus of expected-agreement cases that everything downstream reruns and cites.uncertainty_and_error_budget— the per-row tolerance is the explicit error budget that defines how much divergence still counts as agreement.
It does not implement violation_detection_protocol — actually executing the comparison runs that populate the observed column is the job of Limiting-Case Test Suite. The table is the catalog of expectations and tolerances; the suite is what runs against it.
Related¶
- Instantiates: Correspondence Violation Detection and Theory Refinement — the table is the reference layer that fixes expected correspondence so violations can be judged against it.
- Sibling mechanisms: Anomaly Triage Board · Assumption Audit Worksheet · Boundary Condition Memo · Expert Adjudication Panel · Failure Mode Annotation Card · Limiting-Case Test Suite · Parameter Sweep Matrix · Regression Correspondence Harness · Residual Divergence Map
Editorial Notes¶
Form Classification¶
Form family: Record, Log & Register
Rationale: Overlap-Regime Benchmark Table operates as a persistent ledger, log, register, or case record that preserves history and traceability because it the reference table that catalogues, row by row, each case where two formulations are expected to agree — with inputs, conditions, tolerance, and observed divergence.
Independent corroboration: The frozen evidence defines Overlap-Regime Benchmark Table as 'The reference table that catalogues, row by row, each case where two formulations are expected to agree — with inputs, conditions, tolerance, and observed divergence', so its operative form is Record, Log & Register.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Engineering & Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Engineering verification and validation established controlled reference cases, tolerances, and discrepancy ledgers for demonstrating that old and new formulations agree throughout a declared overlap regime.
Related originating lineages:
- Mathematics — Overlap-Regime Benchmark Table also draws materially on mathematics' axiomatic study of abstract structure, relations, and formal operations, which shaped this mechanism rather than merely adopting it as an application.
- Physics — Overlap-Regime Benchmark Table is most directly rooted in physics' modeling of matter, dynamics, waves, and experimentally constrained systems. The lineage fits its defining practice: The reference table that catalogues, row by row, each case where two formulations are expected to agree — with inputs, conditions, tolerance, and observed divergence.
- Statistics & Experimental Design — Cataloging expected agreement cases, tolerances, and observed divergence follows statistical benchmark and validation practice.
Review resolution: Authoritative-source research resolves the primary-origin disagreement in favor of engineering design. NASA Systems Engineering Handbook documents the formative practice or theory represented here. The retained alternate domains identify material co-development or translation, while current applicability is recorded separately as domain_reach=multi_domain; origin_mode=cross_disciplinary_synthesis describes the historical relationship among lineages.
Attribution caveat: The standing table is a synthesized regression artifact rather than a standard named statistical instrument.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; medium confidence.
Sources consulted:
Notes¶
[n1] Goodhart's law (after economist Charles Goodhart) is usually paraphrased as "when a measure becomes a target, it ceases to be a good measure." A benchmark table is exactly such a measure: the moment refinement is optimized to pass its listed cases, passing the table stops being evidence of genuine correspondence. ↩