Differential Transition Comparison¶
Comparison method — instantiates Deterministic Transition Contract
Runs the same present state through two variants — two machines, two law versions, two builds — and diffs the resulting transitions to localize exactly which uncontrolled factor makes them differ.
When a transition is non-deterministic, the hard question is not whether it drifts but what makes it drift. Differential Transition Comparison answers that by running the identical present state through two variants side by side — two hosts, two versions of the law set, two builds of the same code — and comparing the resulting transitions step by step. Its defining move is that it has no reference answer: it is not asking "did this match the correct output?" but "where, exactly, do these two runs part company, and why?" The divergence point is the evidence. Because everything is held equal except one deliberately varied axis, the first place the two transitions differ names the uncontrolled factor — a hidden read, an environment difference, a rule-version change — turning an invisible influence into a located, explained one.
Example¶
A national weather service runs the same atmospheric forecast model on two supercomputers — an old cluster being retired and a new one being commissioned. Bit-for-bit, the two must agree before the new cluster goes live, because a forecast that differs by machine cannot be trusted or audited. On the first parallel run they diverge: by forecast hour 6 the two clusters predict rainfall bands 40 km apart.
Differential comparison localizes the cause. Both clusters start from the identical initial atmospheric state and run the identical model version, so any difference is machine-attributable. The method diffs the two runs field by field, timestep by timestep, and finds the transitions are identical through timestep 300, then diverge in one convection routine at timestep 301 — where the new cluster's math library evaluates a transcendental function to a different last bit, and the model's chaotic dynamics amplify that speck into a 40 km band by hour 6. The comparison does not say which cluster is "right"; it says here is the exact hidden factor — a library-level rounding difference — that was silently steering the outcome. The team pins the math library across both clusters and the runs converge.
How it works¶
The method isolates a cause by holding everything constant but one axis:
- Vary exactly one axis. Run identical present state and inputs through two configurations that differ in a single deliberate way — the machine, the build, the rule version — so any divergence is attributable to that axis.
- Diff the transitions, not just the endpoints. Compare intermediate states step by step, so the first point of divergence is captured rather than only a different final answer.
- Read the divergence as an exposure. The location and content of the first difference names the uncontrolled factor — an unpinned dependency, a hidden state read, a version-behavior change — and adds it to the register of things that were secretly influencing the transition.[n1]
- Attribute across versions. When the varied axis is the law set itself, the divergence is tied to the specific version change, using the recorded lineage of rule versions.
Tuning parameters¶
- Varied axis — which single dimension differs between the two runs. One axis gives a clean attribution; varying two at once makes the divergence uninterpretable.
- Comparison granularity — whether states are diffed per-field-per-step or only at the end. Fine granularity pinpoints the first divergence; coarse comparison is cheap but only tells you that they differ.
- Equivalence definition — what counts as "the same" (exact bits, or within tolerance). Strict equality catches subtle drift; a tolerance suppresses benign noise but can mask a real cause.
- Pairing choice — old-vs-new, machine-A-vs-B, or version-N-vs-N+1. Each pairing answers a different question about which factor is loose.
- Divergence-first vs. exhaustive — stop at the first difference, or catalog all of them. First-difference is fastest for root cause; exhaustive mapping is better for a full compatibility audit.
When it helps, and when it misleads¶
Its strength is diagnosis: when two runs that should agree don't, it converts a vague "it's flaky" or "it behaves differently on the new box" into a named, located factor. It is the natural tool for platform migrations, cross-machine reproducibility, and validating that a refactor changed nothing observable.
It misleads when the two variants share the same hidden flaw — then they agree, the diff is clean, and the comparison certifies a determinism that both runs merely happen to violate identically. It also cannot tell you which side is correct; it only tells you where they differ, so on its own it localizes a cause without adjudicating a winner. The guarding discipline is to remember that agreement between two related variants is weak evidence — pair a same-family diff with an independent reference (a golden master) when correctness, not just consistency, is the question.
How it implements the components¶
hidden_state_exposure_register— the divergence point is the exposure: each localized difference names a previously invisible factor and records it as something the transition was secretly depending on.law_version_lineage— when the varied axis is the rule set, it attributes the behavioral change to a specific version, exercising the recorded lineage of law versions.
It does not implement determinism_acceptance_test — the pass/fail verdict against a frozen reference is Golden Master Transition Test; differential comparison only localizes where two runs diverge, it renders no correctness judgment. Nor does it implement deterministic_replay_trace — re-executing a single run from its record is Deterministic Replay Harness.
Related¶
- Instantiates: Deterministic Transition Contract — supplies the divergence-localization step the contract's verification depends on.
- Consumes: Deterministic Replay Harness can re-execute each variant so the two runs being diffed are themselves reproducible.
- Sibling mechanisms: Golden Master Transition Test · Deterministic Replay Harness · Transition Audit Log
Editorial Notes¶
Form Classification¶
Form family: Experiment, Test & Rehearsal
Rationale: Differential Transition Comparison operates as a bounded trial, probe, simulation, or rehearsal that generates evidence from performance because it runs the same present state through two variants — two machines, two law versions, two builds — and diffs the resulting transitions to localize exactly which uncontrolled factor makes them differ.
Independent corroboration: The frozen evidence defines Differential Transition Comparison as 'Runs the same present state through two variants — two machines, two law versions, two builds — and diffs the resulting transitions to localize exactly which uncontrolled factor makes them differ', so its operative form is Experiment, Test & Rehearsal.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Computer Science & Software Engineering
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: Differential software testing established same-input execution across versions or environments and localization at the first divergence point.
Review resolution: Differential software testing established same-input execution across versions or environments and localization at the first divergence point. Differential testing is the sole origin lineage; the generalized transition-focused artifact is nevertheless an encyclopedia synthesis.
Attribution caveat: The transition-level trace is a portable encyclopedia extension of differential testing rather than a separate statistical method.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] Differential testing feeds the same input to two or more independent implementations (or versions) that should agree and treats any disagreement as a bug in at least one of them — the technique behind compiler fuzzers like Csmith. It needs no oracle for "correct," only a pair that ought to match. ↩