Differential Testing for Software¶
McKeeman, W. M. (1998). Differential Testing for Software. Digital Technical Journal, 10(1), 100-107.
Cited by¶
4 citations across 4 artifacts.
Each citation links to the sentence it supports in the citing article.
Mechanisms¶
- Differential Equivalence Test
- Two implementations can share a common ancestor, a library, or the same misreading of the spec and agree on the same wrong answer — a correlated failure the test is blind to by construction
This sourceDefines differential testing by output divergence, leaving coincident outputs outside the method’s detection rule.
- Two implementations can share a common ancestor, a library, or the same misreading of the spec and agree on the same wrong answer — a correlated failure the test is blind to by construction
- Dual-Run Equivalence Test
- Its strength is that it needs no theory of why two contexts might differ — it catches coupling bugs that static reasoning misses, which is why parallel-run and shadow comparison are the gold standard before a risky cutover.
This sourceCompares equivalent implementations on the same generated inputs and treats output differences as defect evidence without first specifying an expected-output oracle.
- Its strength is that it needs no theory of why two contexts might differ — it catches coupling bugs that static reasoning misses, which is why parallel-run and shadow comparison are the gold standard before a risky cutover.
- Equivalence Test Suite
- Its fundamental limit is that a passing suite proves equivalence only over the inputs it tried — the untested region of the input space is exactly where a false equivalence hides, so "all green" is a statement about coverage, not about the universe.
This sourceExplains differential testing as comparing systems on generated test cases, so its observed agreements are tied to the cases actually exercised.
- Its fundamental limit is that a passing suite proves equivalence only over the inputs it tried — the untested region of the input space is exactly where a false equivalence hides, so "all green" is a statement about coverage, not about the universe.
- Reference-Implementation Differential Test
- The discipline is to keep the reference genuinely independent and simple, and to treat every divergence as an open question about which side is right, not an automatic verdict against the candidate.
This sourceTreats differential-test disagreements as candidates for investigation: either implementation may be wrong, and even the majority can be wrong.
- The discipline is to keep the reference genuinely independent and simple, and to treat every divergence as an open question about which side is right, not an automatic verdict against the candidate.
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Not recorded. Neither this nor any other of the 4 citations of this work carries a recorded support check.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 2 other ways.
- https://vmssoftware.com/docs/dtj-v10-01-1998.pdf ×2
- https://www.cs.tufts.edu/comp/150FP/archive/bill-mckeeman/DifferentailTesting.pdf ×1
Registry ID ref:e54981b4254e · see in the full table