Skip to content

Controlled Disambiguation Test

A diagnostic protocol — instantiates Grammar-Guided Structure Recovery

Resolves a specific ambiguity by constructing a discriminating probe whose outcome forces one reading over its rivals, and scores the confidence of the verdict.

When a parser leaves two or more readings standing, the Controlled Disambiguation Test resolves the specific case by experiment rather than by prior. It takes the competing parses, finds or constructs a discriminating case — an input feature the readings predict differently — applies it, and lets the outcome force the answer, annotating the once-ambiguous node with the winning reading and a confidence drawn from how decisively the probe separated the rivals. Its defining move is that it manufactures evidence to earn the decision: where the Ambiguity Register merely records what stays unresolved and Probabilistic Grammar Parsing scores readings from a trained model, this mechanism can reach genuine certainty whenever a truly separating case exists — and is honest about staying uncertain when none does.

Example

An ingestion service receives date strings like 01/02/2026 from an unlabelled source. Two readings compete: month-first (US, MM/DD) and day-first (European, DD/MM). Rather than default to one, the test scans the batch for a discriminating token — any date whose first field exceeds 12, such as 13/02/2026. Under MM/DD that string is impossible; its mere presence and validity therefore force the whole source to be DD/MM. One such token collapses the ambiguity for the entire feed, and the verdict carries high confidence because the separation is logically clean.

If the batch contains no field-over-12 anywhere, the probe fails to separate the readings: the test does not guess. It annotates the source as genuinely ambiguous with low confidence and routes it onward — the honest outcome when the input simply cannot distinguish the two. A downstream anchor like ISO 8601 exists precisely because this ambiguity is unresolvable by content once the format label is lost.

How it works

The protocol enumerates the competing readings for a node, then searches for a feature on which they disagree — a value one reading permits and another forbids, or an input the two would process into visibly different structures. That discriminating case is applied (mined from the real input, or synthesized on purpose), and the reading consistent with the observed outcome wins. The node is annotated with the resolved reading and a confidence proportional to how decisively the probe separated the candidates. When no discriminating case is reachable, the test reports low confidence rather than a forced pick — a null result that is itself information about the limits of the input.

Tuning parameters

  • Probe strength — a single logically decisive discriminator (a field-over-12) vs. a battery of weak cues combined into a soft verdict. Decisive probes resolve cleanly but do not always exist.
  • Confidence threshold — how sharply a probe must separate the readings before the verdict is accepted rather than flagged as uncertain.
  • Test source — real discriminating instances mined from the input vs. synthetic cases constructed to force the distinction.
  • Generalization scope — apply the verdict per item, or extend one item's decisive result to a whole source or batch. Generalizing is powerful and dangerous — mixed-format sources punish it.
  • Fallback on a null result — default to a designated reading, or escalate the case unresolved.

When it helps, and when it misleads

Its strength is that it earns a decision from evidence instead of assuming one, and can reach certainty — not just a high probability — when a genuinely separating case exists. Its limits are exactly the availability of such cases: when the competing readings are behaviorally identical on every reachable input, the test is powerless and must say so. Its sharpest failure is over-generalization — extending one decisive probe to a batch that is secretly heterogeneous, producing a confident verdict that is uniformly wrong. And its classic misuse is running backwards: constructing the "discriminating" probe after choosing the answer, so the test is rigged to confirm the reading you already preferred. The discipline that guards against this is to pre-commit the discriminating criterion before applying it, and to route null results to the register rather than forcing a pick.[1]

How it implements the components

The Controlled Disambiguation Test realizes the active-resolution side of the ambiguity machinery — deciding a specific case and quantifying how sure that decision is:

  • uncertainty_annotation — it marks the tested node with its resolved reading and whatever residual uncertainty the probe leaves behind.
  • confidence_score — it quantifies how decisively the discriminating case separated the competing readings, distinguishing a logically forced verdict from a merely-favored one.

It does not maintain the standing record of ambiguities that survive resolution (that is the Ambiguity Register), and it does not score readings from a trained statistical prior (that is Probabilistic Grammar Parsing); it consumes the competing parses produced by Chart Parsing.

  • Instantiates: Grammar-Guided Structure Recovery — this test is the archetype's evidence-based resolver for a single ambiguity.
  • Consumes: Chart Parsing supplies the competing parses the test discriminates between.
  • Sibling mechanisms: Ambiguity Register · Chart Parsing · Probabilistic Grammar Parsing · Interpretation Walkthrough · Grammar Rule Set · Semantic Schema Validation

Notes

The confidence this test reports is earned from an observation, not read from a prior — which is what separates it from a probabilistic parser. The two can disagree: a model may rate a reading likely that a discriminating probe then rules out. When they conflict, the observation should win, and the model's prior is what the register falls back on only when no discriminating case exists.

References

[1] ISO 8601 fixes calendar dates in an unambiguous YYYY-MM-DD order precisely so the MM/DD vs. DD/MM ambiguity cannot arise. It is the standard corrective here: where a discriminating case inside the data is not guaranteed, the durable fix is to remove the ambiguity at the source rather than to keep re-resolving it downstream. ↩