Hallucination Check¶

Protocol — instantiates Cautious Pattern Completion

A review pass over generated or inferred content that flags every unsupported detail and verifies each nontrivial claim against a real source before it is trusted.

Version: v1 · 2026-08-24 · History
Mechanism #: 4005
Type: Protocol
Form family: Assessment, Review & Assurance
Solution family: Learning & Scaffolding
Problem family: Uncertainty, Evidence & Inference Failure
Problem subfamily: Explanatory Hypothesis, Pattern & Case Reasoning
Origin domain: Ethics of Technology & AI Governance
Also from: Computer Science & Software Engineering, Library & Information Science
Instantiates: Cautious Pattern Completion

A Hallucination Check is a review protocol aimed at a specific danger: fluent, confident content that contains details no one can source. Its defining move is to treat generated output — a model's summary, an auto-drafted brief, a smoothed-over reconstruction — as guilty until verified, walking claim by claim and forcing each nontrivial assertion to either point at a real supporting source or be marked as unsupported. The distinguishing feature is the presumption: unlike ordinary editing, which trusts the text and looks for errors, this protocol distrusts the text and looks for the absence of support, because polished prose can make a fabricated citation, a made-up figure, or an invented causal link feel exactly as solid as a true one.

Example¶

A market-research team asks a language model to summarize thirty customer-interview transcripts into a findings memo. The output is clean and quotable: it reports that "68% of enterprise users cited onboarding friction as their top concern" and attributes a vivid complaint to a named account. The Hallucination Check runs before the memo circulates. Each nontrivial claim is lifted out and traced back to the transcripts: the "top concern" theme is genuinely present and survives; the crisp 68% is not — no such tally exists in the source, the model manufactured a precise-sounding number from a qualitative pattern; and the named-account quote is a paraphrase the model attributed to the wrong customer.

The check does not rewrite the memo's argument; it flags the two unsupported details, sends the 68% back as "no source — remove or re-derive," and corrects the misattribution. What ships is the same useful summary with its invented specifics stripped out. The protocol's whole contribution is catching the fluent fabrications before they hardened into "findings" that a downstream reader would cite as fact.

How it works¶

The protocol is a systematic pass, and its steps are shaped by the failure it targets — unsupported specifics dressed as facts.

Decompose into checkable claims. Break the generated content into individual assertions, especially the specific ones: numbers, names, dates, quotations, causal links.
Demand a source for each. For every claim, require a pointer to a real supporting passage. A claim with no locatable support is flagged, not trusted — its very confidence is treated as a warning sign.
Verify the specifics hardest. Fabrications cluster in precise details, so exact figures and named attributions get the closest check against the actual source.
Route, don't just delete. Flagged claims are returned as unsupported — re-derive, source, or cut, so a real but poorly-sourced finding is not thrown out with a fabricated one.

Tuning parameters¶

Claim-granularity threshold — how fine the decomposition goes. Checking every clause catches more fabrication but is slow; checking only load-bearing claims is faster and lets small invented details slip.
Source-strictness — what counts as adequate support (exact passage vs. general presence of the theme). Strict sourcing kills more hallucinations and rejects more true-but-fuzzy claims.
Coverage — full review of every claim versus a sampled audit. Full coverage is safest for high-stakes outputs; sampling scales to volume but leaves gaps.
Specific-detail weighting — how much extra scrutiny precise figures and named quotes receive relative to general statements. Heavier weighting targets where fabrication lives, at some cost in even-handedness.

When it helps, and when it misleads¶

Its strength is catching the exact failure that fluent generation introduces: a confident, well-formed detail with nothing behind it. It is most needed precisely when the output reads well, because polish suppresses the reader's own skepticism — a form of automation bias, the documented tendency to over-trust an automated system's output and under-check it.^[n1]

Its failure modes are over- and under-trust. A shallow check becomes rubber-stamping — a pass performed so lightly that it launders the output as "reviewed" while the fabrications survive, which is worse than no check because it adds false assurance. An over-zealous check produces false-flag fatigue, drowning real content in nitpicks until reviewers start ignoring the flags. And the protocol cannot vouch for what it cannot source-check — a plausible unsupported claim that happens to be true still gets flagged, and one that is checkable but subtly wrong can pass if the source is skimmed. The guarding discipline is to concentrate scrutiny on the specific, high-consequence claims where fabrication actually clusters, and to route rather than reflexively delete, so the check improves the output instead of merely gatekeeping it.

How it implements the components¶

verification_step — the protocol's core is a claim-by-claim verification of generated content against real supporting sources, done before the content is trusted or forwarded.
missing_evidence_marker — any claim that cannot be sourced is explicitly flagged as unsupported, converting a silent fabrication into a visible, actionable gap.

It verifies claims but does not maintain the standing provenance ledger that maps every element to a source type — that is source_trace, held by Source-Tracing Table, its nearest twin: the table is the durable map, this protocol is the active review that consults and tests it. Nor does it attach the travel-with-the-claim uncertainty_labels that persist downstream — that is Uncertainty Tagging.

Instantiates: Cautious Pattern Completion — it applies the archetype's evidence/inference discipline to generated and AI-assisted completions, where fluency masks unsupported detail.
Consumes: Source-Tracing Table supplies the provenance map the check verifies each claim against.
Sibling mechanisms: Hypothesis List · Diagnostic Differential · Disconfirming Evidence Search · Reconstruction Note · Source-Tracing Table · Uncertainty Tagging · Withhold-Conclusion Checkpoint · Assumption Log

Editorial Notes¶

Form Classification¶

Form family: Assessment, Review & Assurance

Rationale: Hallucination Check operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it a review pass over generated or inferred content that flags every unsupported detail and verifies each nontrivial claim against a real source before it is trusted.

Independent corroboration: The frozen evidence defines Hallucination Check as 'A review pass over generated or inferred content that flags every unsupported detail and verifies each nontrivial claim against a real source before it is trusted', so its operative form is Assessment, Review & Assurance.

Review outcome: Independent reviewer agreement; high confidence.

Origin Attribution¶

Primary origin: Ethics of Technology & AI Governance

Origin pattern: Cross-disciplinary synthesis

Present-day reach: Multi-domain

Rationale: Responsible generative-AI practice created a specific verification duty for fluent unsupported model output.

Related originating lineages:

Computer Science & Software Engineering — AI evaluation materially supplies factuality tests and retrieval-backed verification.
Library & Information Science — Source evaluation and reference checking provide claim-level evidentiary discipline.

Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.