Falsification Check¶
Test specification — instantiates Counterexample Search
Restates a confident claim as an explicit rule with a bounded scope and a pre-committed breaking criterion, so later evidence can actually refute it.
Most confident claims are not yet testable — they are slogans with a hidden universal buried inside ("it always works," "we catch everything"). Falsification Check is the front-end move that makes such a claim challengeable at all. It does exactly three things and no more: it writes the claim down as an explicit rule, it states the scope in which the rule is asserted to hold, and — crucially, before anyone goes looking — it fixes the exact observation that would count as a refutation. It searches nothing and revises nothing. Its entire value is that it nails the goalposts in place in advance, so that when a candidate exception later shows up, there is already an agreed answer to "does this break the claim?" rather than a fresh argument.
Example¶
A hospital quality team is about to lean on a claim: "our early-warning score flags every patient who is going to deteriorate." Falsification Check turns that into something a chart audit could actually break. The proposed rule: for an adult, non-ICU inpatient, if the patient suffers a deterioration event within 24 hours, the score crossed its alert threshold beforehand. The scope: adult medical and surgical wards, current score version — explicitly not pediatrics, not the ICU, not the post-discharge window. The falsification condition, written and signed off before a single chart is pulled: one in-scope patient who deteriorated within 24 hours while the score never crossed threshold refutes the claim as stated.
Now the audit has a target it can hit, and — the point of the whole exercise — the team agreed on what "break" means before they saw the charts. When a missed case surfaces, no one can wave it away as "well, that one doesn't really count." The claim has been made honest enough to lose.
How it works¶
- Surface the hidden quantifier. Confident claims conceal an always/every/never. Name it: for all in-scope cases, the rule holds.
- Bound the scope tightly. The narrower and more explicit the scope, the more a breaking case inside it actually counts — and the harder it is to redefine the scope later to dodge one.
- Pre-state the refutation criterion. Write the precise observation that would falsify the claim, and freeze it before any search. For a strict universal a single instance suffices (the decisive form is Proof by Counterexample); for a probabilistic claim the criterion is a rate or threshold.
- Hand off. The output is a testable specification, not a verdict. Downstream search mechanisms consume it.
Tuning parameters¶
- Falsification bar — how severe (one case, or a failure rate) counts as refutation. Too strict and no real claim survives; too loose and nothing is genuinely testable.
- Scope tightness — broad scope is easy to break but yields a weak rule; tight scope is defensible but risks defining away the very cases that worry you.
- Pre-commitment firmness — how hard the criterion is frozen before search. Firm freezing blocks goalpost-moving but risks committing to a poorly chosen test.
- Operationalization grain — how concretely "the rule holds" is measured. Vague terms quietly re-open the goalposts.
When it helps, and when it misleads¶
Its strength is leverage per effort: it costs a paragraph and it forecloses the single most common evasion in the whole archetype — amending the claim after an exception appears. It converts an unfalsifiable boast into a claim with a defined failure.
Its failure mode is falsification theater: a scope drawn so narrow, or a criterion engineered so unreachable, that the claim is technically falsifiable but effectively immune to the cases you actually fear. The classic misuse is stating a clean falsification condition and then, when a candidate case arrives, quietly editing the scope to exclude it — precisely what the pre-commitment is meant to prevent. The guarding discipline is to fix scope and criterion in writing before search, and to treat any post-hoc scope change as a red flag that the test was rigged, not a free save. The anchoring idea is Popper's falsifiability[1]: a claim earns credibility by specifying what would refute it, not by piling up confirmations.
How it implements the components¶
proposed_rule— makes the rule explicit enough that a breaking case can be recognized, extracting the hidden quantifier from the slogan.claim_scope— states the contexts, populations, thresholds, and conditions where the rule is asserted to hold.falsification_condition— pre-states, before any search, the criterion for what would count as a breaking case.
It never goes looking: designing and executing the hunt — counterexample_search_space, counterexample_record — is Exception Search's work, and revising the rule afterward — scope_revision, confidence_update — is Negative Case Analysis's.
Related¶
- Instantiates: Counterexample Search — it supplies the testable claim the rest of the archetype searches against.
- Sibling mechanisms: Exception Search · Adversarial Example Generation · Boundary Condition Matrix · Negative Case Analysis · Proof by Counterexample
Editorial Notes¶
Form Classification¶
Form family: Representation, Specification & Plan
Rationale: Falsification Check operates as a non-executable information artifact that externalizes static or prospective structure because it restates a confident claim as an explicit rule with a bounded scope and a pre-committed breaking criterion, so later evidence can actually refute it.
Independent corroboration: The frozen evidence defines Falsification Check as 'Restates a confident claim as an explicit rule with a bounded scope and a pre-committed breaking criterion, so later evidence can actually refute it', so its operative form is Representation, Specification & Plan.
Nearest alternative: Rule, Policy & Commitment — The artifact externalizes a bounded claim and precommitted breaking criterion; it does not itself impose conduct or execute the later test.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Philosophy
Origin pattern: Single lineage
Present-day reach: Universal
Rationale: Pre-specifying an observation that would refute a bounded claim is canonical philosophy-of-science falsificationism.
Related originating lineages:
- Statistics & Experimental Design — Empirical hypothesis testing materially operationalizes the breaking criterion.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Independent reviewer agreement; high confidence.
References¶
[1] Popper, K. R. The Logic of Scientific Discovery. Hutchinson (1959). Makes exposure to possible empirical refutation—not accumulation of favorable instances—the criterion for an empirical scientific claim. registry ↩