Counterevidence Precommitment Question¶
Checklist — instantiates Epistemic Boundary Permeability Design
Asks participants to state in advance what evidence would change or complicate their view.
A belief that quietly rewrites what would have counted against it can never be disconfirmed. Counterevidence Precommitment Question blocks that rewrite. It is a prospective checklist that captures, before the evidence is in, the specific observations each participant says would change or complicate their view — locking in the falsifiers ahead of time. Its defining move is that it is forward-looking and commits people in advance: it records intentions about what should move a belief, so that when data arrives the bar cannot silently shift. A "nothing would change my mind" answer is not a failure of the exercise — it is its most valuable finding.
Example¶
A research lab is heavily invested in a favored hypothesis about a drug's mechanism. Before the next experiment, the principal investigator runs a precommitment checklist. Each team member writes down, in advance: what result would falsify the hypothesis, what result would merely complicate it, and — reciprocally — whether they would hold a rival lab's identical result to the same evidentiary standard.
One member realizes they cannot name any result that would change their mind, which exposes that the belief has quietly become unfalsifiable. The answers are sealed and revisited when the data lands. The group has surfaced a sealed belief before running the experiment and has set an honest, symmetric bar for what the results must clear.
How it works¶
- Ask before the evidence. Elicit falsifiers while the outcome is still unknown, not after.
- Separate falsify from complicate. Distinguish what would overturn the view from what would merely qualify it.
- Add the reciprocal item. Would the same bar apply to an ally's and a rival's identical result?
- Seal and revisit. Record the answers, then compare them to what actually happens.
Its distinguishing feature is prospective commitment. It captures what people say in advance would move them; it does not look back at what did.
Tuning parameters¶
- Individual vs. group elicitation — private answers surface honest doubt; group answers build shared commitment but invite conformity.
- Specificity required — vague falsifiers are easy but useless; operational thresholds bind but take work.
- Public vs. sealed — sealing protects candor; publishing raises accountability.
- Reciprocal-standard item — whether the same-bar-for-both question is included, which tests for asymmetric skepticism.
- Binding force — how strongly the precommitted answers constrain later interpretation.
When it helps, and when it misleads¶
Its strength is that it reveals sealed, unfalsifiable beliefs before they harden further, and it pre-commits an honest evidence bar that later disconfirmation can be held to. It is an operational form of falsifiability[n1]: a view that names no possible disconfirming observation is, by that criterion, not doing empirical work at all.
Its failure mode is the straw falsifier — people name conditions so extreme they will never occur ("I'd change my mind if the sun rose in the west"), preserving the belief while appearing open — or they quietly move the goalposts when the evidence actually arrives. The classic misuse is performing the ritual and then reinterpreting disconfirming data as confirmation, the precommitment forgotten. The guarding discipline is to require operational, plausible falsifiers, seal them, and let a separate retrospective review hold the group to exactly what it committed to in advance.
How it implements the components¶
Counterevidence Precommitment Question fills the probe-the-sealed-belief side of the archetype:
perceived_completeness_probe— a "nothing would change my mind" answer directly exposes a view experienced as complete and closed, which is precisely what this probe is meant to surface.reciprocal_standard_check— the same-bar-for-allies-and-rivals item applies a symmetric evidentiary standard, catching the asymmetric skepticism that props up a boundary.
It fixes the bar in advance; it does not record what actually changed once the evidence landed. That retrospective log — reflection_and_uptake_loop — is Belief-Update After-Action Review, its nearest twin: precommitment looks forward at what would move a belief, the after-action review looks back at what did.
Related¶
- Instantiates: Epistemic Boundary Permeability Design — the question is the prospective commitment that keeps a belief answerable to evidence before that evidence arrives.
- Sibling mechanisms: Source Diet Audit · Filter Transparency Dashboard · Cross-Cutting Source Rotation · Recommendation Diversity Constraint · Rival Hypothesis Red Team · Bridge Panel or Boundary-Spanner Session · Steelman Counterposition Brief · Caricature Detection Review · Belief-Update After-Action Review
Editorial Notes¶
Form Classification¶
Form family: Protocol, Workflow & Routine
Rationale: Participants state falsifying and complicating evidence before outcomes are known, seal the answers, and later compare them with what occurred, so its operative form is a precommitment-and-revisit procedure.
Nearest alternative: Representation, Specification & Plan — A checklist carries and records the questions, but asking prospectively and revisiting after evidence arrives rather than the static list alone creates the discipline.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Philosophy
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Philosophy of science supplies the falsifiability demand to state what evidence would count against a claim; psychology supplies the precommitment prompt that makes the demand behaviorally effective.
Related originating lineages:
- Psychology — Debiasing research supplied consider-the-opposite prompts and advance specification of disconfirming observations to reduce motivated reasoning.
Review resolution: Popperian falsifiability is the closest origin for the counterevidence criterion, while experimental debiasing research establishes the consider-the-opposite intervention form.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
- Stanford Encyclopedia of Philosophy: Karl Popper
- Lord, Lepper, and Preston, Considering the opposite
Notes¶
[n1] Falsifiability, Karl Popper's demarcation criterion, holds that a claim is empirically meaningful only if some possible observation could count against it. Asking in advance what evidence would change your view operationalizes the criterion for a working group — a belief that admits no falsifier is exposed as immune to correction. ↩