Preregistration Refutation Table¶
Document — instantiates Revision-Readiness Precommitment
A table listing what findings would support, weaken, refute, or leave unchanged a hypothesis.
A Preregistration Refutation Table is a document, written and time-stamped before the data are seen, that lays out every plausible outcome of a study and what each one does to the hypotheses in play — support, weaken, refute, or leave unchanged. Its distinguishing move among its siblings is exhaustive outcome mapping across a set of rival hypotheses: it is not a single if-then trigger and not a running number, but a grid that fixes, in advance, the interpretation of each cell of the possibility space and the analysis window that produces it. By deciding what every result would mean while no result is yet known, it removes the freedom to invent a flattering reading after the fact.
Example¶
A memory-research lab plans a study on whether spaced practice beats massed practice for long-term recall. Before collecting a single data point, the team writes a refutation table. Down the side sit the alternatives they are actually deciding among: H1 spacing helps, H2 no real difference, H3 massing helps for this material. Across the top sit the possible results — a large spacing advantage, a small one, a null, a reversal. In each cell they write the verdict in advance: a large advantage at the pre-set 90-day retention test supports H1 and refutes H3; a null within a defined confidence interval weakens H1 and can't distinguish H2 from underpowering; a reversal refutes H1.
They also fix the observation window: the recall test happens at 90 days (illustrative), analyzed with the model named in the document, on the pre-stated sample. When the data land showing a small, noisy effect, the team cannot quietly promote it to "clear support" — the table already logged a small effect as weakens H1. The document did not make them right; it made their standard of evidence immune to being rewritten by the result.
How it works¶
- Enumerate the rival hypotheses. List the genuine contenders the study is meant to discriminate, not just the favored one and a strawman.
- Enumerate the possible findings. Lay out the outcome space — effect large / small / null / reversed, or the categorical results — so no realized result is a surprise the table hasn't seen.
- Fill every cell with a pre-committed verdict. For each finding, state what it does to each hypothesis: support, weaken, refute, or leave unchanged.
- Fix the window and analysis. Name the measurement point, sample, and analysis path in the same document, so "we'll decide how to analyze once we see it" is foreclosed.
- Time-stamp and lock. Register the table before data collection so the mapping cannot be edited after results appear.
Its engine is completeness: because every outcome already has a meaning, there is no leftover result the analyst gets to interpret freely.
Tuning parameters¶
- Hypothesis-set breadth — how many rival hypotheses the table admits. Wider sets discriminate more but demand more pre-specified cells; a two-row table risks a false binary.
- Outcome granularity — coarse bins (support / refute) versus fine ones (effect-size brackets). Finer cells pin interpretation harder but multiply the pre-work and can over-constrain judgment.
- Window definition — how tightly the measurement point, sample, and analysis are fixed. Tight windows kill flexibility-abuse but forbid legitimate mid-study adaptation; leave named contingencies for that.
- Bindingness — whether the table is an internal note or a publicly registered protocol. Public registration maximizes accountability at the cost of admitting error where others can see it.
When it helps, and when it misleads¶
Its strength is that it makes results hard to retrofit. By committing the meaning of each outcome across a real set of alternatives before the data exist, it directly blocks HARKing — hypothesizing after the results are known — and the softer garden-of-forking-paths problem where an analyst wanders the outcome space to the story they like.[1] The table also clarifies design: filling the cells often reveals that the planned study can't actually distinguish two of the hypotheses, prompting a better one.
Its failure mode is false completeness. A table is only as honest as its rows and columns; if a genuine alternative is left off, or the outcome space is drawn too coarsely, an inconvenient result can still slip into an unmapped cell and get interpreted freely. Rigidly locking the analysis can also force a bad model onto data that revealed a real, unanticipated problem. The classic misuse is a table stacked so every realistic outcome "supports" the favored hypothesis — goalpost-freezing dressed as rigor. The guarding discipline is to include the alternatives you would least like to be true, to pre-register named contingencies for legitimate surprises, and to audit the grid for missing rows and unmapped cells before it is registered.
How it implements the components¶
The table realizes the pre-analysis mapping face of the archetype — the components that fix the meaning of evidence before it arrives:
evidence_update_condition— each cell is a pre-stated condition: "this finding, at this size, does this to this hypothesis," specified before the data exist.observation_window— the document fixes the measurement point, sample, and analysis path, foreclosing "we'll decide how and when to look once we see it."alternative_hypothesis_set— the rows enumerate the genuine rival hypotheses the study discriminates among, so the favored view is judged against real competitors rather than a strawman.
It does not size a numeric revision_threshold-driven trip-wire or enact an update_action_mapping on a strategy (the Strategic Revisit Gate), track a current_confidence_baseline across a stream of forecasts (the Forecast Update Trigger Log), or lock a whole-belief accountability_trace (the Belief Update Review Template). Its nearest twin is the Red-Team Update Condition Session, which also sets update conditions pre-evidence; the difference is that this is a static document mapping the full outcome grid, whereas the session is a live adversarial process that supplies an independent_revision_witness and a dissonance_buffer (components this table leaves to it).
Related¶
- Instantiates: Revision-Readiness Precommitment — it supplies the pre-analysis document that fixes the evidential meaning of every outcome before data collection.
- Sibling mechanisms: Belief Update Review Template · Forecast Update Trigger Log · Red-Team Update Condition Session · Strategic Revisit Gate · If-Then Revision Contract
Editorial Notes¶
Form Classification¶
Form family: Representation, Specification & Plan
Rationale: Preregistration Refutation Table operates as a static representation, map, specification, schema, or prospective plan that externalizes information because it a table listing what findings would support, weaken, refute, or leave unchanged a hypothesis.
Independent corroboration: The frozen evidence defines Preregistration Refutation Table as 'A table listing what findings would support, weaken, refute, or leave unchanged a hypothesis', so its operative form is Representation, Specification & Plan.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Preregistration Refutation Table is most plausibly rooted in the statistics_experimental_design tradition because its characteristic form depends on probability, calibrated inference, experimental design, and uncertainty analysis. The assignment tracks that formative lineage, not the many settings in which the mechanism can now be applied.
Related originating lineages:
- Philosophy — The philosophy tradition materially shaped Preregistration Refutation Table through its own practice of logic, argument, normative analysis, and conceptual distinction.
Review resolution: Both blind reviewers agree that statistics experimental design is the primary origin. Explicit reconciliation resolves reported ambiguity. Formative alternate lineages are retained as philosophy; later breadth of use is recorded separately as domain_reach=multi_domain, while origin_mode=cross_disciplinary_synthesis describes the relationship among origin lineages.
Attribution caveat: The exact encyclopedia label appears to synthesize established practices; the primary domain identifies the strongest formative lineage, while the alternates record material ingredients rather than downstream uses. The exhaustive refutation-table artifact appears encyclopedia-specific though its ingredients are established open-science practices.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; medium confidence.
References¶
[1] HARKing — Hypothesizing After the Results are Known — is presenting a hypothesis constructed from the data as though it had been stated in advance (a term introduced by Norbert Kerr, 1998). A refutation table registered before data collection is a direct structural defense: every outcome already has a pre-committed interpretation, so there is no unassigned result left to hypothesize around. registry ↩