Claim Registry¶
Traceability and governance record — instantiates Multiple-Testing Discipline
A living ledger of every attempted claim — its status, owner, and follow-up burden — so selective memory can't erase the failed tries that made a discovery look surprising.
A Claim Registry is the organizational memory that keeps multiplicity discipline from decaying between studies. Statistical corrections protect a single family analyzed at one moment; a registry protects a program over time, by recording every claim someone attempted, what became of it, who owns it, and what it must still survive before it can drive action. Its defining idea — the thing no sibling does — is that it is a persistent, cross-claim ledger of statuses and follow-up obligations, maintained continuously rather than tied to any one experiment. It fights the failure where only the winners are remembered: without a registry, the null and abandoned attempts quietly disappear, and the surviving claim looks far more surprising than the search that produced it warrants.
Example¶
A quantitative investment firm generates trading signals by backtesting ideas against decades of market data. Analysts test hundreds of candidate signals a year; most look promising in-sample and evaporate out-of-sample. The danger is institutional amnesia: a signal that "worked" is pitched to the investment committee with no visible trail of the ninety-nine cousins that were tried and dropped, so its backtest looks like a rare insight rather than the best of many draws. The firm stands up a claim registry. Every signal an analyst backtests is logged the moment it is proposed — hypothesis, data window, the analyst who owns it, and a live status: proposed, in backtest, live-tracked, retired. A signal cannot be promoted to real capital until its registry entry shows it has met a pre-set follow-up bar (a minimum out-of-sample tracking period). When a committee reviews a candidate, the registry shows not just the winner but the field it emerged from — and the count of siblings is itself a warning that its backtest may be a lucky extreme. The registry converts scattered private notebooks into a shared, auditable record that resists the file-drawer problem.[n1]
How it works¶
- Log on attempt, not on success. Every claim is entered when it is proposed, before its result is known, so failures cannot vanish retroactively.
- Track a live status and owner. Each entry carries a status label that moves through defined states and a named person accountable for it.
- Attach a follow-up obligation. Entries record what a claim must still pass before it earns decision-ready status — the confirmation burden it owes.
- Expose the field, not just the winner. Any review reads the registry to see how many sibling claims shared the same search, so a survivor is judged against its cohort.
Tuning parameters¶
- Entry threshold — how early a claim must be logged (at idea, at first analysis, at proposal). Earlier capture prevents amnesia but raises administrative load.
- Status vocabulary — how many states a claim can occupy, and how it transitions; richer vocabularies carry more information but demand discipline to keep current.
- Access and visibility — private to a team, shared across an org, or public. Wider visibility deters cherry-picking most but resists disclosure most.
- Retention and audit cadence — how long retired claims persist and how often the ledger is reviewed for stale or abandoned entries.
- Enforcement coupling — whether promotion gates are hard-wired to registry status or merely advisory.
When it helps, and when it misleads¶
Its strength is memory: it makes the denominator of attempts durable, so no later reviewer can mistake the luckiest survivor for a lone insight, and it assigns ownership and follow-up obligations that would otherwise be forgotten. It is the right mechanism when the main risk is organizational cherry-picking rather than a single mis-set threshold.
Its failure mode is that a registry records claims but does not, by itself, correct or confirm any of them — a beautifully complete ledger can sit beside decisions that ignore it, which is governance theater. It also imposes real administrative burden, and an under-maintained registry (stale statuses, quietly un-logged attempts) is worse than none because it looks like coverage while missing the very failures it was built to keep. The guarding discipline is to couple promotion gates to registry status so that an unlogged or unconfirmed claim simply cannot advance, and to audit for entries that were never opened.
How it implements the components¶
discovery_record— the registry is the record: it keeps failed, null, abandoned, and selected claims all visible in one durable place.result_status_label— every entry carries an explicit, evolving status (proposed / tracked / confirmed / retired) that travels with the claim.confirmation_requirement— each entry names the follow-up burden a claim must clear before it can guide costly action, and gates promotion on it.
It records what confirmation a claim owes but does not execute it: reserving a holdout_evidence partition is Holdout Validation's work, and freezing a study's confirmatory plan before data via an ex-ante exploratory_confirmatory_boundary is Preregistration's — the registry is an ongoing ledger, not a pre-commitment or a test.
Related¶
- Instantiates: Multiple-Testing Discipline — supplies the governance-memory form of multiplicity control.
- Sibling mechanisms: Alpha-Spending Plan · Bonferroni-Like Correction · False Discovery Rate Control · Confirmatory Follow-Up · Holdout Validation · Metric Hierarchy · Multiverse Analysis Report · Preregistration · Replication Study
Editorial Notes¶
Form Classification¶
Form family: Record, Log & Register
Rationale: A living ledger of every attempted claim — its status, owner, and follow-up burden — so selective memory can't erase the failed tries that made a discovery look surprising, making its operative form a durable record, ledger, register, or trace whose value depends on preserving actual state or history.
Independent corroboration: The frozen evidence defines Claim Registry as 'A living ledger of every attempted claim — its status, owner, and follow-up burden — so selective memory can't erase the failed tries that made a discovery look surprising', so its operative form is Record, Log & Register.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Multiple-testing and preregistration practice established retaining every attempted claim so null and abandoned tests cannot disappear from the evidential denominator.
Related originating lineages:
- Accounting & Auditing — Ledger discipline contributes persistent ownership, status, and follow-up records.
Review resolution: Multiple-testing and preregistration practice is the agreed primary lineage because the ledger preserves the denominator of attempted claims. Accounting contributes persistent ownership and status discipline; the organization-wide promotion-gated registry is an Encyclopedia synthesis.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] The file-drawer problem is the disappearance of null or unsuccessful analyses into researchers' drawers, leaving the published or promoted record biased toward positive results; a registry counters it by logging attempts at proposal time, before their outcome is known. ↩