Detection Power Checklist¶
Checklist — instantiates Null Finding Warrant Calibration
Prompts reviewers to check scope, timing, sensitivity, thresholds, masking, and failure modes before interpreting non-detection.
Most bad null-finding calls are not subtle statistical errors; they are omissions — nobody asked whether the search was in the right place, at the right time, looking for the right thing. The Detection Power Checklist is the cheap, portable guard against that: a short ordered list of yes/no prompts a reviewer runs before a null finding is allowed to carry any weight. Its defining idea is that it is a gate, not a model — it does not quantify anything and it does not draw a grid; it simply forces the reviewer past each question that a false all-clear tends to skip, and its output is a routing decision: every prompt clears → you may interpret the null; any prompt fails → search more before concluding. Unlike the sensitivity matrix, it never decomposes channel by target form; it asks whether the search as a whole cleared each hurdle, which is exactly why it works in any domain with no setup.
Example¶
A systematic-review team is about to conclude "no published evidence exists" for a proposed clinical intervention. Before that sentence ships, they run the checklist. Scope: did the search cover the right databases, plus trial registries and grey literature, or only two convenient indexes? Conditions: right date range, right languages, or an English-only window that drops half the field? Masking: did the search terms catch the synonyms and the older nomenclature the target might hide under, and did they account for publication bias — the studies that were run but never printed? Failure modes: would a relevant trial have been indexed under a heading nobody queried? Two prompts fail — grey literature was skipped and one key synonym was missing — so the checklist's output is not "no evidence" but "expand the search first." The absence claim is held at the gate until the search is repaired.
How it works¶
- Ordered prompts, each tied to a failure. Every item corresponds to one way a search produces a false null: wrong place, wrong time, insufficient sensitivity, wrong threshold, masking, or a known failure mode.
- Binary answers, evidence required. Each prompt is answered yes/no, and a "yes" is supposed to be backed by a fact, not a reflexive tick.
- Any-fail routing. The checklist's product is a route: a clean pass authorizes interpretation; any failed prompt triggers a specific remediation and re-search rather than a conclusion.
- Kept lightweight. It is deliberately shallow enough to run in minutes on any non-detection, which is what makes it get used at all.
Tuning parameters¶
- Item count and granularity — a five-line pocket version or a thirty-line domain checklist. More items catch more, but a checklist too long to run is a checklist nobody runs.
- Strictness rule — any-fail-blocks versus a weighted or discretionary pass. Strict gating is safer; discretion is faster and erodes under pressure.
- Domain customization — how much the generic prompts are specialized to the field's characteristic blind spots (retention windows, seasonal timing, indexing quirks).
- Sign-off — whether the gate needs a second reviewer, raising cost and independence together.
When it helps, and when it misleads¶
Its strength is coverage at near-zero cost: it catches the obvious, embarrassing omissions — the unsearched channel, the wrong season — that sink most false all-clears, and it does so before the conclusion is written rather than after it is challenged. Because it routes rather than concludes, it also breaks the two opposite failures the archetype warns about: it neither waves a weak search through nor demands infinite search.
Its failure mode is checklist theater — boxes ticked without substance, the form completed as ritual while the underlying question goes unexamined. In evidence synthesis the classic masking mode it must not let slide is publication bias, the file-drawer problem[n1]: the studies that were run and found nothing are exactly the ones least likely to be printed, so "found no studies" and "no effect exists" can be miles apart. The guarding discipline is to require a supporting fact for every "yes," and to treat the checklist as a gate on interpretation, never as evidence in itself.
How it implements the components¶
The checklist realizes the pre-interpretation gating side of the archetype — the qualitative hurdles, not the quantitative machinery:
search_scope_boundary— its scope prompts confirm the search reached the right domain and pin down where it stops.observation_condition_record— its timing and condition prompts confirm and note when and how the search ran.non_detection_alternative_set— its masking and failure-mode items enumerate the ways a present target could have left no trace.follow_up_or_stopping_rule— the pass/fail result is the routing rule: interpret now, or search more first.
It does not quantify per-channel sensitivity or assign a graded ceiling: the computed detection_power_model is Likelihood Ratio for Non-Detection; the numeric minimum_detectable_presence_threshold is the Minimum Detectable Presence Table; and the displayed absence_warrant_grade is the Null Finding Warrant Memo. The checklist's nearest twin is the Search Sensitivity Matrix — the matrix decomposes detection cell by cell, whereas the checklist gates the search as a whole.
Related¶
- Instantiates: Null Finding Warrant Calibration — the checklist is the archetype's lightweight, cross-domain entry gate.
- Sibling mechanisms: Null Finding Warrant Memo · Search Sensitivity Matrix · Negative Test Interpretation Protocol · Likelihood Ratio for Non-Detection · Minimum Detectable Presence Table · Coverage Map and Blind-Spot Review · Silent Monitor Assurance Review
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Detection Power Checklist operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it prompts reviewers to check scope, timing, sensitivity, thresholds, masking, and failure modes before interpreting non-detection.
Independent corroboration: The frozen evidence defines Detection Power Checklist as 'Prompts reviewers to check scope, timing, sensitivity, thresholds, masking, and failure modes before interpreting non-detection', so its operative form is Assessment, Review & Assurance.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Statistical inference cohered power, sensitivity, scope, and masking checks before a null result is treated as evidence of absence.
Related originating lineages:
- Library & Information Science — Evidence-search practice supplied coverage and publication-bias checks such as the file-drawer problem.
- Security Studies & Intelligence Analysis — Collection assessment supplied channel and target-form coverage review.
Review resolution: Statistical inference cohered power, sensitivity, scope, and masking checks before a null result is treated as evidence of absence. The retained alternate lineages materially shaped the mechanism's form.
Attribution caveat: The checklist is a portable synthesis of statistical power and search-coverage disciplines.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; medium confidence.
Notes¶
[n1] The file-drawer problem (publication bias) — studies with null or unwelcome results are disproportionately never published, so a literature search that "finds nothing" is systematically biased toward false absence. It is the canonical masking mode a detection checklist in evidence work must force onto the table. ↩