Null-Result Power Check¶

Test or assessment — instantiates Expected-Absence Signal Interpretation

Estimates whether a failed search or null observation had enough sensitivity to count as evidence of absence.

Version: v2 · 2026-08-28 · History
Mechanism #: 5735
Type: Test or Assessment
Form family: Assessment, Review & Assurance
Solution family: Participation, Norms & Culture
Problem family: Uncertainty, Evidence & Inference Failure
Problem subfamily: Evidence Warrant, Source & Observation Chain
Origin domain: Statistics & Experimental Design
Instantiates: Expected-Absence Signal Interpretation

Finding nothing only means something if you were looking hard enough to have found it. Null-Result Power Check takes a search or observation that came up empty and asks a quantitative question the detection-opportunity check does not: given that a channel existed, was it sensitive enough that, had the sought-for thing been there at the expected magnitude, this search would probably have caught it? Its defining idea is the sensitivity contrast — comparing how likely the null was under "the thing is present" versus "the thing is absent." A null from an underpowered search is nearly uninformative (you'd have missed it either way); a null from a well-powered search is real evidence of absence. The check converts a raw "we found nothing" into a graded "this null rules out presence down to level X, and no further." It reasons about how sharp the look was, not whether a look was possible — and it does not act on the result.

Example¶

A radio-astronomy group predicts that a nearby star system, if it hosts a particular kind of energetic phenomenon, should emit a detectable signal in a specific frequency band. They observe for several nights and detect nothing. Before announcing "no such phenomenon here," they run a Null-Result Power Check. The question is not whether the telescope was pointed correctly and recording — that much is established — but whether the integration time, bandwidth, and system noise gave them the sensitivity to have seen the predicted signal at its expected strength.

They compute the expected signal amplitude, compare it to the noise floor achieved over the observing window, and find their sensitivity threshold sat above the predicted amplitude: a real signal at the predicted level would have been buried in noise more often than not. The null is therefore weak — it fails to exclude the phenomenon. The power check reframes the finding from "absent" to "not yet detectable at this sensitivity," and points to the fix: longer integration or a second dish stacked in — a redundant channel — to push the threshold below the predicted amplitude before the null can carry weight.

How it works¶

State the expected effect size. Fix what magnitude the sought event would have if present — the amplitude, rate, or count the search was hunting.
Characterize the search's sensitivity. Establish the detection threshold actually achieved: noise floor, sample size, exposure, minimum detectable effect.
Contrast the likelihoods. Estimate the probability of getting this null if the effect were present at the expected size versus if it were absent. A high probability of null-under-presence means low power and a weak null.
Grade, and prescribe more power if needed. Report the level down to which the null excludes presence; where power is short, recommend more exposure, larger samples, or stacking an independent channel to raise sensitivity.

Tuning parameters¶

Assumed effect size — the magnitude the search is credited with hunting. Assume a large effect and the null looks powerful; assume a small one and the same null looks weak. The most consequential and most arguable dial.
Sensitivity confidence — how conservatively the achieved detection threshold is estimated. Optimistic thresholds overstate power and over-trust the null; conservative ones under-trust it.
Exclusion level — the strength of null the check demands before "evidence of absence" is granted. A strict level protects against false-absence claims but leaves more nulls verdict-less.
Redundancy budget — how much added exposure or how many stacked independent channels you'll spend to lift power. More buys a decisive null but costs time and instrument.
Aggregation method — for repeated or multi-channel searches, how nulls are pooled to raise combined sensitivity. Pooling strengthens the verdict but assumes the channels are truly independent.

When it helps, and when it misleads¶

Its strength is refusing the most seductive scientific and operational error — reading a null as a finding when the search never had the teeth to produce one. It is the operational face of statistical power: a low-power null carries almost no information, and confusing it for evidence of absence is a Type II error dressed as a discovery.^[1] Wherever a "we found nothing" is about to become "there is nothing," this check is the gate that grades how much the null is worth.

Its failure mode is that power estimates are only as honest as their assumed effect size: inflate the magnitude you claim to have been hunting and any null looks decisive, so the check can be gamed to manufacture a strong-sounding absence. It also assumes the sensitivity model is faithful — an unmodeled systematic can leave a search far less powerful than its formula says. And pooling channels to raise power silently fails if the channels share the same blind spot. The guarding discipline is to state the assumed effect size before seeing the result, stress-test it, and verify the independence of any channels stacked to gain power rather than assuming it.

How it implements the components¶

absence_likelihood_baseline — it computes the core contrast, P(null | present) versus P(null | absent), that says how diagnostic this particular null is.
absence_threshold_rule — it sets and applies the sensitivity bar a null must clear before it is admitted as evidence of absence.
redundant_observation_channel — when power falls short, its prescription is to add exposure or stack an independent channel to push sensitivity below the expected effect.

It does not verify that any observing channel existed or was live in the first place (observation_opportunity_window, production_process_reference, false_absence_guardrail) — that is its hazard-twin Detection Opportunity Audit; this check assumes a channel existed and asks only whether it was sensitive enough.

Instantiates: Expected-Absence Signal Interpretation — supplies the sensitivity grading that tells a null how much evidential weight it has earned.
Consumes: Detection Opportunity Audit must first confirm a channel existed; only then is a power check meaningful.
Sibling mechanisms: Absence Likelihood Dashboard · Confirmation Probe Request · Detection Opportunity Audit · Exception-Lag Review Workflow · Expected Event Register · Missing Heartbeat Monitor · No-Response Escalation Protocol · Silence Signal Review Board

Editorial Notes¶

Form Classification¶

Form family: Assessment, Review & Assurance

Rationale: Null-Result Power Check operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it estimates whether a failed search or null observation had enough sensitivity to count as evidence of absence.

Independent corroboration: The frozen evidence defines Null-Result Power Check as 'Estimates whether a failed search or null observation had enough sensitivity to count as evidence of absence', so its operative form is Assessment, Review & Assurance.

Review outcome: Independent reviewer agreement; high confidence.

Origin Attribution¶

Primary origin: Statistics & Experimental Design

Origin pattern: Single lineage

Present-day reach: Multi-domain

Rationale: Power analysis and Type II error theory establish whether a study had a meaningful chance to detect a specified effect before its null result is read as absence.