False Discovery Rate Control¶
Statistical discovery procedure — instantiates Multiple-Testing Discipline
Ranks a whole family of results and draws the significance line to hold the expected share of false discoveries below a chosen rate, trading a little purity for far more power.
False Discovery Rate Control is the multiplicity procedure built for screening, where you expect to keep many discoveries and can afford to follow them up. Instead of protecting against even one false positive, it accepts that some declared discoveries will be wrong and instead caps the expected proportion of false ones among everything it calls significant. The defining idea — the one that separates it sharply from a familywise correction — is that its control target is a rate, not an event: it does not ask "what is the chance of any false alarm?" but "of the results I flag, what fraction do I tolerate being false?" That reframing lets it stay sensitive across thousands of tests where dividing an error budget across the whole family would suppress nearly every real signal.
Example¶
A cancer-genomics lab profiles gene expression across tumor and healthy tissue, testing roughly 20,000 genes at once for differential activity. A strict familywise bar would demand each gene clear a threshold so tiny that only a handful of the strongest could ever pass, and most real disease-linked genes would be missed — useless for a screen whose whole point is to generate a shortlist for follow-up. The lab instead runs the Benjamini–Hochberg procedure.[n1] It ranks all 20,000 p-values from smallest to largest and finds the largest rank at which the p-value still falls under a line that climbs with rank; every gene at or below that rank is declared a discovery. With the false-discovery rate set to 10%, the lab knows that about one in ten of the flagged genes is expected to be a false lead — an acceptable price, because each will be checked in a confirmatory assay anyway. The result is a shortlist of a few hundred candidate genes with a known, controlled contamination level, rather than five survivors or twenty thousand maybes.
How it works¶
- Assemble and rank the family. Collect the p-values of all tests in the family and sort them from smallest to largest.
- Set the tolerated rate. Choose q, the maximum acceptable expected fraction of false discoveries among the flagged results.
- Find the step-up cutoff. Walk the ranked list and identify the largest rank k whose p-value is still below (k / m) × q, where m is the family size; reject everything up to that rank.
- Label discoveries. Each test at or below the cutoff is declared a discovery, the rest are not — and the label carries the promise that the expected false share among the discoveries is at most q. Because the line rises with rank, a family rich in real signal earns a more permissive cutoff than a family of pure noise.
Tuning parameters¶
- Target rate q — the tolerated false-discovery share; raising it flags more candidates but dilutes their purity, lowering it does the reverse. This is the defining dial.
- Dependence variant — the basic procedure assumes independence or positive dependence; a more conservative form (Benjamini–Yekutieli) holds under arbitrary dependence at a power cost.
- Adaptive estimation — estimating the fraction of true nulls (q-value / Storey-style adaptivity) recovers power when many effects are real.
- Family scope — which tests share the ranking; a wider family gives the rate more meaning but must be fixed before ranking.
When it helps, and when it misleads¶
Its strength is power at scale: it keeps far more real signals than a familywise correction while still bounding contamination, which is exactly right when the family is huge and each discovery gets checked downstream. It turns "twenty thousand tests" from a paralyzing multiplicity into a ranked, calibrated shortlist.
Its failure mode is that the guarantee is about the set, not any single member: one flagged discovery is not "90% likely true" — it is a member of a set whose expected false fraction is 10%, and individual flagged results (especially those near the cutoff) can be quite unreliable. Treating any single FDR-passing result as confirmed is the classic misuse; so is re-running with a looser q after the first pass disappoints, which quietly relabels the same noise. And like every threshold procedure it is defenceless against a family gerrymandered after the fact. The guarding discipline is to fix q and the family in advance, treat FDR output as leads for confirmation rather than settled facts, and pair it with independent follow-up before acting on any one hit.
How it implements the components¶
multiplicity_adjustment_rule— the step-up cutoff on the ranked p-values is the adjustment: it is how the significance line bends to the size and signal of the family.error_risk_profile— choosing q is a precise statement of which error matters — tolerating some false positives to avoid missing real effects — the discovery-oriented risk stance.result_status_label— the procedure stamps each test discovery or not, carrying the explicit, set-level guarantee that attaches to that label.
It does not guarantee against even a single false positive across the counted family; that strict familywise protection — dividing by the claim_family's multiplicity_inventory — is Bonferroni-Like Correction's stance, and it is why Bonferroni is far less powerful on a large screen.
Related¶
- Instantiates: Multiple-Testing Discipline — supplies the discovery-rate form of multiplicity control.
- Sibling mechanisms: Alpha-Spending Plan · Bonferroni-Like Correction · Claim Registry · Confirmatory Follow-Up · Holdout Validation · Metric Hierarchy · Multiverse Analysis Report · Preregistration · Replication Study
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: False Discovery Rate Control operates as a computation, comparison, model, or analytic representation used to infer, estimate, or choose because it ranks a whole family of results and draws the significance line to hold the expected share of false discoveries below a chosen rate, trading a little purity for far more power.
Independent corroboration: The frozen evidence defines False Discovery Rate Control as 'Ranks a whole family of results and draws the significance line to hold the expected share of false discoveries below a chosen rate, trading a little purity for far more power', so its operative form is Analysis, Modeling & Optimization.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Specialized
Rationale: False discovery rate control is a canonical statistical multiple-testing procedure formalized by Benjamini and Hochberg.
Review outcome: Independent reviewer agreement; high confidence.
Notes¶
[n1] The Benjamini–Hochberg procedure controls the false discovery rate — the expected proportion of false positives among rejected hypotheses — by comparing ranked p-values against a rank-dependent line; it is uniformly more powerful than familywise correction when the family is large and some effects are real. ↩