Controlling the False Discovery Rate¶
Benjamini, Y., & Hochberg, Y. (1995). Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society: Series B, 57(1), 289-300.
Cited by¶
12 citations across 12 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Clustering Illusion
- Genomics confronts the same structure at massive scale: a genome-wide scan tests hundreds of thousands of variants for association with a trait, and because so many candidates are examined, the sample-size-perversity invariant guarantees that some will exceed any naive threshold by noise; the corrective is multiple-testing correction (Bonferroni or false-discovery-rate control), which is the build-the-null-and-compare discipline applied to the maximum over many comparisons.
This sourceIntroduces false-discovery-rate control for multiple testing — the build-the-null-and-compare discipline applied to maxima over many candidates (genome-wide scans, motif finding).
Supported in partVerified against the publisher's abstract
Its abstract introduces FDR control for multiple testing, including a sequential Bonferroni-type procedure — the corrective half of the claim — but says nothing of genome-wide scans.
“A different approach to problems of multiple significance testing is presented. It calls for controlling the expected proportion of falsely rejected hypotheses — the false discovery rate. ... A simple sequential Bonferronitype procedure is proved to control the false discovery rate for independent test statistics”
- Genomics confronts the same structure at massive scale: a genome-wide scan tests hundreds of thousands of variants for association with a trait, and because so many candidates are examined, the sample-size-perversity invariant guarantees that some will exceed any naive threshold by noise; the corrective is multiple-testing correction (Bonferroni or false-discovery-rate control), which is the build-the-null-and-compare discipline applied to the maximum over many comparisons.
- Effect Size
- Listed in the references but not attached to a specific claim.
- Multiple Comparisons Correction
- The distinguishing commitment is that inferential conclusions must account for the full testing context and family structure, not just a single test's p-value in isolation
This sourceBenjamini Hochberg false discovery rate FDR control multiple testing procedure step-up.
Supported in partVerified against the publisher's abstract
The abstract frames the multiplicity problem and error control over a family of tests (FWER, then FDR), which backs the frame of the claim but not its general normative wording about all inferential conclusions.
“The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach to problems of multiple significance testing is presented. It calls for controlling the expected proportion of falsely rejected hypotheses — the false discovery rate.”
- The distinguishing commitment is that inferential conclusions must account for the full testing context and family structure, not just a single test's p-value in isolation
- Texas Sharpshooter Fallacy
- Suppose a researcher tests \(m = 1000\) independent associations (say, 1000 candidate genes against a trait), each at significance level \(\alpha = 0.05\).
This sourceIntroduces false-discovery-rate control for the multiple-comparisons problem; underpins the multiplicity correction in large-scale (e.g. genome-wide) testing.
Supported in partVerified against the publisher's abstract
Benjamini–Hochberg supplies the multiple-testing frame — many independent tests, no true effects, false rejections — but states no count of expected false positives for 1000 tests at α = 0.05.
“It calls for controlling the expected proportion of falsely rejected hypotheses — the false discovery rate. This error rate is equivalent to the FWER when all hypotheses are true but is smaller otherwise.”
- Suppose a researcher tests \(m = 1000\) independent associations (say, 1000 candidate genes against a trait), each at significance level \(\alpha = 0.05\).
Domain-specific¶
- Benjamini–Hochberg Procedure
- Under independence of the test statistics (or positive regression dependence), the resulting FDR is bounded by α·m₀/m ≤ α
This sourceAbstract reporting that a simple sequential Bonferroni-type procedure controls the false discovery rate for independent test statistics. The paper that presents and proves the step-up procedure controlling the false discovery rate (Benjamini and Hochberg, 1995).
Supported in partVerified against the publisher's abstract
“A simple sequential Bonferronitype procedure is proved to control the false discovery rate for independent test statistics”
- Under independence of the test statistics (or positive regression dependence), the resulting FDR is bounded by α·m₀/m ≤ α
- False Discovery Rate
- False discovery rate is the expected proportion of false rejections among all rejected hypotheses, conventionally \(E[V/\max(R,1)]\)
SupportedVerified against the source
- False discovery rate is the expected proportion of false rejections among all rejected hypotheses, conventionally \(E[V/\max(R,1)]\)
Mechanisms¶
- Conditional-Independence Test Suite
- The suite chooses the right test for each pairing (partial correlation for the roughly-linear continuous signals, a conditional mutual-information test where nonlinearity is suspected) and corrects across the many tests it ran.
This sourceIntroduces a procedure that controls the false discovery rate across a family of multiple significance tests.
- The suite chooses the right test for each pairing (partial correlation for the roughly-linear continuous signals, a conditional mutual-information test where nonlinearity is suspected) and corrects across the many tests it ran.
- Cross-Validated Error-Slice Report
- Its central trap is multiplicity: scan enough slices and one will look bad by pure chance, so an uncorrected report reliably manufactures a false villain.
This sourceExplains that unguarded testing across many comparisons increases false discoveries, motivating explicit control of the false discovery rate.
- Its central trap is multiplicity: scan enough slices and one will look bad by pure chance, so an uncorrected report reliably manufactures a false villain.
- Motif Enrichment Table
- Correct for multiplicity. Because many motifs are tested together, adjust p-values or control the false-discovery rate; an uncorrected table manufactures false positives by sheer count of tests.
This sourceEstablishes false-discovery-rate control for families of simultaneous tests, limiting the false positives that accumulate across many comparisons.
- Correct for multiplicity. Because many motifs are tested together, adjust p-values or control the false-discovery rate; an uncorrected table manufactures false positives by sheer count of tests.
- Multiple-Testing Review
- Permutation Null and Multiplicity Check
- The former controls any false positive and sacrifices power; the latter tolerates a controlled fraction of them and keeps more discoveries.
This sourceContrasts familywise control, which seeks to avoid any false rejection at a power cost, with false-discovery-rate control, which bounds the expected fraction of false rejections and permits more discoveries.
- The former controls any false positive and sacrifices power; the latter tolerates a controlled fraction of them and keeps more discoveries.
- Standardized Residual Score
- Standardize thousands of units and, by chance alone, some will post |z| > 3 with nothing real behind them — the multiple comparisons problem — so a large score is not self-certifying evidence.
This sourceDefines the multiple-testing problem and controls the expected proportion of falsely rejected null hypotheses through the false discovery rate.
- Standardize thousands of units and, by chance alone, some will post |z| > 3 with nothing real behind them — the multiple comparisons problem — so a large score is not self-certifying evidence.
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 5 of 12 citations: 1 supported, 4 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Was it audited? Yes. A second, independent pass read the citation against the article text and recorded a verdict.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:400722d0db02 · see in the full table