Statistical tests, P values, confidence intervals, and power¶
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016). Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. European Journal of Epidemiology, 31(4), 337-350.
Cited by¶
7 citations across 7 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Confidence Intervals
- Immature practice reports CIs mechanically (CI just because it is required) without assessing model adequacy, misinterprets the realized interval as having 1−α probability of containing the parameter, or treats CIs as superfluous decoration alongside the "real" analysis (p-value or point estimate alone)
This sourceCatalogs and corrects common misinterpretations, including the error of treating a realized interval as having 1-alpha probability of containing the parameter. DOI live-confirmed to resolve this session.
Supported in partVerified against the publisher's abstract
Greenland et al.'s abstract shows CI misinterpretation is pervasive and that the paper lists 25 misinterpretations, backing this work's place in the cluster but not the specific realized-interval 1-alpha error.
“We then provide an explanatory list of 25 misinterpretations of P values, confidence intervals, and power.”
- Immature practice reports CIs mechanically (CI just because it is required) without assessing model adequacy, misinterprets the realized interval as having 1−α probability of containing the parameter, or treats CIs as superfluous decoration alongside the "real" analysis (p-value or point estimate alone)
- Distributional Assumption
- This clarity distinguishes between defensible assumptions (supported by prior domain knowledge, physical principles, or extensive historical data) and convenient assumptions (chosen because they are mathematically tractable or computationally fast)—a distinction Greenland et al. (2016) operationalize in their guide to misinterpretation of statistical tests and confidence intervals.
Supported in partVerified against the publisher's abstract
The abstract confirms only that Greenland et al. (2016) is a guide to misinterpretations of P values, confidence intervals and power; it states no distinction between principled and convenience-driven assumptions.
“We then provide an explanatory list of 25 misinterpretations of P values, confidence intervals, and power.”
- This clarity distinguishes between defensible assumptions (supported by prior domain knowledge, physical principles, or extensive historical data) and convenient assumptions (chosen because they are mathematically tractable or computationally fast)—a distinction Greenland et al. (2016) operationalize in their guide to misinterpretation of statistical tests and confidence intervals.
- Effect Size
This sourceReal and authoritative, but it documents 25 misinterpretations of P/CI/power — it does NOT cover Type M/S errors. Demoted from marker 072 to.
- Hypothesis Testing (Null vs. Alternative)
- Without the frame, people treat statistical evidence as binary (significant/not), conflate statistical with practical significance, misinterpret p-values as posterior probabilities, and select hypotheses after seeing data
Supported in partVerified against the publisher's abstract
The publisher abstract documents an epidemic of wrong shortcut interpretations of P values and a 25-item list of misinterpretations, but does not itself state the four specific errors the claim enumerates.
“This high cognitive demand has led to an epidemic of shortcut definitions and interpretations that are simply wrong, sometimes disastrously so-and yet these misinterpretations dominate much of the scientific literature.”
- Without the frame, people treat statistical evidence as binary (significant/not), conflate statistical with practical significance, misinterpret p-values as posterior probabilities, and select hypotheses after seeing data
- Statistical Power
- Without the frame, people run studies whose sample sizes are determined by convenience or convention without regard for what effects they can detect, interpret non-significant results as evidence of no effect regardless of achieved power, and conflate "observed power" with independent diagnostic information
Supported in partVerified against the publisher's abstract
The work's own abstract attests that misuse of statistical power remains rampant and that it catalogues 25 misinterpretations of P values, confidence intervals and power; it names neither convenience sampling nor 'observed power'.
“Misinterpretation and abuse of statistical tests, confidence intervals, and statistical power have been decried for decades, yet remain rampant.”
- Without the frame, people run studies whose sample sizes are determined by convenience or convention without regard for what effects they can detect, interpret non-significant results as evidence of no effect regardless of achieved power, and conflate "observed power" with independent diagnostic information
- Statistical Significance (p-Value)
- It is a continuous summary of sample evidence's incompatibility with the null, ranging from 0 (data impossible under null) to 1 (data exactly typical under null)
Supported in partVerified against the publisher's abstract
The abstract announces an explanatory list of 25 misinterpretations of P values, supporting the claim's frame but not the four confusions it enumerates.
“We then provide an explanatory list of 25 misinterpretations of P values, confidence intervals, and power.”
- It is a continuous summary of sample evidence's incompatibility with the null, ranging from 0 (data impossible under null) to 1 (data exactly typical under null)
- Type I & Type II Errors
- The tension is between the simplicity of individual-test error control and the reality that most studies effectively conduct many tests, requiring multiplicity-adjusted reasoning
Supported in partVerified against the publisher's abstract
Its abstract shows unstated analysis protocols, such as selecting analyses by their P values, yield misleading P values, but never says most studies run many tests or that multiplicity adjustment is required.
“We emphasize how violation of often unstated analysis protocols (such as selecting analyses for presentation based on the P values they produce) can lead to small P values even if the declared test hypothesis is correct, and can lead to large P values even if that hypothesis is incorrect.”
- The tension is between the simplicity of individual-test error control and the reality that most studies effectively conduct many tests, requiring multiplicity-adjusted reasoning
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 6 of 7 citations: 6 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:e91360457d43 · see in the full table