Type M Error¶
Quantify how much a significant effect's reported magnitude is exaggerated by the significance filter under low power, via the exaggeration ratio — the expected significant estimate divided by the true effect — computable from the design before any data exist.
Core Idea¶
A Type M (magnitude) error is the systematic exaggeration of a statistically significant effect's size that results from filtering estimates through a significance threshold under low power. When a study is underpowered, only estimates far out in the sampling distribution's tail clear the threshold, so the distribution of reported effects given significance is shifted upward from the truth. Gelman and Carlin (2014) formalized the exaggeration ratio — the expected significant estimate divided by the true effect — computable prospectively from the design.
Scope of Application¶
Type M error lives across the significance-testing sciences that share the substrate of estimation from a significance-filtered sample; its reach is within that one statistical domain.
- Underpowered psychology, economics, biomedicine — surprising effects that shrink dramatically on higher-powered replication.
- Early-phase clinical trials — effects clearing p < 0.05 that Phase III systematically fails to recover.
- Genome-wide association studies — winner's-curse shrinkage of top-hit effect sizes between discovery and replication.
- A/B testing — stopped-on-significance experiments exaggerating lift.
- Meta-science — PET-PEESE and related methods built to correct this magnitude inflation.
Clarity¶
Naming Type M error breaks the equation the Neyman-Pearson frame quietly encourages: that a significant result is a roughly-right estimate. Type I and II error treat the decision as the unit; Type M shifts the unit to the number itself, conditional on rejection. Its load-bearing service is to make the exaggeration ratio a concrete, prospective quantity, localizing the problem in the selection structure imposed by the significance filter under low power rather than in the estimator or the experimenter's honesty.
Manages Complexity¶
The reasons a reported effect might overstate the truth accumulate into a sprawling catalog — winner's curse in GWAS, early-trial over-statement, stopped-on-significance A/B lift, publication-bias corrections. Type M compresses the whole catalog onto one mechanism and one prospective scalar. The analyst tracks two design inputs — an assumed true effect and the study's power — from which the ratio reads off, with a fixed low-power-versus-adequate fork deciding whether a significant magnitude is trustworthy.
Abstract Reasoning¶
The signature move is conditioning on the filter — reasoning about the distribution of estimates given they cleared significance, reading a headline number as a selected order statistic. This is made prospective and quantitative by the exaggeration-ratio move (from power alone to expected inflation), a boundary-drawing move that re-chooses the unit of analysis, and a retrospective diagnostic (systematic shrinkage from study to higher-powered replication as the observable signature) that needs no appeal to misconduct.
Knowledge Transfer¶
Within statistics and the significance-testing sciences the whole apparatus transfers literally — the condition-on-the-filter move, the prospective exaggeration ratio, the unit-of-analysis reframe, and the replication-shrinkage signature recur identically across underpowered psychology, clinical trials, GWAS, A/B testing, and meta-science, because the substrate (a noisy estimator filtered by a significance threshold under specified power) is held fixed. Beyond that scaffolding, Type M is the statistical-inference face of the winner_s_curse: the portable pattern — the extremum of noisy estimates, selected by a one-sided gate, overstates the truth — is carried by winner_s_curse and the broader selection_bias family, not by "Type M," whose contribution is operationalising that pattern for the NHST significance filter.
Relationships to Other Abstractions¶
Current abstraction Type M Error Domain-specific
Parents (5) — more general patterns this builds on
-
Type M Error is a kind of Selection on Noisy Estimates Prime
Type M is the significance-threshold species in which selection inflates a reported effect magnitude.
-
Type M Error is part of Effect Size Prime
Type M Error contains observed and assumed true Effect Sizes whose magnitudes form the selected estimate and comparison target.
-
Type M Error is part of Ratio Prime
Type M contains an exaggeration ratio whose numerator is expected significant effect magnitude and whose denominator is true effect magnitude.
-
Type M Error is part of Statistical Power Prime
Type M Error contains the design's Statistical Power as the regime variable governing how far admitted estimates must lie from the truth.
-
Type M Error is part of Statistical Significance (p-Value) Prime
Type M Error contains a statistical-significance gate that selects the tail estimates whose conditional magnitude is evaluated.
Hierarchy paths (24) — routes to 8 parentless roots
- Type M Error → Selection on Noisy Estimates → Selection Bias → Bias
- Type M Error → Effect Size → Scale
- Type M Error → Statistical Significance (p-Value) → Statistical Inference → Inductive Reasoning
- Type M Error → Effect Size → Comparison → Self Checking
- Type M Error → Ratio → Comparison → Self Checking
- Type M Error → Statistical Significance (p-Value) → Statistical Inference → Uncertainty
- Type M Error → Selection on Noisy Estimates → Selection Bias → Statistical Inference → Inductive Reasoning
- Type M Error → Statistical Significance (p-Value) → Hypothesis Testing (Null vs. Alternative) → Statistical Inference → Inductive Reasoning
- Type M Error → Statistical Power → Experimental Design → Comparison → Self Checking
- Type M Error → Statistical Power → Probability → Measure → Set and Membership
- Type M Error → Statistical Significance (p-Value) → Probability → Measure → Set and Membership
- Type M Error → Selection on Noisy Estimates → Selection Bias → Statistical Inference → Uncertainty
- Type M Error → Statistical Significance (p-Value) → Hypothesis Testing (Null vs. Alternative) → Statistical Inference → Uncertainty
- Type M Error → Selection on Noisy Estimates → Selection Bias → Vantage-Induced Omission → Viewpoint
- Type M Error → Statistical Power → Probability → Measure → Aggregation → Micro Macro Linkage
- Type M Error → Statistical Significance (p-Value) → Probability → Measure → Aggregation → Micro Macro Linkage
- Type M Error → Statistical Power → Experimental Design → Control Sample → Comparison → Self Checking
- Type M Error → Statistical Significance (p-Value) → Statistical Inference → Probability → Measure → Set and Membership
- Type M Error → Statistical Significance (p-Value) → Statistical Inference → Probability → Measure → Aggregation → Micro Macro Linkage
- Type M Error → Statistical Significance (p-Value) → Hypothesis Testing (Null vs. Alternative) → Verification → Evaluation → Comparison → Self Checking
- Type M Error → Selection on Noisy Estimates → Selection Bias → Statistical Inference → Probability → Measure → Set and Membership
- Type M Error → Statistical Significance (p-Value) → Hypothesis Testing (Null vs. Alternative) → Statistical Inference → Probability → Measure → Set and Membership
- Type M Error → Selection on Noisy Estimates → Selection Bias → Statistical Inference → Probability → Measure → Aggregation → Micro Macro Linkage
- Type M Error → Statistical Significance (p-Value) → Hypothesis Testing (Null vs. Alternative) → Statistical Inference → Probability → Measure → Aggregation → Micro Macro Linkage
Neighborhood in Abstraction Space¶
Type M Error sits in a moderately populated region (55th percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.
Family — Unclustered & Miscellaneous (309 abstractions)
Nearest neighbors
- Type S Error — 0.89
- Funnel Plot Asymmetry — 0.85
- File Drawer Problem — 0.85
- Small-Study Effects — 0.83
- Jeffreys-Lindley Paradox — 0.82
Computed from structural-signature embeddings · 2026-07-12