Type M Error¶

Quantify how much a significant effect's reported magnitude is exaggerated by the significance filter under low power, via the exaggeration ratio — the expected significant estimate divided by the true effect — computable from the design before any data exist.

Version: v3 · 2026-09-28 · History · 4 corrections
Domain-specific #: 1147
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomain: Estimation Under Selection → Experimental Design & Statistics

Core Idea¶

A Type M (magnitude) error is the systematic exaggeration of a statistically significant effect's size that results from filtering estimates through a significance threshold under low power. When a study is underpowered, only estimates far out in the sampling distribution's tail clear the threshold, so the distribution of reported effects given significance is shifted upward from the truth. Gelman and Carlin (2014) formalized the exaggeration ratio — the expected significant estimate divided by the true effect — computable prospectively from the design.

Scope of Application¶

Type M error lives across the significance-testing sciences that share the substrate of estimation from a significance-filtered sample; its reach is within that one statistical domain.

Underpowered psychology, economics, biomedicine — surprising effects that shrink dramatically on higher-powered replication.
Early-phase clinical trials — effects clearing p < 0.05 that Phase III systematically fails to recover.
Genome-wide association studies — winner's-curse shrinkage of top-hit effect sizes between discovery and replication.
A/B testing — stopped-on-significance experiments exaggerating lift.
Meta-science — PET-PEESE and related methods built to correct this magnitude inflation.

Clarity¶

Naming Type M error breaks the equation the Neyman-Pearson frame quietly encourages: that a significant result is a roughly-right estimate. Type I and II error treat the Decision as the unit; Type M shifts the unit to the number itself, conditional on rejection. Its load-bearing service is to make the exaggeration ratio a concrete, prospective quantity, localizing the problem in the selection structure imposed by the significance filter under low power rather than in the estimator or the experimenter's honesty.

Manages Complexity¶

The reasons a reported effect might overstate the truth accumulate into a sprawling catalog — winner's curse in GWAS, early-trial over-statement, stopped-on-significance A/B lift, publication-bias corrections. Type M compresses the whole catalog onto one mechanism and one prospective scalar. The analyst tracks two design inputs — an assumed true effect and the study's power — from which the ratio reads off, with a fixed low-power-versus-adequate fork deciding whether a significant magnitude is trustworthy.

Abstract Reasoning¶

The signature move is conditioning on the filter — reasoning about the distribution of estimates given they cleared significance, reading a headline number as a selected order statistic. This is made prospective and quantitative by the exaggeration-ratio move (from power alone to expected inflation), a boundary-drawing move that re-chooses the unit of analysis, and a retrospective diagnostic (systematic shrinkage from study to higher-powered replication as the observable signature) that needs no appeal to misconduct.

Knowledge Transfer¶

Within statistics and the significance-testing sciences the whole apparatus transfers literally — the condition-on-the-filter move, the prospective exaggeration ratio, the unit-of-analysis reframe, and the replication-shrinkage signature recur identically across underpowered psychology, clinical trials, GWAS, A/B testing, and meta-science, because the substrate (a noisy estimator filtered by a significance threshold under specified power) is held fixed. Beyond that scaffolding, Type M is the statistical-inference Face of its parent Selection on Noisy Estimates, as Winner's Curse is its auction face: the portable pattern — the extremum of noisy estimates, selected by a one-sided gate, overstates the truth — is carried by Selection on Noisy Estimates (with its auction instance Winner's Curse and the broader Selection Bias family), not by "Type M," whose contribution is operationalising that pattern for the NHST significance filter.

Relationships to Other Abstractions¶

Current abstraction Type M Error Domain-specific

Parents (5) — outgoing typed relations

Type M Error is a kind of Selection on Noisy Estimates Prime

Type M is the significance-threshold species in which selection inflates a reported effect magnitude.
Type M Error includes Effect Size Prime

Type M Error contains observed and assumed true Effect Sizes whose magnitudes form the selected estimate and comparison target.
Type M Error includes Ratio Prime

Type M contains an exaggeration ratio whose numerator is expected significant effect magnitude and whose denominator is true effect magnitude.
Type M Error includes Statistical Power Prime

Type M Error contains the design's Statistical Power as the regime variable governing how far admitted estimates must lie from the truth.
Type M Error includes Statistical Significance (p-Value) Prime

Type M Error contains a statistical-significance gate that selects the tail estimates whose conditional magnitude is evaluated.

Hierarchy paths (19) — routes to 7 parentless roots

Type M Error → is a kind of Selection on Noisy Estimates → is a kind of Selection Bias → is a kind of Bias

Show alternative paths (18)

Neighborhood in Abstraction Space¶

Type M Error sits in a sparse region of the domain-specific corpus (67^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Causal Inference & Identification Design (7 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25