Type S Error¶
Quantify the risk that a statistically significant estimate points the wrong way by computing, before data collection, the probability that a two-sided significance filter is cleared from the opposite tail when the true effect is near zero relative to noise.
Core Idea¶
A Type S (sign) error is the failure mode in which a statistically significant estimate has the wrong sign. When a study's power is low relative to the true effect, the significance threshold is cleared almost only by large draws, and when the true effect is near zero relative to sampling noise both tails reach significance with comparable probability. Gelman and Carlin's prospective diagnostic, Pr(sign correct | significant), asks: given that I will only act on a significant result, how likely is its direction to be right?
Scope of Application¶
Requires a two-sided significance filter, a p-value, and a power calculation — the significance-testing sciences with true effects near zero relative to noise.
- Behavioural-science replication — priming and embodied-cognition reversals against high-powered replications.
- Health-policy A/B tests — underpowered studies reporting the opposite direction of the truth.
- Genome-wide association studies — top discovery-stage SNP hits flipping sign at replication.
- Election-poll subgroups — small-cell swings reversing direction across consecutive surveys.
- Exploratory contrasts — interaction terms and subgroup comparisons read off noisy cells.
Clarity¶
The construct punctures the comforting fallback "at least the direction is right," showing this is exactly the assumption that fails under low power. It separates two questions the classical frame fuses — "is there an effect?" versus "is the direction of a significant result trustworthy?" — and converts an apparently safe positive claim into one with a computable failure probability available at design time.
Manages Complexity¶
Scattered direction-of-effect anxieties across interactions, subgroups, and discovery-stage hits collapse onto one mechanism and one prospective scalar. What the analyst tracks reduces to a single ratio — true effect size relative to sampling noise — because that alone sets how the two tails contribute significant draws. A near-zero-versus-far-from-zero fork then reads the verdict off, prescribing more power rather than better magnitude measurement.
Abstract Reasoning¶
The defining move conditions the sign on the filter: not "is the direction right?" but "given I only report significance, how often does it point wrong?" A boundary-drawing move locates the regime where the "at least the sign is right" comfort fails, fixing the remedy by diagnosis. A retrospective move reads replication sign-reversals as materialized Type S risk, not scandal — computed jointly with its magnitude sibling, Type M.
Knowledge Transfer¶
Within the significance-testing sciences the whole apparatus transfers literally — the two-tail geometry, the prospective Pr(wrong sign), the power-not-precision remedy — because the substrate of a noisy estimator near zero under a two-sided filter is held fixed. Beyond that scaffolding the portable content is the parent Selection on Noisy Estimates (shared with its sibling Winner's Curse, within the Selection Bias family): selection on a noisy estimator near zero can flip the sign. Type S operationalises that flip for the frequentist significance filter.
Relationships to Other Abstractions¶
Current abstraction Type S Error Domain-specific
Parents (4) — outgoing typed relations
-
Type S Error is a kind of Selection on Noisy Estimates Prime
Type S is the near-zero two-sided-threshold species in which the selected estimate can enter from the tail opposite the true effect and reverse its sign.
-
Type S Error includes Effect Size Prime
Type S Error contains an estimated and true Effect Size whose signed directions are compared after significance selection.
-
Type S Error includes Statistical Power Prime
Type S Error contains Statistical Power because the effect-to-noise detection regime determines whether the opposite tail remains reachable.
-
Type S Error includes Statistical Significance (p-Value) Prime
Type S Error contains a two-sided statistical-significance gate whose opposite-tail admissions can carry the wrong sign.
Hierarchy paths (18) — routes to 7 parentless roots
- Type S Error → is a kind of Selection on Noisy Estimates → is a kind of Selection Bias → is a kind of Bias
- Type S Error → includes Effect Size → presupposes Comparison
- Type S Error → includes Effect Size → presupposes Scale
- Type S Error → includes Statistical Power → presupposes Experimental Design → is a decomposition of Comparison
- Type S Error → includes Statistical Significance (p-Value) → is a kind of Statistical Inference → is a kind of Inductive Reasoning
- Type S Error → includes Statistical Significance (p-Value) → is a kind of Statistical Inference → presupposes Uncertainty
- Type S Error → includes Statistical Power → presupposes Experimental Design → includes, typical Control Sample → presupposes Comparison
- Type S Error → is a kind of Selection on Noisy Estimates → is a kind of Selection Bias → presupposes Statistical Inference → is a kind of Inductive Reasoning
- Type S Error → includes Statistical Significance (p-Value) → presupposes Hypothesis Testing (Null vs. Alternative) → is a kind of Statistical Inference → is a kind of Inductive Reasoning
- Type S Error → includes Statistical Power → presupposes Probability → is a kind of Measure → presupposes Set and Membership
- Type S Error → includes Statistical Significance (p-Value) → presupposes Probability → is a kind of Measure → presupposes Set and Membership
- Type S Error → is a kind of Selection on Noisy Estimates → is a kind of Selection Bias → presupposes Statistical Inference → presupposes Uncertainty
- Type S Error → includes Statistical Significance (p-Value) → presupposes Hypothesis Testing (Null vs. Alternative) → is a kind of Statistical Inference → presupposes Uncertainty
- Type S Error → is a kind of Selection on Noisy Estimates → is a kind of Selection Bias → is a kind of Vantage-Induced Omission → presupposes Viewpoint
- Type S Error → includes Statistical Significance (p-Value) → presupposes Hypothesis Testing (Null vs. Alternative) → is a kind of Verification → is a kind of Evaluation → presupposes Comparison
- Type S Error → includes Statistical Significance (p-Value) → is a kind of Statistical Inference → presupposes Probability → is a kind of Measure → presupposes Set and Membership
- Type S Error → is a kind of Selection on Noisy Estimates → is a kind of Selection Bias → presupposes Statistical Inference → presupposes Probability → is a kind of Measure → presupposes Set and Membership
- Type S Error → includes Statistical Significance (p-Value) → presupposes Hypothesis Testing (Null vs. Alternative) → is a kind of Statistical Inference → presupposes Probability → is a kind of Measure → presupposes Set and Membership
Neighborhood in Abstraction Space¶
Type S Error sits in a sparse region of the domain-specific corpus (74th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.
Family — Unclustered & Miscellaneous (2769 abstractions)
Nearest neighbors
- Type M Error — 0.89
- Quadrant Count Ratio — 0.84
- Underfitting — 0.82
- Jeffreys-Lindley Paradox — 0.82
- Attenuation Bias — 0.82
Computed from structural-signature embeddings · 2026-10-25