Effect Size¶

Prime #: 447
Origin domain: Statistics & Experimental Design
Aliases: Cohens D, Standardized Mean Difference, Practical Significance, Magnitude of Effect, Effect Estimate
Related primes: Statistical Power, Statistical Significance (p-Value), Confidence Intervals, Hypothesis Testing (Null vs. Alternative), Type I & Type II Errors, Reproducibility & Replicability, Regression to the Mean

Core Idea¶

Effect Size quantifies the magnitude of a relationship or difference—e.g., using Cohen's d, correlation coefficients, or odds ratios—beyond simple statistical significance, illuminating the practical importance of an observed effect.

How would you explain it like I'm…

How Big Is It

If two kids race, asking who won is one question. Asking by how much one beat the other, like one step or one whole block, is a different question. Knowing the size of the gap tells you way more than just knowing there was a gap. That bigger picture matters.

Size Of The Difference

When scientists test something, they often ask two different questions. First: 'Is there any effect at all?' Second: 'How big is the effect?' The second question is called effect size. A medicine might really lower blood pressure, but only by a tiny amount that no patient would notice. Effect size tells you the size of the change in real-world units, so you can decide if it actually matters for your life, not just whether a study counts it as 'significant.'

Measuring Magnitude, Not Just Yes/No

Effect size is a number that tells you the magnitude of a difference or relationship in units you can interpret. It is separate from statistical significance, which only tells you whether an effect probably exists. With a huge sample, even a tiny effect can be 'statistically significant'; with a tiny sample, even a big effect can fail that test. So significance alone is a poor guide to whether something matters in practice. To draw a real conclusion you need three pieces together: the size of the effect, the uncertainty around it (a confidence interval), and the direction. Collapsing all that into a yes/no verdict throws away the information you actually need.

Effect size quantifies the magnitude of an observed relationship or difference in substantive, interpretable units, independently of sample size and separately from the question of statistical significance. The conceptual move is from a *dichotomous* hypothesis test (does the effect differ from zero?) to a *continuous* estimation question (how large is the effect, and how precisely have we estimated it?). This matters because the null-hypothesis significance test (NHST) is sample-size dependent: in a large enough study, even a trivially small effect crosses the significance threshold, while a substantively important effect can fail to reach significance in a small study. Reporting standardized effect sizes (Cohen's d, Pearson's r, odds ratios, eta-squared) together with confidence intervals or posterior distributions allows readers to evaluate practical importance, perform meta-analytic synthesis across studies, and conduct power analyses for replication. The discipline is to report magnitude, uncertainty, and direction jointly rather than collapse them into a single reject/do-not-reject verdict.

Broad Use¶

Psychology: Cohen's d or r-values measure how strongly an intervention or trait is associated with changes in behavior or cognition.
Medical Research: Odds ratios or relative risks show how much a treatment raises or lowers disease incidence compared to control.
Education: An effect size for a new teaching method clarifies whether the improvement in test scores is modest or substantial.
Marketing: Gauging the "real impact" on sales or click-through rates rather than only declaring results "statistically significant."

Clarity¶

Statistical significance alone doesn't tell how big an effect is—effect size ensures interpretation of results in terms of real-world relevance or practical difference.

Manages Complexity¶

Distilling data into a numeric measure of impact (rather than just p-values) prevents misinterpretations where very large sample sizes detect tiny, insignificant differences as "significant."

Abstract Reasoning¶

Underscores that measuring how large a difference or correlation is can be more vital than verifying its non-zero nature, a universal principle for focusing on meaningful changes across fields.

Knowledge Transfer¶

Software A/B Testing: Reporting a 5% improvement in user engagement (effect size) is more interpretable than "p \< 0.05."
Organizational Behavior: Summarizing how an HR policy change affects morale or retention with a standardized effect measure fosters consistent cross-study comparisons.

Example¶

A meta-analysis might reveal that an intervention consistently yields moderate effect sizes (e.g., Cohen's d ≈ 0.5), implying a meaningful but not revolutionary impact across studies.

Relationships to Other Primes¶

Parents (2) — more general patterns this builds on

Effect Size presupposes Comparison — Effect size presupposes comparison because magnitude is read off the relation between two or more co-considered quantities.
Effect Size presupposes Scale — Effect size presupposes scale because it quantifies the magnitude of an observed relationship in substantive units of measurement.

Path to root: Effect Size → Comparison

Not to Be Confused With¶

Effect Size quantifies the magnitude of a relationship or treatment effect in standardized, interpretable units. Proportion and Scale concern visual or relational sizing and ratios. One is a statistical magnitude measure, the other is a compositional relationship.
Effect Size specifies the magnitude of a measured phenomenon in substantive units. Scale specifies the ontological band at which a system is described and the fact that laws differ across bands. One is about effect magnitude, the other about system description level.
Effect Size quantifies the magnitude of a relationship. Statistical Significance quantifies whether the relationship is distinguishable from zero. They are orthogonal questions; significance can be high with trivial effect size, or vice versa.
Effect Size measures the magnitude of a difference or relationship abstracted from context. Dose-Response Relationship maps the quantitative input-output function across a range, characterizing the functional form and shape. One is a magnitude scalar, the other is a curve.