Skip to content

Effect Size

Prime #
447
Origin domain
Statistics & Experimental Design
Aliases
Cohens D, Standardized Mean Difference, Practical Significance, Magnitude of Effect, Effect Estimate
Related primes
Statistical Power, Statistical Significance (p-Value), Confidence Intervals, Hypothesis Testing (Null vs. Alternative), Type I & Type II Errors, Reproducibility & Replicability, Regression to the Mean

Core Idea

Effect Size quantifies the magnitude of a relationship or difference—e.g., using Cohen's d, correlation coefficients, or odds ratios—beyond simple statistical significance, illuminating the practical importance of an observed effect.

How would you explain it like I'm…

How Big Is It

If two kids race, asking who won is one question. Asking by how much one beat the other, like one step or one whole block, is a different question. Knowing the size of the gap tells you way more than just knowing there was a gap. That bigger picture matters.

Size Of The Difference

When scientists test something, they often ask two different questions. First: 'Is there any effect at all?' Second: 'How big is the effect?' The second question is called effect size. A medicine might really lower blood pressure, but only by a tiny amount that no patient would notice. Effect size tells you the size of the change in real-world units, so you can decide if it actually matters for your life, not just whether a study counts it as 'significant.'

Measuring Magnitude, Not Just Yes/No

Effect size is a number that tells you the magnitude of a difference or relationship in units you can interpret. It is separate from statistical significance, which only tells you whether an effect probably exists. With a huge sample, even a tiny effect can be 'statistically significant'; with a tiny sample, even a big effect can fail that test. So significance alone is a poor guide to whether something matters in practice. To draw a real conclusion you need three pieces together: the size of the effect, the uncertainty around it (a confidence interval), and the direction. Collapsing all that into a yes/no verdict throws away the information you actually need.

 

Effect size quantifies the magnitude of an observed relationship or difference in substantive, interpretable units, independently of sample size and separately from the question of statistical significance. The conceptual move is from a *dichotomous* hypothesis test (does the effect differ from zero?) to a *continuous* estimation question (how large is the effect, and how precisely have we estimated it?). This matters because the null-hypothesis significance test (NHST) is sample-size dependent: in a large enough study, even a trivially small effect crosses the significance threshold, while a substantively important effect can fail to reach significance in a small study. Reporting standardized effect sizes (Cohen's d, Pearson's r, odds ratios, eta-squared) together with confidence intervals or posterior distributions allows readers to evaluate practical importance, perform meta-analytic synthesis across studies, and conduct power analyses for replication. The discipline is to report magnitude, uncertainty, and direction jointly rather than collapse them into a single reject/do-not-reject verdict.

Broad Use

  • Psychology: Cohen's d or r-values measure how strongly an intervention or trait is associated with changes in behavior or cognition.

  • Medical Research: Odds ratios or relative risks show how much a treatment raises or lowers disease incidence compared to control.

  • Education: An effect size for a new teaching method clarifies whether the improvement in test scores is modest or substantial.

  • Marketing: Gauging the "real impact" on sales or click-through rates rather than only declaring results "statistically significant."

Clarity

Statistical significance alone doesn't tell how big an effect is—effect size ensures interpretation of results in terms of real-world relevance or practical difference.

Manages Complexity

Distilling data into a numeric measure of impact (rather than just p-values) prevents misinterpretations where very large sample sizes detect tiny, insignificant differences as "significant."

Abstract Reasoning

Underscores that measuring how large a difference or correlation is can be more vital than verifying its non-zero nature, a universal principle for focusing on meaningful changes across fields.

Knowledge Transfer

  • Software A/B Testing: Reporting a 5% improvement in user engagement (effect size) is more interpretable than "p \< 0.05."

  • Organizational Behavior: Summarizing how an HR policy change affects morale or retention with a standardized effect measure fosters consistent cross-study comparisons.

Example

A meta-analysis might reveal that an intervention consistently yields moderate effect sizes (e.g., Cohen's d ≈ 0.5), implying a meaningful but not revolutionary impact across studies.

Relationships to Other Abstractions

Local relationship map for Effect SizeParents appear above the current abstraction, mutual partners to the right, and children below. Node labels state whether each abstraction is prime or domain-specific; colors identify relation types.Effect SizePRIMEPrime abstraction: Comparison — presupposesComparisonPRIMEPrime abstraction: Scale — presupposesScalePRIMEDomain-specific abstraction: Small-Study Effects — is part ofSmall-StudyEffectsDOMAINDomain-specific abstraction: Type M Error — is part ofType M ErrorDOMAINDomain-specific abstraction: Type S Error — is part ofType S ErrorDOMAIN

Current abstraction Effect Size Prime

Parents (2) — more general patterns this builds on

  • Effect Size presupposes Comparison Prime

    Effect size presupposes comparison because magnitude is read off the relation between two or more co-considered quantities.

  • Effect Size presupposes Scale Prime

    Effect size presupposes scale because it quantifies the magnitude of an observed relationship in substantive units of measurement.

Children (3) — more specific cases that build on this

  • Small-Study Effects Domain-specific is part of Effect Size

    Small-Study Effects contains effect-size estimates as the magnitude coordinate whose systematic relationship with study precision defines the pattern.

  • Type M Error Domain-specific is part of Effect Size

    Type M Error contains observed and assumed true Effect Sizes whose magnitudes form the selected estimate and comparison target.

  • Type S Error Domain-specific is part of Effect Size

    Type S Error contains an estimated and true Effect Size whose signed directions are compared after significance selection.

Hierarchy paths (2) — routes to 2 parentless roots

Not to Be Confused With

  • Effect Size quantifies the magnitude of a relationship or treatment effect in standardized, interpretable units. Proportion and Scale concern visual or relational sizing and ratios. One is a statistical magnitude measure, the other is a compositional relationship.
  • Effect Size specifies the magnitude of a measured phenomenon in substantive units. Scale specifies the ontological band at which a system is described and the fact that laws differ across bands. One is about effect magnitude, the other about system description level.
  • Effect Size quantifies the magnitude of a relationship. Statistical Significance quantifies whether the relationship is distinguishable from zero. They are orthogonal questions; significance can be high with trivial effect size, or vice versa.
  • Effect Size measures the magnitude of a difference or relationship abstracted from context. Dose-Response Relationship maps the quantitative input-output function across a range, characterizing the functional form and shape. One is a magnitude scalar, the other is a curve.