Jeffreys-Lindley Paradox

The result that a frequentist significance test and a Bayesian posterior-odds comparison of the same data against the same point null can reach opposite verdicts, with the disagreement growing without bound as sample size increases — because the two answer different questions.

Version

v4 · 2026-09-28 · History · 4 corrections

Domain-specific #

584

Domain group

Formal Sciences

Origin domain

Experimental Design & Statistics

Subdomain

Statistical Inference and Bayesian Methods → Experimental Design & Statistics

Core Idea

The Jeffreys-Lindley paradox (Jeffreys 1939; Lindley 1957) is the result that a frequentist significance test and a Bayesian posterior-odds comparison of the same data against the same point-null hypothesis can give opposite verdicts, and that this disagreement grows without bound as sample size increases. With a result just past threshold (p ≈ 0.05), the p-value rejects the null while the Bayes factor favors it, because the diffuse prior on the alternative pays a heavy Occam penalty on concentrated data. The disagreement is structural: the two procedures answer different questions.

How would you explain it like I'm…

 

No faithful explanation at this level. At this level the story becomes 'one of two judges made a mistake.' In fact both methods are computed correctly; they answer different questions and diverge more as data grow.

Two Judges, Two Questions

Scientists have two main ways to check whether an idea like 'this coin is perfectly fair' holds up against data. One asks: if the idea were true, how surprising would results this extreme be? The other asks: do these results fit the idea better than they fit a vague 'something else', where that something else is spread over lots of possibilities? With huge amounts of data showing only a tiny difference, the first way can say 'reject the idea' while the second says 'the idea looks even better'. Neither one made a math mistake. They are answering different questions.

P-Value vs Bayes Factor Clash

The Jeffreys-Lindley Paradox is when a significance test and a Bayesian comparison, applied to the same data and the same exact null hypothesis (for example 'the effect is exactly zero'), reach opposite conclusions. Suppose the result sits just past the usual cutoff, with p ≈ 0.05, so the test rejects the null. The Bayesian alternative spreads its belief over a wide range of possible effects, so data very close to zero actually fit the null better than they fit that spread-out average, and the Bayes factor favors the null. If the sample size keeps growing while the result stays right at the cutoff, the p-value stays near 0.05 but the Bayesian support for the null keeps getting stronger. The gap between the two verdicts grows without limit. It is not an arithmetic error: the p-value and the Bayes factor measure different things.

 

The Jeffreys-Lindley paradox, from Jeffreys (1939) and Lindley (1957), concerns testing a point null H₀: θ = θ₀ against an alternative H₁ with a diffuse prior on θ. If the test statistic sits just past the significance threshold (z ≈ 1.96, p ≈ 0.05), the frequentist rejects H₀. The Bayes factor can still strongly favor H₀, because the diffuse prior spreads H₁'s probability over many values far from θ₀. The data, concentrated near θ₀, are therefore more probable under H₀ than under H₁ averaged over its prior; this is an Occam penalty on the vague alternative. Now let n → ∞ with z held fixed. The observed effect shrinks like 1/√n and the p-value stays near 0.05, but the Bayes factor for H₀ grows like √n, so the two verdicts diverge without bound. The cause is structural: a p-value is a tail probability under H₀, while a Bayes factor is a ratio of marginal likelihoods that depends on the prior. The paradox shows that choosing between the frameworks means choosing which question to ask.

Scope of Application

The paradox surfaces in every field doing null-hypothesis testing with large samples, but each imports the same inferential machinery — one substrate.

• Bayesian statistics — the canonical motivating example for the p-value crisis literature.
• Frequentist statistics — the standard warning that p ≈ 0.05 at large n carries little weight.
• Model selection — small point null versus parameter-rich alternative under diffuse priors.
• Psychology and the replication crisis — many significant originals carried weak evidence, though from small samples rather than the paradox's large n.
• Particle physics — the paradox as a criticism of any fixed discovery threshold, 5σ included.
• Genomics — genome-wide association significance thresholds that should fall as sample size grows.

Clarity

Naming the paradox makes legible that statistical significance and evidential weight are not the same quantity and at large n can point opposite ways. It converts prior specification from an invisible default into a consequential decision, sharpens the strict-point-null versus interval-null distinction, and exposes the frequentist-versus-Bayesian choice as a substantive question about which question is being asked.

Manages Complexity

A scatter of separate puzzles — negligible-effect significant trials, "failed" replications, BIC-versus-AIC disagreements — compresses into one structural signature checked by three parameters: large sample, diffuse alternative prior, rejection just past threshold. When all hold, the divergence is predictable behavior, and sample size alone predicts its direction.

Abstract Reasoning

The paradox licenses a diagnostic move inferring a paradoxical artifact from the joint signature; a predictive monotone-in-n move (more data widens the divergence); interventionist moves on its two levers (tighten the prior, replace the point null with an interval null, pre-register); and a boundary-drawing move treating "frequentist or Bayesian" as a choice of which question is asked.

Knowledge Transfer

Within statistical inference the paradox transfers as the full result — the signature, the monotone-in-n divergence, and the levers carry across biomedicine, physics, genomics, and psychology, all importing the same machinery. It does not transfer outside that substrate: it is a result about inference, not a phenomenon in any domain. What generalizes is a failure of Commensurability between the questions of the parents Hypothesis Testing (Null vs. Alternative) and Bayesian Updating — two valid procedures answering different questions can diverge, and more data magnifies it.

Relationships to Other Abstractions

Current abstraction Jeffreys-Lindley Paradox Domain-specific

Parents (2) — outgoing typed relations

• Jeffreys-Lindley Paradox includes Bayesian Updating Prime

  The Jeffreys-Lindley construction contains a Bayesian posterior-odds update whose prior-averaged likelihood supplies the opposed verdict.

• Jeffreys-Lindley Paradox includes Hypothesis Testing (Null vs. Alternative) Prime

  The Jeffreys-Lindley construction contains a frequentist point-null hypothesis test whose tail-area verdict supplies one side of the disagreement.

Hierarchy paths (7) — routes to 4 parentless roots

• Jeffreys-Lindley Paradox → includes Bayesian Updating → is a kind of Inductive Reasoning

Show alternative paths (6)

Neighborhood in Abstraction Space

Jeffreys-Lindley Paradox sits in a moderately populated region (58^th percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.

Family — Unclustered & Miscellaneous (2769 abstractions)

Nearest neighbors

• Benford's Law — 0.86
• Bayes Factor — 0.86
• Monotone Likelihood Ratio Property — 0.85
• Benjamini–Hochberg Procedure — 0.85
• Variational Bayesian Methods — 0.84

Computed from structural-signature embeddings · 2026-10-25