File Drawer Problem

Recognize that studies with null results disproportionately go unpublished while significant ones enter the literature, so any synthesis treating the published record as the full population of conducted research systematically overestimates effect sizes toward the filter.

Version

v3 · 2026-09-28 · History · 1 correction

Domain-specific #

399

Domain group

Formal Sciences

Origin domain

Experimental Design & Statistics

Subdomain

Meta Research and Publication → Experimental Design & Statistics

Core Idea

The file drawer problem, named by Rosenthal in 1979, is the meta-research phenomenon in which studies with null or unsurprising results disproportionately go unpublished while significant ones enter the literature, making the published record a biased sample of the conducted research. Its mechanism has three parts: a population of conducted studies, a publication filter correlated with effect size, and a downstream synthesis that mistakes the filtered record for the full population.

How would you explain it like I'm…

The Hidden Failures Drawer

Imagine a friend tries a magic trick lots of times but only tells you about the times it worked. You'd think the trick almost always works! Scientists have the same problem. Tests that found nothing often get hidden in a drawer, so we mostly hear about the exciting ones.

The Missing Boring Studies

Scientists run many experiments. The ones that find something exciting usually get published. The ones that find nothing often get stuck in a drawer and forgotten. When someone later reads all the published studies, they only see the exciting ones. That makes an effect look bigger and surer than it really is. It can even make something look real when it isn't real at all. This is called the file drawer problem.

Publication Bias by Omission

The file drawer problem, named by Robert Rosenthal in 1979, is when studies with null or unsurprising results go unpublished much more often than studies with statistically significant or exciting results. This makes the published research a biased sample of all the research actually done. When someone reviews or combines the published studies, they overestimate how big an effect is and how reliable it is. A weak or even nonexistent effect can look solid if many null results are hidden. Fixes include pre-registering studies before collecting data, publishing based on the research plan rather than the results, requiring drug trials to report results, and statistical tools that estimate what's missing.

 

The file drawer problem, named by Robert Rosenthal in 1979, is a form of publication bias in which null, negative, or unsurprising results disproportionately remain unpublished. It has three parts. First, a population of conducted studies, with true effects plus sampling variation. Second, a publication filter selecting on criteria such as statistical significance, novelty, or agreement with expectations, which correlate with the observed effect size. Third, downstream synthesis such as meta-analyses, reviews, guidelines, and policy that treats the filtered record as the whole population. The result is systematic overestimation of effect sizes, understated uncertainty, and overstated reliability, sometimes by large margins. Rosenthal's key point is that what is absent from the record drives conclusions drawn from it. Responses either remove the filter (pre-registration, registered reports, reporting mandates for pharmaceutical sponsors) or estimate and correct for it (funnel plots, trim-and-fill, p-curve, Egger regression, selection models).

Scope of Application

The file drawer problem lives across the subfields of meta-research, evidence synthesis, and scientific reform — the substrate of science as a publication-mediated knowledge system.

• Meta-analysis methodology — file-drawer correction as a routine step: funnel-plot asymmetry, trim-and-fill, p-curve, Egger regression, selection models.
• Clinical-trial registries — registration before data collection instituted to make the conducted population visible regardless of outcome.
• The replication crisis — the file drawer as one proximate mechanism behind over-stated, non-replicating effects.
• Drug-trial regulation — mandates that sponsors report all trial results, targeting selective publication of favourable trials.
• Pre-registration and registered reports — editorial models that decide publication before results exist, removing the filter for accepted work.

Clarity

Naming the file drawer problem moves the locus of suspicion outside the individual study to the record itself, treated as a measurement that can be biased independently of the quality of any study in it. It converts a vague unease — "can I trust this literature?" — into a sharp question: is the visible record a fair sample of the conducted research, or a filtered one?

Manages Complexity

The concept replaces an unbounded study-by-study audit with a three-part generative model and one latent quantity: the hidden mass of filed-away results and its zero-ward pull. The bias reads off a low-dimensional summary the record already exposes — effect-versus-precision asymmetry — and the remedy forks on timing relative to the filter.

Abstract Reasoning

The concept licenses a diagnostic move from a fingerprint the record exposes (funnel asymmetry implies a removed mass of nulls), an interventionist move forked on timing (eliminate the filter prospectively versus estimate and correct after the fact), and boundary-drawing on the locus of the defect, including a signed prediction that the bias on the synthesised estimate is always away from zero.

Knowledge Transfer

Within meta-research the file drawer transfers as mechanism and is the connective tissue of a cluster of publication pathologies. Beyond that substrate the deeper pattern — a selection filter on visibility biases the visible record — is carried by parents Selection Bias, survivorship_bias, and selection_effect, of which the file drawer is the publication-pipeline case. Its named machinery and pipeline framing stay home.

Relationships to Other Abstractions

Current abstraction File Drawer Problem Domain-specific

Parents (1) — outgoing typed relations

• File Drawer Problem is a kind of Publication Bias Domain-specific

  Drawer Problem is the whole-study null-suppression species of the broader results-dependent publication and reporting filter.

Hierarchy paths (10) — routes to 5 parentless roots

• File Drawer Problem → is a kind of Publication Bias → is a kind of Selection Bias → is a kind of Bias

Show alternative paths (9)

Neighborhood in Abstraction Space

File Drawer Problem sits in a sparse region of the domain-specific corpus (63^rd percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (2769 abstractions)

Nearest neighbors

• Funnel Plot Asymmetry — 0.91
• Small-Study Effects — 0.86
• Type M Error — 0.85
• Systematic Review — 0.84
• Clinical Study Design — 0.83

Computed from structural-signature embeddings · 2026-10-25