HARKing (Hypothesizing After the Results are Known)¶

The research practice of building a hypothesis by inspecting already-collected data and then presenting it as if it had been specified in advance, silently inflating the reported false-positive rate because the test's independence assumption is violated.

Version: v4 · 2026-09-28 · History
Domain-specific #: 487
Domain group: Formal Sciences
Origin domain: Experimental Design & Statistics
Subdomain: Research Integrity and Questionable Research Practices → Experimental Design & Statistics

Core Idea¶

HARKing, coined by Norbert Kerr in 1998, is formulating a hypothesis from inspection of already-collected data, then presenting it in publication as if it had been specified before any data were seen. It typically bundles with selective reporting. The distortion is precise: the nominal type I error rate assumes the hypothesis was fixed before the data, so a hypothesis extracted from the data makes the reported p-value understate the true false-positive probability — inflation no analysis-side correction can reach.

How would you explain it like I'm…

Painting the Target Later

Imagine shooting an arrow at a wall, then painting a target right around where it landed and saying, "Bullseye! I aimed there!" That's not fair, because you didn't really aim. HARKing is when scientists do that with their ideas: they look at the results first, then pretend they guessed it all along.

Guessing After Peeking

Scientists are supposed to make a guess, called a hypothesis, before they look at their data, and then check if the guess was right. HARKing is when a scientist looks at the data first, finds something interesting, and then writes the paper as if they had predicted it from the start. This is a problem because if you look at lots of things, some will look exciting just by luck. Pretending you predicted the lucky one makes it seem much more convincing than it is. A fix is writing down your guess publicly before collecting data.

Disguised Post-Hoc Hypotheses

HARKing, a term coined by Norbert Kerr in 1998, means forming a hypothesis after seeing the data, then presenting it as if it had been stated beforehand. It often comes with selective reporting: running many analyses, keeping the significant ones, and writing a theory around them. The p-value assumes the hypothesis was fixed in advance; once the hypothesis comes from the data, the 'prediction' was guaranteed to match. The real chance that at least one of many patterns looks significant by luck is much higher than the reported p-value suggests. As a result, findings fail to replicate, readers can't tell real tests from exploration, and contradictory theories all seem confirmed. Preregistration fixes this by time-stamping hypotheses before data collection.

HARKing (Hypothesizing After the Results are Known), named by Norbert Kerr in 1998, is formulating a hypothesis from inspection of already-collected data and then reporting it as an a priori prediction. It is typically bundled with selective reporting across many analyses and outcome measures. The inferential distortion is precise: the nominal type I error rate assumes the hypothesis was fixed before the data were seen. When the hypothesis is extracted from the data, the "prediction" is guaranteed to match, so the reported p-value answers the wrong question; the relevant probability is that at least one of the many examined patterns would reach nominal significance by chance, which is far larger. This hidden multiple-comparison problem inflates false positives. Consequences include replication rates below what alpha implies, epistemic opacity about which studies were confirmatory versus exploratory, and spurious accumulation of theory, since nearly any post-hoc account can be fitted, letting contradictory theories all look confirmed. Preregistration, a public time-stamped commitment to hypotheses and analysis plans before data collection, restores the temporal order that hypothesis-testing inference depends on.

Scope of Application¶

HARKing lives across the empirical-research-practice subfields of science — every field reporting quantitative hypothesis-confirmation tests against collected data — one substrate where the temporal order of hypothesis and result can be hidden.

Experimental psychology — the birthplace and densest habitat, a leading contributor to the replication crisis.
Biomedical and clinical research — outcome switching, with trial preregistration as the institutional cure.
Neuroimaging — where voxel-by-condition cells make the garden of forking paths most severe.
Economics — the identical move called specification searching.
Machine-learning evaluation — test-set tuning or fitting the leaderboard.

Clarity¶

Naming HARKing makes legible a distinction the literature had blurred: the same analysis on the same data is confirmatory or merely exploratory depending entirely on the temporal order of hypothesis and result. It converts vague unease about a too-tidy paper into a specific, measurable practice with a specific remedy. The sharper question is "when was this hypothesis fixed relative to the data?" — clarifying that the cure is procedural, not a multiple-comparisons correction.

Manages Complexity¶

Before HARKing was named, judging a literature's trustworthiness meant weighing dozens of incommensurable signs study by study. HARKing collapses that sprawl onto one binary: was the hypothesis fixed before or after the data were seen? Once an analyst tracks that variable, the consequences follow without re-derivation — inflated false-positive rate, exploratory column, discount to the replication base rate. It also carves the hypothesis-side member cleanly out of the questionable-research-practices family.

Abstract Reasoning¶

HARKing licenses diagnostic reasoning — detecting reverse-engineered hypotheses from a fixed signature and inferring inferential damage from the ordering. It supports interventionist reasoning that restores the temporal order procedurally because no analysis-side fix reaches it, boundary-drawing that separates it from p-hacking and makes confirmatory-versus-exploratory a property of procedure, and predictive reasoning that a HARKed literature will under-replicate and accumulate contradictory theory.

Knowledge Transfer¶

Within empirical science HARKing transfers literally as mechanism across every field reporting quantitative hypothesis tests, even as the vocabulary changes (specification searching, test-set tuning) — but this is one substrate, which keeps it domain-specific. Beyond it, loose uses in legal or journalistic narrative are analogy, dropping the p-value machinery. The general lessons travel better through the parent patterns it instantiates: Selection Bias, Hindsight Bias/narrative_fallacy, and commitment-before-evidence.

Relationships to Other Abstractions¶

Current abstraction HARKing (Hypothesizing After the Results are Known) Domain-specific

Parents (3) — outgoing typed relations

HARKing (Hypothesizing After the Results are Known) includes Hypothesis Testing (Null vs. Alternative) Prime

The HARKing practice contains a hypothesis test whose apparent prespecification and nominal error rate are the objects being falsified.
HARKing (Hypothesizing After the Results are Known) is a decomposition of, typical Hindsight Bias Domain-specific

HARKing typically recruits hindsight by making an explanation fitted after the result appear to have been predictable beforehand.
HARKing (Hypothesizing After the Results are Known) is a decomposition of Selection Bias Prime

HARKing selects the claim by inspecting which outcomes favored it, so the reported hypothesis is conditioned on the evidence later used to test it.

Hierarchy paths (20) — routes to 11 parentless roots

HARKing (Hypothesizing After the Results are Known) → includes Hypothesis Testing (Null vs. Alternative) → is a kind of Statistical Inference → is a kind of Inductive Reasoning

Show alternative paths (19)

Neighborhood in Abstraction Space¶

HARKing (Hypothesizing After the Results are Known) sits in a sparse region of the domain-specific corpus (80^th percentile for distinctiveness): few abstractions share its structure, so a faithful description tends to retrieve it precisely.

Family — Unclustered & Miscellaneous (2769 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25