Increasing Transparency Through a Multiverse Analysis

Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing Transparency Through a Multiverse Analysis. Perspectives on Psychological Science, 11(5), 702-712.

Type

Journal article

Intellectual base

Primary research

Year

2016

Volume

11

Issue

5

Pages

702-712

DOI

10.1177/1745691616658637

DOI registrant

SAGE Publications

Link

https://doi.org/10.1177/1745691616658637

Cited by

5 citations across 5 artifacts.

Each citation links to the sentence it supports in the citing article.

Mechanisms

• Basis Sensitivity Review
  • Its failure mode is the sampling trap: you can only compare the bases you thought to try, so "robust across the ones we tested" is not "robust across all reasonable bases," and a fragility that lives between your samples goes unseen.
• Distributional Sensitivity Grid
  • The honest version of the same combinatorial machinery is multiverse analysis — reporting the decision across the whole set of defensible specifications

    This sourceDefines multiverse analysis as evaluating results across the full set of reasonable analytical specifications rather than selectively reporting one analysis.

• Model Comparison Table
  • It is the natural home for a multiverse analysis — reporting the result across the reasonable set of analytic choices rather than cherry-picking one — which is the discipline's answer to selective specification.

    This sourceIntroduces multiverse analysis as evaluating results across the full set of reasonable data-processing choices to expose sensitivity to arbitrary analytic decisions and reduce selective reporting.

• Selection–Transmission Sensitivity Analysis
  • The method is a specification-curve or multiverse sweep applied to the selection–transmission split: rather than defend one pipeline, it shows the conclusion across many and lets the spread speak.

    This sourceDefines multiverse analysis as running an analysis across many reasonable data-processing choices to reveal the robustness and spread of conclusions.

• Sensitivity Analysis by Group
  • The provenance log records every specification and its result, and the decision rule delivers the verdict: the conclusion is specification-dependent, with a modal estimate closer to 3%, and the brief is rewritten to report the range and name the choices the top figure depended on.

    This sourceUses a multiverse of reasonable specifications to show how analytic choices can make a conclusion specification-dependent.

Verification

Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.

Does it back the claim? Not recorded. Neither this nor any other of the 5 citations of this work carries a recorded support check.

Was it audited? Yes. A second, independent pass read the citation against the article text and recorded a verdict.

Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.

See how references were verified.

Registry ID ref:552e39a5fb2b · see in the full table