Missing-Data Sensitivity Analysis¶
Robustness analysis — instantiates Attrition and Dropout Monitoring
Re-runs the conclusion under a range of assumptions about the missing outcomes — including deliberately adverse ones — to see whether the finding survives the people who are gone.
The people who left took their endpoints with them, and no table can measure what was never recorded. Missing-Data Sensitivity Analysis confronts that head-on: instead of pretending the missing outcomes look like the observed ones, it varies the assumption about them and watches what happens to the result. It re-computes the study's conclusion across a spectrum — from missing-completely-at-random, through missing-at-random, to explicitly missing-not-at-random scenarios in which the departed did systematically worse (or better) than the stayers — anchored on the full assigned denominator rather than the surviving completers. Its defining move is reasoning about outcomes you cannot see: it asks not "what did the completers show" but "how wrong would my assumption about the missing have to be before my finding flips." An effect that holds across plausible assumptions is credible; one that survives only under the convenient assumption is not.
Example¶
A labor-economics field experiment tested a job-training program, but twelve-month earnings were captured only for participants the survey team could still reach. Complete-case analysis — using only those reached — shows a healthy earnings gain. The sensitivity analysis refuses to stop there. Working from the full randomized denominator, it re-estimates the effect under several missingness assumptions: MAR imputation from baseline characteristics leaves the gain roughly intact; then a pattern-mixture step imposes progressively worse assumed outcomes on the unreached, and a tipping-point search finds how adverse the unobserved earnings would have to be to erase the effect. The tipping point turns out to be modest — well within what the reason-for-leaving evidence suggests is plausible. The verdict changes from "the program worked" to "the program's estimated effect is fragile to who we couldn't reach," which is a truer statement of what the data can support.
How it works¶
Its distinguishing discipline is conclusion-under-varied-assumption:
- Anchor on the full assigned denominator. The analysis is run over everyone randomized, not just the reachable, keeping the intention-to-treat frame intact so missingness cannot be quietly analyzed away.
- Enumerate and record the missingness assumptions. MCAR, MAR, MNAR, and censoring are each written down as an explicit interpretation, with the evidence for or against each noted rather than assumed.
- Re-estimate under each. Impute or model the missing outcomes per assumption and recompute the effect.
- Find the tipping point. Push the MNAR assumption until the conclusion overturns, and report how extreme that departure had to be — the headline output is the finding's fragility, not a single number.
Tuning parameters¶
- Assumption range — how far the MNAR scenarios reach; a timid range flatters the result, an honest one includes genuinely adverse departures.
- Imputation or modeling method — multiple imputation, pattern-mixture, selection models; each carries its own assumptions into the estimate.
- Tipping-point granularity — how finely the adverse assumption is swept before the conclusion flips.
- Denominator anchor — intention-to-treat versus per-protocol; the former resists survivorship, the latter answers a narrower question.
- Estimand — which quantity the conclusion is stated in, since robustness can differ across effect measures.
When it helps, and when it misleads¶
Its strength is honesty about the limits of the data: it is the mechanism that can say "this finding does not survive plausible assumptions about the people we lost" while there is still a chance to qualify the claim rather than overstate it. It converts an unknowable — the missing outcomes — into a bounded statement about how much they could matter.
It misleads when the assumption range is drawn too narrow, producing false comfort: sweep only gentle departures and everything looks robust. It also misleads when quietly re-anchored on completers, smuggling survivorship back in. The classic misuse is presenting a bare complete-case analysis as if missingness were random — the very assumption a tipping-point analysis[n1] exists to interrogate. The guarding discipline is to pre-specify the assumption range before seeing results, include genuinely adverse MNAR scenarios, and report the tipping point plainly rather than only the base case.
How it implements the components¶
missingness_assumption_record— it enumerates the MCAR/MAR/MNAR/censoring interpretations, records which the evidence supports, and tests the conclusion under each.denominator_transition_frame— it anchors every re-estimate on the full assigned denominator, keeping the analysis from collapsing onto the surviving completers.
It does not produce the observed baseline comparison of leavers versus stayers that calibrates its adverse assumptions (completer_non_completer_contrast, its nearest twin Completer Balance Table) — it consumes that contrast rather than building it — nor track loss rates as they diverge in real time (differential_attrition_monitor, Attrition Dashboard).
Related¶
- Instantiates: Attrition and Dropout Monitoring — the sensitivity analysis is the archetype's inferential honesty step, bounding what the conclusion can claim.
- Consumes: Completer Balance Table — its observed completer–non-completer differences calibrate how adverse the MNAR scenarios should reach.
- Sibling mechanisms: Participant Flow Diagram · Attrition Dashboard · Withdrawal Reason Survey or Interview · Completer Balance Table · Retention Outreach Protocol · Data Monitoring Review
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Missing-Data Sensitivity Analysis operates as a computation, comparison, model, or analytic representation used to infer, estimate, or choose because it re-runs the conclusion under a range of assumptions about the missing outcomes — including deliberately adverse ones — to see whether the finding survives the people who are gone.
Independent corroboration: The frozen evidence defines Missing-Data Sensitivity Analysis as 'Re-runs the conclusion under a range of assumptions about the missing outcomes — including deliberately adverse ones — to see whether the finding survives the people who are gone', so its operative form is Analysis, Modeling & Optimization.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Sensitivity analysis under alternative missing-data assumptions is a canonical statistical robustness method.
Related originating lineages:
- Medicine & Healthcare — Clinical trials made adverse dropout scenarios and attrition robustness especially consequential.
Review resolution: Both independent reviews place the primary provenance in statistics_experimental_design. The queued differences (alternate_origin_disagreement, origin_mode_disagreement) concern secondary metadata, not primary lineage. The final retains medicine_healthcare only where a reviewer supplied a formative-lineage rationale; downstream use or broad applicability by itself is not treated as origin. origin_mode=cross_disciplinary_synthesis because the supplied rationales identify formative contributions that are composed in the mechanism's present form. domain_reach=multi_domain records established application breadth separately from provenance. confidence=high preserves the more cautious evidence assessment. encyclopedia_synthesis=false records whether either reviewer identified deliberate corpus-level composition.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
The distinction from the Completer Balance Table is the crux of the archetype's diagnostic pair. The table reports the observed difference between leavers and stayers and stops; this analysis takes that difference as a clue and extrapolates into the unobserved, asking how the finding behaves if the missing outcomes were worse than the observed ones imply. Description ends where inference begins, and that boundary is exactly where these two mechanisms hand off.
[n1] A tipping-point analysis (a form of pattern-mixture sensitivity analysis) progressively worsens the assumed outcomes of missing units until the study's conclusion reverses, then reports how large that departure from missing-at-random had to be. It is the standard tool for asking whether a result depends on optimistic assumptions about who was lost, which is why it anchors the honest end of this mechanism's assumption range. ↩