Counterfactual Comparison¶
Compare what happened with a plausible alternative to isolate causal effect or decision value.
The Diagnostic Story¶
Symptom: An outcome is evaluated on its own terms — success or failure, guilty or blameless — without naming what would have happened under any alternative condition. The baseline is unstated or cherry-picked, and the comparison being made sounds causal but tracks something else entirely. Hindsight locks the story: the outcome was obvious, the decision was reckless, the result was inevitable. Disagreements about value persist even after everyone agrees on what actually happened.
Pivot: Replace outcome-only judgment with an explicit actual-versus-counterfactual comparison. Name the actual path, define the plausible alternate condition, select and justify a reference baseline, and limit all causal, responsibility, or value claims to what the comparison can genuinely support.
Resolution: Better causal attribution and fairer decision evaluation become possible because the missing path is made visible and its plausibility is checked. Opportunity costs are clearer, learning under uncertainty improves, and speculative what-if reasoning is bounded by the comparison's honest scope.
Reach for this when you hear…¶
[public policy] “We're calling the program a success because outcomes went up, but we never said what we expected to happen without it.”
[litigation] “You can't say the surgeon caused the harm without first saying what the outcome would have been had she done nothing.”
[investment review] “The fund beat the benchmark, sure — but nobody agreed on the benchmark before we wrote the check.”
When This Archetype Applies¶
Partial catalog groundingSome structural conditions are represented by existing abstractions, but no sufficient condition set is fully represented.
Diagnostic problem
Actors evaluate an action, policy, intervention, exposure, or decision using only the observed outcome. They infer success, failure, causation, harm, inevitability, or responsibility without comparing the actual path to a plausible alternative path that did not occur. This makes interpretation vulnerable to hindsight bias, selection effects, baseline manipulation, regression to the mean, and stories that sound causal but do not identify the difference made by the focal condition.
What this problem means
The structural problem is outcome-only interpretation. People see what happened and treat it as the full evidence of what a decision was worth or what an intervention caused. This is attractive because observed outcomes are concrete, while the unchosen path is invisible. But the invisible path is often the path that matters.
A good outcome after an action does not prove the action helped. A bad outcome after a decision does not prove the decision was poor. A decline after a policy does not prove the policy caused decline; the no-policy path might have been worse. An improvement after a program does not prove the program caused improvement; the same trend might have happened anyway. Without a counterfactual comparison, actors can mistake luck, trend, selection, regression to the mean, or baseline choice for evidence.
Show the applicability expression
Applicability expression5 distinct conditions
groundedpartly groundedopen
5 conditions, all required.
5Required in every casenumbered 1–5
These hold no matter which pattern applies.
Attributed intervention outcome · grounded
An outcome is being attributed to a decision or intervention.
A bad outcome after a decision does not prove the decision was poor. The narrower requirement in this condition set is: An outcome is being attributed to a decision or intervention.
Outcome-only decision judgment · open
A decision is judged by the final result alone.
The pattern is especially important when a decision is being judged by its result alone, when a program is being evaluated by pre/post change, when a product or policy change is credited for a metric movement, or when a retrospective review is assigning blame or praise. The narrower requirement in this condition set is: A decision is judged by the final result alone.
Contested implicit baseline · grounded · any one of 2
A baseline is implicit or contested.
This is a load-bearing situation condition in the diagnostic expression. The condition is: A baseline is implicit or contested. If it does not hold, this particular condition set is incomplete.
Unobservable alternative path · grounded
The relevant alternative path cannot be directly observed.
It is also useful when the relevant alternative is contested. The narrower requirement in this condition set is: The relevant alternative path cannot be directly observed.
Assumed outcome inevitability · grounded · any one of 2
An outcome is treated as inevitable or impossible to avoid.
This is a load-bearing situation condition in the diagnostic expression. The condition is: An outcome is treated as inevitable or impossible to avoid. If it does not hold, this particular condition set is incomplete.
Other requirements and context (1)
Why these sit outside the expression
Supporting context — it may accompany or help interpret the situation, but it is not a load-bearing condition in a sufficient diagnostic set.
Supporting contextThe decision use is high-stakes enough that casual what-if reasoning is inadequate.
People see what happened and treat it as the full evidence of what a decision was worth or what an intervention caused. In this archetype, the relevant contextual consideration is: The decision use is high-stakes enough that casual what-if reasoning is inadequate. It helps interpret the situation or strengthens the practical case for examining the archetype.
Coverage
4 of 5 conditions grounded · 1 open.
Mechanisms / Implementations¶
- What-If Analysis: Uses a structured hypothetical prompt to define an alternate condition and reason through likely outcome differences; it becomes Counterfactual Comparison only once the alternate is plausibility-checked and used for disciplined comparison.
- Control Group Comparison: Compares treated units against otherwise-similar untreated ones to recover what total use would have been without the efficiency program — separating the real saving from the rebound and from what would have happened anyway.
- Synthetic Control Method: Builds a weighted comparison case from multiple units when a single natural control is unavailable, often in policy, economics, public health, or regional intervention evaluation.
- A/B Test: An A/B test implements counterfactual comparison by assigning comparable units to different versions.
- Baseline Comparison: Compares actual outcomes with a pre-action baseline, expected trend, benchmark, or no-action projection when direct controls are unavailable.
- Scenario Contrast: Contrasts a focal path with one or more explicitly described alternatives, often in strategy, planning, design, or historical interpretation where controlled testing is impossible.
- Matched Case Comparison: Pairs cases or periods that are similar on key attributes so differences in outcomes can be interpreted relative to a more credible counterfactual baseline.
- Counterfactual History Review: Uses historically plausible alternatives to test whether an outcome depended on a decision, constraint, accident, or structural condition without treating imaginative speculation as proof.
Related Abstractions¶
Abstractions this archetype builds on — directly (a source ingredient) or as a related pattern. Links follow the typed catalog namespace.
Built directly on (2)
- Causality: Cause-effect relationships.
- Counterfactuals: Alternate hypothetical scenarios.
Also references 8 related abstractions
- Confounding: Hidden variable interference.
- Counterfactual Reasoning: Hypothetical alternatives.
- Effect Size: Magnitude of effect.
- Hypothesis Testing (Null vs. Alternative): Null vs alternative evaluation.
- Regression To Mean
- Scenario Planning: Construct plausible futures.
- Selection Bias: Skewed sampling.
- Uncertainty: Incomplete knowledge.
Variants¶
Narrower or domain-specific specializations that share this archetype's core structure. Recognized variants are established; candidate variants are provisional.
Causal Effect Counterfactual Comparison · subtype · recognized
Compares observed outcome under an action or exposure with the outcome expected under no action or a different exposure in order to estimate causal effect.
Decision-Value Counterfactual Comparison · subtype · recognized
Compares the actual decision with a feasible alternative to judge value, regret, opportunity cost, avoided harm, or future decision policy.
Historical Counterfactual Comparison · temporal variant · candidate
Uses historically plausible alternatives to test contingency, agency, structural force, or decision significance in events that cannot be experimentally replayed.
Baseline Selection Counterfactual Comparison · implementation variant · likely subtype
Focuses the counterfactual work on choosing and defending the baseline that stands in for the unobserved alternative.
Counterfactual Contingency Testing · temporal variant · merge review
Uses plausible what-if alternatives to test whether an outcome depended on a specific condition, choice, timing, or contingent event.
Editorial Notes¶
Problem Classification¶
Classification: Uncertainty, Evidence & Inference Failure → Causal, Counterfactual & Attribution Validity
Problem kernel: observed outcome is evaluated without an alternative path
Rationale: Success, harm, inevitability, or responsibility is inferred from actuality alone rather than a plausible counterfactual under comparable background conditions.
Independent corroboration: The earliest necessary condition in the frozen evidence is: Actors evaluate an action, policy, intervention, exposure, or decision using only the observed outcome. That is a causal counterfactual and attribution validity problem because Association or observed outcome is assigned causal meaning without a mechanism, valid counterfactual, confounder control, or correct level of change.
Review outcome: Independent reviewer agreement; high confidence.