What-If Analysis¶
Elicitation method — instantiates Counterfactual Comparison
Uses a structured hypothetical prompt to define an alternate condition and reason through likely outcome differences; it becomes Counterfactual Comparison only once the alternate is plausibility-checked and used for disciplined comparison.
What-If Analysis is the lightest, most upstream member of the family: it starts from a single structured hypothetical prompt — "what would have happened if this one condition had been different?" — specifies that one alternate concretely, checks that it is plausible, and reasons through the likely difference in outcome from known causal drivers. It needs no comparison group, no donor pool, and no data model, which is exactly its value: it can be run in minutes to surface the "compared with what" a bare outcome hides. But that same lightness is its risk. The prompt only becomes Counterfactual Comparison when the alternate is pinned down and passed through a plausibility gate; left vague, a what-if is just speculation with a confident tone.
Example¶
A product team ships a checkout redesign on an e-commerce site without running an A/B test — the release was urgent and there was no time to split traffic. Two weeks later, conversion is up, and someone declares the redesign a success. The lead pushes back with a structured what-if: what would conversion be right now if we had kept the old checkout? That single alternate condition is made concrete — same traffic mix, same promotion calendar, old flow — and then plausibility-checked: is it fair to assume everything else would have moved the same way? Reasoning from known drivers, the team notes that a large seasonal promotion ran during the same two weeks and typically lifts conversion on its own, and that mobile traffic (which converts higher) was unusually heavy. Netting those out, the redesign's own likely contribution is a modest, bounded improvement — not the full jump. The output is a hypothetical, honestly hedged: "the redesign probably helped a little, but most of the lift was the promotion; treat this as a guess until we can test it." That single reframing stops the team from over-crediting an untested change.
How it works¶
The method is disciplined reasoning, not measurement. Frame the what-if as one clearly-held alternate condition rather than a fog of "things could have gone differently." Specify that alternate concretely enough that its consequences can be reasoned about. Gate it through a plausibility check — could the alternate actually have held, and is it a fair comparison, or does it quietly change more than one thing? Then reason the likely outcome difference from known causal drivers, subtracting anything (a promotion, a trend, a mix shift) that would have moved the outcome regardless. Finally, state the result as a bounded estimate, not a fact. The whole discipline is what separates a usable what-if from a daydream: a specific alternate, a plausibility gate, and an explicitly hedged difference.
Tuning parameters¶
- Prompt specificity — how concretely the single alternate is defined. Vague prompts invite motivated reasoning; a sharply specified alternate can actually be reasoned about.
- Single versus swept condition — one fixed what-if, or the same prompt run across a small range of the varied input. Sweeping surfaces how fragile the conclusion is; a single point is faster but blinder.
- Plausibility strictness — how hard the alternate is interrogated before it is used. Stricter gating rejects fantasy alternates but slows the quick-look value the method exists for.
- Reasoning depth — back-of-envelope heuristic versus a lightly modeled estimate. Deeper reasoning tightens the answer but starts to cost the speed that distinguishes this mechanism.
When it helps, and when it misleads¶
Its strength is speed and reach: it is the fastest way to force "compared with what" into a conversation, it needs no data or comparison unit, and it is the natural tool for a quick evaluation or a pre-mortem when a heavier method is not warranted or not yet possible.
Its failure modes come from the same lightness. Hindsight bias makes the counterfactual feel obvious in retrospect — once you know the outcome, the alternate seems to lead there inevitably — and lends unearned confidence.[n1] The fantasy counterfactual (an alternate that could never have held) and motivated framing (choosing the what-if that flatters a decision already made) are close behind. The classic misuse is treating a vivid, reasoned-through what-if as if it were evidence rather than a structured guess. The guarding discipline is to force the alternate to be specific and explicitly plausibility-checked, to state the answer as a bounded estimate rather than a finding, and to escalate to a heavier sibling — a matched comparison, a synthetic control, an actual test — as the stakes rise.
How it implements the components¶
counterfactual_condition— the single alternate the structured prompt defines and holds fixed, made concrete enough to reason about.plausibility_check— the gate that turns a daydream into a usable comparison, testing whether the alternate could truly have held and whether it changes only one thing.outcome_comparison— the reasoned, bounded estimate of the likely difference between the actual outcome and the what-if.
It does not lay out and choose among several fully-described alternatives for a decision_use_case — that is Scenario Contrast, its nearest twin, which trades this method's single elicited prompt for a multi-path decision comparison; nor does it issue a documentary contingency causal_or_value_claim_revision about a past era — that is Counterfactual History Review.
Related¶
- Instantiates: Counterfactual Comparison — What-If Analysis is the elicitation step that surfaces and specifies the alternate the heavier siblings then discipline.
- Sibling mechanisms: Scenario Contrast · Counterfactual History Review · Baseline Comparison · Matched Case Comparison · Synthetic Control Method · Control Group Comparison · A/B Test
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: What-If Analysis operates as an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution because it uses a structured hypothetical prompt to define an alternate condition and reason through likely outcome differences; it becomes Counterfactual Comparison only once the alternate is plausibility-checked and used for disciplined comparison.
Independent corroboration: The frozen evidence defines What-If Analysis as 'Uses a structured hypothetical prompt to define an alternate condition and reason through likely outcome differences; it becomes Counterfactual Comparison only once the alternate is plausibility-checked and used for disciplined comparison', so its operative form is Analysis, Modeling & Optimization.
Nearest alternative: Representation, Specification & Plan — What-If Analysis includes features of a static representation, map, specification, schema, or prospective plan that externalizes information, but its defining operation is an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Operations Research
Origin pattern: Convergent development
Present-day reach: Universal
Rationale: Defining an alternate condition and calculating the resulting change is the sensitivity and scenario-analysis tradition of operations research. NASA explicitly defines what-if analysis as sensitivity analysis, and GAO requires varying assumptions and tracing effects on modeled outcomes; organizational facilitation is a portable application of that analytic operation.
Related originating lineages:
- Economics & Finance — Economics, finance, and mechanism-design practice has a distinct contributing or parallel lineage for the mechanism's defining operation: uses a structured hypothetical prompt to define an alternate condition and reason through likely outcome differences; it becomes Counterfactual Comparison only once the alternate is….
- Futurism & Strategic Foresight — Strategic foresight, scenario planning, and anticipatory governance has a distinct contributing or parallel lineage for the mechanism's defining operation: uses a structured hypothetical prompt to define an alternate condition and reason through likely outcome differences; it becomes Counterfactual Comparison only once the alternate is….
- Organizational & Management Science — Organizational design, management, and operational governance has a distinct contributing or parallel lineage for the mechanism's defining operation: uses a structured hypothetical prompt to define an alternate condition and reason through likely outcome differences; it becomes Counterfactual Comparison only once the alternate is….
- Systems Thinking & Cybernetics — Systems science's feedback, boundaries, stocks, flows, and regulation tradition supplies an independent formative lineage for the mechanism's what if analysis logic.
Review resolution: The blind reviewers disagree on primary lineage (organizational_management versus operations_research). Authoritative or primary research supports operations_research as the best historical origin: Defining an alternate condition and calculating the resulting change is the sensitivity and scenario-analysis tradition of operations research. NASA explicitly defines what-if analysis as sensitivity analysis, and GAO requires varying assumptions and tracing effects on modeled outcomes; organizational facilitation is a portable application of that analytic operation. The cited NASA OCFO, Program Planning and Control Glossary: What-If Analysis; U.S. GAO, Cost Estimating and Assessment Guide directly supports the mechanism's defining operation. All independently supported contributing domains are retained without an arbitrary cap. origin_mode=convergent records lineage, while domain_reach=universal records later applicability separately from provenance.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
- NASA OCFO, Program Planning and Control Glossary: What-If Analysis
- U.S. GAO, Cost Estimating and Assessment Guide
Notes¶
[n1] Hindsight bias — the tendency, once an outcome is known, to see it as having been predictable all along. In counterfactual reasoning it quietly inflates confidence that the imagined alternate would have led where the reasoner expects, which is why an explicit plausibility check and a hedged estimate are the discipline that keeps a what-if honest. ↩