Skip to content

Counterfactual Contrast Memo

Counterfactual reasoning method — instantiates Structured Comparative Case Design

Argues one case's causal claim by spelling out what would have happened absent the cause, anchored to a closely matched case where the cause was in fact missing.

A single case can carry a causal claim only if it answers a question it usually leaves implicit: compared to what? The Counterfactual Contrast Memo makes that answer explicit and checkable. In a page or two it states the counterfactual — "had the cause not occurred, the outcome would have been X" — and grounds it in a closely comparable contrast case that actually lacked the cause. Its distinguishing move is to work at the grain of one focal contrast rendered in prose: it names the comparison logic out loud and then defends the comparability of the foil it leans on. Where the systems-design mechanisms choose the comparison logic for a whole study and build its full matrices, the memo argues a single counterfactual and stands or falls on it.

Example

A mid-size city credits its new downtown bike-lane network for a fall in traffic injuries. The memo asks: compared to what? It pairs the city with a neighbouring one of similar size, climate, downtown density, and — crucially — a matching pre-period injury trend, which had considered the same network but did not build it. It spells out the counterfactual: absent the lanes, this city's injuries would most likely have tracked the comparison city's, which stayed flat; instead they fell.

Then it does the honest half of the work. It lists the ways the two cities differ that could carry the outcome instead — the comparison city opened a new transit line the same year — and states how far the claim can be pushed given those threats. The output is a short, contestable argument that turns "the lanes worked" into an explicit counterfactual anchored to a defensible foil, with its confounds on the table rather than under it.

How it works

  • Name the focal case and its causal claim in one sentence.
  • Make the comparison logic explicit: what is the counterfactual, and which real case stands in for the world where the cause was absent?
  • Profile the contrast case's comparability — the ways it matches (the controls) and the ways it differs (the threats).
  • State the counterfactual conclusion with its residual confounds and the scope over which it holds.

Tuning parameters

  • Contrast source — a real matched case versus a reasoned scenario; real foils are far stronger but harder to find.
  • Comparability bar — how similar the foil must be before the contrast counts; stricter is cleaner but narrows the field of candidates.
  • Rewrite discipline — how minimal the counterfactual change is (alter only the cause, hold the rest fixed); larger rewrites weaken the argument fast.
  • Confound disclosure — how exhaustively the memo lists what else differs between focal and contrast case.

When it helps, and when it misleads

Its strength is that it turns an implicit "it worked" into an explicit, falsifiable counterfactual and forces the "compared to what?" that single-case stories skip; a well-chosen foil does real inferential work at low cost. Its failure mode is that the whole argument lives or dies on the foil's comparability, and a plausible-sounding contrast can conceal a decisive difference — while counterfactuals themselves tempt speculation beyond the evidence. The classic misuse is picking the foil that most flatters the claim. The discipline that guards against this is to choose the contrast case by explicit comparability, keep the counterfactual to a minimal rewrite, and disclose the differences that could have carried the outcome on their own.[1]

How it implements the components

  • comparison_logic_choice — the memo makes the contrast logic explicit for one focal inference: what is compared to what, and why that foil rather than another.
  • case_comparability_profile — it profiles the single contrast case's similarities (the controls) and differences (the threats) to defend, or qualify, the counterfactual.

It applies these at the grain of one written argument, not the study-wide design of most-similar and most-different logic with their full matrices (that's most-similar and most-different systems design), nor the operational routine for matching cases in bulk (that's the matched case pairing protocol).

References

[1] Counterfactual analysis — judging a cause by asking what would have happened in its absence, disciplined by the "minimal-rewrite" rule: change only the putative cause, and as little else as possible. A closely matched real case is the most credible stand-in for the counterfactual world. ↩