Intervention Test¶
Experimental test — instantiates Causal Mechanism Mapping
Deliberately changes a chosen intervention point and watches whether intermediate and final outcomes move as the mechanism predicts.
An Intervention Test stops arguing about a causal claim and acts on it: it deliberately changes one chosen link in the mechanism — the intervention point — and observes whether the intermediate and final outcomes move the way the mapped pathway predicted. Its defining move is manipulation. Where a review scrutinizes existing evidence and a map decomposes a story, the intervention test manufactures new evidence by intervening and comparing against what happens without the change. Its signature discipline is that a real test predicts before it looks: the mechanism says which signals should move, in which direction, and — where the intervention can be delivered in varying strength — by how much more at higher doses. An outcome that shifts is not enough; it must shift the predicted way, or the mechanism, not just the metric, is what failed.
Example¶
A retail company believes its slow checkout page is the mechanism behind cart abandonment: shoppers add items, hit a laggy multi-step checkout, and leave. Rather than redesign everything on faith, the team runs an intervention test on one link — page speed — via a staged rollout. Half of arriving sessions are randomly served a pre-loaded, streamlined checkout (the intervention point); the other half get the existing flow (the counterfactual). Crucially they specify predictions from the mechanism first: if slowness truly drives abandonment, the intermediate signal — time-to-payment — should fall, and completion rate should rise, and the effect should be larger on mobile where the lag was worst.
They also vary the dose: a subset gets a partially optimized flow, letting them trace whether completion improves smoothly with speed or only past a threshold. The result is instructive precisely because it is mixed — completion rises on mobile as predicted and scales with the speed gain, but desktop barely moves. That pattern does more than confirm a number; it refines the mechanism: latency was the operative link for mobile users but not desktop, where something else drives abandonment. Because the test manipulated a single point and predicted the intermediate signal in advance, the team learns not just that speed matters but for whom and how much — and where to look next.
How it works¶
- Pick one link to move. Choose a single intervention point on the mapped pathway and change only it, so any downstream movement can be attributed to that change rather than to a bundle.
- Establish the counterfactual. Hold a comparison — randomized control, staggered rollout, or a credible before/after baseline — so "what happened" can be read against "what would have happened."
- Predict intermediate signals first, then look. State which mediating and final measures should move, and how, before seeing results; a test that checks the mechanism's predicted signals is far harder to fool than one that only watches the endpoint.
- Vary the dose where possible. Deliver the intervention at different strengths and trace whether the response scales as the mechanism implies — a monotone dose-response is strong corroboration; a flat or non-monotone one flags a misread link.
Tuning parameters¶
- Isolation strength — how cleanly the intervention point is separated from other changes (full randomization vs. a rollout confounded with time). More isolation buys cleaner causal reading but is slower and sometimes infeasible.
- Dose resolution — how many intervention strengths are tested. More levels reveal the shape of the response curve but split the sample and dilute power at each level.
- Intermediate-signal coverage — how many mediating measures are instrumented, not just the endpoint. Richer instrumentation catches a mechanism that "worked for the wrong reason" but adds measurement burden.
- Duration — how long the test runs before reading. Too short misses delayed effects and novelty decay; too long delays the decision and risks contamination.
- Reversibility / safety margin — how easily the change can be rolled back and how large a fraction is exposed. Tighter margins protect against a harmful intervention at the cost of statistical power.
When it helps, and when it misleads¶
Its strength is the strongest evidence this archetype can produce: by manipulating the pathway rather than observing it, a well-isolated intervention test cuts through confounders that stump every observational method, and its intermediate-signal predictions let it distinguish "the outcome moved" from "the mechanism was right." Dose variation adds a second, hard-to-fake corroboration.
Its failure mode is the outcome-only test that confirms the metric while missing the mechanism — the number moves, everyone declares the theory proven, and the reason it moved (a novelty effect, a coincident promotion, a different mediator entirely) goes unexamined, so the result fails to transfer.[1] The classic misuse is peeking at results and stopping the moment the endpoint looks favorable, converting a test into a search for a flattering snapshot. The guarding discipline is to commit predictions and stopping rules in advance, instrument the intermediate signals the mechanism names, and treat a bare endpoint move as unexplained until the predicted pathway signals move with it.
How it implements the components¶
Intervention Test fills the manipulate-and-observe components — the ones only acting on the system provides:
intervention_point— it operationalizes the single chosen link, delivering a concrete manipulation to exactly that point and nothing else.causal_test— its core service: a designed comparison that challenges the mechanism's directional prediction rather than merely recording activity.counterfactual_anchor— the control, staggered baseline, or before/after comparison against which the intervened outcome is read.dose_response_trace— where strength can be varied, it traces whether the response scales with dose as the mechanism implies, a corroboration observation alone cannot give.
It does not decompose or evidence the full pathway — mechanism_chain and causal_evidence_record are filled by Mechanism Map — and it acts rather than audits: the paper scrutiny of an existing claim's confounder_check belongs to Causal Inference Review. The test supplies fresh manipulated evidence; it does not adjudicate someone else's estimate.
Related¶
- Instantiates: Causal Mechanism Mapping — supplies the archetype's learning loop by manipulating a link and checking the predicted response.
- Consumes: Mechanism Map or Causal Diagram — supplies the pathway from which the intervention point and predicted signals are chosen.
- Sibling mechanisms: Causal Inference Review · Contribution Analysis · Mechanism Map · Causal Diagram · Causal-Loop Map · Theory of Change Model · Failure Tree Analysis · Process Tracing · Root-Cause Analysis
Editorial Notes¶
Form Classification¶
Form family: Experiment, Test & Rehearsal
Rationale: Intervention Test operates as a bounded trial, probe, simulation, or rehearsal that generates evidence from performance because it deliberately changes a chosen intervention point and watches whether intermediate and final outcomes move as the mechanism predicts
Independent corroboration: The frozen evidence defines Intervention Test as 'Deliberately changes a chosen intervention point and watches whether intermediate and final outcomes move as the mechanism predicts', so its operative form is Experiment, Test & Rehearsal.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Convergent development
Present-day reach: Universal
Rationale: Manipulating a chosen variable and checking predicted mediators and outcomes is a core experimental test of causal mechanism.
Related originating lineages:
- Data Science & Analytics — A/B testing and causal product experimentation provide a modern high-volume implementation.
- Systems Thinking & Cybernetics — Intervention-point and feedback-map reasoning materially determine where the system is perturbed and which intermediate signals should move.
Review resolution: Both independent reviews place the primary lineage in statistics_experimental_design. The queued differences (alternate_origin_disagreement, origin_mode_disagreement) concern secondary metadata rather than primary provenance. The final retains systems_cybernetics, data_science only where a reviewer supplied a formative-lineage rationale; this does not convert downstream applicability into origin. origin_mode=convergent because the reviewers document independently established or materially co-developing traditions. domain_reach=universal records application breadth separately from provenance.
Review outcome: Reconciled after independent review; high confidence.
References¶
[1] Deaton, A., and Cartwright, N. "Understanding and Misunderstanding Randomized Controlled Trials". Social Science & Medicine 210, 2–21 (2018). Warns that an estimated treatment effect does not by itself identify the causal mechanism. registry ↩