Sensitivity Testing¶
Sensitivity analysis — instantiates False Convergence Prevention
Sweeps a model's assumptions and parameters across their plausible ranges to find whether a conclusion is robust or hinges on a knife-edge choice, then turns that fragility verdict into an explicit stop condition for commitment.
A model can produce a confident recommendation that rests entirely on one optimistic number. Sensitivity Testing finds out by sweeping the model's own inputs: it varies each uncertain assumption and parameter across its plausible range and watches whether the conclusion stays put or flips. Its defining idea is parametric fragility — the concern that an apparent convergence is an artifact of a single knife-edge choice rather than a property of the situation. The sweep produces two things the raw recommendation hides: a ranked register of which inputs the result actually leans on, and a robustness verdict that becomes a gate condition — commit only if the conclusion survives the plausible range. Where a live-system shock would test a system against reality, this tests a model against its own inputs, on paper, before any commitment is made.
Example¶
A corporate development team runs a discounted-cash-flow model on a proposed acquisition. The base case returns a comfortably positive net present value, and the deal memo reads as settled: acquire. But the base case is a single column of point estimates — a chosen discount rate, an assumed terminal growth rate, a projected synergy-realization schedule — and the "settled" recommendation is only as solid as those choices.
Sensitivity Testing sweeps them. Holding the rest fixed, the analyst walks the discount rate across its plausible band and finds that NPV crosses from positive to negative well inside that band; the same happens when synergy realization is dialed from optimistic to merely typical. Arrayed as a tornado diagram, the inputs rank by how far each swings the outcome, and two of them are wide enough to flip the decision on their own.[n1] The convergence on acquire turns out to hinge on a knife-edge: the recommendation holds only if discount rate and synergies both land near their optimistic ends. That verdict becomes a commitment gate — do not proceed unless NPV stays positive across the plausible range of the two dominant inputs — which reframes the boardroom conversation from "is the deal good?" to "can we actually defend those two assumptions?"
How it works¶
- Enumerate the uncertain inputs. List the assumptions and parameters the conclusion depends on, each with a plausible range set before results are seen, so the ranges are honest rather than chosen to protect the answer.
- Sweep and record the swing. Vary each input across its range — one at a time to isolate its effect, and jointly where interactions matter — and record how far the conclusion moves.
- Rank by influence. Order the inputs by how much they move the result (the tornado), separating the handful that actually drive the outcome from the many that do not.
- Find the switching points and set the gate. Locate where the recommendation flips, and express the robustness verdict as a stop condition the decision must clear before it commits.
The distinguishing discipline is that it operates on a model's parameters within their plausible ranges and converts the result into a go/no-go condition — not by shocking a live system, and not by qualitatively auditing a plan's premises for evidence (that is Assumption Audit).
Tuning parameters¶
- Range width — how wide a band each input is swept across. Plausible ranges reveal real fragility; ranges drawn too narrow manufacture false robustness, while paranoid ranges make every decision look fragile.
- One-at-a-time vs. joint — whether inputs are varied singly or in combination. One-at-a-time is legible and cheap but blind to interactions; joint sweeps catch compounding assumptions but are harder to read.
- Input coverage — how many uncertain inputs are swept. Sweeping only a few keeps the analysis tidy but can miss the driver that actually matters; sweeping everything buries the signal in noise.
- Gate stringency — how much stability the conclusion must show to pass. A strict gate blocks fragile recommendations but can stall on irreducible uncertainty; a lax one lets knife-edge conclusions through as if robust.
When it helps, and when it misleads¶
Its strength is that it exposes the recommendation that is technically positive but structurally fragile — the conclusion that depends on one hopeful input — and it does so cheaply, before any resource is committed, by ranking exactly which assumptions the decision is betting on.
Its failure modes track its dials. A one-at-a-time sweep misses interactions, so a model can pass every single-input test yet flip when two plausible moves combine. Ranges set too narrow produce false robustness, and the whole exercise is readily gamed by choosing ranges that keep the recommendation inside the safe zone. The classic misuse is decorative sensitivity — sweeping only the inputs already known not to matter, so the analysis reliably confirms the base case and launders a fragile decision as a tested one. The guarding discipline is to fix the plausible ranges before seeing which way they cut, include joint moves for inputs that plausibly co-vary, and set the gate stringency in advance so the robustness bar is a standard rather than a result negotiated after the fact.
How it implements the components¶
Sensitivity Testing realizes the parametric-robustness and decision side of the archetype — the parts that measure how much a conclusion leans on its inputs and gate commitment on the answer:
assumption_register— the sweep produces a ranked register of the assumptions and parameters the conclusion depends on, recording how far each moves the result so the load-bearing few are explicit rather than buried in a base case.commitment_gate— the fragility verdict becomes the gate's stop condition: proceed only if the conclusion survives the plausible range of its dominant inputs, otherwise hold, revise, or reopen.
It does not inject a live disturbance into a running system or define robustness-under-shock (perturbation_test, genuine_convergence_standard) — that is Perturbation Probe, and the nearest-twin difference is that this sweeps a model's parameters on paper while the probe shocks the actual system; nor does it reproduce the result through an independent actor or fresh sample (independent_check, out_of_sample_check — Independent Replication).
Related¶
- Instantiates: False Convergence Prevention — this is the mechanism that measures whether a modeled conclusion is robust to its own assumptions and gates commitment on the answer.
- Sibling mechanisms: Perturbation Probe · Independent Replication · Stratified Residual Review · Appeal or Reopening Review · Assumption Audit
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Sensitivity Testing operates as an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution because it sweeps a model's assumptions and parameters across their plausible ranges to find whether a conclusion is robust or hinges on a knife-edge choice, then turns that fragility verdict into an explicit stop condition for commitment.
Independent corroboration: The frozen evidence defines Sensitivity Testing as 'Sweeps a model's assumptions and parameters across their plausible ranges to find whether a conclusion is robust or hinges on a knife-edge choice, then turns that fragility verdict into an explicit stop condition for commitment', so its operative form is Analysis, Modeling & Optimization.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Operations Research
Origin pattern: Convergent development
Present-day reach: Universal
Rationale: Testing a decision or model over alternative assumptions and parameter values is operations-research robustness analysis. NIST and NASA define sensitivity analysis around input-output influence, with experimental design supplying disciplined test selection.
Related originating lineages:
- Data Science & Analytics — Data science, analytics, and operational monitoring supplies a parallel or contributing lineage for the mechanism's defining operation: sweeps a model's assumptions and parameters across their plausible ranges to find whether a conclusion is robust or hinges on a knife-edge choice, then turns that fragility verdict….
- Economics & Finance — Stress testing asks whether commitments remain viable under adverse parameter shifts.
- Engineering & Design — Qualification testing converts fragility limits into go/no-go criteria.
- Mathematics — Mathematical modeling, proof, and abstract-structure practice supplies a parallel or contributing lineage for the mechanism's defining operation: sweeps a model's assumptions and parameters across their plausible ranges to find whether a conclusion is robust or hinges on a knife-edge choice, then turns that fragility verdict….
- Statistics & Experimental Design — Uncertainty and robustness methods quantify how conclusions depend on plausible inputs.
Review resolution: The blind reviewers disagree on primary lineage (operations_research versus statistics_experimental_design). Authoritative or primary research supports operations_research as the best historical origin: Testing a decision or model over alternative assumptions and parameter values is operations-research robustness analysis. NIST and NASA define sensitivity analysis around input-output influence, with experimental design supplying disciplined test selection. The cited NIST, Guide for the Use of the International System of Units: Model Sensitivity; NASA, Sensitivity Analysis Overview directly supports the mechanism's defining operation. All independently supported contributing domains are retained without an arbitrary cap. origin_mode=convergent records the lineage relationship, while domain_reach=universal records later applicability separately from provenance.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
- NIST, Guide for the Use of the International System of Units: Model Sensitivity
- NASA, Sensitivity Analysis Overview
Notes¶
Sensitivity Testing and Assumption Audit both scrutinize assumptions, but they operate in different registers and belong to different archetypes. This mechanism sweeps a model's parameters numerically to measure how much each moves a quantitative conclusion; the audit is a qualitative reasoning pass over a plan's whole premise-set, triaging by materiality and grading evidence. Treat the tornado's ranked drivers here as complementary to — not a substitute for — the audit's list of load-bearing beliefs.
[n1] A tornado diagram is the standard visualization of one-at-a-time sensitivity analysis: each uncertain input is swept across its range and the resulting swing in the output is drawn as a horizontal bar, with the bars sorted longest-to-shortest so the shape tapers like a tornado. It makes visible at a glance which few assumptions a conclusion actually leans on. ↩