Sensitivity Analysis Protocol¶
Essence¶
Sensitivity Analysis Protocol is the archetype for testing whether a conclusion survives plausible changes in the assumptions that support it. It is not merely a chart, a table, or a solver setting. The core move is to make uncertainty operational: choose the assumptions that might matter, vary them across defensible ranges, watch the decision-relevant outcome, and identify which assumptions are harmless, material, critical, or invalidating.
The archetype is especially valuable when a model or plan produces a confident recommendation but the input assumptions are uncertain, contested, or likely to move. The goal is not to eliminate uncertainty. The goal is to learn which uncertainty changes the decision.
Compression statement¶
When a model, plan, or decision appears justified but depends on uncertain inputs, systematically vary plausible assumptions, record outcome response, identify critical parameters, and update confidence or follow-up action.
Canonical formula: Given baseline conclusion C from model or plan M with parameters P, vary selected p_i within plausible ranges R_i, observe outcome response ΔC or decision state D, and classify assumptions as stable, material, critical, or invalidating relative to a materiality threshold T.
When to Use This Archetype¶
Use this archetype when a decision depends on forecasts, estimates, weights, thresholds, or assumptions that could plausibly be wrong. It is a strong fit when the decision is costly, risky, irreversible, politically contested, safety-relevant, or likely to become locked in.
It is also useful when stakeholders distrust a conclusion but cannot yet say what would change it. A sensitivity protocol turns that disagreement into testable claims: which assumption, over what range, changes which outcome?
Do not start here when the baseline objective or decision rule is still unclear. In that case, objective and constraint formulation should happen first. Do not treat this as robust selection either: sensitivity analysis diagnoses fragility; robust solution selection chooses an option after fragility is understood.
Structural Problem¶
The structural problem is conditional confidence. A plan, model, forecast, ranking, or recommendation appears settled, but its conclusion may depend on uncertain inputs whose influence is hidden. This creates two bad options: either actors overtrust the baseline answer, or they reject the analysis wholesale because uncertainty is present.
Sensitivity analysis creates a middle path. It keeps the baseline explicit but treats it as conditional. The question becomes: under what plausible assumption changes does the conclusion remain stable, and under what changes does it fail?
Intervention Logic¶
The intervention begins by stabilizing a baseline case. The baseline records the current model, assumption set, watched outcome, and decision rule. Next, the analyst or decision group selects key parameters: inputs that are uncertain, influential, contested, or consequential if wrong. Each parameter receives a variation range, scenario bundle, distribution, or threshold that can be defended.
The protocol then runs variation tests. Some tests vary one input at a time; others vary combinations or coherent scenarios. The important output is not simply a recalculated number. The important output is the outcome response: what changes, how much it changes, whether the recommendation flips, and whether the change is decision-relevant.
Finally, the results are translated into action. Stable conclusions may proceed with documented confidence. Fragile conclusions may need validation, monitoring triggers, redesign, contingency planning, or robust solution selection.
Key Components¶
Sensitivity Analysis Protocol works as a diagnostic loop that converts a settled-looking conclusion into one whose conditions of validity are explicit. The Baseline Case pins down the reference point — model, assumptions, conclusion, and decision rule — so that any later movement can be interpreted as movement away from a known starting state. From there, the analyst selects each Key Parameter, filtering all inputs down to the ones whose uncertainty actually intersects with consequence. Each chosen parameter receives a Variation Range drawn from data, expert judgment, scenarios, or stress logic; the range must be wide enough to expose fragility but defensible enough to avoid manufactured alarm. The watched Outcome Measure stays fixed during the test so that any change can be attributed to the assumption being varied rather than to a moving target.
The remaining components turn the test mechanics into a decision-grade verdict. The Outcome Response records how the watched measure actually moved under each variation — the evidentiary core that links assumptions to conclusion confidence. The Materiality Threshold tells the protocol which movements count as meaningful, filtering out numerical noise while flagging small changes that flip a recommendation. A Critical Assumption is then identified as any input whose plausible variation crosses that threshold, reverses the recommendation, or violates a constraint; these become the assumptions deserving validation, monitoring, or redesign. Finally, the Robustness Conclusion translates the full pattern of responses into decision language — stable, conditionally stable, fragile, or invalid — so the protocol ends not in a chart but in updated confidence and a clear next move.
| Component | Description |
|---|---|
| Baseline Case ↗ | The baseline case is the reference point for the analysis. It records the model, plan, input assumptions, conclusion, and decision rule before variation begins. Without a baseline, sensitivity testing becomes unstructured tinkering. |
| Key Parameter ↗ | A key parameter is an assumption, coefficient, forecast, threshold, weight, or condition that could materially influence the outcome. The point is not to vary every possible input. The point is to select inputs where uncertainty and consequence intersect. |
| Variation Range ↗ | The variation range defines how far an assumption will be moved. Ranges can come from historical data, expert judgment, external scenarios, policy limits, or deliberate stress cases. A narrow range can hide fragility; an unrealistic range can create irrelevant alarm. |
| Outcome Measure ↗ | The outcome measure is what the protocol watches. It may be a score, ranking, feasibility status, cost, benefit, risk level, recommendation, or decision state. The measure must remain stable during the test so that outcome movement can be interpreted. |
| Outcome Response ↗ | Outcome response records how the watched outcome changes under variation. This is the evidentiary core of the archetype: it connects assumptions to conclusion confidence. |
| Critical Assumption ↗ | A critical assumption is one whose plausible change materially alters the decision, reverses the recommendation, violates a constraint, or substantially changes confidence. Critical assumptions should trigger validation, monitoring, review, or redesign. |
| Materiality Threshold ↗ | The materiality threshold defines what counts as a meaningful change. This keeps the protocol from overreacting to numerical noise or ignoring small changes that flip a decision. |
| Robustness Conclusion ↗ | The robustness conclusion states whether the original conclusion is stable, fragile, conditionally stable, or invalid under plausible variation. It should be written in decision language, not only technical language. |
Common Mechanisms¶
| Mechanism | Description |
|---|---|
| One-way Sensitivity Analysis ↗ | One-way sensitivity analysis varies one parameter at a time. It is useful for first-pass driver ranking and for explaining which inputs matter most. It can mislead when parameters are dependent or when interaction effects drive the result. |
| Two-way or Multi-way Sensitivity Analysis ↗ | Multi-way analysis varies combinations of inputs. It implements the archetype when assumptions interact, such as demand and staffing, cost and schedule, or adoption and retention. |
| Scenario Variation ↗ | Scenario variation groups assumptions into coherent worlds. It is useful when uncertainty is bundled: a high-demand scenario may also involve different costs, capacity pressures, or policy constraints. |
| Threshold Analysis ↗ | Threshold analysis searches for the point where the conclusion changes. It is especially useful for monitoring: if a future value crosses the flip point, the decision posture should change. |
| Tornado Chart ↗ | A tornado chart visualizes which inputs produce the largest outcome movement. It is a mechanism and artifact, not the archetype. It helps communicate results after the sensitivity logic has been applied. |
| Sensitivity Table ↗ | A sensitivity table records parameters, ranges, outcome responses, materiality judgments, and critical assumptions. It makes the analysis auditable. |
| Probabilistic Sensitivity Simulation ↗ | Probabilistic sensitivity simulation samples uncertain inputs from distributions or weighted cases. It can estimate how often a conclusion changes under plausible combinations, but it must not launder weak assumptions into false rigor. |
| Assumption Stress-test Workshop ↗ | A workshop mechanism brings stakeholders or experts into range-setting and interpretation. It is useful when assumptions are contested or value-laden. |
Parameter / Tuning Dimensions¶
The main tuning dimensions are parameter selection, range width, variation structure, outcome granularity, and materiality threshold. A protocol can be narrow and fast, using one-way tests on a few inputs, or broad and expensive, using multi-way scenarios or probabilistic sampling.
Other tuning decisions include whether ranges should represent ordinary uncertainty or extreme stress, whether parameters should vary independently or as linked bundles, whether results should be interpreted numerically or categorically, and whether the output should trigger validation, monitoring, redesign, deferral, or robust solution selection.
Invariants to Preserve¶
The baseline must remain fixed during the test. The watched outcome must remain stable. Variation ranges must be documented and defensible. Critical assumptions must be visible rather than hidden inside aggregate output. Materiality must be tied to decision relevance.
The most important invariant is diagnostic honesty. The protocol should not be used to protect a preferred conclusion. It should reveal the conditions under which that conclusion holds and the conditions under which it fails.
Target Outcomes¶
A good sensitivity analysis produces conditional confidence. It tells decision-makers what they can trust, what they should monitor, what they should validate, and what could change the decision.
The target outcomes are visible fragility, prioritized validation effort, clearer stakeholder disagreement, better monitoring triggers, and stronger downstream robust selection. The output should reduce false certainty without collapsing into indecision.
Tradeoffs¶
Sensitivity analysis trades speed for confidence, simplicity for coverage, and legibility for interaction realism. One-way tests are easy to explain but may miss joint effects. Scenario bundles are more realistic but can be contested. Probabilistic simulations can be rich but may be opaque.
There is also a social tradeoff. A rigorous sensitivity protocol may expose that a preferred recommendation is fragile. That is uncomfortable, but it is the point of the intervention.
Failure Modes¶
Common failures include range laundering, where analysts choose convenient ranges; parameter theater, where many irrelevant inputs are varied while critical ones are avoided; one-way blindness, where dependent parameters are varied separately; and chart substitution, where a tornado chart is mistaken for the full protocol.
Another failure is analysis paralysis. Sensitivity analysis is not complete until results change confidence, monitoring, validation, or decision posture. Endless variation without decision implication is not the archetype working; it is avoidance.
Neighbor Distinctions¶
Sensitivity Analysis Protocol is distinct from Constrained Resource Allocation. Allocation decides how to distribute resources under assumptions; sensitivity analysis tests whether the allocation conclusion survives plausible changes in those assumptions.
It is distinct from Robust Solution Selection. Robust selection chooses an option that remains acceptable across scenarios. Sensitivity analysis identifies the assumptions, thresholds, and scenarios that should inform such a choice.
It is distinct from Uncertainty Explicitness. Making uncertainty visible is useful, but sensitivity analysis actively tests what uncertainty does to the conclusion.
It is distinct from Perturbation Testing. Perturbation testing disturbs a system or model generally; sensitivity analysis varies assumptions with the specific goal of assessing conclusion fragility.
Variants and Near Names¶
Common variants include one-way sensitivity protocols, threshold sensitivity analysis, scenario sensitivity protocols, and probabilistic sensitivity protocols. Near names include assumption stress testing, robustness checks, sensitivity testing, and parameter sweeps.
Tornado charts and sensitivity tables should be retained as mechanisms or artifacts. They support the archetype but do not replace the protocol.
Cross-Domain Examples¶
In budget planning, a city can vary demand growth, labor cost, inflation, and grant timing to see whether a proposed service remains affordable. In infrastructure planning, engineers can vary load forecasts, outage probabilities, and material costs to see whether a design has adequate margin. In public health, analysts can vary prevalence, uptake, treatment effect, and adherence assumptions to see whether a screening recommendation remains beneficial.
In product strategy, a team can vary adoption, churn, support cost, and pricing assumptions before committing to launch. In procurement, a buyer can vary demand, supplier lead time, exchange rates, and defect rates to see whether a sourcing recommendation still holds.
Non-Examples¶
A spreadsheet that recalculates when someone edits cells is not enough. Without defined ranges, watched outcomes, materiality thresholds, and decision implications, it is not a sensitivity protocol.
A solver output under fixed assumptions is not sensitivity analysis. It may be optimization, but the archetype begins when assumptions are varied.
A team choosing the option that performs best across scenarios is closer to robust solution selection. Sensitivity analysis is the diagnostic step that reveals which assumptions and scenarios matter.