Sensitivity Probe¶
Test or assessment — instantiates Bounded Approximation
Varies key assumptions or inputs to see whether the approximate conclusion changes materially.
A Sensitivity Probe takes an existing approximate conclusion and systematically varies its key assumptions and inputs to find out whether the conclusion changes in a way that matters — turning "how wrong might the estimate be?" into "does the wrongness change the decision?" Its defining feature, and what sets it apart from every sibling here, is that it produces no approximation of its own: it consumes another mechanism's output and stress-tests its robustness. The probe tells you which assumptions the decision actually rests on — the ones that, if wrong, flip it — so precision can be spent there and nowhere else, and it triggers escalation to exact analysis precisely when the approximate answer proves fragile.
Example¶
A climate economist's model recommends adopting a moderate carbon price now rather than waiting. The recommendation rests on three chosen inputs: the discount rate, the damage function, and the assumed economic growth path. On its own the model outputs a single tidy verdict — but is that verdict robust, or an artifact of the inputs? The probe re-runs the model across a plausible range of each input. It finds the recommendation stable across the growth and damage assumptions — vary them widely and "act now" still wins — but fragile to the discount rate: somewhere between 1.5% and 4%, the recommendation flips from "act now" to "wait." That is the payoff. The discount rate is the binding assumption, so it is escalated to careful, exact analysis and open debate, while the other inputs can safely stay rough. The probe did not make the estimate better; it revealed where the estimate's roughness was and was not decision-relevant.
How it works¶
- List the assumptions and inputs. Enumerate what the approximate conclusion depends on.
- Vary each across a plausible range. Sweep inputs one at a time, or jointly, over ranges chosen from evidence rather than convenience.
- Watch the decision, not the number. Track whether the decision changes, not merely whether the output moves — small output shifts that never cross a decision boundary do not matter.
- Flag and escalate the binding inputs. Inputs that flip the decision are marked binding and sent to exact analysis; the rest are left at rough values.
Tuning parameters¶
- Inputs probed — which assumptions get varied. Covering only comfortable inputs manufactures false robustness; the honest sweep includes the ones you would rather not test.
- Range width — how far each input is pushed. Too narrow a range declares robustness cheaply and falsely; too wide wastes effort on implausible extremes.
- One-at-a-time vs joint — whether inputs move singly or together. One-at-a-time is cheap and readable but blind to interactions; joint variation catches inputs that are safe alone but dangerous in combination.
- Decision-flip threshold — how large a change counts as "material." A tight threshold surfaces more binding inputs; a loose one keeps focus on the few that truly decide.
When it helps, and when it misleads¶
Its strength is separating the assumptions that deserve precision from the many that do not — it tells a team exactly where to spend its scarce exactness budget, and it is what keeps an approximation from being trusted or distrusted wholesale. A common way to present its results is a tornado diagram, which ranks inputs by how much swinging each one moves the outcome.[n1]
Its failure mode is the blind spot of the cheapest version: varying inputs one at a time misses interaction effects, where two assumptions are each safe alone but jointly flip the decision, so an all-clear from a one-at-a-time sweep can be falsely reassuring. The classic misuse is probing only the inputs that are convenient or already favourable, staging a "robustness check" that was never at risk of failing. The guarding discipline is to probe joint variation wherever interactions are plausible and to choose ranges from evidence, not from what keeps the conclusion intact.
How it implements the components¶
validation_check— its core act: testing whether the approximation's conclusion holds up under input and assumption variation.acceptable_error— it operationalizes acceptable error as how much input error changes the decision, turning an abstract tolerance into a measured, decision-relevant quantity.exactness_escalation_rule— the inputs that flip the decision are exactly the ones flagged for escalation to exact analysis.
It implements no approximation_method of its own — it produces no simplified representation. Every sibling here, such as Back-of-Envelope Estimate or Surrogate Model, supplies the approximation this probe stress-tests.
Related¶
- Instantiates: Bounded Approximation — the validation mechanism that decides whether an approximation's roughness is decision-relevant.
- Consumes: Back-of-Envelope Estimate or Surrogate Model — it takes their approximate output as the thing to be probed.
- Sibling mechanisms: Back-of-Envelope Estimate · Rough Order-of-Magnitude Estimate · Surrogate Model · Simplified Simulation · Algorithmic Relaxation · Prototype Test · Policy Pilot
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Sensitivity Probe operates as an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution because it varies key assumptions or inputs to see whether the approximate conclusion changes materially.
Independent corroboration: The frozen evidence defines Sensitivity Probe as 'Varies key assumptions or inputs to see whether the approximate conclusion changes materially', so its operative form is Analysis, Modeling & Optimization.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Operations Research
Origin pattern: Convergent development
Present-day reach: Universal
Rationale: A small, deliberate perturbation used to expose which variables control an outcome is local sensitivity analysis. NASA describes sensitivity in terms of changing inputs and observing output contribution; operations research applies the probe to decision models.
Related originating lineages:
- Data Science & Analytics — Data science, analytics, and operational monitoring supplies a parallel or contributing lineage for the mechanism's defining operation: varies key assumptions or inputs to see whether the approximate conclusion changes materially.
- Engineering & Design — Back-of-envelope tolerance checks serve the same early-stage screening role.
- Mathematics — Mathematical modeling, proof, and abstract-structure practice supplies a parallel or contributing lineage for the mechanism's defining operation: varies key assumptions or inputs to see whether the approximate conclusion changes materially.
- Statistics & Experimental Design — Robustness probing gauges whether observed conclusions depend on analytic choices.
Review resolution: The blind reviewers disagree on primary lineage (operations_research versus statistics_experimental_design). Authoritative or primary research supports operations_research as the best historical origin: A small, deliberate perturbation used to expose which variables control an outcome is local sensitivity analysis. NASA describes sensitivity in terms of changing inputs and observing output contribution; operations research applies the probe to decision models. The cited NIST, Guide for the Use of the International System of Units: Model Sensitivity; NASA, Sensitivity Analysis Overview directly supports the mechanism's defining operation. All independently supported contributing domains are retained without an arbitrary cap. origin_mode=convergent records the lineage relationship, while domain_reach=universal records later applicability separately from provenance.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
- NIST, Guide for the Use of the International System of Units: Model Sensitivity
- NASA, Sensitivity Analysis Overview
Notes¶
[n1] A tornado diagram is a bar chart that ranks a model's inputs by how much varying each one, across its plausible range, moves the output — the widest bars on top, tapering downward like a funnel. It is the standard one-at-a-time visualization of sensitivity, and its shape makes the binding assumptions immediately visible while, by construction, saying nothing about interactions between inputs. ↩