Monte Carlo Robustness Screen¶
Simulation method — instantiates Robust Solution Selection
Samples many plausible parameter combinations to estimate how often each candidate remains acceptable, with sampling assumptions documented.
The Monte Carlo Robustness Screen replaces a handful of named scenarios with thousands of sampled ones. Instead of asking "how does each candidate do in the drought case, the surge case, the disruption case," it puts distributions on the uncertain parameters, draws many random combinations from them, and evaluates every candidate on each draw. The output is not a verdict but a frequency: the share of sampled futures in which a candidate stays above the acceptability line — a probabilistic robustness metric. Its distinctive move is turning "acceptable across scenarios" into an estimated probability of acceptability, and, as a by-product, revealing which uncertain inputs actually drive the failures. It measures how often an option survives, not how badly it loses or how far it falls short of the best.
Example¶
A general contractor is choosing between two delivery plans for a mid-rise build: a fast-track plan that overlaps design and construction, and a sequential plan that finishes design first. The owner cares about one acceptability line — finishing within 14 months and under a hard budget cap. Rather than argue three point-estimate scenarios, the planner puts distributions on the drivers: permit-approval time, subcontractor availability, weather delay days, and material-price escalation, each ranged from the firm's own historical project records.
The screen draws ten thousand combinations. In each draw it runs both plans through the schedule-and-cost model and marks pass or fail against the 14-month, under-cap line. Fast-track passes in 62% of draws; sequential passes in 81%. Crucially, the run also reports which input separates the outcomes: fast-track's failures cluster almost entirely on permit-approval delay, because its overlap assumes permits land on time. That single finding reframes the decision — the owner can either accept the sequential plan's higher pass rate or keep fast-track and spend effort de-risking permits specifically. The numbers are illustrative, but the shape of the answer is what the method delivers.
How it works¶
The method is a loop over sampled worlds. Assign each uncertain parameter a distribution grounded in data or elicited ranges, and specify the correlations between them where they matter. Draw a large sample of joint parameter vectors. For each draw, evaluate every candidate through the outcome model and test it against the acceptability threshold, tallying pass/fail. Aggregate to a robustness metric — the estimated probability of acceptability per candidate — and, from the failed draws, attribute failure to inputs (which parameters, in which ranges, break which candidate). The documented sampling assumptions travel with the result; without them the frequency is a number with no provenance.
Tuning parameters¶
- Sample size — more draws tighten the probability estimate but cost compute. Match precision to how close the candidates' pass rates are.
- Input distributions — the shapes and ranges assigned to each parameter. This is where most of the answer is decided; fat tails and correlations change conclusions more than the model does.
- Acceptability line — the pass/fail test applied to each draw. A single threshold gives a clean frequency; multiple lines give a richer but busier picture.
- Correlation structure — whether inputs are drawn independently or jointly. Ignoring correlation between, say, weather and subcontractor delay flatters robustness.
- Failure attribution depth — a simple driver ranking vs. a full variance decomposition. Deeper attribution guides de-risking but costs analysis time.
When it helps, and when it misleads¶
Its strength is coverage: it explores a combinatorially large space of futures no named-scenario set could enumerate, and it distinguishes an option that survives 90% of plausible worlds from one that survives 60% — a distinction a pass/fail scenario table cannot make. The Monte Carlo method's power is exactly this estimation of a distribution of outcomes by repeated random sampling.[1] The failure-attribution by-product is often more valuable than the headline probability, because it names what to fix.
It misleads through the garbage-in problem: the frequency is only as credible as the input distributions, and analysts routinely under-specify tails and ignore correlations, manufacturing false confidence. The tidy percentage also invites treating a modeled probability as an empirical one. The classic misuse is reporting "83% robust" without disclosing the assumed distributions, so a reader cannot see that the answer was baked into the priors. The guarding discipline is to publish the distributions and correlations alongside the metric, stress the tails deliberately, and treat the output as conditional on the sampling assumptions rather than as a fact about the world.
How it implements the components¶
uncertainty_scenario_set— the sampled parameter draws are the scenario set, generated from distributions rather than hand-named.robustness_metric— the estimated probability of remaining acceptable across draws is the transparent robustness figure the screen reports per candidate.sensitivity_analysis_input— attributing failed draws to the inputs that caused them feeds directly back as sensitivity evidence for where to de-risk.
This is a sampling screen, not a selection or a scorer: it does not compute a regret_measure against the scenario-best — that's Regret Analysis — nor search for and generate the optimal candidate_solution_set over an uncertainty set, which is Robust Optimization Model.
Related¶
- Instantiates: Robust Solution Selection — supplies the probabilistic robustness evidence over a sampled uncertainty envelope.
- Sibling mechanisms: Regret Analysis · Robust Optimization Model · Decision Matrix Under Uncertainty · Maximin / Satisficing Rule · Minimax Decision Rule · Robust Policy Design Review · Scenario Robustness Check · Stress-Tested Plan Review
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: The mechanism evaluates candidates over sampled parameter worlds to estimate acceptability probabilities and attribute failure regions to uncertain inputs.
Nearest alternative: Experiment, Test & Rehearsal — The sampled worlds are an offline uncertainty calculation that produces robustness estimates, not a controlled exposure of a real target or practiced execution.
Review outcome: Quality-audited after independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Operations Research
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Sampling uncertain parameters to rank candidates by frequency of acceptable performance is a standard robust-decision and operations-research method.
Related originating lineages:
- Engineering & Design — Reliability and sensitivity analysis materially apply such screens to engineered designs.
- Statistics & Experimental Design — Distributional sampling and uncertainty quantification supply the probabilistic experiment.
Review resolution: Both independent reviews agree on primary origin operations_research; reconciliation resolves secondary fields (origin_mode_disagreement). Alternate origins retained (engineering_design, statistics_experimental_design) are the union of reviewer-supported formative lineages with explicit rationales, not a list of later application domains. Present-day breadth is represented separately as domain_reach=multi_domain; origin_mode=cross_disciplinary_synthesis records the historical relationship among lineages. Confidence is conservatively reconciled to high, and encyclopedia_synthesis=false preserves either reviewer's finding that the encyclopedia generalized the mechanism.
Review outcome: Reconciled after independent review; high confidence.
References¶
[1] Metropolis, N., & Ulam, S. "The Monte Carlo Method". Journal of the American Statistical Association 44(247), 335–341 (1949). Describes Monte Carlo computation as proceeding by repeated random sampling. registry ↩