Distributional Sensitivity Grid¶
Sensitivity review — instantiates Distributional-Assumption Governance
Runs each plausible family, tail, dependence, and parameter choice all the way through to the final decision output to see whether the conclusion actually moves.
Diagnostics that stop at fit answer the wrong question. The question a decision cares about is not "does this distribution fit?" but "if I had chosen a different-but-plausible distribution, would I have acted differently?" Distributional Sensitivity Grid answers exactly that. It takes the set of defensible modeling choices — family, tail, dependence structure, transformation, parameter values — as given, and pushes each combination all the way through to the actual decision output: the threshold that gets crossed, the ranking that gets published, the count that gets funded, the interval that gets quoted. Then it reports whether the consequential conclusion is stable or swings, and, crucially, which assumption is doing the swinging. Its defining move is that sensitivity must reach the decision, not stop at a fit statistic — a model can be wrong in ways that never touch the action, and right in ways that do. The grid holds the data fixed and varies the modeling commitments; it does not assemble the rival families and it does not resample the observations.
Example¶
A social-policy agency estimates how many households fall below the income cutoff for a benefit, from a model of the household-income distribution. Rather than argue about the "right" model, analysts build a sensitivity grid over the plausible choices: lognormal versus a two-component mixture; independent household income sources versus correlated ones; with and without a log transform; and a range of parameter values. Every combination is run through to the one number that governs the program — the count of eligible households.
The grid's finding is not that one family fits better. It is that the eligible-household count barely moves when the family changes but swings by tens of thousands when income sources are modeled as correlated rather than independent, because correlation thickens the lower joint tail where eligibility is decided. The load-bearing assumption turns out to be dependence, not the marginal family everyone was debating. The agency responds by widening the reported uncertainty, flagging the dependence assumption to the reviewer, and examining which subgroups the swing falls on — a governance response the raw fit statistics would never have prompted.
How it works¶
- Fix the menu, vary systematically. The plausible commitments are taken as inputs (from the comparison of families) and crossed into a grid of scenarios.
- Dependence is a first-class axis. Because correct marginals do not validate a joint model, the grid varies independence-versus-clustering and tail-dependence explicitly, not just the marginal family.
- Propagate to the decision, not the fit. Each cell is carried through the real decision rule, and the output recorded is the action, not a goodness-of-fit number.
- Name the dominant assumption. The grid reports which axis moves the decision most, so the argument and any fallback attach to that variable.
Tuning parameters¶
- Axis breadth — how many families, tails, dependence structures, and parameter values enter the grid. Broader grids are more honest but explode combinatorially.
- Decision output chosen — which downstream quantity is treated as "the decision." Choosing a mild output hides sensitivity the real action would feel.
- Dependence structures included — how many clustering and tail-dependence patterns are tried; omitting them is the most common way a grid misses the load-bearing axis.
- Material-movement tolerance — how large a swing counts as "the decision changed." Set it loose and everything looks robust; set it tight and nothing does.
When it helps, and when it misleads¶
Its strength is that it closes the "decision disconnect" — the failure where diagnostics are dutifully reported but plausible alternatives are never propagated to the action. It tells you not just that misspecification exists but whether it matters, and which assumption to worry about.
Its failure mode is the garden of forking paths run in reverse: with enough axes, an analyst can hunt for the specification that yields the wanted answer and report only that cell. The honest version of the same combinatorial machinery is multiverse analysis — reporting the decision across the whole set of defensible specifications[1] rather than cherry-picking one. The discipline that keeps the grid honest is to fix the axes before seeing the outcomes, report the full spread rather than a favorable corner, and treat a decision that flips across defensible specifications as a signal to widen uncertainty or escalate, not as a menu to choose from.
How it implements the components¶
decision_consequential_sensitivity_map— the grid's core output: a map of how the estimate, threshold, ranking, or interval moves across plausible commitments and parameter uncertainty.joint_dependence_and_conditional_structure— dependence is an explicit axis the grid varies and propagates, since clustering and tail dependence are often the assumptions that actually move a pooled or multi-stage decision.
The grid consumes rival families rather than assembling them — building the credible candidate set (alternative_family_and_assumption_light_comparator) is the Candidate-Family Comparison Grid — and it holds the sample fixed rather than perturbing it, so the finite-sample resampling of empirical_shape_and_diagnostic_profile belongs to the Resampling Robustness Audit.
Related¶
- Instantiates: Distributional-Assumption Governance — the grid is the step that propagates misspecification into the decision so acceptance and fallback can be evidence-based.
- Consumes: Candidate-Family Comparison Grid — the credible families it varies across.
- Sibling mechanisms: Candidate-Family Comparison Grid · Resampling Robustness Audit · Tail and Boundary Stress Scenario · Independent Assumption-Challenge Gate
Draft mechanism page for the Encyclopedia of Abstractions.
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Distributional Sensitivity Grid operates as a computation, comparison, model, or analytic representation used to infer, estimate, or choose because it runs each plausible family, tail, dependence, and parameter choice all the way through to the final decision output to see whether the conclusion actually moves.
Independent corroboration: The frozen evidence defines Distributional Sensitivity Grid as 'Runs each plausible family, tail, dependence, and parameter choice all the way through to the final decision output to see whether the conclusion actually moves', so its operative form is Analysis, Modeling & Optimization.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Multi-domain
Rationale: Robustness and multiverse analysis cohered carrying all defensible distribution families, tail assumptions, dependencies, and parameters through to the decision outcome.
Related originating lineages:
- Data Science & Analytics — Model-risk practice operationalizes specification sweeps at scale and visualizes decision fragility across cells.
Review resolution: Both current reviews place distributional_sensitivity_grid primarily in statistics_experimental_design; the reconciled classification retains only lineages that materially shaped the mechanism and keeps breadth of origin separate from reach.
Review outcome: Reconciled after independent review; high confidence.
References¶
[1] Steegen, S., Tuerlinckx, F., Gelman, A., and Vanpaemel, W. "Increasing Transparency Through a Multiverse Analysis". Perspectives on Psychological Science 11(5), 702–712 (2016). Defines multiverse analysis as evaluating results across the full set of reasonable analytical specifications rather than selectively reporting one analysis. registry ↩