Aggregation Sensitivity Test¶
Sensitivity analysis — instantiates Part-Level Explanatory Reduction
Varies the aggregation and bridge-rule assumptions to reveal how much a whole-level result is an artifact of how the parts were combined.
Every reduction hides a bridge rule — the assumption about how part-level facts add up into a whole-level claim — and that rule is usually invisible until you disturb it. The Aggregation Sensitivity Test leaves the parts exactly as they are and perturbs the composition instead: it re-derives the whole-level result under different weightings, different assumptions of additivity versus interaction, different groupings, and different scale transitions, then measures how much the answer moves. Its defining move is that the object of the experiment is the bridge rule, not the constituents. A whole-level conclusion that flips or swings wildly when you change how the parts are combined was never a fact about the parts; it was an artifact of the aggregation. The test's job is to tell the two apart and to log which composition choices the whole-level claim actually depends on.
Example¶
A national statistics office publishes a single "wellbeing index" and reports that wellbeing rose this year, a whole-level claim that policy hangs on. The index aggregates part-level indicators — income, health, education, safety — but the choice of how to combine them is buried. The Aggregation Sensitivity Test holds the underlying indicators fixed and re-computes the index under alternative bridge rules: equal weights versus expert weights, a simple sum versus a rule that penalizes imbalance across dimensions, national aggregation versus a population-weighted regional roll-up.
Under equal weights the index rises; under the imbalance-penalizing rule it falls, because gains concentrated in income masked a decline in safety. The whole-level story — "wellbeing rose" — turns out to be an artifact of one aggregation choice, and the office learns that its headline claim is not robust to how the parts are combined. This is a live instance of Simpson's paradox territory[n1]: recomposed one way the trend points up, recomposed another it points down, from the very same parts. The test does not decide which aggregation is right — that is a normative judgment it deliberately refuses — it only exposes how much rides on the choice and logs the composition assumptions the claim is fragile to.
How it works¶
- Surface the bridge rule. Extract the currently-implicit aggregation — the weights, the additivity assumption, the grouping, the scale transition — and make it an editable object.
- Enumerate plausible alternatives. List the other defensible ways the same parts could be composed, spanning the assumptions most likely to be contestable.
- Recompose and record the swing. Re-derive the whole-level result under each alternative, logging the recomposition and how far the answer moves each time.
- Report the dependence, not a verdict. Name which composition choices the claim is sensitive to; the test flags fragility, it does not pick the correct rule.
Tuning parameters¶
- Perturbation breadth — how many aggregation assumptions you vary (weights only, versus weights and additivity and grouping). Broader coverage finds more fragility but costs runs and can overwhelm interpretation.
- Alternative plausibility filter — how defensible an alternative rule must be to include. A permissive filter stress-tests hard but risks strawman aggregations; a strict one may miss a real vulnerability.
- One-at-a-time vs. joint — varying assumptions singly versus in combination. Joint variation catches interactions between composition choices but explodes the design.
- Robustness threshold — how large a swing counts as "the claim is fragile." A tight threshold flags almost everything; a loose one may bless a genuinely brittle result.
- Level granularity — the scale at which recomposition happens (individual, regional, national). Coarser aggregation smooths variation but is where ecological artifacts hide.
When it helps, and when it misleads¶
Its strength is that it targets the one thing most reductions hide — the bridge rule — and separates whole-level conclusions that are real (stable across defensible aggregations) from those that are artifacts of an arbitrary composition choice. It is the natural guard against bridge-rule smuggling, the failure where aggregation assumptions ride along unexamined.
Its failure mode is that a sensitivity sweep can devolve into paralysis or false reassurance. Vary enough assumptions and every claim looks fragile; vary only flattering ones and a brittle result looks robust — the test is only as honest as the alternatives you were willing to try. It also quantifies how much the answer moves without judging which composition is correct, so a team can mistake "robust across the rules I tried" for "true." And it says nothing about whether the leftover, unexplained variation is emergent or contextual — that is a different mechanism's job. The guarding discipline is to pre-register the set of defensible aggregations before seeing results, report the full swing rather than the most convenient run, and route any large residual to a review rather than treating stability as proof.
How it implements the components¶
Aggregation Sensitivity Test fills the composition-robustness side of the archetype:
aggregation_bridge_rule— it makes the bridge rule the explicit object of experiment, surfacing and varying the assumptions that turn parts into a whole-level claim.reversibility_and_recomposition_log— each re-derivation under a new rule is a logged recomposition of the same parts, recording how the whole reassembles differently.scope_and_level_boundary— varying the level of aggregation (individual → regional → national) tests how sensitive the claim is to where the level boundary is drawn.
It does NOT implement counterfactual_part_substitution_set — that's Ablation or Knockout Test, its nearest twin: Ablation perturbs the parts while holding composition fixed, whereas this test holds the parts fixed and perturbs the composition. It also leaves emergence_residual_register and holism_escape_condition to Residual Explanation Review, which judges what the leftover residual means rather than measuring the aggregation swing.
Related¶
- Instantiates: Part-Level Explanatory Reduction — supplies the robustness check that keeps a bridge rule from smuggling in the answer.
- Consumes: Bottom-Up Simulation — recomposes the simulation's or model's part-level outputs under varied aggregation rules.
- Sibling mechanisms: Part Inventory Matrix · Interaction Graph Analysis · Mechanism Chain Diagram · Bottom-Up Simulation · Ablation or Knockout Test · Residual Explanation Review · Scope Clause and Exception Note
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Varies the aggregation and bridge-rule assumptions to reveal how much a whole-level result is an artifact of how the parts were combined, making its operative form a computation or analytic transformation that produces an inference, comparison, or optimized result.
Independent corroboration: The frozen evidence defines Aggregation Sensitivity Test as 'Varies the aggregation and bridge-rule assumptions to reveal how much a whole-level result is an artifact of how the parts were combined', so its operative form is Analysis, Modeling & Optimization.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Multi-domain
Rationale: Sensitivity analysis in statistics varies defensible model and aggregation assumptions while holding observations fixed to measure how inferential conclusions depend on those choices.
Related originating lineages:
- Data Science & Analytics — Model robustness practice supplies reproducible recomputation across alternative pipeline and grouping choices.
- Economics & Finance — Composite indicators and policy indices made weighting and aggregation-rule sensitivity a prominent applied concern.
- Mathematics — Formal aggregation operators and invariance provide its structural basis.
Review resolution: Both reviews place the method in statistical sensitivity analysis. Data science, mathematical aggregation, and economic composite indicators materially contribute, and the page's explicit bridge-rule perturbation is a synthesized generalization of that established single lineage.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; high confidence.
Notes¶
[n1] Simpson's paradox is the reversal in which a trend that appears in aggregated data disappears or flips when the data are grouped differently. It is the sharpest illustration of aggregation sensitivity: the same part-level facts support opposite whole-level conclusions depending on how they are combined, which is exactly the artifact this test is built to detect. ↩