Weight-Sweep Sensitivity Table¶
Test or assessment — instantiates Aggregation Function Design and Weighting
Re-runs an existing composite score across a plausible range of weights and records where the ranking holds and where it flips.
A Weight-Sweep Sensitivity Table takes an aggregation whose weights are already set and stress-tests those weights: it varies each weight across the range a reasonable person might defend, recomputes the ranking each time, and tabulates which conclusions survive and which invert. Its whole identity — the thing that separates it from the scoring template it inspects — is that it never assigns a weight; it perturbs weights it is handed, so it can report whether the headline result is a robust finding or an artifact of one arbitrary coefficient. The output is a verdict about fragility, not a score: "the top choice is stable across every plausible weighting" is a very different claim from "the top choice changes if cost weight drops five points."
Example¶
A national magazine publishes a college ranking built from a weighted score — graduation rate, faculty resources, selectivity, alumni giving, reputation survey. A skeptical analyst does not argue about which weights are "correct"; she builds a sweep. Holding the published formula as the base case, she moves each weight up and down across the band the editors could plausibly justify — say, graduation rate anywhere from 15% to 30% of the total — and recomputes the top-25 each time.
The table shows the result: the top five schools never move, but ranks 12 through 20 reshuffle constantly, and one school swings from #14 to #24 depending only on how heavily alumni giving is weighted. The finding is not "the ranking is wrong" but "the ranking is decisive at the top and noise in the middle" — and the audit trace records exactly which weight each flip depended on, so a reader can see that a school's precise rank is an artifact of an editorial choice rather than a fact about the school.
How it works¶
- Freeze everything but the weights. The input set, scales, and rule family stay fixed so any change in output is attributable to weights alone.
- Define plausible ranges, not point values. Each weight gets a defensible interval; the sweep explores that space rather than one guess.
- Recompute and log outcomes. One-at-a-time moves show single-weight fragility; joint moves catch interactions the one-at-a-time view misses.[1]
- Report stability, not a new winner. Summarize which rankings hold across the whole space and flag every pair whose order flips, with the weight that caused it.
Tuning parameters¶
- Sweep width — how far each weight is allowed to move. Narrow bands understate fragility; overly wide bands manufacture instability from weights no one would defend. Anchor the band to what stakeholders could actually justify.
- One-at-a-time vs joint — perturbing one weight or several together. Joint sweeps catch interaction effects but explode combinatorially; one-at-a-time is cheap but can miss a flip that needs two weights to move at once.
- Flip threshold — how large a rank change counts as "the result is fragile." A strict threshold (any top-3 change) flags more; a loose one (only the winner) reports less.
- Resolution — how finely the range is sampled. Finer grids locate the exact tipping point but cost compute and can over-report near-ties.
When it helps, and when it misleads¶
Its strength is turning "is this ranking real?" into a checkable answer: it separates conclusions that hold across every reasonable weighting from those that ride on one contestable coefficient, and it does so without anyone having to win the argument over the "true" weights. It is the honest companion to any weighted score, and its verdict rightly downgrades a fragile winner from decision to coin-flip.
Its failure mode is that a sweep is only as honest as its ranges: draw the bands too narrow and everything looks robust; draw them to include only the weightings that keep the favored answer on top and the "test" becomes theater. It can also drown a decision in caveats — reporting that something changes somewhere is easy and useless; the value is in saying what decision-relevant conclusion is fragile. The guarding discipline is to fix the plausible ranges from stakeholder reasoning before running the sweep, and to report fragility against the actual decision at stake, not against every reshuffle in the table.
How it implements the components¶
sensitivity_check— this is the mechanism's core: it asks whether plausible weight changes materially alter the result and answers with the range over which each conclusion holds.audit_trace— the table itself is a reproducible record of every weighting tried and every ranking it produced, so a reader can see and rerun exactly which coefficient drove each flip.
It does not implement weight_assignment_scheme or aggregation_rule — it perturbs weights it does not set and re-runs a rule it does not define; both belong to Weighted Scoring Rubric. Its qualitative counterpart, which checks for named bias patterns rather than numeric fragility, is Aggregation Bias Audit.
Related¶
- Instantiates: Aggregation Function Design and Weighting — the sweep is the robustness test of the pattern.
- Consumes: Weighted Scoring Rubric — it takes a finished composite score as its input and stress-tests that score's weights.
- Sibling mechanisms: Weighted Scoring Rubric · Median, Trimmed-Mean, or Quantile Rule · Ranked-Choice or Approval Voting Rule · Ensemble Weighting Table · Dashboard Rollup Formula · Aggregation Bias Audit
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Weight-Sweep Sensitivity Table operates as an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution because it re-runs an existing composite score across a plausible range of weights and records where the ranking holds and where it flips.
Independent corroboration: The frozen evidence defines Weight-Sweep Sensitivity Table as 'Re-runs an existing composite score across a plausible range of weights and records where the ranking holds and where it flips', so its operative form is Analysis, Modeling & Optimization.
Nearest alternative: Assessment, Review & Assurance — Weight-Sweep Sensitivity Table includes features of a bounded evaluation of existing evidence or work that produces a finding or disposition, but its defining operation is an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Universal
Rationale: OECD/JRC Handbook on Constructing Composite Indicators documents that composite-indicator practice requires explicit weights and sensitivity analysis because rankings can change with value judgments. This is direct, mechanism-specific evidence for statistics experimental design as the best-evidenced historical home of the operation—Re-runs an existing composite score across a plausible range of weights and records where the ranking holds and where it flips.—rather than evidence merely that the operation is useful there. The retained alternates record genuine adjacent lineages; later portability is represented separately by domain_reach=universal.
Related originating lineages:
- Data Science & Analytics — Data science, analytics, and operational monitoring supplies a parallel or contributing lineage for the mechanism's defining operation: re-runs an existing composite score across a plausible range of weights and records where the ranking holds and where it flips.
- Mathematics — Mathematical modeling, proof, and abstract-structure practice supplies a parallel or contributing lineage for the mechanism's defining operation: re-runs an existing composite score across a plausible range of weights and records where the ranking holds and where it flips.
- Organizational & Management Science — Organizational Management supplies a historically relevant adjacent lineage or formative practice for the operation—Re-runs an existing composite score across a plausible range of weights and records where the ranking holds and where it flips.—but the adjudicated evidence more directly locates the defining lineage in statistics experimental design.
- Systems Thinking & Cybernetics — Systems science's feedback, boundaries, control, and regulation tradition contributes a separate formative lineage to the mechanism's weight sweep sensitivity table logic.
Review resolution: The blind reviewers disagree on primary lineage (organizational_management versus statistics_experimental_design). The defining operation is: Re-runs an existing composite score across a plausible range of weights and records where the ranking holds and where it flips. The researched OECD/JRC Handbook on Constructing Composite Indicators establishes that composite-indicator practice requires explicit weights and sensitivity analysis because rankings can change with value judgments. That source therefore supports statistics experimental design as the historical origin. organizational management remains in the uncapped alternates where it contributes a formative practice, but application or governance is not itself proof of origin. origin_mode=single_lineage records lineage construction; domain_reach=universal separately records later applicability.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
References¶
[1] Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. Global Sensitivity Analysis: The Primer. John Wiley & Sons (2008). Explains how one-at-a-time analysis isolates individual-input effects and how joint or total-effect analysis reveals interactions it can miss. registry ↩