Subgroup Disaggregation Audit¶
Test or assessment — instantiates Welfare Analysis and Distributional Effects Assessment
Breaks down aggregate impacts by subgroup, geography, role, income, exposure, access, or vulnerability to reveal hidden losses and uneven gains.
The Subgroup Disaggregation Audit takes a reported aggregate and drills into it, splitting the affected population along candidate cut lines — age, geography, income, disability, language, exposure, access — to detect losses and exclusions that the average conceals. Its defining idea is descriptive discovery: it asks where impact actually lands and whether an apparent subgroup gap is real signal or small-sample noise, but it renders no verdict on whether any given outcome is acceptable. It is a measurement that surfaces concentration inside a headline, not a rule that says a subgroup outcome is out of bounds. Finding the loss is its job; judging it belongs to a different sibling.
Example¶
A regional health provider replaces many in-person appointments with telehealth. The aggregate looks great: patient-reported access and satisfaction both rise, satisfaction by roughly eight points. The audit refuses to stop at the mean. It disaggregates by age band, by broadband availability, by primary language, and by disability status. The picture inverts in one corner: the gains are driven by younger urban patients, while patients over seventy-five in low-broadband rural areas show falling completed-visit rates and rising missed follow-ups — a real loss the eight-point average had swallowed whole.
Because those rural-elderly cells are small, the audit checks whether the drop survives as signal or dissolves as noise; it survives. It then validates the cut lines with a patient advisory group, who note that "rural" is bundling two very different populations that should be split. The output is a documented map of exactly who lost access — passed to the guardrail test to judge against the provider's access commitments, and to compensation review. The audit describes the loss; it deliberately does not declare a floor breached.
How it works¶
- Select candidate cut lines. Choose the axes most likely to hide harm — exposure, access, vulnerability, geography — and mark which are pre-specified versus exploratory.
- Drill and compare. Compute the incidence within each subgroup and hunt for the "average-driven-by-the-majority" pattern where a strong mean masks a losing minority.
- Separate signal from noise. Test whether a subgroup gap survives sampling variation and the particular cut chosen, before it is reported as real.
- Validate the categories. Check the subgroup definitions against affected groups so the cuts reflect lived, experienced distinctions rather than convenient analytic buckets.
Tuning parameters¶
- Cut-line set — which axes to split on. More axes catch more hidden harm but raise the chance of a spurious finding and the cost of the audit.
- Disaggregation depth — single-axis versus intersectional cuts (e.g. rural and elderly and low-income). Intersections find the sharpest concentration but shrink cell sizes fast.
- Noise threshold — the minimum cell size or confidence a gap must clear to count. Loose thresholds cry wolf; strict ones miss small but real harms.
- Category authorship — analyst-defined versus participatory categories. Participatory cuts find the real fault line but slow the audit and add coordination cost.
- Stopping rule — how deep to keep slicing. Slice forever and you will always "find" a losing subgroup somewhere.
When it helps, and when it misleads¶
Its strength is that it defeats aggregate masking with evidence rather than assertion: it converts "on average it helps" into a concrete map of who, specifically, is worse off behind a favorable mean.
Its failure mode is the mirror image of that power — slice finely enough and a "losing" subgroup always appears, most of them noise. This is the multiple-comparisons trap[1], and it makes unplanned subgroup findings notoriously unreliable; fine granularity also raises privacy risk, and analyst-chosen categories can quietly hide the fault line that matters. The classic misuse is fishing a subgroup story to order — to kill a policy or to sell one. The guarding discipline is to pre-specify the primary cuts, treat any post-hoc split as a hypothesis rather than a result, and validate the categories with the people they describe.
How it implements the components¶
This audit realizes the discovery side of incidence — finding concentration, not ruling on it:
benefit_burden_incidence_model— it computes incidence within each subgroup, splitting the aggregate into per-group gains and losses.uncertainty_and_sensitivity_frame— it tests whether a subgroup gap survives sampling noise and the cut-line choice before reporting it, applying sensitivity to an empirical, not a normative, question.participation_and_validation_channel— it checks that the subgroup categories reflect lived distinctions by validating them with affected groups.
It does not decide whether a surfaced loss violates a floor or proportionality limit (equity_guardrail_set, minimum_floor_constraint) — that verdict is Equity Guardrail Test, its nearest twin. The audit measures where impacts land; the guardrail test judges those impacts against fixed, declared rules.
Related¶
- Instantiates: Welfare Analysis and Distributional Effects Assessment — it supplies the disaggregated evidence the rest of the assessment reasons over.
- Sibling mechanisms: Distributional Incidence Matrix · Counterfactual Welfare Comparison · Value-Weight Sensitivity Analysis · Compensation Adequacy Review · Externality and Spillover Inventory · Equity Guardrail Test · Public Reason Disclosure Protocol
Editorial Notes¶
Form Classification¶
Form family: Assessment, Review & Assurance
Rationale: Subgroup Disaggregation Audit operates as a bounded evaluation of existing evidence or work that produces a finding or disposition because it breaks down aggregate impacts by subgroup, geography, role, income, exposure, access, or vulnerability to reveal hidden losses and uneven gains.
Independent corroboration: The frozen evidence defines Subgroup Disaggregation Audit as 'Breaks down aggregate impacts by subgroup, geography, role, income, exposure, access, or vulnerability to reveal hidden losses and uneven gains', so its operative form is Assessment, Review & Assurance.
Nearest alternative: Analysis, Modeling & Optimization — Subgroup Disaggregation Audit includes features of an analytical, modeling, inference, comparison, or optimization procedure that derives insight or a solution, but its defining operation is a bounded evaluation of existing evidence or work that produces a finding or disposition.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Public Administration & Policy
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Universal
Rationale: Disaggregating impacts reveals uneven public gains and losses.
Related originating lineages:
- Economics & Finance — Economics, finance, and mechanism-design practice supplies a parallel or contributing lineage for the mechanism's defining operation: breaks down aggregate impacts by subgroup, geography, role, income, exposure, access, or vulnerability to reveal hidden losses and uneven gains.
- Law & Governance — Legal doctrine, regulatory governance, and procedural accountability supplies a parallel or contributing lineage for the mechanism's defining operation: breaks down aggregate impacts by subgroup, geography, role, income, exposure, access, or vulnerability to reveal hidden losses and uneven gains.
- Organizational & Management Science — Organizational design, management, and operational governance supplies a parallel or contributing lineage for the mechanism's defining operation: breaks down aggregate impacts by subgroup, geography, role, income, exposure, access, or vulnerability to reveal hidden losses and uneven gains.
- Sociology & Anthropology — Sociology and anthropological study of institutions and social relations supplies a parallel or contributing lineage for the mechanism's defining operation: breaks down aggregate impacts by subgroup, geography, role, income, exposure, access, or vulnerability to reveal hidden losses and uneven gains.
- Statistics & Experimental Design — Valid denominators support estimates.
- Ethics of Technology & AI Governance — Vulnerable groups may be hidden.
Review resolution: The blind reviewers agree that public_administration_policy is the primary origin and differ only on reported ambiguity, alternate origin disagreement, origin mode disagreement, domain reach disagreement, encyclopedia synthesis disagreement. I preserve every independently explained alternate from both records rather than imposing a numeric cap. I retain cross_disciplinary_synthesis because the combined evidence shows material contributions from several lineages. The broader reach of universal records portability separately from historical provenance; encyclopedia_synthesis=true preserves the affirmative synthesis judgment where either reviewer identified one.
Attribution caveat: Distributional disaggregation spans policy evaluation, welfare economics, statistics, and sociology.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Reconciled after independent review; medium confidence.
References¶
[1] Wang, R., Lagakos, S. W., Ware, J. H., Hunter, D. J., and Drazen, J. M. “Statistics in Medicine — Reporting of Subgroup Analyses in Clinical Trials.” The New England Journal of Medicine 357, no. 21 (2007): 2189–2194. Warns that post hoc subgroup analyses are vulnerable to multiplicity and inflated false-positive findings. registry ↩