Subgroup Residual Heatmap¶
Diagnostic visualization — instantiates Residual-Driven Model Refinement
Tiles average residual across two crossed segmentations so a subgroup the overall fit hides lights up as a hot cell.
The Subgroup Residual Heatmap finds where the misfit lives. Its defining move is to break the residuals out along two crossed segmentations — region by product, age-band by channel, cohort by site — and colour each resulting cell by its average signed residual, so that a subgroup the model systematically over- or under-serves appears as a saturated tile even when the residual pooled over everyone is essentially zero. Its distinct question is who, not what shape: where a residual-versus-fitted plot asks whether the mean function bends and a Q-Q plot asks about the tails, the heatmap asks which slice of the population the model quietly fails — the question a single average is structurally incapable of answering.
Example¶
An education-technology company builds a model predicting whether a student will complete an online course. Overall it looks excellent: calibration is tight and the mean residual is essentially zero. Then the team tiles the residuals by school-funding tier crossed with learner age-band, and one corner of the grid glows deep red — adult learners at low-funding schools are consistently predicted to complete at far higher rates than they actually do. The flattering aggregate was two large errors cancelling: over-prediction in that corner offset by mild over-correction elsewhere.
The hot cell is simultaneously a fairness flag and a directed revision target. It says a feature the model needs for that subgroup — perhaps available support hours, or a prior-preparation gap — is missing or mis-weighted, and it says so precisely enough to act on. The heatmap turned a reassuring headline metric into a specific, accountable failure attached to an identifiable group.[n1]
How it works¶
- Cross two segmentations that matter to the domain and to fairness, then aggregate the signed residual within each cell (a mean, or a calibration gap).
- Colour by magnitude and sign so over- and under-prediction read as opposite hues, and shade or size cells by their sample count so a two-record cell cannot shout as loudly as a thousand-record one.
- Let the eye go to saturated, well-populated cells — those are systematic subgroup misfits rather than noise.
- Protect sensitive slices explicitly — a cell that is small in the data but high-stakes in the world is examined regardless of its colour, not allowed to hide behind a low count.
Tuning parameters¶
- Segmentation choice — which two axes are crossed; the axes you decline to draw conceal the failures they would have revealed, so the fairness-relevant ones are included on purpose.
- Cell aggregate — mean signed residual versus mean absolute error versus a calibration gap; the signed form finds direction and bias, the absolute form finds magnitude.
- Minimum cell size — the count below which a cell is greyed rather than trusted; set it too low and sampling noise masquerades as a hot spot.
- Colour scale and centring — a diverging scale centred at zero versus a sequential one; centring at zero is what makes a systematic bias legible as colour rather than washing it out.
When it helps, and when it misleads¶
Its strength is catching concentrated subgroup failures and fairness problems that every aggregate statistic averages away — it turns "the model is well-calibrated" into the sharper "well-calibrated for whom," which is often the question that actually matters.
Its weaknesses come with fine slicing. Cut the data on enough axes and some cell will look bad purely by chance — a multiple-comparisons trap — and the boundaries of the bins can be drawn to manufacture a hot cell or to dissolve one. A red cell also shows only that a subgroup fails, never why. The classic misuse runs in both directions: dicing until a cell supports a pre-chosen narrative, or coarsening the bins until a real disparity disappears. The discipline is to fix the segmentations and the cell-size threshold in advance, correct for the number of cells scanned, examine high-stakes slices regardless of count, and confirm any flagged cell on held-out data before acting on it.
How it implements the components¶
stratified_residual_view— it is the stratified view: residuals resolved across crossed strata instead of pooled into one number.evaluation_population_and_alignment_boundary— defining the cells is defining the population partition and the boundary within which each subgroup's fit is judged.safety_and_fairness_slice_guardrail— by surfacing per-slice bias it enforces the guardrail that no protected subgroup is allowed to hide inside a healthy-looking average.
It shows where misfit concentrates but does not size a single influential record — that is the Influence and Leverage Diagnostic — nor test slice error out-of-sample, which is the Cross-Validated Error-Slice Report; interpreting a hot cell into a cause is Residual Root-Cause Review.
Related¶
- Instantiates: Residual-Driven Model Refinement — it localizes the loop's misfit to the subgroups that carry it, and flags the fairness slices that must not be averaged away.
- Sibling mechanisms: Residual Root-Cause Review · Residual-versus-Fitted Plot · Cross-Validated Error-Slice Report · Influence and Leverage Diagnostic · Quantile-Quantile Residual Check · Posterior-Predictive Residual Check
Editorial Notes¶
Form Classification¶
Form family: Interface, Display & Cue
Rationale: Subgroup Residual Heatmap operates as a user-facing prompt, display, template, or perceptual cue that shapes attention and action at the point of use because it tiles average residual across two crossed segmentations so a subgroup the overall fit hides lights up as a hot cell.
Independent corroboration: The frozen evidence defines Subgroup Residual Heatmap as 'Tiles average residual across two crossed segmentations so a subgroup the overall fit hides lights up as a hot cell', so its operative form is Interface, Display & Cue.
Nearest alternative: Structure, Architecture & Configuration — Subgroup Residual Heatmap includes features of a configured physical, technical, or logical arrangement whose structure creates the effect, but its defining operation is a user-facing prompt, display, template, or perceptual cue that shapes attention and action at the point of use.
Review outcome: Independent reviewer agreement; medium confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Specialized
Rationale: A matrix of signed residuals across subgroup and condition cells is statistical residual diagnostics rendered as a heatmap. NIST residual-analysis guidance uses residual plots to expose structure, nonconstant variance, and model failure; fairness work determines salient groups.
Related originating lineages:
- Data Science & Analytics — Data science, analytics, and operational monitoring supplies a parallel or contributing lineage for the mechanism's defining operation: tiles average residual across two crossed segmentations so a subgroup the overall fit hides lights up as a hot cell.
- Human-Computer Interaction — Heatmaps make hot cells legible.
- Mathematics — Mathematical modeling, proof, and abstract-structure practice supplies a parallel or contributing lineage for the mechanism's defining operation: tiles average residual across two crossed segmentations so a subgroup the overall fit hides lights up as a hot cell.
- Ethics of Technology & AI Governance — tech_ethics_ai_governance contributes technology ethics and AI governance to this mechanism's defining operation—Tiles average residual across two crossed segmentations so a subgroup the overall fit hides lights up as a hot cell—without displacing the selected primary historical lineage.
Review resolution: The blind reviewers disagree on primary lineage (data_science versus statistics_experimental_design). Authoritative or primary research supports statistics_experimental_design as the best historical origin: A matrix of signed residuals across subgroup and condition cells is statistical residual diagnostics rendered as a heatmap. NIST residual-analysis guidance uses residual plots to expose structure, nonconstant variance, and model failure; fairness work determines salient groups. The cited NIST/SEMATECH, Residual Analysis directly supports the mechanism's defining operation. All independently supported contributing domains are retained without an arbitrary cap. origin_mode=cross_disciplinary_synthesis records lineage, while domain_reach=specialized records later applicability separately from provenance.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
Notes¶
The heatmap is in-sample and descriptive: a hot cell is a hypothesis about a subgroup, not yet a confirmed defect. Before it drives a model change it should be reproduced out-of-sample — that is the Cross-Validated Error-Slice Report's job — and interpreted in a review, since the tile shows the failure but never its cause.
[n1] Simpson's paradox — a relationship, or here a near-zero average, in the aggregate can reverse or vanish once the data are disaggregated, because opposing subgroup effects cancel. It is the standard reason a globally good residual can conceal a systematically failing subgroup. ↩