Deviation Residual Table¶
Diagnostic table — instantiates Deviant Case Analysis
Displays the predicted or expected pattern against the observed case evidence so the magnitude and type of deviation are explicit.
Before anyone argues about why a case deviates, it helps to see, laid out plainly, how far and in what direction it does. The Deviation Residual Table puts the predicted or expected value beside the observed one for each case or period, computes the signed gap between them — the residual — and ranks and flags those gaps so their size, sign, and persistence are readable at a glance. Its defining move is that it is a display that also screens: alongside each large residual it records a measurement and data-quality check, so a gap is not treated as a real anomaly until it has survived the boring explanation that the data are simply wrong. It quantifies the deviation and separates signal from artifact; it does not decide the selection rule and does not explain the gap.
Example¶
A regional grocery chain's demand model forecasts weekly unit sales per store from seasonality, promotions, and weather. One suburban store's frozen-dessert sales run roughly 40% above forecast for six consecutive weeks (an illustrative figure). The residual table renders the situation in columns: predicted units, observed units, the raw residual, that residual as a percentage and as a standardized score, a persistence count, and — critically — a data-quality flag. The flag column is where the table earns its keep: was a promotion mis-coded to the wrong store, did a competitor two blocks away close, did a point-of-sale timestamp glitch double-count a shift? Reconciling the POS logs clears the artifact hypotheses. What the table now shows is unambiguous — the residual is large (beyond three standard deviations), persistent (six weeks, not one), and real (not a coding error). That is exactly what promotes this store from "doing great" to "worth explaining," and it hands the next step a screened, sized anomaly rather than a rumor.
How it works¶
- Line up expected against observed. One row per unit or period; the predicted value beside the actual one.
- Compute the residual in comparable form. Raw gap, standardized score, and percentage, so residuals across differently-scaled units can be ranked together.
- Rank and flag by magnitude and persistence. A one-week spike and a six-week drift are different animals; the table makes that visible.
- Screen each flagged residual for artifact. Attach a measurement and data-quality check — coding, timing, definition changes, missingness — before the gap is treated as substantive.
The table stops at this gap is real and this big. It never says what caused it.
Tuning parameters¶
- Residual metric — raw versus standardized versus percentage. Standardized scores are comparable across units; raw gaps are more intuitive to a domain reader.
- Flag threshold — how large or persistent a residual must be to be flagged. A low bar catches subtle deviants but floods the table with noise.
- Artifact-screen strictness — how hard each flagged residual is checked for measurement error. Strict screening avoids chasing ghosts but costs effort.
- Aggregation grain — per-case versus per-cell (case × period), trading resolution against readability.
When it helps, and when it misleads¶
Its strength is turning an impression — "that store is on fire" — into a sized, screened, ranked residual, and doing the unglamorous work of catching a coding error before anyone builds a theory on top of it. Its honest failure mode is that a table shows magnitude, not validity: a huge residual can come from a single high-leverage observation that is distorting the fitted model rather than revealing a substantive anomaly — an influential point, not an insight.[1] The classic misuse is fixating on the one giant residual while ignoring that the model is biased across a whole subgroup of smaller ones. The guarding discipline is built into the table itself: every flagged residual carries its measurement-and-data-quality screen, so an artifact is caught in the same artifact that surfaces the anomaly, rather than discovered three steps later.
How it implements the components¶
The table realizes the archetype's make-the-deviation-explicit step — the display and screen that stand between noticing and explaining:
residual_pattern_scan— the table is the residual scan rendered: expected minus observed, sized, signed, and ranked across cases and periods.data_quality_and_measurement_check— each flagged residual carries a coding, timing, and measurement screen before it is called substantive.
It does not set the rule for what counts as deviant (deviation_criterion — the Deviant Case Selection Protocol), arrange the matched comparison cases (matched_or_contrast_comparison_set — the Matched Case Pairing Matrix), or list the candidate causes of the gap (rival_explanation_inventory — the Omitted Variable Probe).
Related¶
- Instantiates: Deviant Case Analysis — supplies the archetype's deviation-display-and-screen step.
- Consumes: Deviant Case Selection Protocol for the case and the expected pattern it tabulates against.
- Sibling mechanisms: Deviant Case Selection Protocol · Matched Case Pairing Matrix · Omitted Variable Probe · Within-Case Process-Tracing Memo · Follow-Up Case Sampling Plan · Boundary Condition Revision Workshop · Rival Explanation Elimination Table
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: Deviation Residual Table operates as a computation, comparison, model, or analytic representation used to infer, estimate, or choose because it displays the predicted or expected pattern against the observed case evidence so the magnitude and type of deviation are explicit.
Independent corroboration: The frozen evidence defines Deviation Residual Table as 'Displays the predicted or expected pattern against the observed case evidence so the magnitude and type of deviation are explicit', so its operative form is Analysis, Modeling & Optimization.
Review outcome: Independent reviewer agreement; high confidence.
Origin Attribution¶
Primary origin: Statistics & Experimental Design
Origin pattern: Single lineage
Present-day reach: Multi-domain
Rationale: Regression diagnostics cohered residual tables that compare predicted with observed values, rank signed gaps, and check leverage and data quality.
Related originating lineages:
- Data Science & Analytics — Applied analytics operationalized residual review as a sortable diagnostic display.
Review resolution: Regression diagnostics cohered residual tables that compare predicted with observed values, rank signed gaps, and check leverage and data quality. The retained alternate lineages materially shaped the mechanism's form.
Review outcome: Reconciled after independent review; high confidence.
References¶
[1] Cook, R. D. "Detection of Influential Observation in Linear Regression". Technometrics 19(1), 15–18 (1977). Defines an influential observation by how materially its deletion changes the fitted least-squares model. registry ↩