Decomposition Residual Reconciliation Workflow¶
Reconciliation workflow — instantiates Selection–Transmission Change Attribution
Takes the leftover after selection and transmission are subtracted from the observed change and attributes it to unmatched units, scale drift, or normalization rather than substance.
Every decomposition leaves a leftover, and the leftover is where analyses quietly go wrong. Decomposition Residual Reconciliation Workflow starts from the residual an accounting mechanism already reported — the part of the observed change the selection and transmission terms did not absorb — and decides whether that gap is a data artifact or genuine unexplained substance. Its defining discipline is directional: it works backward from a number it is handed, tracing the residual to unmatched units, measurement-scale drift, or a normalization choice, so the leftover is never mistaken for real change (and real change is never buried inside "residual"). It recomputes no terms of its own; it interrogates what the terms failed to explain.
Example¶
A retail chain's average transaction value moved between two quarters. The decomposition attributes most of the shift to selection (new high-ticket stores opened) and transmission (existing stores' baskets grew), but leaves a stubborn residual equal to about 40% of the total move — too large to ignore. The reconciliation workflow traces it. First, correspondence: twelve stores were remodeled and re-issued new IDs mid-quarter, breaking their linkage, so their change fell into neither term. Second, the value baseline: the point-of-sale system changed how returns net against sales partway through, drifting the value scale so the "same" metric meant different things across states. Third, normalization: a handful of closed stores were dropped inconsistently between the two snapshots. After re-linking the remodeled stores and harmonizing the returns definition, the residual shrinks to 4%, and that remainder is declared immaterial and set aside.
How it works¶
Begin with the reported residual and run three probes against it. Correspondence gaps: find unmatched, split, merged, or re-IDed units whose change escaped both terms, re-linking them where a defensible match exists. Value-baseline drift: test whether the measure meant the same thing in both states — measurement invariance — since a rescaled or redefined value fabricates residual that looks substantive.[n1] Normalization sensitivity: perturb the weighting convention and watch whether the residual moves, which reveals a leftover that is really an artifact of the baseline choice. The residual is attributed across these sources, fixes are applied, and the workflow re-runs to a cleaned residual it then declares material or immaterial. Crucially, it never recomputes the selection or transmission terms — those are inputs, and the workflow's job is to explain the gap they left.
Tuning parameters¶
- Materiality threshold — how large a cleaned residual is tolerable before the decomposition is declared unreliable. A tight threshold catches subtle artifacts but rejects more real-world messiness.
- Re-linking aggressiveness — how hard to work at matching unmatched units. Aggressive re-linking closes the residual but risks forcing matches that were never real.
- Invariance-test strictness — how much value-scale drift must be shown before it is blamed for the residual. Strict tests avoid over-attributing to measurement but may leave real drift unflagged.
- Probe order — which source is interrogated first. Front-loading the likeliest artifact resolves fast but can anchor the analyst on it and under-explore the others.
When it helps, and when it misleads¶
Its strength is that it stops the archetype's two mirror-image failures: reading a data artifact as real change, and hiding real change inside an unexamined residual. It is the auditor that keeps a decomposition honest after the arithmetic is done.
Its failure mode is that the same tools cut both ways. An aggressive analyst can explain away a substantive residual by attributing it to phantom artifacts — re-linking units that were genuinely replaced, or blaming "scale drift" that isn't there — and a lax one leaves a large residual undiagnosed and the decomposition unsupported. The guarding discipline is to fix causes and re-run rather than reclassify by assertion: every attribution must survive an actual re-linking or an actual invariance check, and the audit trail must show the residual shrinking because a real artifact was corrected, not because a label was changed.
How it implements the components¶
unit_correspondence_map— it probes correspondence gaps, finding unmatched, split, or re-IDed units whose change fell outside both terms and re-linking the defensible ones.counterfactual_weight_or_value_baseline— it tests the residual against the weighting and value baseline, perturbing the normalization and checking value-scale invariance to see whether the leftover is an artifact of those choices.stratification_layer— it localizes the residual by segment to find where the non-closure concentrates, narrowing the search for its source.
It does not produce the recomposition or the residual number itself (decomposition_identity_and_residual_check — that is produced by the Price Equation Decomposition Table, whose residual this workflow consumes) and it recomputes no channel (selection_term_definition, transmission_term_definition). Its nearest twin is that table: the table produces the identity and its residual, while this workflow takes that residual as input and attributes it to unmatched units, scale drift, or normalization — never recomputing the terms.
Related¶
- Instantiates: Selection–Transmission Change Attribution — it keeps unmatched units, drift, and normalization error from being read as substantive change.
- Consumes: Price Equation Decomposition Table supplies the terms and the residual this workflow reconciles.
- Sibling mechanisms: Price Equation Decomposition Table · Covariance Selection-Term Calculation · Within-Unit Change Assay · Lineage or Panel Correspondence Matrix · Composition-vs-Transformation Dashboard · Entry/Exit Normalization Protocol · Selection–Transmission Sensitivity Analysis
Editorial Notes¶
Form Classification¶
Form family: Analysis, Modeling & Optimization
Rationale: The method probes correspondence gaps, value-baseline drift, and normalization sensitivity, attributes the residual among them, and recomputes a cleaned remainder, so its operative form is reconciliation analysis.
Nearest alternative: Protocol, Workflow & Routine — The probes and rerun are ordered, but decomposition and attribution of the residual rather than coordination of a handoff sequence is the defining work.
Review outcome: Adjudicated after independent review; high confidence.
Origin Attribution¶
Primary origin: Biology & Ecology
Origin pattern: Cross-disciplinary synthesis
Present-day reach: Multi-domain
Rationale: Evolutionary biology is primary because the entry's named selection and transmission terms reproduce the Price-equation partition of total population change, including a remainder encompassing change not attributed to selection. Statistical measurement-invariance and sensitivity methods, together with audit reconciliation, supplied the probes that trace an anomalous remainder to correspondence, scale, and normalization defects.
Related originating lineages:
- Accounting & Auditing — Reconciliation practice supplied backward tracing from a non-closing remainder to unmatched units and inconsistent treatment.
- Statistics & Experimental Design — Measurement invariance and sensitivity analysis supplied tests for changed scales, definitions, and baseline-dependent residuals.
Review resolution: Evolutionary biology is primary because the entry's named selection and transmission terms reproduce the Price-equation partition of total population change, including a remainder encompassing change not attributed to selection. Statistical measurement-invariance and sensitivity methods, together with audit reconciliation, supplied the probes that trace an anomalous remainder to correspondence, scale, and normalization defects.
Attribution caveat: The decomposition is Price-equation lineage; the diagnostic cleanup workflow is an encyclopedia synthesis of statistical and audit controls.
Encyclopedia synthesis: The exact catalogued form synthesizes established practice rather than reproducing a single standard historical label.
Review outcome: Researched adjudication after independent review; high confidence.
Sources consulted:
- Frank (2012): Natural selection IV—the Price equation
- Frank (1997): The Price Equation, Fisher's Fundamental Theorem, Kin Selection, and Causal Analysis
Notes¶
[n1] Measurement invariance is the property that a measure means the same thing across the groups or times being compared. When it fails — a value scale is redefined or drifts between states — an apparent change is an artifact of the instrument, not the units, which is exactly the kind of false residual this workflow is built to catch. ↩