Skip to content

Label Shift

The distribution shift in which the label marginal P(Y) changes between training and deployment while P(X|Y) stays fixed, so a classifier's discrimination survives but its calibration and thresholds miscalibrate — correctable by re-estimating the deployment prior rather than retraining.

Core Idea

Label shift is the form of distribution shift in which the label marginal P(Y) changes between development and deployment while the conditional P(X|Y) is assumed fixed. Classes become more or less prevalent than in training, so thresholds and calibrations tuned to the training prior quietly miscalibrate without any weight change. Because P(X|Y) holds, the model's discrimination is preserved — only calibration is wrong — so the fix is prior correction from a small unlabelled sample, not retraining.

Scope of Application

Label shift applies wherever a learned classifier has a prior-dependent calibration object and P(Y) can move while P(X|Y) holds — the precondition its confusion-matrix estimators need.

  • Clinical prediction — disease prevalence differing between a referral cohort and a community hospital.
  • Fraud and AML detection — the base rate moving with adversary activity and defensive posture.
  • Survey weighting — samples over- or under-representing classes; post-stratification is the analogue.
  • Hiring analytics — a changing proportion of qualified applicants in the pool.

Clarity

Naming label shift makes legible that "distribution shift" is three problems, not one — label, covariate, and concept shift — each with a different invariance assumption and a non-interchangeable correction. Its sharpest contribution is to make a puzzling signature interpretable: ROC AUC is prior-invariant and holds steady while precision collapses, converting an anomaly into a fingerprint that points at the moved prior and its cheap remedy.

Manages Complexity

"Distribution shift" as one undifferentiated category is unmanageable, with the default remedy — retrain — both expensive and often wrong. Label shift's factorisation collapses the space to a three-way sort keyed off which factor moved, reading the correct fix off the cell. Within its cell it decouples discrimination (prior-invariant, untouched) from calibration (mis-set), turning a baffling failure into a fingerprint whose minimal correction follows directly.

Abstract Reasoning

The concept is organized by a factorisation discipline. It licenses boundary-drawing / diagnostic-selection (identify which marginal or conditional moved, read off the cell's remedy), a decoupling of discrimination from calibration justified by fixed P(X|Y), a diagnostic-fingerprint move (unchanged AUC with collapsed precision points uniquely at the moved prior), and an interventionist move computing the minimal sufficient correction — with the P(X|Y)-fixed assumption itself treated as falsifiable.

Knowledge Transfer

Within supervised-learning evaluation the concept transfers as mechanism across every setting sharing one substrate — a learned classifier with a prior-dependent calibration object — so the factorisation, the discrimination/calibration decoupling, the fingerprint, and the BBSE/RLLS/MLLS correction carry intact from clinical prediction to fraud to survey weighting. Beyond that substrate the honest account is shared-abstract-mechanism: the base-rate-moves-decisions core belongs to the parent bundle base rate / Bayesian inference and updating / stationarity, which carries the cross-domain lesson; the estimators do not float free of the learned-classifier substrate.

Relationships to Other Abstractions

Local relationship map for Label ShiftParents appear above the current abstraction, mutual partners to the right, and children below. Node labels state whether each abstraction is prime or domain-specific; colors identify relation types.Label ShiftDOMAINDomain-specific abstraction: Probability Distribution — is part ofProbabilityDistributionDOMAIN

Current abstraction Label Shift Domain-specific

Parents (1) — more general patterns this builds on

  • Label Shift is part of Probability Distribution Domain-specific

    Label shift contains paired training and deployment probability distributions whose label marginals differ while their feature-given-label conditionals remain invariant.

Hierarchy paths (5) — routes to 3 parentless roots

Neighborhood in Abstraction Space

Label Shift sits in a moderately populated region (54th percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.

Family — Statistical Inference & Model Failure Modes (16 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-07-12