Ground-Truth Drift¶

The model-evaluation failure in which the operational definition of the correct answer drifts on a clock the monitoring apparatus cannot see, so metrics keep scoring against a moved target while the dashboards stay green — invisible because every detector consumes current ground truth as its reference.

Core Idea¶

Ground-truth drift is the model-evaluation failure in which the operational definition of the correct answer — the label that a supervised learning, monitoring, or scoring system treats as truth — changes over time while the measurement apparatus surrounding it appears constant. Labels continue to be produced, metrics continue to be computed, dashboards continue to update, but what the label means is no longer what it meant when the model was trained. The apparatus does not flag the change because it consumes current ground truth as its reference; comparison to historical ground truth is not performed.

The mechanism is a mismatch between two clocks: the clock on which the labeling rule, taxonomy, clinical criteria, or policy threshold is updated by external actors, and the clock on which the model's performance is tracked. Because the labeling pipeline's machinery — the team, the software, the reporting cadence — stays nominally unchanged, nothing in the operational monitoring signals that the target has moved. The harm is systematic: apparent changes in model performance (an apparent accuracy drop, a rising positive rate, a changing calibration curve) are in fact changes in what positive means, and the diagnostic ambiguity — is the model degrading, or is the label definition expanding? — is unresolvable without explicit versioning of the labeling rule and retrospective comparison of labels applied to identical cases under old and new definitions. The distinction from concept drift is precise: concept drift is P(Y|X) changing because the underlying phenomenon changed; ground-truth drift is the Y-definition itself changing, with the underlying phenomenon and the feature space potentially unchanged. Catching it requires institutional inspection of the labeling process, not statistical monitoring of model outputs.

Structural Signature¶

Sig role-phrases:

the supervised apparatus — a model, monitor, or scoring system whose metrics are computed against a label it treats as truth
the labeling pipeline — the operational machinery (team, software, reporting cadence) that produces the reference label, nominally unchanged
the operational definition — the contingent rule, taxonomy, clinical criteria, or policy threshold that defines what a positive case is
the definition's hidden clock — the schedule on which external actors revise that definition, distinct from the performance-tracking clock
the silent change — a revision to the definition that leaves no trace in the monitoring infrastructure because every detector consumes current ground truth as its reference
the metrics-against-moving-truth — performance numbers scored against a target whose meaning has moved while the dashboards stay green
the diagnostic ambiguity — an apparent performance change that is in fact a definition expansion ("is the model worse, or did 'positive' expand?"), resolvable only off the labeling process, not output statistics
the four-way decomposition — the closed sort of "why is performance different?": covariate shift (P(X) moved), label shift (P(Y) moved), concept drift (P(Y|X) moved because the world changed), and this — label-definition drift (the Y-definition itself moved while world and features may be untouched)
the like-for-like catch — versioning the labeling rule and re-scoring identical cases under old and new definitions, the only operations that compare against historical rather than current ground truth

What It Is Not¶

Not model degradation. The defect lives in the definition, not the model: the apparatus is scoring against a target whose meaning has moved. The operational reflexes — the model is decaying, retrain it; the population shifted, recollect — are misdirected, because nothing about the model or the inputs need have changed.
Not concept drift. Concept drift is P(Y|X) moving because the world changed; ground-truth drift is the Y-definition itself moving while the world and the feature space may be untouched. This is the boundary practitioners most need drawn — the same accuracy curve, two utterly different causes and remedies.
Not covariate or label shift. Covariate shift is P(X) moving and label shift is P(Y) moving; ground-truth drift is the fourth, distinct box — label-definition drift — in the decomposition of "why is performance different?" Sorting an anomaly into the wrong box sends the wrong investigation.
Not detectable by statistical output monitoring. Every standard detector consumes current ground truth as its reference, so a change in what "positive" means is structurally invisible to it. Only versioning the labeling rule and re-scoring identical cases under old and new definitions — comparisons against historical ground truth — can catch it.
Not Goodhart's law. Goodhart is a measure becoming a target and being gamed; ground-truth drift is upstream of that — the measure's definition itself changes, with no optimising agent required. The label moves on an external clock, not in response to pressure on the metric.
Not gold-standard erosion. Its twin emphasizes the authority of the reference declining; ground-truth drift emphasizes the silent change in the operational definition. Both are faces of reference-standard decay, but the diagnostic here is a moved definition, not a weakened one — and the catch is a like-for-like re-score, not a re-validation of the reference's standing.

Scope of Application¶

Ground-truth drift lives across the application areas of supervised model evaluation and monitoring — wherever a model is scored against a contingent operational label that external actors can quietly revise on a clock the apparatus does not see; its reach is within that one substrate (supervised evaluation against a contingent label) sampled across many domains. The cross-substrate analogues (revised legal categories, DSM/ICD criteria changes) belong to the parent reference_standard_decay, not to this P(Y)-definition machinery.

Clinical prediction — the operational definition of "sepsis," "stroke," or "readmission" revised as clinical criteria update, so post-revision labels score the model against a moved target.
Hiring analytics — shifting HR taxonomies of "successful hire," "high performer," or "voluntary exit," expanding or contracting the positive class beneath the model.
Forecasting evaluation — realized economic series revised retrospectively, so today's "ground truth" for a past quarter's GDP differs from the value available when the forecast was scored.
Fraud detection — the operational rule for flagging "fraud" expanding (e.g. adding first-party fraud), inflating the apparent positive rate as a definition change rather than a detection improvement.
Content moderation — an evolving definition of "policy violation," so old labels were applied under older rules than the ones now scoring the classifier.
NLP and data annotation — tightened annotator instructions, so new annotations are produced under a different rule than the cohort the model was trained on.

Clarity¶

Naming ground-truth drift makes visible a failure that leaves no trace in the monitoring infrastructure: the labeling pipeline is running, the metrics are computed, the dashboards are green — yet the numbers are being scored against a target whose meaning has moved. The label pins the failure to the definition rather than the operation, which is exactly where the standard ML toolkit cannot look: every detector consumes current ground truth as its reference, so a change in what "positive" means is invisible unless someone has versioned the labeling rule and re-scored identical cases under old and new definitions. Without the name, a team facing a shifting accuracy curve reaches for the operational explanations — the model is degrading, retrain it; the input population shifted — none of which touch a defect that lives in the label, not the model or the data.

Its sharpest clarifying work is to add a fourth, distinct entry to the decomposition of "why is performance different?" Alongside input-distribution drift (covariate shift), output-prior drift (label shift), and conditional drift (concept drift) sits label-definition drift — and the distinction from concept drift is the one practitioners most need drawn. Concept drift is P(Y|X) moving because the world changed; ground-truth drift is the Y-definition itself moving while the world and the feature space may be untouched. That separation tells the analyst that the diagnostic — is the model worse, or did "positive" expand? — is unresolvable by more statistical monitoring of outputs, and redirects the investigation to where the answer actually lives: institutional inspection of how the labeling rule, taxonomy, or threshold has been revised, on whose clock, and whether the old and new labels agree on the same case.

Manages Complexity¶

When a model's metrics move — accuracy slips, the positive rate climbs, the calibration curve bends — a monitoring team confronts an open-ended list of candidate causes and a matching list of reflexes: the model is decaying, retrain it; the input population shifted, recollect; the instruments are noisy, recalibrate; a fraud spike, a seasonal effect, a pipeline bug. Each shifting curve invites its own investigation, and because every standard detector consumes current ground truth as its reference, none of these reflexes can surface a defect that lives in the label itself. Ground-truth drift compresses this by completing a small closed decomposition of the single question "why is performance different?" into exactly four mutually exclusive sources: input-distribution drift (covariate shift, P(X) moved), output-prior drift (label shift, P(Y) moved), conditional drift (concept drift, P(Y|X) moved because the world changed), and label-definition drift (the Y-definition itself moved while world and features may be untouched). The analyst stops enumerating bespoke operational stories and instead asks which of four boxes a given anomaly falls in.

That decomposition does the load-bearing work because the fourth box behaves unlike the other three and the concept tells the analyst exactly how. Once a performance change is suspected of being ground-truth drift, the relevant state collapses to two clocks and one agreement test: the clock on which external actors revise the labeling rule, taxonomy, clinical criteria, or policy threshold, the clock on which the model's performance is tracked, and whether old and new labels, applied to identical cases, still agree. Those few facts settle the otherwise-unresolvable diagnostic ambiguity — is the model worse, or did "positive" expand? — and they redirect effort away from the entire apparatus of statistical output monitoring (which is structurally blind here, since it scores against the very target that moved) toward institutional inspection of the labeling process. So a high-dimensional "what went wrong with this model" investigation reduces to a four-way sort plus, in the drift branch, a two-clock-and-one-comparison check from which the qualitative answer and the correct line of inquiry both follow.

Abstract Reasoning¶

Ground-truth drift licenses reasoning moves built on one decomposition and one anchoring insight: every supervised performance change has exactly one of four sources, and the label a system treats as truth is a contingent operationalization that drifts on its own clock. The most distinctive is a diagnostic relocation move that points where the standard toolkit cannot look. Facing a shifting accuracy curve, rising positive rate, or bending calibration, the analyst reasons that the defect may live in the definition rather than the operation — the labeling pipeline is running, the metrics are computed, the dashboards are green, yet the numbers are scored against a target whose meaning has moved. The reasoning explicitly rejects the operational reflexes (the model is degrading, retrain it; the input population shifted, recollect) as misdirected when the fault is in the label, and predicts that no statistical detector can surface it, because every detector consumes current ground truth as its reference and so is structurally blind to a change in what "positive" means unless someone versioned the labeling rule and re-scored identical cases under old and new definitions.

The governing move completes a small closed decomposition of "why is performance different?" into four mutually exclusive sources — input-distribution drift (covariate shift, P(X) moved), output-prior drift (label shift, P(Y) moved), conditional drift (concept drift, P(Y|X) moved because the world changed), and label-definition drift (the Y-definition itself moved while world and features may be untouched) — and reasons by sorting any anomaly into one of four boxes rather than enumerating bespoke operational stories. The boundary-drawing sub-move the analyst most needs is the line between the third and fourth boxes: concept drift is P(Y|X) moving because the world changed; ground-truth drift is the Y-definition moving while the world and the feature space may be untouched. Reasoning from that distinction, the analyst concludes that the diagnostic question — is the model worse, or did "positive" expand? — is unresolvable by more output monitoring and must be answered elsewhere.

Once the fourth box is suspected, a two-clocks-and-an-agreement-test move collapses the relevant state to three facts and reasons from them. The analyst compares the clock on which external actors revise the labeling rule, taxonomy, clinical criteria, or policy threshold against the clock on which the model's performance is tracked, and asks whether old and new labels, applied to identical cases, still agree. Reasoning from a clock mismatch plus disagreement on identical cases, the analyst attributes the performance change to a label-definition expansion rather than a model change — and predicts the apparatus stayed silent precisely because the labeling pipeline's machinery (the team, the software, the reporting cadence) is nominally unchanged, so nothing in operational monitoring signals that the target moved. This is the move that resolves the otherwise-unresolvable ambiguity.

The interventionist and order-of-investigation consequences follow directly. Because the defect lives in the label, the analyst redirects effort away from the entire apparatus of statistical output monitoring toward institutional inspection of the labeling process: how the rule, taxonomy, or threshold has been revised, on whose clock, and whether the revision is documented (often only in passing, in a fraud team's quarterly briefing or an annotator-instruction update). The prevention prediction is that only explicit versioning of the labeling rule plus retrospective re-scoring of identical cases under old and new definitions can catch this class, since those are the only operations that compare against historical rather than current ground truth. The recurring habit the concept installs is to treat every "truth" a supervised system consumes as a construct operationalization that can drift on a clock the apparatus does not see, and so to ask of any performance change not only "is the model worse?" but "did the meaning of the label move, on whose schedule, and do old and new labels still agree on the same case?"

Knowledge Transfer¶

Within supervised evaluation and monitoring the concept transfers as mechanism, and the transfer is literal because every instance is the same substrate: a supervised apparatus scored against a contingent operational label. The same four-way decomposition of "why is performance different?" (covariate shift, label shift, concept drift, label-definition drift), the same boundary it draws against concept drift (the Y-definition moved, not P(Y|X)), the same two-clocks-and-an-agreement-test (compare the labeling rule's revision clock against the performance-tracking clock; ask whether old and new labels agree on identical cases), and the same prevention prescription (version the labeling rule; re-score identical cases under old and new definitions, because only that compares against historical rather than current ground truth) carry without translation across clinical prediction (revised criteria for "sepsis," "stroke," "readmission"), hiring analytics (shifting taxonomies of "successful hire" or "voluntary exit"), forecasting evaluation (retrospectively revised economic series), fraud detection (an expanded "fraud" rule), content moderation (an evolving "policy violation"), and NLP annotation (tightened annotator instructions). The vocabulary does not strain across these because they share the one feature the mechanism needs: a model consuming a label it treats as truth, where that label is an operationalization external actors can quietly revise.

Beyond ML the honest account is a shared abstract mechanism — case (B). Strip the model-evaluation idiom and a substrate-neutral structure remains: an apparatus is anchored to a reference standard whose definition drifts on a clock the apparatus does not see, so the apparatus keeps scoring against a moved target while its machinery looks unchanged. That structure genuinely recurs across distinct substrates as co-instances, not resemblances — legal categories used in enforcement whose statutory or interpretive definition shifts, medical diagnostic criteria revised across DSM/ICD editions, "proficient" redefined in educational standards, "revenue" redefined in accounting rules. In each, the general pattern — reference-standard decay, where a series is silently being measured against a definition that no longer matches the one it was anchored to — is the thing that travels and carries a real, transferable lesson: version the reference, and compare like-for-like across the change rather than trusting that a stable-looking pipeline means a stable target. What does not travel is the ML-specific cargo: the P(X)/P(Y)/P(Y|X)/Y-definition decomposition, label versioning and re-scoring of cohorts, the supervised-monitoring apparatus, and the "ground-truth drift" name are data-science furniture that does not survive extraction. So when the cross-domain lesson is wanted, the honest move is to carry the parent — reference-standard decay (the unified emergent reference_standard_decay that subsumes ground-truth drift and its twin gold_standard_erosion), connected upstream to calibration (since if the reference drifts, calibration to it follows) — not the term "ground-truth drift," whose machinery is substrate-bound, and whose faithful instances are one substrate (supervised evaluation against a contingent label) sampled across many domains (see Structural Core vs. Domain Accent).

Examples¶

Canonical¶

Clinical sepsis prediction is a clean instance. In 2016 the "Sepsis-3" consensus redefined sepsis from the older SIRS-based criteria (two or more of abnormal temperature, heart rate, respiratory rate, or white-cell count with suspected infection) to a definition centered on organ dysfunction, operationalized by an acute rise of 2 or more points in the SOFA score. A model trained and validated on patient records labeled "septic" under the SIRS-era rule, then evaluated on records labeled under Sepsis-3, is scored against a materially different positive class — some SIRS-positive patients are no longer septic, and some organ-dysfunction patients newly are. Its apparent accuracy, positive rate, and calibration shift, but the model, the features, and the underlying patient physiology may be unchanged; only the definition of the label moved, on the clock of a consensus committee, not the monitoring dashboard.

Mapped back: "Septic" is the operational definition; the 2016 consensus revision is a change on the definition's hidden clock, invisible to output monitoring. The resulting metric shift is the metrics-against-moving-truth, and the unanswerable "is the model worse or did sepsis get redefined?" is the diagnostic ambiguity — resolvable only by re-scoring identical patients under both definitions, the like-for-like catch.

Applied / In Practice¶

Macroeconomic forecast evaluation lives with this routinely because official statistics are revised retrospectively. The U.S. Bureau of Economic Analysis releases GDP in an advance estimate, then revises it in second and third estimates and again in annual and comprehensive benchmark revisions, sometimes years later. A forecaster whose model predicted last quarter's GDP is scored against a "ground truth" figure that itself keeps changing: the real-time vintage available when the forecast was made can differ substantially from today's revised value. If evaluation silently uses the latest revised series, the model's apparent error reflects both genuine forecast error and the drift in the target figure. Careful forecast-evaluation practice therefore uses real-time "vintage" databases (such as the Philadelphia Fed's real-time dataset) to score each forecast against the data as it stood then.

Mapped back: The published GDP value is the operational definition of truth for a quarter; BEA's revision schedule is the definition's hidden clock. Scoring against the latest vintage is the metrics-against-moving-truth, conflating forecast error with target drift — the diagnostic ambiguity — and using vintage databases to compare like-for-like is precisely the like-for-like catch.

Structural Tensions¶

T1: The stable-looking pipeline versus the moved definition (stability as camouflage). The labeling pipeline keeps running, metrics keep computing, dashboards stay green — and it is precisely this nominal unchangedness of the machinery (the team, the software, the reporting cadence) that hides the drift. In almost every other failure mode a steady, healthy-looking apparatus is reassuring evidence that nothing is wrong; here it is the camouflage, because the defect lives in the definition the apparatus consumes, not in any operation it performs. The tension is that the very operational stability a monitoring team is trained to check for and take comfort in is, for this failure, the thing that guarantees it stays invisible — a green board is consistent with a target that has silently walked away. Diagnostic: Is the reassurance coming from the pipeline's operational steadiness, or from a demonstration that the label's definition has not moved?

T2: Current ground truth as reference versus historical like-for-like (the blindness is designed in). Every detector scores against current ground truth — the property that makes real-time evaluation possible at all and also makes the apparatus structurally unable to see a change in what "positive" means. The failure cannot be fixed by monitoring harder, because more monitoring consumes the same moved reference. The only catch compares against historical ground truth: version the labeling rule and re-score identical cases under old and new definitions. The tension is that the reference choice that makes ongoing evaluation tractable is the same choice that blinds it to definition drift, and the remedy requires stepping outside the live pipeline to reconstruct a rule that no longer runs. Diagnostic: Is this comparison scored against the label as it is defined now, or against the definition as it stood when the model was trained?

T3: Concept drift versus label-definition drift (the boundary practitioners most need drawn). The same accuracy slip, positive-rate climb, or bending calibration is produced by two utterly different causes: P(Y|X) moving because the world changed (concept drift, box three) or the Y-definition itself moving while world and features are untouched (ground-truth drift, box four). They demand opposite remedies — retrain on fresh data versus audit the labeling rule — so sorting the anomaly into the wrong box sends the entire investigation the wrong way. The tension is that the two are empirically indistinguishable from output statistics alone; the surface signal is identical, and only inspection of the labeling process, off the dashboard, separates them. Diagnostic: Did the phenomenon behind the label change, or did the rule for assigning the label change, on the same underlying cases?

T4: Statistical sophistication versus institutional inspection (the wrong instrument, however sharp). The entire apparatus of statistical output monitoring — drift detectors, calibration tracking, distribution tests — is structurally the wrong instrument here, because it scores against the very target that moved. The answer lives in a low-tech place the ML toolkit does not point to: institutional inspection of how the labeling rule, taxonomy, or threshold was revised, on whose clock, documented perhaps only in a fraud team's quarterly briefing or an annotator-instruction update. The tension is that the more sophisticated the statistical monitoring, the more it can reassure a team it has looked everywhere, when the one place holding the answer is a committee's meeting notes, not a metric. Diagnostic: Is the investigation still interrogating model outputs, or has it moved to who revised the definition and when?

T5: Analytic exclusivity versus empirical superposition (four clean boxes, one messy anomaly). The four-way decomposition presents covariate shift, label shift, concept drift, and label-definition drift as mutually exclusive sources — which they are analytically. But a single real performance change can be a genuine model decay and a definition expansion at once, superposed in the same curve. The two-clocks-and-agreement test attributes the change to the definition when clocks mismatch and old and new labels disagree on identical cases, yet that attribution does not by itself certify the model is otherwise fine. The tension is that the decomposition's clean partition invites a single-cause verdict, while an anomaly may draw from several boxes at once, and only the residual after re-scoring like-for-like reveals whether anything remains to explain. Diagnostic: After re-scoring identical cases under both definitions, does the performance gap close, or is there residual change no definition move accounts for?

T6: A silently moved definition versus a gamed or eroded reference (bounding the diagnosis). Ground-truth drift is flanked by look-alikes it must be pried from. It is not Goodhart's law: Goodhart needs an optimizing agent gaming a measure that became a target, whereas here the measure's definition itself changes on an external clock with no agent pressing on the metric. It is not gold-standard erosion, its twin: that emphasizes the authority of the reference declining, catchable by re-validating its standing, whereas this emphasizes a silent change in the operational definition, catchable by a like-for-like re-score. The tension is that all three are faces of a reference going wrong, and folding any in prescribes the wrong catch — an anti-gaming control, a re-validation, or a re-score. Diagnostic: Did the definition move (re-score like-for-like), lose authority (re-validate the reference), or get gamed under pressure (anti-Goodhart control)?

T7: Autonomy versus reduction (its own named ML failure or an instance of reference-standard decay). "Ground-truth drift" is a specific model-evaluation failure with proprietary machinery — the P(X)/P(Y)/P(Y|X)/Y-definition decomposition, label versioning, cohort re-scoring, the supervised-monitoring apparatus — and inside supervised evaluation it transfers intact across clinical prediction, hiring analytics, forecasting, fraud, moderation, and annotation, because all are one substrate: a model scored against a contingent label. But beyond ML the named thing does not travel; what carries is the parent it instantiates — reference_standard_decay (which subsumes it and its twin gold_standard_erosion), connected upstream to calibration, the substrate-neutral pattern of an apparatus scoring against a definition that silently moved. Revised legal categories and DSM/ICD editions are co-instances of that parent, not of the label machinery. Diagnostic: Resolve toward the parent (reference_standard_decay) when asking what travels beyond model monitoring; toward ground-truth drift when diagnosing a supervised metric scored against a moved label.

Structural–Framed Character¶

Ground-truth drift sits on the framed side of the spectrum — best read as framed-leaning, and (being the twin of gold-standard erosion) it lands in exactly the same place for the same reasons: a methodological diagnostic wholly constituted by the human practice of supervised evaluation, whose portable core is a real relational shape that is itself intrinsically about measuring, never observer-free. On evaluative_weight it reads mildly framed: the construct names a failure — a pitfall to detect and avoid — so it carries more normative freight than a neutral mechanism, though (like its twin) it is more a diagnostic category than a moral verdict. On human_practice_bound it reads strongly framed: the phenomenon exists only inside an evaluation practice — there is no ground-truth drift without a supervised apparatus being scored against a label it treats as truth, all acts of a measuring pipeline; remove the practice of monitoring models against labels and nothing remains. Institutional_origin is likewise framed: the drifting definitions themselves are institutional artifacts (Sepsis-3 consensus criteria, HR taxonomies, BEA revision schedules, annotator instructions), and the diagnostic apparatus (the four-way P(X)/P(Y)/P(Y|X)/Y-definition decomposition, label versioning, cohort re-scoring) is furniture of model-evaluation methodology. On vocab_travels it reads framed: covariate shift, label shift, concept drift versus label-definition drift, like-for-like re-score are pinned to supervised ML and have no literal referent off it. And on import_vs_recognize the transfer is the entry's explicit "case-B" pattern — within supervised-evaluation fields (clinical prediction, hiring, forecasting, fraud, moderation, annotation) the full diagnostic is recognized intact as one substrate sampled across domains, but beyond ML only the thinner shape travels, and non-ML cases (revised legal categories, DSM/ICD editions, redefined "revenue") inherit none of the decomposition/versioning/re-scoring machinery.

The structural glint is that its portable core — an apparatus scoring against a reference whose definition silently moves on a clock the apparatus cannot see — is a genuine relational pattern recurring across measurement regimes; but it does not pull the entry toward the structural pole, because even that abstract shape is inherently about judging-against-a-standard, a measuring practice rather than an observer-free mechanism. The portable structural skeleton is reference_standard_decay — the unified parent that subsumes ground-truth drift and its twin gold_standard_erosion — connected upstream to calibration (if the reference drifts, calibration to it follows). That skeleton genuinely spans measurement domains, but it is exactly what ground-truth drift specializes from its parent, not what makes "ground-truth drift" itself travel: the cross-domain reach belongs to reference-standard decay, while the four-way decomposition, the two-clocks-and-agreement test, and the label-versioning cure stay home as data-science accent. Its character: a practice-constituted, methodology-furnished evaluation diagnostic — framed-leaning by its mildly-evaluative "failure" framing, its ML-bound vocabulary, and its institutional reference definitions — structural only in the reference-standard-decay shape it specializes, itself a pattern about measuring rather than about the world.

Structural Core vs. Domain Accent¶

This section decides why ground-truth drift is a domain-specific abstraction and not a prime, and (being the twin of gold-standard erosion) carries the case for its domain-specificity on the same fault line.

What is skeletal (could lift toward a cross-domain prime). Strip the model-evaluation idiom and one thin relational shape survives: an apparatus is anchored to a reference standard whose definition drifts on a clock the apparatus cannot see, so it keeps scoring against a moved target while its machinery looks unchanged. The portable pieces are abstract — a thing under judgment, a reference operationalizing the correct answer, a hidden revision clock external to the monitoring, and the silent scoring-against-a-moved-target that follows. Nothing there mentions models or labels. This is exactly reference_standard_decay — the unified parent that subsumes ground-truth drift and its twin gold_standard_erosion — connected upstream to calibration (if the reference drifts, calibration to it follows). It genuinely recurs across measurement regimes as co-instances (revised legal categories, DSM/ICD editions, redefined "revenue" or "proficient"). But that reference-drift shape is what ground-truth drift specializes from, not what makes it ground-truth drift — and even the "portable core" is intrinsically about a measuring practice, not an observer-free mechanism.

What is domain-bound. Almost all the diagnostic content is supervised-ML furniture, and none of it survives extraction. The apparatus is not generic — it is a model, monitor, or scoring system whose metrics are computed against a label it treats as truth. The decomposition is a precise probabilistic four-way sort — covariate shift (P(X)), label shift (P(Y)), concept drift (P(Y|X) because the world changed), and this fourth box, label-definition drift (the Y-definition itself moved). Its catch is worked ML practice — versioning the labeling rule and re-scoring identical cases under old and new definitions, the only operation comparing against historical rather than current ground truth. Its distinctions (from Goodhart, from its twin gold-standard erosion) are drawn inside model-evaluation methodology, its reference definitions are institutional artifacts (Sepsis-3 criteria, HR taxonomies, BEA revision vintages, annotator instructions), and its worked cases (SIRS→Sepsis-3, GDP vintage databases) are all supervised evaluation. The decisive test: remove the supervised apparatus scoring against a contingent label and there is no ground-truth drift left — a revised statute or a new DSM edition is a co-instance of the parent but inherits no P(Y)-decomposition, no cohort re-scoring, no "positive class"; what remains is the bare reference-drift shape, a looser thing.

Why this does not clear the prime bar. A prime's vocabulary travels and its transfer is recognition of the same mechanism, not analogy. Ground-truth drift's transfer is bimodal. Within supervised evaluation it moves intact — the four-way decomposition, the boundary it draws against concept drift, the two-clocks-and-agreement test, and the version-and-re-score cure carry without translation across clinical prediction, hiring analytics, forecasting evaluation, fraud detection, content moderation, and NLP annotation, because these are one substrate (a model consuming a contingent label) sampled across domains, not distinct domains. That is genuine within-domain transfer. Beyond ML the named thing does not travel — revised legal categories or DSM/ICD editions are co-instances of the parent, inheriting none of the decomposition/versioning/re-scoring machinery. And when the bare cross-domain lesson is wanted — version the reference and compare like-for-like across the change rather than trusting that a stable-looking pipeline means a stable target — it is already carried, in more general form, by reference_standard_decay (with gold_standard_erosion as its twin and calibration upstream). The cross-domain reach belongs to that parent; "ground-truth drift," as named, carries the P(X)/P(Y)/P(Y|X)/Y-definition decomposition, the label versioning, and the cohort re-scoring that should stay home in data science.

Relationships to Other Abstractions¶

Current abstraction Ground-Truth Drift Domain-specific

Parents (1) — more general patterns this builds on

Ground-Truth Drift is a decomposition of Reference Standard Decay Prime

Removing supervised-model terminology from ground-truth drift leaves an apparatus scoring against a reference whose definition moved invisibly.

Hierarchy path (1) — routes to 1 parentless root

Ground-Truth Drift → Reference Standard Decay → Proxy-Target Divergence → Proxy–Target Fidelity → Representation → Abstraction

Not to Be Confused With¶

Concept drift. The neighbor practitioners most need to separate: concept drift is P(Y|X) moving because the world changed — the real relationship between features and outcome shifted — whereas ground-truth drift is the Y-definition itself moving while the world and feature space may be untouched. The same accuracy slip flows from both, but they demand opposite remedies: retrain on fresh data (concept drift) versus audit the labeling rule (ground-truth drift). Tell: did the phenomenon behind the label change, or did the rule for assigning the label change, on the same underlying cases?
Covariate shift and label shift. The other two boxes of the four-way "why is performance different?" decomposition: covariate shift is P(X) moving (the input distribution changed) and label shift is P(Y) moving (the outcome's base rate changed). Both are genuine distributional moves in data the definition still scores the same way; ground-truth drift is the fourth, distinct box, where the definition of the positive class itself moved. Tell: did the inputs or their base rate shift under a fixed labeling rule (covariate/label shift), or did the labeling rule itself get rewritten (ground-truth drift)?
Label noise (annotation error). Random, per-case mislabeling — an annotator misreads a case, a pipeline flips a value — scattered around a stable definition. Ground-truth drift is not noise but a systematic move of the definition itself on an external clock, so old and new labels disagree on identical cases in a directional, not random, way. Tell: are the mislabels scattered and definition-stable (noise), or coherent and traceable to a revised rule (drift)?
Gold-standard erosion. Its twin under the same reference_standard_decay parent, but with a different emphasis: gold-standard erosion is the authority of the reference declining — the standard weakening — caught by re-validating its standing; ground-truth drift is the operational definition silently changing, caught by a like-for-like re-score under old and new rules. Tell: did the reference lose its standing (re-validate it) or shift its meaning (re-score like-for-like)?
Goodhart's law. A measure that became a target and is gamed by an optimizing agent responding to pressure on the metric. Ground-truth drift is upstream of that and agentless — the measure's definition changes on an external clock with no one pressing on it. Tell: is a strategic actor optimizing against the metric (Goodhart), or did the definition move with no agent gaming anything (drift)?
The reference_standard_decay parent (with calibration upstream). The substrate-neutral umbrella ground-truth drift specializes — an apparatus scoring against a reference whose definition silently moves on a clock it cannot see, of which revised statutes and DSM/ICD editions are co-instances. Tell: strip the P(X)/P(Y)/P(Y|X)/Y-definition decomposition and cohort re-scoring and what remains is bare reference-drift — the parent, treated more fully elsewhere, not ground-truth drift.

Neighborhood in Abstraction Space¶

Ground-Truth Drift sits in a moderately populated region (59^th percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.

Family — Unclustered & Miscellaneous (309 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-07-12