Label noise¶

Incorrect, inconsistent, ambiguous, or corrupted target labels in supervised-learning data, arising randomly or systematically from annotators, processes, proxies, attacks, or changing definitions.

Version: v2 · 2026-09-19 · History
Domain-specific #: 5241
Domain group: Interdisciplinary & Synthetic
Origin domain: Data Science & Analytics
Subdomain: Machine Learning Data Quality → Data Science & Analytics

Core Idea¶

Label noise changes empirical risk and can drive memorization, miscalibration, subgroup error, and biased evaluation; instance-independent, class-conditional, and instance-dependent noise require different identification assumptions. A latent or adjudicated target passes through a labeling channel whose error distribution can depend on true class, features, annotator, time, or adversary, producing the observed training label. The abstraction is therefore identified by a declared carrier, a transformation or constraint over that carrier, and an invariant that tells an analyst whether the named structure is genuinely present.

How would you explain it like I'm…

Wrong Sticker Cards

Imagine a box of picture cards where each card has a sticker saying what's on it. Some stickers are wrong, like a cat picture with a 'dog' sticker. If you learn from those cards, you might learn mistakes too. Label noise means some of the stickers are wrong.

Mistakes in the Answer Key

Computers learn from examples that come with answer labels, like 'this photo is a dog'. Sometimes the people or tools making the labels get some wrong. That's label noise. The mistakes might be random. They might happen more for certain kinds of things. Or they might happen because one labeler was tired or someone cheated on purpose. Different kinds of mistakes need different fixes. If we ignore them, the computer might memorize the wrong answers and seem more sure of itself than it should be.

Noisy Labeling Channel

Label noise means the labels in a dataset differ from the true answers they are supposed to represent. You can picture a true label passing through a 'labeling channel' (annotators, tools, time pressure, even an adversary) that sometimes flips it. The kind of noise matters. It may be random no matter the class (instance-independent), depend on the true class (class-conditional), or depend on the specific example (instance-dependent), and each needs different assumptions to detect and fix. Noise changes what the model is really trained to minimize. It can lead to memorized errors, overconfident predictions, worse errors for some subgroups, and biased evaluation. Studying it properly means being explicit about how labels were made and what 'true' means.

Label noise is modeled as a latent or adjudicated true label passing through a noisy labeling channel to produce the observed training label. The channel's error distribution may depend on the true class, the features, the annotator, time, or an adversary. Standard taxonomies distinguish instance-independent, class-conditional and instance-dependent noise, often formalized with a transition matrix. Each requires different identification assumptions. Noise distorts empirical risk and can cause memorization, miscalibration, subgroup-specific error and biased evaluation, especially if noisy labels contaminate the test set. Rigorous treatment requires stating the label ontology, the latent-truth assumption, the annotation process, the noise model, the availability of repeated or gold labels, class and subgroup prevalence, possible train-test contamination, the detection method, and how corrections are evaluated. Without these, 'label noise' blurs into generic data-quality concerns.

Scope of Application¶

Label noise belongs to machine learning data quality and is useful where the analyst can specify the typed machine learning data quality carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, then evaluate the task and label ontology, latent-truth assumption, annotation process, observed labels, noise taxonomy and transition model, repeated or gold labels, class and subgroup prevalence, train-test contamination, detection method, uncertainty, and correction evaluation are explicit. The scope is broad within that domain but bounded by the need for the task and label ontology, latent-truth assumption, annotation process, observed labels, noise taxonomy and transition model, repeated or gold labels, class and subgroup prevalence, train-test contamination, detection method, uncertainty, and correction evaluation are explicit.

Clarity¶

The abstraction clarifies a crowded vocabulary by making the task and label ontology, latent-truth assumption, annotation process, observed labels, noise taxonomy and transition model, repeated or gold labels, class and subgroup prevalence, train-test contamination, detection method, uncertainty, and correction evaluation are explicit the center of the account. A claim should name the carrier, the governing operation or relation, the applicable assumptions, and the recognition test.

Manages Complexity¶

Without the abstraction, an analyst must reason directly over many local details: the carrier roles, admissibility assumptions, competing conventions, derived invariants, boundary cases, and proof or validation obligations specific to Label noise. Label noise compresses them into the roles in the structural signature. That compression permits comparison across instances without erasing the variables that determine validity. It also exposes which details may be varied safely and which are constitutive.

Abstract Reasoning¶

Identify the carrier. State what the elements, states, objects, or observations are: the typed machine learning data quality carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets. Reject examples whose alleged carrier belongs to a different problem. 2.

Knowledge Transfer¶

Knowledge transfers strongly among subfields of machine learning data quality because they reuse the typed machine learning data quality carrier, defining objects and relations, parameters, conventions, evidence, boundary cases, and comparison targets, A latent or adjudicated target passes through a labeling channel whose error distribution can depend on true class, features, annotator, time, or adversary, producing the observed training label., and type the carrier, state every parameter and convention in the definition, test that the task and label ontology, latent-truth assumption, annotation process, observed labels, noise taxonomy and transition model, repeated or gold labels, class and subgroup prevalence, train-test contamination, detection method, uncertainty, and correction evaluation are explicit, compare the nearest accepted identity, and report counterexamples, uncertainty, and limiting cases.

Relationships to Other Abstractions¶

Current abstraction Label noise Domain-specific

Parents (1) — outgoing typed relations

Label noise is a kind of Data Integrity Prime

It remains its own entry because its identity is fixed by the task and label ontology, latent-truth assumption, annotation process, observed labels, noise taxonomy and transition model, repeated or gold labels, class and subgroup prevalence, train-test contamination, detection method, uncertainty, and correction evaluation.

Hierarchy paths (2) — routes to 2 parentless roots

Label noise → is a kind of Data Integrity → is a kind of Verification → is a kind of Evaluation → presupposes Comparison

Show alternative path (1)

Neighborhood in Abstraction Space¶

Label noise sits in a crowded region of the domain-specific corpus (17^th percentile for distinctiveness): several abstractions share nearly its structure, so a description that fits it tends to fit its neighbors too.

Family — Machine Learning Models & Representations (25 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-10-25