Skip to content

Data Poisoning Attack

The adversarial-ML failure mode where an attacker corrupts training inputs so the model learns an attacker-chosen behaviour — an upstream contamination that runtime defenses are structurally powerless against and that clean benchmark accuracy cannot detect.

Core Idea

A data poisoning attack is the adversarial-ML failure mode in which an attacker corrupts the inputs to a learning process so the trained model's behaviour is shifted in an attacker-chosen direction at inference — often undetected at training time, since poisoned samples may be unremarkable and clean-benchmark accuracy stays high. It is upstream of inference, targeting the training pipeline itself. Variants include label-flipping, clean-label backdoors with imperceptible triggers, availability attacks, and federated-learning gradient poisoning.

Scope of Application

Data poisoning lives across the subfields of adversarial machine learning, within training-time integrity.

  • Supervised-learning red-teaming — poisoning-fraction threat models and benchmark attacks (BadNets, Hidden Trigger).
  • LLM training-set contamination — poisoned web pages, false facts, and jailbreak samples in RLHF data.
  • Federated learning — a client submitting malicious gradient updates, met by Byzantine-robust aggregation.
  • Recommendation and ranking systems — review fraud and shill attacks poisoning the learned signal.
  • Spam and abuse classifiers — borderline content submitted to drift the decision boundary over time.

Clarity

Naming data poisoning fixes the pipeline stage where the attack lives, making the defensive landscape legible. A model can misbehave from a runtime adversarial example, from extraction, or because it learned the wrong function first — three failures that look alike but yield to different controls. Holding poisoning as the training-time case tells the defender the entire inference-time toolkit is structurally powerless against it. Its second contribution is breaking the assumption that benchmark performance certifies integrity: the failure is silent.

Manages Complexity

The named attacks differ on every visible axis, and a defender reasoning attack by attack faces a fast-multiplying catalogue. Naming the data poisoning attack collapses that zoo onto a single locus: all are contamination of the training substrate, upstream of inference. The defender tracks one stage and one trust relation, from which the impotence of runtime defenses, the location of effective upstream ones, and the unreliability of benchmark accuracy all follow without re-derivation per variant.

Abstract Reasoning

The concept licenses diagnostic reasoning (inferring a training-time origin from a misbehaviour runtime monitoring can't see, and reading a behavioural signature back to a variant), interventionist reasoning (moving defense upstream, predicting runtime controls are futile), boundary-drawing (separating poisoning from sibling attacks by stage, "behaves correctly on the eval distribution" from "contains no attacker-chosen behaviour"), and prediction (silent failure, small-fraction efficacy, and the intractability of auditing at scale).

Knowledge Transfer

Within adversarial ML the concept transfers as mechanism, intact — the training-time diagnostic, signature-to-variant attribution, upstream-defense prescription, and silent-failure predictions carry across supervised red-teaming, LLM contamination, federated learning, and recommender fraud. Beyond ML the pattern — contaminate the substrate a system learns from — recurs vividly (propaganda, fake reviews, Batesian mimicry, benchmark gaming), and precisely because none would be called "data poisoning," the AI name should not carry it. The travelling primes are corruption, feedback_loop, selective_propagation, and trust_boundary.

Relationships to Other Abstractions

Local relationship map for Data Poisoning AttackParents appear above the current abstraction, mutual partners to the right, and children below. Node labels state whether each abstraction is prime or domain-specific; colors identify relation types.Data Poisoning AttackDOMAINPrime abstraction: Learning-Substrate Contamination — is a decomposition ofLearning-Substr…PRIMEDomain-specific abstraction: Model Skewing — is a kind ofModel SkewingDOMAIN

Current abstraction Data Poisoning Attack Domain-specific

Parents (1) — more general patterns this builds on

  • Data Poisoning Attack is a decomposition of Learning-Substrate Contamination Prime

    Data poisoning is learning-substrate contamination specialized to ML training data, labels, gradients, triggers, and training-pipeline defenses.

Children (1) — more specific cases that build on this

  • Model Skewing Domain-specific is a kind of Data Poisoning Attack

    Model skewing is data poisoning specialized to a clean-launched, continuously or periodically updating deployment whose production stream is biased gradually after launch.

Hierarchy paths (2) — routes to 2 parentless roots

Neighborhood in Abstraction Space

Data Poisoning Attack sits in a moderately populated region (49th percentile for distinctiveness): it has near-neighbors but no dense thicket of look-alikes.

Family — Statistical Inference & Model Failure Modes (16 abstractions)

Nearest neighbors

Computed from structural-signature embeddings · 2026-07-12