AI Models Collapse When Trained on Recursively Generated Data¶
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024). AI Models Collapse When Trained on Recursively Generated Data. Nature, 631, 755-759.
Cited by¶
2 citations across 2 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Deception Blowback
- In AI training, synthetic data injected to fill a gap re-enters a later model's training set as ground truth, compounding errors at scale.
This sourceDemonstrates model collapse when synthetic/model-generated outputs re-enter later training corpora as if authentic, causing irreversible degradation — the self-poisoning blowback of unsegregated generated data re-ingested by a later training run.
Supported in partVerified against the publisher's abstract
The paper's own abstract states that model-generated content used indiscriminately in training causes irreversible 'model collapse', but it does not state the gap-filling mechanism or a quantified error-compounding rate.
“We find that indiscriminate use of model-generated content in training causes irreversible defects in the resulting models, in which tails of the original content distribution disappear. We refer to this effect as 'model collapse' and show that it can occur in LLMs as well as in variational autoencoders (VAEs) and Gaussian mixture models (GMMs).”
- In AI training, synthetic data injected to fill a gap re-enters a later model's training set as ground truth, compounding errors at scale.
Mechanisms¶
- Contaminated Record Quarantine
- Where synthetic artifacts can feed back into training sets, isolating them by lineage is the concrete guard against a system learning from its own fabrications.
This sourceShows that synthetic outputs fed recursively into training can cause a model to learn from generated artifacts and collapse.
- Where synthetic artifacts can feed back into training sets, isolating them by lineage is the concrete guard against a system learning from its own fabrications.
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 1 of 2 citations: 1 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:1bd0f3e40333 · see in the full table