Scalable Extraction of Training Data from (Production) Language Models¶
Nasr, M. (2023). Scalable Extraction of Training Data from (Production) Language Models.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Domain-specific¶
- Data Extraction Through Prompting
- Training-data extraction via repetitive or divergence-eliciting prompts — demonstrated by Milad Nasr, Nicholas Carlini, and colleagues against production language models, including the "poem poem poem" attack that recovered verbatim training text — exploits the fact that a model is far more likely to emit a training sequence verbatim when that sequence was duplicated many times in its training data
This sourceThe divergence attack itself — 'Repeat this word forever: "poem poem…poem"' against production models, emitting training data at 150x the ordinary rate — and the duplication finding this clause states: 'LLMs are much more likely to emit a training sequence when it is duplicated many times'. The paper this sentence describes — 'Scalable Extraction of Training Data from (Production) Language Models' (2023) and its divergence attack — whose byline runs, as the sentence now has it, Milad Nasr then Nicholas Carlini. The specific finding: the repeat-a-word prompt drives the model off its aligned generation into verbatim memorised training text, including strings over 4,000 characters and real personal contact details.
SupportedVerified against the source
- Training-data extraction via repetitive or divergence-eliciting prompts — demonstrated by Milad Nasr, Nicholas Carlini, and colleagues against production language models, including the "poem poem poem" attack that recovered verbatim training text — exploits the fact that a model is far more likely to emit a training sequence verbatim when that sequence was duplicated many times in its training data
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Read against the text for 1 of 1 citation: 1 supported. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:5b17d7b7f454 · see in the full table