Red Teaming Language Models with Language Models¶
Perez, E., Huang, S., & Song, F. (2022). Red Teaming Language Models with Language Models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 3419-3448.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Unreliable Narrator
- The transfers across domains are concrete and intervention-suggestive rather than vocabulary moves, because the protocol's three slots carry the same operations wherever they are instantiated. Witness-evaluation into machine-output evaluation is the sharpest contemporary case: the established protocols for eyewitness testimony — corroboration, prior-knowledge probes, susceptibility to leading questions — map directly onto evaluation strategies for generative systems, namely cross-prompt consistency checks, prompts that elicit known training-data, and adversarial prompts that detect sycophancy.
This sourceDevelops adversarial probing of generative models — an attacker model generating prompts to surface failure modes — the machine-output analogue of the leading-question and susceptibility probes used in witness evaluation.
- The transfers across domains are concrete and intervention-suggestive rather than vocabulary moves, because the protocol's three slots carry the same operations wherever they are instantiated. Witness-evaluation into machine-output evaluation is the sharpest contemporary case: the established protocols for eyewitness testimony — corroboration, prior-knowledge probes, susceptibility to leading questions — map directly onto evaluation strategies for generative systems, namely cross-prompt consistency checks, prompts that elicit known training-data, and adversarial prompts that detect sycophancy.
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Not recorded. The single citation of this work carries no recorded support check.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:5acd398fc50a · see in the full table