Training language models to follow instructions with human feedback¶
Ouyang, Wu, Jiang, Almeida, Wainwright, Mishkin, Zhang, et al. (2022). Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Preference
- The ML case also makes the Inference visible: the reward model is a learned approximation of the underlying preference relation, and the quality of the approximation depends on how the ordering itself is structured (transitivity helps; intransitive cycles do not).
This sourceScales the RLHF preference-learning pipeline to large language models (InstructGPT): collects pairwise human comparisons over model completions, fits a reward model, and optimizes the policy via PPO; canonical industrial application of preference learning at scale.
- The ML case also makes the Inference visible: the reward model is a learned approximation of the underlying preference relation, and the quality of the approximation depends on how the ordering itself is structured (transitivity helps; intransitive cycles do not).
Verification¶
Does it exist? Not checked yet. This entry carries no identifier to resolve. It was extracted from the citation as written in the article, normalized, and deduplicated against the rest of the registry.
Does it back the claim? Not recorded. The single citation of this work carries no recorded support check.
Was it audited? Yes. A second, independent pass read the citation against the article text and recorded a verdict.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:b10b86608b7b · see in the full table