Language Models Don't Always Say What They Think¶
Turpin, M., Michael, J., Perez, E., & Bowman, S. R. (2023). Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting. Advances in Neural Information Processing Systems 36.
Cited by¶
2 citations across 2 artifacts.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Enthymeme
- In AI reasoning, models produce arguments with hidden premises, and faithfulness research asks whether the stated reasoning is the actual reasoning.
This sourceShows that a model's stated chain-of-thought can systematically misrepresent its actual reasoning — supporting that AI reasoning carries hidden premises and that faithfulness asks whether the stated reasoning is the actual reasoning.
- In AI reasoning, models produce arguments with hidden premises, and faithfulness research asks whether the stated reasoning is the actual reasoning.
- Informal Fallacy
- In an AI deployment, a language model is asked whether a policy is sound and returns a fluent, confident argument that turns on the claim "every credible economist agrees," a fabricated-but-relevant-sounding appeal to authority whose persuasive force (it sounds authoritative and well-formed) far exceeds its logical force
This sourceDocuments large-language-model reasoning that is fluent and plausible yet unfaithful/content-defective, generating well-formed arguments whose stated grounds do not actually warrant the conclusion — the form-passes-content-fails structure in generated reasoning.
Supported in partVerified against the work's full text
Turpin et al. show chain-of-thought explanations that look plausible yet misrepresent why the model answered; they do not catalogue fabricated premises, false dilemmas or equivocation.
“Despite the lack of errors, model explanations rationalize incorrect answers by giving incon- sistent subjective assessments (Table 4, Ruin Names) or by exploiting ambiguity in the task definition (Navigate).”
- In an AI deployment, a language model is asked whether a policy is sound and returns a fluent, confident argument that turns on the claim "every credible economist agrees," a fabricated-but-relevant-sounding appeal to authority whose persuasive force (it sounds authoritative and well-formed) far exceeds its logical force
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 1 of 2 citations: 1 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Links previously used in the corpus¶
Before the registry existed this work was also linked 1 other way.
Registry ID ref:9bbfe4fa685a · see in the full table