Attention Is Not Explanation¶
Jain, S., & Wallace, B. C. (2019). Attention Is Not Explanation. Proceedings of NAACL-HLT 2019.
Cited by¶
1 citation across 1 artifact.
Each citation links to the sentence it supports in the citing article.
Primes¶
- Emic And Etic
- In AI interpretability, a model's emitted reasoning (emic — what it says it is doing) and its mechanistic description via probes (etic — what the circuits implement) are explicitly mapped in faithfulness work.
This sourceRepresentative faithfulness work testing whether a model's attended/emitted rationale matches its mechanistic computation — the emic self-report versus etic mechanistic account.
Supported in partVerified against the work's full text
A faithfulness study testing whether attention weights (a presented rationale) track model outputs — relevant background for an emic/etic contrast, but it never mentions probes or any explicit emic-to-etic mapping.
“Attention mechanisms have seen wide adoption in neural NLP models. In addition to improving predictive performance, these are often touted as affording transparency: models equipped with attention provide a distribution over attended-to input units, and this is often presented (at least implicitly) as communicating the relative importance of inputs. However, it is unclear what relationship exists between attention …”
- In AI interpretability, a model's emitted reasoning (emic — what it says it is doing) and its mechanistic description via probes (etic — what the circuits implement) are explicitly mapped in faithfulness work.
Verification¶
Does it exist? Confirmed. This work's DOI resolves to a registered record, which fixes its identity. That is all it fixes.
Does it back the claim? Read against the text for 1 of 1 citation: 1 supported in part. Each verdict is shown under its citation below, with what in the work backs the sentence.
Support is checked per citation rather than per work — the same source can be cited soundly in one article and wrongly in another. Per-citation recording began recently, so a citation with no recorded check is a gap in the record rather than evidence it went unchecked.
See how references were verified.
Registry ID ref:59ed2af65418 · see in the full table