Attention is all you need.

Vaswani, A., Shazeer, Parmar, Uszkoreit, Jones, Gomez, & Kaiser. (2017). Attention is all you need. Advances in Neural Information Processing Systems 30 (NeurIPS 2017).

Type

Preprint

Intellectual base

Grey literature

Year

2017

Link

https://arxiv.org/abs/1706.03762

Cited from

cognitive_science, linguistics_semiotics, psychology

Cited by

3 citations across 3 artifacts.

Each citation links to the sentence it supports in the citing article.

Primes

• Attention

  • This sourceIntroduces the Transformer with multi-head attention as the sole sequence-mixing mechanism

• Attentional Capacity
  • The intervention family transfers from the human-factors literature with structural fidelity: filter upstream (RAG), automate per-task draw (caching), route across modality-analogs (mixture-of-experts), design the inference duty cycle to manage exhaustion (context windowing).

    This sourceIntroduces the Transformer with multi-head attention as the sole sequence-mixing mechanism; attention heads constitute a bounded per-layer per-token selection budget

• Paradigmatic vs. Syntagmatic Relations
  • The transformer architecture (Vaswani et al. 2017

    This sourceIntroduces the Transformer architecture with multi-head attention as the sole sequence-mixing mechanism; attention heads constitute a bounded per-layer per-token selection budget — a non-biological instance of the attentional-capacity pattern.

Verification

This reference passed the adversarial substantiation pipeline: it was checked to exist and to support the claim it is attached to. See how references were verified.

Registry ID ref:1515a41bc48d · see in the full table