Attention is a mechanism that lets a model dynamically weight which parts of its input are most relevant to producing each part of its output, rather than treating all input positions equally. Introduced for sequence-to-sequence translation, it became the foundation of the Transformer architecture via self-attention (scaled dot-product attention). Multi-head attention allows the model to attend to different relationship types simultaneously. Attention patterns are sometimes used for interpretability, though they don't always correspond to human notions of importance.