Introduction to the Attention Mechanism — Roadmap Step & Resources
Letting a decoder look back at all encoder states instead of a single fixed vector
- Level: advanced
- Estimated time: 1 days
- Roadmap: Deep Learning & Neural Networks
Before this step
Interview questions
- What limitation of vanilla seq2seq models does attention directly address?
Study resources
- Attention for Neural Networks, Clearly Explained!!! (Video) — StatQuest's widely-loved, clear explanation of the attention mechanism.
- Neural Machine Translation by Jointly Learning to Align and Translate (Article) — The original Bahdanau attention paper that introduced attention to sequence models.
Part of
- Recurrent Neural Networks & Sequence Models — section
- Deep Learning & Neural Networks roadmap — the full learning path