The Transformer Architecture (Encoder-Decoder) — Roadmap Step & Resou…
Multi-head attention, positional encoding and the full encoder-decoder stack
- Level: advanced
- Estimated time: 2 days
- Roadmap: Deep Learning & Neural Networks
Before this step
Interview questions
- Why do transformers need explicit positional encoding when RNNs didn't?
Study resources
- Attention Is All You Need (Article) — The original Transformer paper that introduced the architecture.
- But what is a GPT? Visual intro to transformers (Video) — 3Blue1Brown's legendary, visually-driven introduction to the Transformer architecture.
Part of
- Transformers & Modern Architectures — section
- Deep Learning & Neural Networks roadmap — the full learning path