GPT & Decoder-Only Models — Roadmap Step & Resources
Autoregressive, decoder-only models built for generation tasks
- Level: advanced
- Estimated time: 1 days
- Roadmap: Deep Learning & Neural Networks
Before this step
Interview questions
- Why is a decoder-only architecture naturally suited to text generation?
Study resources
- Language Models are Few-Shot Learners (GPT-3) (Article) — The original GPT-3 paper describing decoder-only, autoregressive language models.
- But what is a GPT? Visual intro to transformers (Video) — 3Blue1Brown's legendary explanation of the decoder-only GPT architecture.
Part of
- Transformers & Modern Architectures — section
- Deep Learning & Neural Networks roadmap — the full learning path