Vanishing & Exploding Gradients — Roadmap Step & Resources
Why very deep networks can fail to train at all without careful design
- Level: intermediate
- Estimated time: 1 days
- Roadmap: Deep Learning & Neural Networks
Before this step
Interview questions
- What architectural tricks help combat vanishing gradients in very deep networks?
Study resources
- Vanishing and Exploding Gradients Explained (Video) — A clear explanation of why deep networks struggle to train without care.
- Deep Learning Book: Optimization for Training Deep Models (Article) — The free official Deep Learning book's coverage of gradient instability issues.
Part of
- Training Deep Neural Networks — section
- Deep Learning & Neural Networks roadmap — the full learning path