Regularization (L1/L2, Dropout) — Roadmap Step & Resources
Techniques that discourage a network from memorizing the training set
- Level: intermediate
- Estimated time: 1 days
- Roadmap: Deep Learning & Neural Networks
Interview questions
- How does dropout act like an implicit ensemble of smaller networks?
Study resources
- Ridge, Lasso and Elastic-Net Regression, Clearly Explained!!! (Video) — StatQuest's widely-loved explanation of Ridge and Lasso regularization.
- Linear Models (Article) — The official scikit-learn documentation on Ridge, Lasso and Elastic-Net.
- Dropout: A Simple Way to Prevent Overfitting Explained (Video) — A clear explanation of dropout regularization specific to neural networks.
- Deep Learning Book: Regularization (Article) — The free official Deep Learning book's dedicated chapter on regularization for deep networks.
Part of
- Training Deep Neural Networks — section
- Deep Learning & Neural Networks roadmap — the full learning path