Learning Rate Scheduling — Roadmap Step & Resources
Decaying, warming up or cycling the learning rate over training
- Level: intermediate
- Estimated time: 1 days
- Roadmap: Deep Learning & Neural Networks
Before this step
Study resources
- torch.optim: How to adjust learning rate (Article) — The official PyTorch documentation on learning rate schedulers.
- Learning Rate Scheduling Explained (Video) — A practical explanation of decaying the learning rate over training.
Part of
- Training Deep Neural Networks — section
- Deep Learning & Neural Networks roadmap — the full learning path