Mixed Precision & Performance Basics — Roadmap Step & Resources
Training faster and using less memory with lower-precision floating point
- Level: advanced
- Estimated time: 1 days
- Roadmap: Deep Learning & Neural Networks
Before this step
Study resources
- Automatic Mixed Precision (Article) — The official PyTorch documentation on mixed precision training for faster training.
- Mixed Precision Training Explained (Video) — Covers why lower-precision floats speed up training without much accuracy loss.
- Hugging Face Documentation (Article) — Official documentation covering Transformers, PEFT, and fine-tuning workflows for open-weight models. Background reading/viewing for: Mixed Precision Training.
- PyTorch Documentation (Article) — Official documentation on distributed training relevant to full fine-tuning. Relevant to: Mixed Precision Training.
Part of
- Deep Learning Frameworks in Depth — section
- Deep Learning & Neural Networks roadmap — the full learning path