Activation Functions — Roadmap Step & Resources
ReLU, sigmoid and tanh, and why non-linearity matters
- Level: advanced
- Estimated time: 1 days
- Roadmap: Machine Learning
Before this step
Interview questions
- Why does a neural network need non-linear activation functions to be useful?
Study resources
- Neural Networks Pt. 3: ReLU In Action!!! (Video) — StatQuest's widely-loved explanation of activation functions including ReLU.
- Activation Functions Explained (Video) — A clear comparison of sigmoid, tanh and ReLU activation functions.
- CampusX Deep Learning (Neural Networks, Activation Functions) — YouTube playlist (verified) (Video) — CampusX Deep Learning (Neural Networks, Activation Functions) — YouTube playlist (verified) — resource aligned with Activation Functions.
- 3Blue1Brown Neural Networks — YouTube playlist (verified) (Video) — 3Blue1Brown Neural Networks — YouTube playlist (verified) — resource aligned with Activation Functions.
Part of
- Introduction to Neural Networks — section
- Machine Learning roadmap — the full learning path