Neural Network Fundamentals — Deep Learning & Neural Networks Roadmap
How a neural network actually learns from data, from first principles
Steps in Neural Network Fundamentals
- The Perceptron & Biological Inspiration — intermediate · The simplest possible neural unit and the loose analogy to biological neurons
- Feedforward Neural Networks — intermediate · Stacking layers of neurons to approximate complex, non-linear functions
- Activation Functions — intermediate · Sigmoid, tanh, ReLU, Leaky ReLU and softmax, and why the choice matters
- Loss Functions — intermediate · MSE, cross-entropy and choosing the right loss for a given task
- Backpropagation Explained — intermediate · How gradients flow backward through a network to update every weight
- Gradient Descent Variants (SGD, Momentum, RMSprop, Adam) — intermediate · How different optimizers navigate the loss surface differently
- Weight Initialization Strategies — intermediate · Xavier/Glorot and He initialization, and why starting weights matter
Part of
- Deep Learning & Neural Networks roadmap — the full learning path