The maths behind machine learning
Four ideas that most of modern machine learning rests on. Each page shows the formula, works a small example you can check with a calculator, and then lets you change the numbers.
- Explainer
Gradient descent
Drag the starting point, set the learning rate, and watch it converge or blow up.
Try it: Draggable loss curve with learning-rate and step controls - Explainer
Vectors and embeddings
Bend the plane with a 2×2 matrix and measure how close toy word vectors are.
Try it: 2D matrix-transform playground and cosine-similarity explorer - Explainer
Softmax and cross-entropy
Edit the logits and watch the probabilities, the loss and the gradient update live.
Try it: Editable logits with live probabilities, loss and gradient - Explainer
Attention
Q, K and V for three tokens, a live score heatmap, and the weighted sum of values.
Try it: Three-token attention calculator with a score heatmap
The approach
Each page shows the formula, then works a small example with real numbers that you can check with a calculator, then gives you an interactive version that computes exactly the same numbers and lets you change them. The worked examples and the widgets share one implementation, so they always agree.
Read them in this order for the full picture of how a classifier or a transformer is trained: vectors and embeddings for how data is represented, softmax and cross-entropy for how predictions are scored, gradient descent for how the parameters are improved, and attention for how a transformer combines information across a sequence.
What you need first
Comfort with algebra and the idea of a derivative as a slope. The pages explain the linear algebra they use.