LearnDeep learning
Backpropagation
Follow the error backwards through a tiny network and see how the chain rule tells every weight which way to move.
Press Forward, then Backward. Click any weight to see its chain rule. Update moves every weight a little against its gradient.
How it works
Blame, passed backwards.
This network has two inputs, two hidden neurons and one output, all with sigmoid activations. The forward pass computes the prediction ŷ, and the loss L = ½(ŷ − t)² measures how far it is from the target t.
Backpropagation applies the chain rule from the output backwards. The gradient of each weight is a product of local slopes along its path: how the loss reacts to ŷ, how ŷ reacts to its input, and so on back to the weight. Hidden weights reuse the factors already computed for the layer above, which is what makes the method fast.
Turn on "Check with finite differences" to compare each gradient with a numerical estimate made by nudging the weight a tiny amount. They agree, which is also how engineers test their own code. Every modern network, however large, is trained with this same algorithm.