LearnMachine learning
Gradient descent
Watch a model learn by walking downhill on its own error, one step at a time.
Tap anywhere on the map to drop the ball there. The rings work like a contour map: the centre of each ring is a valley.
How it works
Walk downhill, one step at a time.
Training a model means finding the settings with the lowest error. Picture that error as a landscape: gradient descent stands somewhere, feels which way is downhill (the gradient) and takes a step that way.
The learning rate is the size of each step. Too small and training crawls; too large and it overshoots and bounces between the walls. Momentum lets it keep some speed, so it rolls through small bumps like a ball.
It only feels the slope where it stands, so it can settle in a valley that is not the deepest one. Almost every modern AI model, including large language models, is trained with a version of this idea.