LearnMachine learning
Overfitting
Make a curve more and more flexible and find the point where it stops learning and starts memorising.
Filled dots train the curve; hollow dots test it. Drag any dot to change the data.
How it works
Fit the pattern, not the noise.
Every dataset is a real pattern plus random noise. A model that is too simple misses the pattern (underfitting), and a model that is too flexible learns the noise as well (overfitting).
Here the model is a curve, and its degree controls how flexible it is. Training error keeps falling as the degree rises, but the error on unseen test points falls and then rises again. The best model sits at the bottom of that U.
This trade-off, often called bias and variance, applies to every model in machine learning, from straight lines to huge neural networks. It is the reason data is split into training and test sets. Strictly speaking, points used to choose a setting such as the degree are validation data: an honest final score needs a separate test set that played no part in any choice.