Learning objectives
- Define overfitting and underfitting
- Recognize symptoms of each problem
- Understand why generalization matters
Introduction
A good model should perform well not only on the data it has already seen but also on new data. Two common problems interfere with this goal: overfitting and underfitting. These ideas are fundamental in machine learning and worth mastering early.
Overfitting happens when a model learns the training data too closely, including noise or random quirks, so it performs poorly on new cases. Underfitting happens when a model is too simple or too weak to capture the important patterns in the data.
The ideal model balances learning and generalization. It should capture real structure without memorizing accidental details.
What overfitting looks like
An overfit model may score extremely well on training data but noticeably worse on validation or test data. This is a warning sign that the model has learned specifics that do not generalize.
Overfitting is more likely when the model is very flexible, the dataset is small, or training continues too long without proper controls.
What underfitting looks like
An underfit model performs poorly even on the training data. It has not learned enough to capture the real structure of the problem.
Underfitting may happen when the model is too simple, the features are weak, or the training process is insufficient.
Why generalization is the goal
In applied AI, the real test is how the model behaves on fresh data after deployment. A model that memorizes the past without understanding underlying patterns is not reliable.
This is why validation, regularization, good feature selection, and appropriate model complexity matter.
Examples
Student analogy
A student who memorizes exact practice questions but cannot answer new ones is overfitting. A student who barely understands the topic at all is underfitting.
Image model
A vision model trained on a very small dataset performs perfectly on those images but fails when new photos have different lighting and backgrounds.
Sales forecast
A simple model that predicts nearly the same sales number every week may underfit because it ignores seasonal patterns and promotions.
Exercises
- Define overfitting and underfitting in your own words.
- How can you tell from validation results that a model may be overfitting?
- Why can a very simple model underfit?
- Give one everyday analogy for each problem.
- Why is generalization more important than training accuracy alone?
Key takeaway
The best models neither memorize noise nor ignore real patterns; they generalize well to the new cases that matter most.