Overfitting and underfitting describe two common ways a machine-learning model can fail to generalize. Overfitting occurs when a model learns the training data too closely, including noise and accidental patterns. Underfitting occurs when the model is too simple or insufficiently trained to capture the real relationships in the data.
Overfitting: memorizing instead of generalizing
An overfit model performs very well on its training data but noticeably worse on unseen validation or test data. It may have learned noise, outliers, or details that do not repeat in new examples. A classic clue is a large gap between training performance and validation performance.
Underfitting: the model has not learned enough
An underfit model performs poorly on both training and validation data. It may be too simple, have insufficiently informative features, be trained for too few steps, or use an unsuitable learning setup. Both training and validation errors remain high.
How to diagnose the difference
| Pattern | Training error | Validation error | Likely diagnosis |
|---|---|---|---|
| A | Low | High | Overfitting |
| B | High | High | Underfitting |
| C | Low | Low and similar | Good generalization, subject to test-set confirmation |
Ways to reduce overfitting
- More representative training data: broader data can help the model learn patterns that generalize.
- Regularization: techniques such as L1 or L2 penalties discourage overly complex parameter values.
- Reduce model complexity: use a simpler model or fewer parameters when the task does not justify a complex model.
- Early stopping: monitor validation performance and stop training when it stops improving.
- Data augmentation: create realistic variations of training examples where appropriate.
- Dropout: during training, randomly deactivate selected units in neural networks to reduce reliance on particular pathways.
Ways to address underfitting
- Use a model with enough capacity to capture the task’s patterns.
- Improve feature quality or add relevant input information.
- Train longer or adjust optimization settings when the model has not converged.
- Reduce excessive regularization if it prevents the model from learning.
- Check whether the data and labels contain enough signal for the task.
Why the validation set matters
Use separate training, validation, and test data. Train on the training set, use the validation set to compare models and tune hyperparameters, and reserve the test set for a final, unbiased estimate after choices are complete. Repeatedly tuning against the test set leaks information from the test data into model selection.
Exam traps
- Low training error plus high validation error points to overfitting.
- High training and validation errors point to underfitting.
- Regularization generally helps control overfitting; too much regularization can contribute to underfitting.
- More training epochs do not always improve generalization.
- A good training score alone does not prove that a model will perform well on new data.
Test yourself
Question 1
A model achieves 99% accuracy on training data but only 72% on validation data. What is the most likely issue?
- Underfitting
- Overfitting
- Perfect generalization
- Insufficient model capacity only
Reveal answer
Correct answer: B. The large training-validation gap suggests the model has learned training-specific patterns that do not generalize.
Question 2
A model performs poorly on both training and validation data. Which diagnosis is most likely?
- Overfitting
- Underfitting
- Data leakage from the test set
- High precision only
Reveal answer
Correct answer: B. Poor performance on both sets indicates the model has not captured the underlying patterns.
Key takeaway
Overfitting means the model fits training data too closely and fails on new data; underfitting means it fails to learn the training patterns in the first place. Compare training and validation performance before choosing a fix.
Related AWS topics
Precision, Recall, F1 and AUC · SageMaker Clarify · Amazon SageMaker
✦ CloudExamPro Premium
Ready for the AWS Certified AI Practitioner (AIF-C01)?
Move beyond memorizing terms. Practice with realistic, scenario-based questions and detailed explanations that help you understand the AWS AI services and concepts behind each answer.
Original practice questions. No dumps. No shortcuts.