Precision, recall, F1 score, and AUC are evaluation metrics used to understand classification models. They answer different questions, so choosing the right metric depends on the cost of false positives, false negatives, and the business objective.
Start with the confusion matrix
| Outcome | Meaning | Example: fraud detection |
|---|---|---|
| True positive (TP) | Predicted positive and actually positive | Fraud correctly flagged |
| False positive (FP) | Predicted positive but actually negative | Legitimate payment flagged as fraud |
| True negative (TN) | Predicted negative and actually negative | Legitimate payment correctly allowed |
| False negative (FN) | Predicted negative but actually positive | Fraudulent payment missed |
Precision: how trustworthy are positive predictions?
Precision answers: “Of everything the model flagged as positive, how many were actually positive?”
Precision = TP / (TP + FP)
High precision means fewer false alarms among the items flagged as positive. Prioritize precision when false positives are expensive—for example, when incorrectly blocking legitimate customers creates significant friction.
Recall: how many actual positives did the model find?
Recall (also called sensitivity or true positive rate) answers: “Of all actual positive cases, how many did the model identify?”
Recall = TP / (TP + FN)
High recall means fewer missed positive cases. Prioritize recall when missing a true positive is especially costly, such as failing to detect a serious safety issue or a fraudulent transaction.
F1 score: balance precision and recall
F1 is the harmonic mean of precision and recall:
F1 = 2 × (Precision × Recall) / (Precision + Recall)
F1 is useful when you want one score that balances precision and recall. It does not include true negatives directly, and it can hide the fact that one of the two component metrics is much weaker than the other. Always inspect precision and recall separately.
ROC curve and AUC
The Receiver Operating Characteristic (ROC) curve plots the true positive rate (recall) against the false positive rate across classification thresholds. AUC is the area under that curve. A larger ROC AUC generally indicates better ability to rank positive examples above negative examples across thresholds. AUC does not select the operating threshold for your application and does not tell you the business cost of errors.
How threshold changes affect the metrics
Lowering a classification threshold often increases recall because more cases are labeled positive, but it can also increase false positives and reduce precision. Raising the threshold often reduces false positives and can improve precision, but may miss more true positives and reduce recall. The exact behavior depends on the model and data.
Quick comparison
| Metric | Question it answers | Focus |
|---|---|---|
| Precision | When the model predicts positive, how often is it right? | False positives |
| Recall | Of all actual positives, how many did it find? | False negatives |
| F1 | What is the harmonic balance of precision and recall? | Combined precision/recall |
| ROC AUC | How well does the model rank positives above negatives across thresholds? | Ranking discrimination |
Exam traps
- Precision is not the same as accuracy. Precision looks only at predicted positives.
- Recall is not the same as specificity. Recall measures the true positive rate; specificity measures the true negative rate.
- F1 is the harmonic mean, not the simple arithmetic average.
- ROC AUC evaluates ranking across thresholds; it does not identify the best threshold for a particular business cost.
- For highly imbalanced datasets, consider whether precision-recall analysis is more informative than ROC analysis for the positive class.
Test yourself
Question 1
A medical screening model must minimize missed positive cases, even if some healthy patients are incorrectly flagged for follow-up. Which metric should the team emphasize?
- Precision
- Recall
- Specificity
- True negative rate only
Reveal answer
Correct answer: B. Recall emphasizes finding actual positive cases and reducing false negatives.
Question 2
A team needs one metric that combines precision and recall using their harmonic mean. Which metric should it use?
- ROC AUC
- Accuracy
- F1 score
- Specificity
Reveal answer
Correct answer: C. F1 is the harmonic mean of precision and recall.
Key takeaway
Use precision when false positives matter, recall when false negatives matter, F1 when you need a combined precision/recall score, and ROC AUC to evaluate ranking discrimination across thresholds. Choose based on the business cost—not because one metric is universally best.
Related AWS topics
Overfitting vs. Underfitting · Evaluating Foundation Models · SageMaker Clarify
✦ CloudExamPro Premium
Ready for the AWS Certified AI Practitioner (AIF-C01)?
Move beyond memorizing terms. Practice with realistic, scenario-based questions and detailed explanations that help you understand the AWS AI services and concepts behind each answer.
Original practice questions. No dumps. No shortcuts.