Learning objectives
- Define accuracy, precision, and recall clearly
- Understand trade-offs among these metrics
- Apply the metrics to real decision scenarios
Introduction
When evaluating classification models, three common metrics appear again and again: accuracy, precision, and recall. These metrics help describe different aspects of performance, especially when not all mistakes are equally serious.
Accuracy measures overall correctness. Precision tells you how often positive predictions are actually correct. Recall tells you how many of the real positive cases the model successfully found.
A beginner mistake is to focus only on accuracy. In many important applications, precision or recall may matter more depending on the consequences of errors.
Accuracy: overall correctness
Accuracy is the proportion of all predictions that were correct. It is easy to understand and useful when classes are balanced and error costs are similar.
However, if positive cases are rare, a model can have high accuracy while still failing at the task. For example, predicting 'not fraud' for almost everything could still look accurate if fraud is uncommon.
Precision: trust in positive predictions
Precision answers the question: when the model predicts positive, how often is it right? High precision matters when false alarms are costly or disruptive.
In a spam filter, low precision could mean many legitimate emails are wrongly marked as spam. In content moderation, low precision could lead to unfair takedowns.
Recall: finding the real positives
Recall answers the question: of all the real positive cases, how many did the model catch? High recall matters when missing positives is costly.
In medical screening or fraud detection, low recall can be dangerous because important cases may be missed.
Examples
Medical screening
A system screening for disease often prioritizes recall because missing a serious case may be more harmful than flagging some healthy cases for extra review.
Spam filtering
A mail system may care a lot about precision because incorrectly hiding important emails creates frustration and business risk.
Fraud monitoring
A bank may try to balance both metrics, since too many false alarms annoy customers while missed fraud causes financial loss.
Exercises
- Explain accuracy, precision, and recall in simple language.
- Which metric would matter most in cancer screening, and why?
- Which metric might matter more in email filtering, and why?
- Why can a model have high accuracy but poor recall?
- Create a real-world case where high precision is more important than high recall.
Key takeaway
Accuracy gives a broad summary, but precision and recall reveal whether a classifier is trustworthy and effective for the cases that matter most.