Precision and RecallIntroduction
It is September in Kochi, and the monsoon has brought mosquitoes — and dengue. Dr. Anjali runs a free screening camp in the community hall. In one day 200 people take a quick finger-prick test. A small computer model reads each test strip and gives a dengue score from 0 to 100: the higher the score, the more likely the person has dengue.
A week later the full lab reports come back: 10 of the 200 really had dengue.
The quick test puts every person into one of two groups, dengue or no dengue. You met this kind of job in Logistic Regression. It is called
Like Kavya ma'am's pass-or-fail model, the test turns its score into a decision with a threshold: a score of 50 or more means positive — "you may have dengue, please see a doctor" — and anything lower means negative. But how do we know whether this test is any good?
The 95 % Trap
The obvious score is accuracy: the fraction of people a model gets right. Here is a lazy rival that does no work at all — it tells everyone they are healthy. Each dot is one person at the camp: 10 had dengue and 190 were healthy. People a model gets wrong turn red, and an amber ring means the model said "dengue". Run both models.
Both models are right about 190 people, so both are 95 % accurate. Yet the lazy model sends all 10 dengue patients home, while the real test finds 9 of them. When one group is far bigger than the other — 190 against 10 — the data is called imbalanced, and accuracy is a poor judge. We have to look at which mistakes a model makes.
The Confusion Matrix
Every person ends up in one of four boxes, depending on what was really true and what the test said:
- True positive (TP): has dengue, and the test says positive — a case caught.
- False negative (FN): has dengue, but the test says negative — a dangerous miss.
- False positive (FP): healthy, but the test says positive — a false alarm, a worried family and an extra lab test.
- True negative (TN): healthy, and the test says negative.
The first word says whether the test was right (true) or wrong (false); the second word is what the test said. Laid out as a 2 × 2 grid — rows for what was true, columns for what the test said — the four boxes are called the confusion matrix.
Here are eight people from the camp, one at a time. Tap the box where each person belongs.
Now a quick check from everyday life. A spam filter hides a perfectly normal email from your school in the junk folder. Counting "spam" as positive, which box is that?
A false alarm: the filter said "spam" (positive) and was wrong (false).