Curvingo · Made in India

Glossary

Select one of the keywords on the left…

Precision and RecallPrecision and Recall

Reading time: ~10 min

Accuracy mixes all four boxes together. Precision and recall each ask one sharper question. At threshold 50, the camp's confusion matrix is:

Test: positiveTest: negative
Has dengueTP = 9FN = 1
HealthyFP = 9TN = 181

Both scores come from two different piles of people:

The top pile is everyone the test called positive: 9 real cases and 9 false alarms. The bottom pile is everyone who really had dengue: the same 9 caught, plus 1 missed patient.

Precision looks at the top pile and asks: when the test says dengue, how often is it right? Of the 9 + 9 = 18 people who tested positive, 9 really had dengue.

\text{precision} = \dfrac{\text{TP}}{\text{TP} + \text{FP}} = \dfrac{9}{9 + 9} =

Recall looks at the bottom pile and asks: of everyone who really had dengue, how many did the test catch? Of the 10 patients, 9 were caught.

\text{recall} = \dfrac{\text{TP}}{\text{TP} + \text{FN}} = \dfrac{9}{9 + 1} =

So a positive result is right only half the time, but the test catches 90 % of the cases. And the lazy model? It never says positive, so it catches of the 10 patients: its recall is 0. Its 95 % accuracy hid that completely.

Note

Doctors call recall sensitivity. The lazy model has no positives at all, so its precision would be 0 \div 0 — not defined.

Moving the Threshold

Dr. Anjali chooses the threshold. Here everyone sits on the score line: patients with dengue above it, healthy people below. Everyone at or to the right of the amber threshold tests positive, and the people the test gets wrong turn red. Drag the threshold and complete the three missions.

Lower the threshold and recall rises — but more healthy people cross the line, so precision falls. Raise it and precision rises — but patients slip through. No threshold makes both perfect at once. This tug-of-war is the precision–recall trade-off.

Which Mistake Is Worse?

At the dengue camp a miss is far worse than a false alarm: a false alarm costs one extra lab test, a miss can cost a life. So Dr. Anjali keeps the threshold low and cares most about recall.

Other jobs are the other way round. Sort each job by the mistake it can least afford.

A spam filter that hides emails from your inbox
A smoke alarm in a school
A traffic camera that fines drivers for jumping a red light
An airport scanner looking for knives in bags
The first page of results in a search engine
A factory camera checking bike helmets for cracks
Recall matters more: a miss is worse
Precision matters more: a false alarm is worse
Sina