Popular Courses
Beginner – Linear Regression
Introduction
Beginner – Decision Trees
Introduction
Beginner – Neural Networks
Introduction

Sign in to Curvingo

or

New Account     Reset Password     

Plans

Details
Precision and Recall
Introduction
Precision and Recall
The F1-Score
The Tradeoff... Again
Not the End
Code with Python
Summary
Curvingo · Made in India

Share

Send us Feedback

Please let us know if you have any feedback and suggestions, or if you find any errors and bugs in our content.

Sorry, your message couldn’t be submitted. Please try again!

Thanks for your feedback!

Reset Progress

This will delete your progress and chat data for all chapters in this course, and cannot be undone!

Glossary

Select one of the keywords on the left…

Precision and RecallThe F1-Score

Reading time: ~10 min

Precision and recall are two numbers. To rank several tests, it helps to have just one. Why not simply take their average? Drag the precision and recall bars and complete the missions.

The plain average is fooled by one big number: precision 1 with recall 0 averages to 0.5, though such a test finds almost nobody. The F1-score is not fooled — it is only high when both scores are high.

The Harmonic Mean

Think of a teacher marking a project done in pairs who gives the pair one mark — and that mark is high only if both partners did well. One brilliant partner cannot carry a partner who did nothing.

The F1-score does this with the harmonic mean of precision P and recall R:

F_1 = \dfrac{2PR}{P + R}

At threshold 50, P = 0.5 and R = 0.9:

F_1 = \dfrac{2 \times 0.5 \times 0.9}{0.5 + 0.9} = \dfrac{0.9}{1.4} \approx

Put the formulas for precision and recall into it, and F1 can be read straight off the confusion matrix:

F_1 = \dfrac{2\,\text{TP}}{2\,\text{TP} + \text{FP} + \text{FN}} = \dfrac{2 \times 9}{18 + 9 + 1} = \dfrac{18}{28} \approx 0.64

The same answer both ways. The plain average of 0.5 and 0.9 would be 0.7, which flatters the test.

The F1 Map

Here is F1 for precision and recall in steps of 0.2 — the darker the cell, the higher the F1. Tap cells to complete the missions.

High F1 lives only in the corner where both scores are high. Along the edges, where one score is near 0, F1 stays near 0 however good the other score is.

Does It Have Any Flaws?

Look again at F_1 = \dfrac{2\,\text{TP}}{2\,\text{TP} + \text{FP} + \text{FN}}. Which box of the confusion matrix does it never use?

TP
FP
FN
TN

F1 ignores the true negatives completely: the 181 healthy people correctly sent home do not move it at all. When those matter, look at other scores too.

F1 also treats precision and recall as important.

Tip

At the dengue camp a miss is far worse than a false alarm, so Dr. Anjali should pick her threshold by recall first, and use F1 only to compare tests that catch enough patients.

Sina