Popular Courses
Beginner – Linear Regression
Introduction
Beginner – Decision Trees
Introduction
Beginner – Neural Networks
Introduction

Sign in to Curvingo

or

New Account     Reset Password     

Plans

Details
Equality Of Odds
Introduction
Measuring Fairness
How Can We Actually Fix the Bias?
Conclusion
Code with Python
Summary
Curvingo · Made in India

Share

Send us Feedback

Please let us know if you have any feedback and suggestions, or if you find any errors and bugs in our content.

Sorry, your message couldn’t be submitted. Please try again!

Thanks for your feedback!

Reset Progress

This will delete your progress and chat data for all chapters in this course, and cannot be undone!

Glossary

Select one of the keywords on the left…

Equality Of OddsMeasuring Fairness

Reading time: ~15 min

To give a model a fairness score, an auditor works out each group's error rates and subtracts. The chart shows both schools' rates at every threshold: solid lines are TPRs, dashed lines are FPRs. Read them at the amber line, threshold 60.

At Hill School, 7 of the 10 qualified students are shortlisted, so its TPR is 7 ÷ 10 = 0.7. At River School only 3 of the 10 are, so its TPR is 3 ÷ 10 = .

The TPR gap is 0.7 − 0.3 = . For false positives: 3 of Hill School's 20 not-qualified students are shortlisted (FPR 0.15) and 1 of River School's 20 (FPR 0.05), so the FPR gap is .

Note

Equalized odds needs both gaps to be 0; equal opportunity needs only the TPR gap to be 0. On the chart, the two solid lines stay far apart at almost every threshold — one threshold for both schools cannot close the gap.

Interactive Simulation

One threshold cannot close both gaps. But the trust can give each school its own threshold — a fix applied after training, called post-processing. The bars show the rates for Hill School and River School. Complete both missions, and keep an eye on the number of correct decisions.

Hill School threshold: ${hill}

River School threshold: ${river}

Equal opportunity has several answers — for example Hill School at 70 and River School at 50, where both TPRs are 0.6. Equalized odds has only one useful answer here: Hill School at 60 and River School at 45, where both schools have TPR 0.7 and FPR 0.15. Now look at the counter. The best thresholds, ignoring fairness, make 52 correct decisions out of 60; equalized odds makes 48.

The Price of Fairness

At the fair point, Hill School gets 7 qualified and 17 not-qualified students right: 24 correct decisions. River School also gets 7 + 17 = 24. So the accuracy is (24 + 24) ÷ 60 = , against 52 ÷ 60 ≈ 0.87 for the most accurate thresholds.

Fairness here costs 4 correct decisions out of 60 — and an honest audit says so openly. There is also a way to make both gaps exactly 0 for free: set both thresholds to 0. What is wrong with that?

Nothing, because both schools now have exactly the same rates
It shortlists every applicant, so the model no longer decides anything
The FPR cannot be worked out at threshold 0

At threshold 0 everyone is shortlisted (TPR 1, FPR 1); at a very high threshold no one is (TPR 0, FPR 0). Both are perfectly "fair" and perfectly useless. They are called lazy solutions: demanding equal rates without also asking for a useful model can push you straight into one.

Sina