Equality Of OddsIntroduction
Every year, the Vidya Scholarship Trust in Sina's district helps students from two schools — Hill School and River School — go to college. This year 60 students apply, far too many to interview, so the trust tries a model. It reads each application and gives a score from 0 to 100: how likely the student is to do well in college.
The trust shortlists everyone at or above a threshold, just like the pass/fail threshold in Logistic Regression. To check the model, it uses last year's applicants, whose college results are known: a student who did well is qualified.
Models learn from past records, and past records can carry old unfairness — a school with fewer records, a style of writing the data rarely saw. If we are not careful, a perfectly logical formula can end up
Hill School and River School are made up. Real fairness audits compare groups defined by protected attributes — characteristics such as gender, age or disability that the law protects from discrimination — but the maths is exactly the same.
One Threshold for Everyone
You met two rates in ROC and AUC. Here the true positive rate (TPR) is the share of qualified students who are shortlisted, and the false positive rate (FPR) is the share of not-qualified students who are shortlisted anyway. On the score lines below, qualified students sit above each line and not-qualified students below it; everyone the threshold gets wrong turns red. One threshold applies to both schools. Complete both missions.
The model is not equally good for both schools. Most of its training records came from Hill School, so it learned to read Hill School applications well, while River School applicants get lower and more muddled scores. Nobody meant this to happen, and no student did anything wrong. But with one threshold, a qualified River School student is far more likely to be turned away — at 70, nine out of ten are — and lowering the threshold to rescue them floods Hill School with false alarms.
Equalized Odds: Making It Fair
What does it mean for the model to be fair? Not that both schools get the same number of scholarships, but that the model's mistakes fall equally on both. A model can make two kinds of mistake: it can turn away a student who would have done well (a false negative), or shortlist a student who would have struggled (a false positive). Equalized odds (EO) says a model is fair only if both kinds of mistake happen at the same rate in every group.
The Intuition
If the model turns away 3 in 10 qualified Hill School students, it should turn away 3 in 10 qualified River School students too — no more. And if it shortlists 3 in 20 Hill School students who are not ready, it should do the same at River School. No group should carry more of the model's mistakes than another.
The Mathematics
Write for what really happened (1 = qualified), for the model's decision (1 = shortlisted) and for the group. Equalized odds asks that, for both values of ,
With this says the two TPRs are equal; with , the two FPRs. A relaxed version asks only for , equal TPRs. It is called equal opportunity: every qualified student has the same chance, whichever school they come from.
Sort each card into the right box.