Logistic RegressionIntroduction
Kavya ma'am teaches Class 12. Before every unit test she wants to know which students need extra classes, so she can help them before they fail. She has last term's records: how many hours each student studied in the week before the test, and whether they passed.
In the last chapter, Riya's model predicted a number — 128 kulfis. Kavya ma'am does not want a number. She wants an answer to a yes/no question: will this student pass?
Choosing between groups is called classification. Logistic regression is the classic method for it, and most of the time it chooses between exactly
Choosing between two groups is called binary classification: spam or not spam, rain or no rain, pass or fail. We write one group as 1 (passed) and the other as 0 (failed).
The Problem with Straight Lines
Here are 14 of Kavya ma'am's students. Those who passed sit on the top row at 1; those who failed sit on the bottom row at 0. Let's try the tool from the last chapter: the best straight line through these dots, reading its height as the chance of passing. Drag the amber marker along the hours.
A chance must lie between 0 and 1 (0 % and 100 %). A straight line keeps rising or falling forever, so sooner or later it escapes. We need a line that bends to stay inside.
The Sigmoid "S" Curve
Here is the trick. First compute a straight-line number exactly as before, . Then squash it with the sigmoid function:
Whatever number goes in, a number between 0 and 1 comes out. Drag z and complete both missions.
Check the middle by hand: , so
Put the two pieces together and you have the whole model:
where is the chance that the answer is 1 ("passed"). The word regression in the name comes from the straight line hidden inside; the job it does is classification.