Popular Courses
Beginner – Linear Regression
Introduction
Beginner – Decision Trees
Introduction
Beginner – Neural Networks
Introduction

Sign in to Curvingo

or

New Account     Reset Password     

Plans

Details
Logistic Regression
Introduction
How It Works
Scoring Our Model
Finding the Best S-Curve
Understanding Our S-Curve
Code with Python
Summary
Curvingo · Made in India

Share

Send us Feedback

Please let us know if you have any feedback and suggestions, or if you find any errors and bugs in our content.

Sorry, your message couldn’t be submitted. Please try again!

Thanks for your feedback!

Reset Progress

This will delete your progress and chat data for all chapters in this course, and cannot be undone!

Glossary

Select one of the keywords on the left…

Logistic RegressionFinding the Best S-Curve

Reading time: ~10 min

How do we find the w and b that make the log-loss smallest? The same way as for lines: gradient descent. Start with a flat curve (w = 0, b = 0, so every student gets a 50 % chance) and keep stepping downhill on the log-loss. This time there are two numbers to learn, so each step works out the slope in both directions.

Method 1: Gradient Descent

Press the buttons to train. The top figure shows the S-curve; the bottom one shows the log-loss after each step.

Note

For log-loss the downhill step turns out to be very neat. With p_i the model's chance for student i:

w_{\text{new}} = w - \alpha \cdot \text{mean of } (p_i - y_i)\,x_i, \qquad b_{\text{new}} = b - \alpha \cdot \text{mean of } (p_i - y_i)

Each student pushes the curve by their error p_i - y_i: students the model is unsure about, or wrong about, push hardest.

Method 2: Maximum Likelihood

Here is another way to ask the same question: which curve makes what actually happened most likely? Take three students. Aman passed and the model gave him a 0.9 chance of passing. Bela failed and the model gave her a 0.2 chance of passing — so the chance it gave to what really happened is . Chirag passed with a 0.6 chance.

If the three results are independent, the chance of all three happening is the product: 0.9 \times 0.8 \times 0.6 = . This product is called the likelihood, and a good curve makes it large.

Now take -\ln of it. The logarithm turns a product into a sum: -\ln(0.9 \times 0.8 \times 0.6) = -\ln 0.9 - \ln 0.8 - \ln 0.6, which is just the sum of the three penalties. So the curve with the highest likelihood is the curve with the lowest — two ways of saying the same thing. Statisticians call this method maximum likelihood estimation.

Sina