Popular Courses
Beginner – Linear Regression
Introduction
Beginner – Decision Trees
Introduction
Beginner – Neural Networks
Introduction

Sign in to Curvingo

or

New Account     Reset Password     

Plans

Details
Decision Trees
Introduction
Decision Tree
Where To Partition?
Information Gain
Another Look At Our Decision Tree
The Problem of Perturbations
Going Beyond A Single Tree
Code with Python
Summary
Curvingo · Made in India

Share

Send us Feedback

Please let us know if you have any feedback and suggestions, or if you find any errors and bugs in our content.

Sorry, your message couldn’t be submitted. Please try again!

Thanks for your feedback!

Reset Progress

This will delete your progress and chat data for all chapters in this course, and cannot be undone!

Glossary

Select one of the keywords on the left…

Decision TreesAnother Look At Our Decision Tree

Reading time: ~10 min

Here is Meena's tree again, this time with the entropy H of every node written under it. As you go down, the boxes get smaller and the entropy falls. But should a tree really keep going until every leaf is pure? To find out, Meena buys 10 new mangoes whose kinds she knows and checks how many each tree gets right. They are the hollow rings on the chart. The line chart at the bottom keeps score: training mangoes and new mangoes. Complete both missions.

Maximum depth: ${depth}

The training score only climbs, but the new-mango score rises and then falls. The depth-3 tree is perfect on its 20 training mangoes, yet it gets two new mangoes wrong. Its third question, "weight ≤ 337.5", only exists to carve out one unusually long, heavy Alphonso — and new Totapuris of about 330 g fall into that thin strip. The tree learned noise, not a pattern.

How Deep Is Deep Enough?

Why might we stop a decision tree from growing before every leaf is pure?

So that it does not overfit and then fail on new data
Because working out entropy takes too long
Because it runs out of features

The danger of going too deep

A tree that splits until every leaf is pure learns its training data by heart, freak mangoes included. We stop it earlier so that it — so that it works on mangoes it has never seen. A model that fits its training data perfectly but does badly on new data is overfitting, just like the memoriser in Train, Test and Validation.

Sina