Decision TreesCode with Python

In scikit-learn, a decision tree takes a few lines: make the model, fit it to the data, then print it or use it. criterion="entropy" asks for the entropy from this chapter (the default is Gini), and max_depth is the pruning rule. export_text prints the tree as indented questions. Run the program, then pick a different max_depth and run it again.

Entropy and the Best Cut, Without a Library

Here is the heart of the algorithm in plain Python: an entropy function, the information gain of a cut, and a loop that tries every cut on one feature. Run it for the lengths, then for the weights.

The loop finds the same numbers you found by dragging: a gain of 0.531 for the length cut at 10.25 cm, and 0.493 for the weight cut at 230 g. A library runs this loop for every feature, inside every box of the tree.