Popular Courses
Beginner – Linear Regression
Introduction
Beginner – Decision Trees
Introduction
Beginner – Neural Networks
Introduction

Sign in to Curvingo

or

New Account     Reset Password     

Plans

Details
Random Forest
Introduction
Ensemble Learning
Bagging Method
Variance in Composition
Conclusion
Code with Python
Summary
Curvingo · Made in India

Share

Send us Feedback

Please let us know if you have any feedback and suggestions, or if you find any errors and bugs in our content.

Sorry, your message couldn’t be submitted. Please try again!

Thanks for your feedback!

Reset Progress

This will delete your progress and chat data for all chapters in this course, and cannot be undone!

Glossary

Select one of the keywords on the left…

Random ForestBagging Method

Reading time: ~10 min

If every tree learned from the same 20 mangoes in the same way, every tree would come out the same — a crowd of copycats. So the first trick gives every tree its own bootstrap sample, or bag: draw 20 mangoes from the 20 at random, with replacement. After each draw, the mango goes back, so it can be drawn again. Try it with 10 mango cards.

Every bag has repeats, and every bag misses some cards — about a third of them on average. So each tree learns from a slightly different dataset, and each tree ends up a little different.

How Many Are Left Out?

Why about a third? Take one card, say card 4. On a single draw, the chance of not picking it is \tfrac{9}{10}. The 10 draws are independent, so the chance that card 4 is never picked is

\left( \dfrac{9}{10} \right)^{10} \approx

The same reasoning works for any number of rows n: the chance of being left out is (1 - \tfrac{1}{n})^n. The purple dots show it for 1 to 30 rows — it quickly settles at about 0.37, a little over a third, however big the data is.

So each tree sees roughly two thirds of the different rows. The rows a tree never saw are its out-of-bag rows, and they can be used to test that tree — a free validation set. Drawing bags and then combining the trees' votes is called bagging, short for bootstrap aggregating.

Trick Two: A Random Question Each Time

Different bags help, but most trees would still ask the same strong first question — length. So a random forest adds a second trick: at every split, a tree may look at only a random handful of features. Meena's mangoes have just two features, so at each split a coin decides whether the tree may ask about weight or about length. Grow trees until both missions are done. Each tree grows from its own bag; mangoes that are not in the bag are faded, and a small number shows a mango that was drawn more than once.

Two random choices — which mangoes go into the bag, and which feature each question may use — make every tree different. Each tree is a little worse than one grown with care, and that is fine: what matters is that their mistakes are different. (To keep the diagrams readable, these trees stop after three levels of questions; in a real forest each tree grows until its leaves are pure.)

Sina