Popular Courses
Beginner – Linear Regression
Introduction
Beginner – Decision Trees
Introduction
Beginner – Neural Networks
Introduction

Sign in to Curvingo

or

New Account     Reset Password     

Plans

Details
Random Forest
Introduction
Ensemble Learning
Bagging Method
Variance in Composition
Conclusion
Code with Python
Summary
Curvingo · Made in India

Share

Send us Feedback

Please let us know if you have any feedback and suggestions, or if you find any errors and bugs in our content.

Sorry, your message couldn’t be submitted. Please try again!

Thanks for your feedback!

Reset Progress

This will delete your progress and chat data for all chapters in this course, and cannot be undone!

Glossary

Select one of the keywords on the left…

Random ForestEnsemble Learning

Reading time: ~10 min

Ask one person to guess the number of laddoos in a big jar and they may be far off. Ask a hundred people and average their guesses: some guess too high, some too low, and the errors largely cancel. A team of models that vote (for a kind) or average (for a number) is called an ensemble, and building one is ensemble learning. Instead of one deep, overfitted tree, we train many weaker models and combine their answers.

Let's check the crowd's numbers with three helpers, each right with probability 0.6, each deciding alone. The majority is right when all three are right, or when exactly two are.

  • All three right: 0.6 \times 0.6 \times 0.6 =
  • Exactly two right: there are 3 ways to choose the one who is wrong, so 3 \times 0.6 \times 0.6 \times 0.4 =

So the majority of three is right with probability 0.216 + 0.432 = — already better than one helper's 0.6.

Note

This is the binomial distribution from Class 12. For a bigger crowd, add up the chances of every result in which more than half of the helpers are right — exactly what the chart above did for every crowd up to 101. In 1785 the French thinker Condorcet noticed that when each voter is right more than half of the time, and the voters decide on their own, this chance keeps climbing towards 1 as the crowd grows. It is called Condorcet's jury theorem.

Many Trees, One Answer

What does an ensemble use to make its final prediction?

It combines the predictions of many models, for example by a majority vote
It finds one perfect model and uses only that
It asks a person to check every answer

What is a random forest?

A random forest is an in which every voter is a decision tree. The trees are made different from each other on purpose, with two tricks: bagging and random feature choice.

Sina