In supervised machine learning, there is a plethora of machine learning models like linear regression, logistic regression, decision tree and others. we use these models to resolve classification or regression problems, and ensemble learning is a part of supervised learning that gives us models that are built using several base models. Random forest is one of those ensemble learning models that are popular in the data science field for its high performance.

Technically, random forest models are built on top of decision trees and we have already covered the basics of a decision tree in one of our articles, so we recommend reading the article once to understand this article’s topic clearly. In this article, we will talk about random forests using the following points.

Table of content

• What is Random Forest?
• How Does a Random Forest Work?
• Important Features
• Important Hyperparameters
• Code Example
• Pros and Cons of Random Forest

What is Random Forest?

Random forest is a supervised machine-learning algorithm that comes under the ensemble learning technique. In supervised machine learning, a random forest can be used to resolve both classification and regression problems.

As discussed above that, it comes under the ensemble learning technique, so it works on top of many decision trees. We can say that decision trees are the base model of a random forest. The algorithm simply builds many decision trees on different data samples, and using the majority vote system solves the classification problem. In the case of regression, it uses the average of the decision trees.

How does a Random Forest Work?

When we talk about the working of the random forest, we can say that it gives outcomes by ensembling the results of many decision trees. Here if we talk about a classification problem, each decision tree predicts an outcome and whatever the class gets majority votes comes out as the final result of a random forest. Let’s take a look at the below image.

When we talk about the working of the random forest, we can say that it gives outcomes by ensembling the results of many decision trees. Here if we talk about a classification problem, each decision tree predicts an outcome and whatever the class gets majority votes comes out as the final result of a random forest. Let’s take a look at the below image.

The above image also gives the intuition behind the ensemble learning technique, where the final prediction is made by combining the results of several other models. The ensemble learning technique can be followed using two ways:

1. Bagging: this way, we divide data into various subsets and train the base models like decision trees in a random forest, and the majority vote for any class comes out as the final result.
2. Boosting: this way, we combine weak learners with strong learners and make a sequence of the model so that the final model is most accurate amongst every learner. For example, XG boost and ADA Boost models.

Random forest in ensemble learning uses the bagging method. We can say that every decision tree under the random forest uses a few samples from the whole training data to get trained and give predictions. Let’s talk about the steps involved in training the random forest algorithm.

Steps involved

1. First, it extracts n number of subsets from the dataset with k number of data points that we call n subsets.
2. n number of decision trees are constructed to get trained using n subsets.
3. Each decision tree gives predictions.
4. Final predictions are generated using the majority voting system for the classification problem and an averaging system for the regression problem.

Using the above four steps working of a random forest gets completed. Next, let’s discuss the important features of a random forest.

Important features

1. Highly immune to dimensionality: Since all data features are not considered in the making of decision trees, the whole random forest gives high performance even in a situation where data is high-dimensional.
2. Diversity: every decision tree uses some of the features from the data. That’s why the training procedure becomes different for the different decision trees. At final, we get more optimum results.
3. Data split: while making a random forest, we don’t really need to spit data in train and test because there will always be some percentage of data unknown for a decision tree.
4. Stable: random forests are stable algorithms when modelled because the majority voting or averaging system is used to make the final prediction.
5. Parallelization: as we know, every individual decision tree uses a part of the main data. It makes full use of the CPU to train random forests.
6. No overfitting: as the final results from the random forest come from the majority voting or averaging system and the decision tree uses subsets to get trained, there are fewer chances of overfitting.

Important Hyperparameters

In the above we have discussed the working and features of random forests, here we will discuss the important hyperparameters of any random forest using which we can control the random forest while increasing its performance and making it’s working or calculation faster.

1. n_estimators- The number of decision trees required to build the random forest.
2. max_features- Maximum number of features that random forest will use from data to split the data.
3. mini_sample_leaf — minimum number of leaves is required to split the decision tree node.
4. n_jobs — we use it to speed up the calculation of random forest because it tells the number of processors a system needs to train the model.
5. random_state- just like for other models, it controls the randomness of the sample.

Code Example

In the above discussion, we have seen how random forest work and their important hyperparameters. Now after knowing this, we need to know how it works using any tool. So here we will look at the simple implementation of the random forest using the python programming language.

We will use randomly generated data and the sklearn library in this implementation. So let’s start with generating data.