AdaBoost So now, let's see what the AdaBoost algorithm is. There are several machine learning algorithms available to address your problem statement, and AdaBoost is one of these powerful, productive modeling techniques known as Adaptive Boosting. AdaBoost is an ensemble method in machine learning. It commonly uses decision trees with just one split, often called decision stumps, as its base estimator. In this approach, the model initially assigns equal weights to all the data points. It then increases the weight of the incorrectly classified points in subsequent models. These points, with higher weight, are given more attention. The process continues, training new models, until the overall error is minimized. We have taken multiple samples from the training data and then combined the results of individual trees to create a single predictive model. This method runs in parallel, as each bootstrap sample does not depend on others. Boosting is another state-of-the-art model that is being used by many data scientists to win so many competitions. Boosting is a general approach that can be applied to many statistical models. Boosting works in a sequential manner and does not involve bootstrap sampling. Instead, each tree is fitted on a modified version of the original data set and finally added up to create a strong classifier. Let’s see the sample methodology on how AdaBoost works. Initially, a simple classifier has been fitted on the data, also called a decision stump, which splits the data into just two regions. Whatever the class is correctly classified will be given less weight in the next iteration (iteration two) and higher weight from misclassified classes. Again, another decision stump or weak classifier will be fitted on the data and will change the weights again for the next iteration (iteration three). Here, check the minus symbols for which weight has been increased. Once it finishes the iterations, these are combined with weights. That is, weights are automatically calculated for each classifier at each iteration based on the error rate to come up with a strong classifier that predicts the classes with surprising accuracy. Let’s have a look at the algorithm for AdaBoost. First, we will initialize the observation weights W to the base I, which is equal to 1nn1, where I=1 to n, and n is the number of observations. Now, for M=1 to MM, we have to fit a classifier G_M(x) to the training data using weights W. After that, we will compute the error, then we will compute α_2M. Finally, we will set W to 1 as the output.we will see G of X all the observations are given equal weight.