Ensemble method(Bagging and Boosting) You have a dataset, and for simplicity, we are here taking just four variables: chest pain, blood circulation (some parameter, yes or no), blocked arteries, and the weight of the person. So, somebody is trying to predict whether a person has heart disease or not based on these four attributes. What we do is, we have a lot of data that has already been labeled, like heart disease is available. We train the models and then we take this data, take a subset of all these data, and train multiple models on top of it. Then, finally, we take a vote from each model when we pass one specific row. For example, suppose there were, say, 1000 data rows with these many columns. We trained 100 different decision trees or models on this dataset, and finally, we ask every individual decision tree, "What do you predict? What should come here—yes or no?" We are saying that, seeing that 95 trees predict the value to be yes for this data which we are passing, yes, yes, yes, and 5 say no. So, we’ll go with yes, and that’s what bagging means. We are running 100 parallel algorithms and then we are aggregating the results based on the votes here. So let's see this in detail now. What happens is, you take your entire dataset and create n subsets of your data. Now, the interesting thing is the size of the data remains the same, so all your subsets are equal in terms of the number of rows, which means you are taking subsets on random selection with replacement. I'll explain that in the next figure, but just remember that we are creating n subsets, where n can be 100, 250, or whatever number makes sense. So, those many subsets of your data, but all of the subsets have the same number of rows as your main dataset. Then, what you do is you create one individual tree on each dataset. Now, what is the difference between these trees? One is they are getting different data because, even though the size is the same, there would be a lot of repetition of rows, and then two subsets will not be the same or equal to each other in terms of the data. They are the same in data count, but they are not the same data. Another difference is that every tree is not exposed to all the columns. So, if there were 10 columns in your dataset, your subsets will also have 10 columns, for example. But the decision tree will not be supplied with 10 columns. We would decide how many columns we want to give to every node while splitting. It can be 2, 3, 4, or whatever number we think is right. Then, the tree would be predicting based on different columns and different data rows. After which, we are going to test, and I'll explain how we do this process in detail—how we create these datasets and how we decide how many columns and which columns to pick. But this is the general idea of bagging in a random forest. Right, so just to reiterate: take the entire dataset, create n subsets of the same size. Of course, the data is different because what we are doing is random sampling with replacement. Then, we pass this data to different trees, where we have different columns for the tree, and which is at random. Then, we are going to train these individual trees, and we will take the vote of these trees, as we have something called bagging. There is another concept called boosting for ensemble learning, but in boosting, we do not run parallel algorithms; we run sequential algorithms. And there are different algorithms for this. Random forest does not work on boosting; it works on bagging. So, in boosting, what we do is we take the dataset, we train one model, we learn from this model, and then we change a few things. We change some data, knowing wherever we have the maximum confidence, and there are other things that happen. Then, we train the next model. After that, we get the output, learn from it, and then train another model. So, even here, we are using multiple algorithms and models, but we are doing that in sequence rather than in parallel. Eventually, we’ll have a final outcome. So, this is the concept of boosting. Both bagging and boosting are ensemble algorithms.