Implementation of GOSS let's implement GOSS as code. Let's use two methods. The first is to add the GOSS feature to the XGBClassifier. The second method is to use the LGBMClassifier library. You can easily implement this using the LGBM library, but to better understand how GOSS algorithm works, let's implement it from scratch. The data we will use is like this. Set the number of boosting to 50, and set the learning rate to 0.3. And the depth of the tree is set to 2.Create a base model for the first method. We will use the XGBClassifier in XGBoost library as the base model. Here, the number of boosting is set to 1. We will repeat the training 50 times later. Set the learning rate and the max_depth as follows, and set the tree_method to histogram-based. And the number of bins is set to 20. We then fit this model to the training data, x and y, with the weights and the logodds values. The weights and the logodds are passed through arguments. This weight is w, which we saw on the previous page.This w represents the output value of the leaf node in the tree. And F zero is the logodds value predicted from the previous iteration round. Since we use F zero when fitting the model, we do not use the base_score, y-hat 0, here. Please refer to the XGBoost videos for logodds prediction. Now let's implement Algorithm 2, which we looked at on the previous page. Set the parameters a and b to 0.3 and 0.2, respectively. And we calculate the fact using this formula. Calculate the topN and the randN like this. And initialize the logodds value, F zero, to 0. And we set the logodds value from the previous round to F zero. Create 50 models through 50 iterations. Convert logodds values to predicted probabilities, y, using the logistic function. This is the y-prev from the previous round. And we calculate the first-order gradient, g, like this. This is a negative residual. And we initialize the weights, w, to 1.0. This is the initial sample weights. Next, we sort the absolute values of g in descending order. Then construct topSet and randSet as follows. The topSet contains the topN data points, and the randSet contains the randN sampled data points. The two subsets are combined to form a usedSet. Next, update the weights for the data points in the randSet. Since the weights are initialized to 1, there is no need to update the weights for the topSet data. Then create a base model using the function defined on the left. The usedSet's training data, weights and logodds values, F are passed. Next, we predict the logodds, Fm, for all training data points in this way. This will be used in the next round. Save the created model to the models list. Now that the training process is complete, we can use the list of models to predict the targets of the test data. We randomly generated 1000 test data points. Set the initial logodds for the test data to zero. Read the models stored in the model list one by one and predict the logodds, F, for the test data. We then use the logistic function to convert the predicted logodds values for the test data into probabilities.If this probability is greater than 0.5, the target of the test data is predicted to be 1. Otherwise, it is predicted to be 0. Create a function to check the prediction result and decision boundary. Let's use this function to visually check the result. The result is like this. The yellow with black border is part of the training data. And red and blue are test data. Red is the area where the target is predicted to be 0, and blue is the area where the target is predicted to be 1. You can see that the decision boundary appears like this. The decision boundary for the entire training data is like this. Next, let's use the LightGBM library. Set the basic parameters to the same as before and set the number of histogram bins to 20. And we set the boosting_type to GOSS. Set the parameters a and b as follows. Fit this model to the training data, and predict the target class of the test data, and then check the result. The result is like this. This result is similar to the one on the left. The reason the results are not the same is because the data was sampled.