Introduction to Tree-based Algorithms Answers 1. What is a primary advantage of using tree-based algorithms? A. They require no data preprocessing. B. They create complex models that are hard to interpret. C. They can handle both numerical and categorical data. D. They always guarantee high accuracy. Answer: They can handle both numerical and categorical data. (C) Tree-based algorithms are versatile and can effectively manage both types of data. 2. Which statement describes the structure of a decision tree? A. All nodes work independently of one another. B. Internal nodes contain the final prediction values. C. Branches in the tree represent the outcomes of decisions. D. Each leaf node represents a decision based on multiple features. Answer: Branches in the tree represent the outcomes of decisions. (C) In a decision tree, branches represent the outcomes of specific decisions based on feature values. 3. What is a significant drawback of decision trees? A. They can only handle categorical data. B. They are inherently robust and not sensitive to outliers. C. They are generally less interpretable than linear models. D. They tend to overfit, especially on complex datasets. Answer: They tend to overfit, especially on complex datasets. (D) Decision trees are prone to overfitting, particularly when dealing with intricate datasets. 4. How does a Random Forest reduce overfitting? A. By eliminating all outliers in the training data. B. By applying stringent criteria to a single tree. C. By using a single decision tree with complex branches. D. By averaging predictions from multiple decision trees. Answer: By averaging predictions from multiple decision trees. (D) Random Forest constructs many trees and averages their predictions, which typically leads to models that are less prone to overfitting. 5. What is the key feature of Gradient Boosting? A. It creates independent trees that do not interact. B. It generates a single tree trained on the entire dataset. C. It sequentially builds trees that correct the errors of previous trees. D. It solely focuses on minimizing the depth of each tree. Answer: It sequentially builds trees that correct the errors of previous trees. (C) Gradient Boosting constructs trees sequentially, with each new tree aimed at correcting the errors made by previous trees. 6. Which factor significantly contributes to the instability of decision trees? A. The dependency on starting conditions during training. B. The use of uniform splits across all features. C. Sensitivity to small changes in the training data. D. Their ability to capture non-linear relationships. Answer: Sensitivity to small changes in the training data. (C) Decision trees can be significantly unstable, meaning that minor variations in input data can lead to drastically different trees. 7. What distinguishes Random Forest from Gradient Boosting? A. Random Forest is only used for regression tasks. B. Random Forest builds trees in a sequential manner, while Gradient Boosting builds them independently. C. Random Forest uses fewer trees than Gradient Boosting. D. Random Forest aggregates predictions of separately trained trees, while Gradient Boosting focuses on correcting previous errors. Answer: Random Forest aggregates predictions of separately trained trees, while Gradient Boosting focuses on correcting previous errors. (D) Random Forest aggregates the predictions from multiple trees, whereas Gradient Boosting builds trees with a focus on correcting errors from prior trees. 8. What is an inherent characteristic of tree-based algorithms regarding model interpretability? A. They require extensive statistical knowledge for interpretation. B. They provide a clear visualization of decision-making paths. C. They only output numerical values with no explanations. D. They are inherently complex and difficult to interpret. Answer: They provide a clear visualization of decision-making paths. (B) Tree-based algorithms, particularly decision trees, are favored for their ability to create interpretable models through visual decision-making paths.