Well, a decision tree is a graphical representation of all the possible solutions to a decision based on certain conditions. Now, you might be wondering why this thing is called a decision tree. Well, it is called so because it starts with a root and then branches off to a number of solutions, just like a tree, right? Even the tree starts from a root and it starts growing its branches once it gets bigger and bigger. Similarly, in a decision tree, it has a root, which keeps on growing with an increasing number of decisions and conditions. Now, let me tell you a real-life scenario. I won’t say that all of you, but most of you must have used it. Remember, whenever you dial the toll-free number of your credit card company, it redirects you to its intelligent computerized assistant, where it asks you questions like: "Press 1 for English," or "Press 2 for Hindi," "Press 3 for this," "Press 4 for that," right? Now, once you select one, it again redirects you to a certain set of questions like: "Press 1 for this," "Press 1 for that," and similarly, right? So, this keeps on repeating until you finally get to the right person. You might think that you were caught in a voicemail hell, but what the company was actually doing was just using a decision tree to get you to the right person. I'd like you to focus on this particular image for a moment. On this particular slide, you can see an image where the task is: "Should I accept a new job offer or not?" So, you have to decide that. For that, what you did was create a decision tree, starting with the base condition or the root node, which was that the basic salary or the minimum salary should be $50,000. If it’s not $50,000, then you are not at all accepting the offer, alright? So, if your salary is greater than $50,000, then you will further check whether the commute is more than one hour or not. If it’s more than one hour, you will just decline the offer. If it is less than one hour, then you are getting closer to accepting the job offer. Then further, what you’ll do is check whether the company is offering free coffee or not. Right? If the company is not offering free coffee, then you will just decline the offer. And if it is offering free coffee, then yeah, you will happily accept the offer. Right, this is just an example of a decision tree. Now, let's move ahead and understand a decision tree. Well, here is a sample dataset that I'll be using to explain the decision tree. Alright, in this dataset, each row is an example, and the first two columns provide features or attributes that describe the data, and the last column gives the label or the class we want to predict. And if you like, you can just modify this data by adding additional features and more examples, and our program will work in exactly the same way. Fine. Now, this dataset is pretty straightforward, except for one thing. I hope you have noticed that it is not perfectly separable. Let me tell you something more about that. As in the second and fifth examples, they have the same features but different labels. Both have yellow as their color and diameter as 3, but the labels are mango and lemon. Fine, let's move on and see how a decision tree handles this case. Alright, in order to build a tree, we'll use a decision tree algorithm called CART. The CART algorithm stands for Classification and Regression Tree Algorithm. All right, let's see a preview of how it worked. All right, to begin with, we'll add a root node for the tree, and all the nodes receive a list of rows as input, and the root will receive the entire training dataset. Now, each node will ask a true and false question about one other feature, and in response to that question, we'll split or partition the dataset into two different subsets. These subsets then become input to child nodes, or to the tree, and the goal of the question is to finally unmix the labels as we proceed down, or in other words, to produce the purest possible distribution of the labels at each node. For example, the input of this node contains only one single type of label, so we could say that it's perfectly unmixed. There is no uncertainty about the type of label as it consists of only grapes. Right? On the other hand, the labels in this node are still mixed up, so we would ask another question to further drill it down. Right? But before that, we need to understand which question to ask and when, and to do that, we need to quantify how much a question helps to unmix the label. We can quantify the amount of uncertainty at a single node using a metric called Gini impurity, and we can quantify how much a question reduces that uncertainty using a concept called information gain. We'll use these to select the best question to ask at each point, and then what we'll do is iterate the steps. We'll recursively build the tree on each of the new nodes. We'll continue dividing the data until there are no further questions to ask, and finally, we reach our leaf. All right, so this was about decision tree So, in order to create a decision tree, first of all, what do you have to do? You have to identify a different set of questions that you can ask the tree, like "Is this color green?" And what will these questions be? These questions will be decided by your dataset, like "Is the color green?" "Is the diameter greater than or equal to 3?" "Is that color yellow?" Right? Questions resemble your dataset. Remember that. Alright, so if my color is green, then what will it do? It will divide into two parts. First, the Green Mango will be in the true, while on the false, we have lemon and Mac Holland. If the color is green, or the diameter is greater than or equal to 3, or the color is yellow. Now, let's move on and understand about decision tree terminologies. Alright, so starting with **root node**: The root node is the base node of a tree. The entire tree starts from the root node. In other words, it is the first node of a tree. It represents the entire population or sample, and this entire population is further segregated or divided into two or more homogeneous sets. Fine. Next is the **leaf node**: Well, the leaf node is the one when you reach the end of the tree. Right? That is, you cannot further segregate it down to any other level. That is the leaf node. Next is **splitting**: Splitting is dividing your root node or your node into different subparts on the basis of some condition. Alright, then comes the **branch** or the **subtree**: Well, this branch or subtree gets formed when you split the tree. Suppose when you split a root node, it gets divided into two branches or two subtrees, right? Next is the concept of **pruning**: Well, you can say that pruning is just the opposite of splitting. What we are doing here is just removing the subnode of a decision tree. Let's move on ahead. Next is **parent and child node**: Well, first of all, the root node is always the parent node, and all other nodes associated with that are known as child nodes. Well, you can understand it in a way that all the top nodes belong to a parent node, and all the bottom nodes, which are derived from a top node, are child nodes. The node producing a child for the node is a child node, and the node which is producing it is a parent. Simple concept, right?