Naïve Bayes Classifier The Bayes' theorem and the naive assumption are together used in the form of a classifier, which is the Naive Bayes classifier. How they are actually used, let's look at that. So, we have seen Naive Bayes classifier as a conditional probability model that uses Bayes' theorem. So, what happens if we have a vector \( X \)? Vector \( X \) can be thought of as a training or test instance. So, we have an instance, and we have to classify it into one of the classes. Then, we will use this formula, which is very similar to—or which is actually the Bayes' theorem formula—but articulated in a way where it can help us predict the probability of the classes. So, given the test instance, we have to classify it into one of the classes. What is the probability that it belongs to one of the \( K \) classes? The probability that the instance belongs to class \( C_K \) is given by the probability of class \( C_K \) multiplied by the probability of \( X \) given \( C_K \), divided by the probability of \( X \). So, what are these terminologies called in the domain of Naive Bayes classifier? Let's look into that. This probability is known as the **posterior probability**, and this probability \( P(C_K) \) is known as the **prior probability**. The probability \( P(X|C_K) \) is called the **likelihood**, and the probability \( P(X) \) is called the **evidence**. So, how is it applied? When we have to classify an instance into one of the classes, we first find the prior probability, which is how frequently that class occurs in the dataset. So, if you have, let's say, 10 data points, and in 2 of them Class A occurs and in 4 of them Class B occurs, you can simply calculate that \( \frac{2}{10} \) for Class A and \( \frac{4}{10} \) for Class B. This is how you calculate the prior probability. Now, given the class, how likely the feature is becomes easier to calculate because of the Naive assumption—that all features are independent. So, these things become independent. Given class \( C_K \), what is the probability of \( X_1 \) given class \( C_K \)? What is the probability of \( X_2 \)? And so on. Because of this assumption, we are able to break it or write it into independent terms, and we can drop the term \( P(X) \) because it's common across the calculation of all the classes. So, the instance will get classified to the class with the highest posterior probability, which is nothing but the prior multiplied by the likelihood. Now, enough being said, let's look at how it is actually used. So, let's say a person has a 170 lb weight, 5'9" height, and the country of origin is the US. We have to find whether the person is male or female. So, what we will do is, first of all, we will have to see in the dataset what is the prior probability of male and female—how frequently male occurs and how frequently female occurs. Then, we can use the likelihood to reverse calculate the probabilities. That is, given a male, how likely is this height (5'9")? Given a male, how likely is weight (170 lbs)? How likely is the country the US? Similarly, we can do the same for female. We can find which probability is bigger, and the instance (the person) will get classified to that sex—male or female. So, we looked at Naive Bayes classifier. This is the formula for Naive Bayes classifier: prior * likelihood, and the class with the highest posterior gets the instance which we are trying to classify. But depending on the feature type, the likelihood calculation may differ. So, what is the likelihood? How likely is this feature given class \( C \)? Right, if the feature is continuous data, like weight or height of the person, the type of Naive Bayes variant we use is **Gaussian Naive Bayes**. So, basically, we fit a distribution, a Gaussian distribution, to the feature, and we determine the parameters of that Gaussian distribution by maximum likelihood estimation. And, like I told you, weight, height of the person—these are examples of continuous data. And when the feature is categorical, for example, country of origin and all, we see how likely different values within that feature occur, and we can find the probabilities. So, there, the version we use, or the variant we use, is **Categorical Naive Bayes**. For text and NLP data, we transform the text and NLP data into words, and we see whether the presence of a word is there in that instance or not. So, presence of a word and frequency of the word is used to calculate the probability. An example could be: given a document, we have to classify whether it's a spam or non-spam mail. The variant for this of Naive Bayes is **Bernoulli** and **Multinomial Naive Bayes**.