Bernoulli and Multinomial Naïve Bayes Classifier First, we'll look at the **Bernoulli Naive Bayes** classifier. So, this is our training data where we have some text content, and we have to classify whether it's Spam or non-spam. First, we will learn from the training data so that we can generalize for some new test instance. "The offer for limited period" is a test instance we have, which we have to classify into one of the classes, either Spam or non-spam. So, how the **Bernoulli Naive Bayes** classifier or the variant of Naive Bayes for text data works is: we first break down the text into features, and each word becomes a feature. This is called the "bag of words" approach, so the sequence of the words is not considered. We are saying that all words randomly occur, so it’s not able to capture the context. Next, what we do is also remove stop words and do some kind of stemming and other preprocessing. But the main idea is that we use this function or this formula of Bayes' theorem, where we calculate the prior probabilities and we calculate the likelihood. For the likelihood, we use a **Bernoulli distribution**, meaning that even if the word occurs multiple times for a test instance, we just see whether it has occurred or not. That is, success or failure—whether the word is present or not. That is what a **Bernoulli distribution** is. So, we just take all the words and check if that word has occurred in the test instance or not. For example, "promotion" has occurred twice, but still, we just give it a value of 1, meaning it either occurred or it didn’t. Then, we can calculate the prior and likelihood. For the prior, out of 4 instances, 2 are spam, so \( \frac{2}{4} \) for spam and \( \frac{2}{4} \) for non-spam, giving equal probabilities for spam and non-spam. Now, for calculating the likelihood: How likely is the word "promotion" given class spam? Out of the two instances of the spam class, "promotion" has occurred only in one of the instances, so \( \frac{1}{2} \). Similarly, we can calculate for all other words. Given this test instance, we can find the likelihood for the given class spam and so on. We can also do the same for non-spam. Finally, we calculate the posterior probability, which is the prior multiplied by the likelihood. For which class this product is greater, the test instance will be classified into that class. For **Multinomial Naive Bayes**, everything is similar with breaking down the text into features, which are words. However, here the likelihood we use is the **Multinomial distribution**. That is, the probability of a word given a particular class will be equal to the number of times the word has occurred divided by all the words within that class. So, we also capture the frequency of how many times the word has occurred. Calculating the prior is the same: out of 4 classes, 2 are Spam, and out of 4 classes, 2 are non-spam. So, the prior probability for spam and non-spam is \( \frac{2}{4} \) for each. For the likelihood, the probability of the word "promotion" given the spam class becomes \( \frac{2}{15} \), because out of all the 15 words that occur across the spam class, the word "promotion" occurs 2 times. Similarly, for the probability of the word "task" given non-spam, it becomes \( \frac{2}{23} \), because out of the 23 words across the non-spam class, the word "task" appears twice. In a similar manner, we can calculate the likelihood for all other words. Given a test instance, we can multiply the likelihoods to find the posterior, and the instance will get classified into the class with the higher posterior probability. Some necessary adjustments that we have to do in the **Bernioulli** and **Multinomial Naive Bayes** variants as well: it also faces a zero-frequency problem that one word has never occurred in the training data for one of the classes. So, if that word occurs in the test instance, then the probability will become zero for the class where it has never occurred, even though all the other words have occurred in that class. So, what we do is, for all the unique words, we add a Lambda number of times to all the classes before running the classifier. This solves the problem, and this is called **adaptive smoothing** or **Laplace smoothing**. Similar to what we have seen in the categorical Naive Bayes, another thing is that when we break the text into features, that is, each word, the text can be very long. So, multiplication of all the likelihoods, or all the probabilities of the words, can become a very small value, and there can be numerical instability. To bring numerical stability, what we can do is we can do the calculation in logarithmic scale. We know that the log is a monotonically increasing function, so we can just apply a log to it. All the multiplication terms will become additive, so it solves the problem of numerical instability and provides stability. So, these are the two things: the zero-frequency problem and underflow problem, which we can tackle by adaptive smoothing and using the log for calculation.