The Sum Rule, Conditional Probability, and the Product Rule

The Sum Rule, Conditional Probability, and the Product Rule The Sum Rule is a method of finding the probability of two or more independent events occurring together. It states that the probability of two independent events happening at the same time is equal to the sum of the probabilities of them happening separately. Conditional Probability is the possibility that a certain event will occur given that another event has occurred. In other words, conditional probability is the likelihood that an event will happen if another event has happened. The Product Rule is the sum of products rule for differentiation. It is used to calculate the derivative of a composite function, which is a function formed by multiplying two or more functions together. Next what is coming up is marginal probabilities and the sum rule. In probability problems where the joint probabilities of two events are known, one may wish to find out the individual probability of one event occurring regardless of whether or not the other event occurs. Suppose we have probabilities for events x1, x2, x3… and y1, y2, y3... Then we can refer to the probability of x1 as the marginal probability of x1. Something like being out on the margins of this matrix. The sum rule tells us that the marginal probability of x 1, given that y is a proper probability distribution, is equal to the sum of the joint probabilities. Thus, the probability of x1 = 1 +, 1% + 10% + 4% = 15%. Similarly, the marginal probability of y2 equals the sum of the marginal probabilities of x1y2, x2y2, and x3y2. This would be 79 percent or .79. That we can add together joint probabilities to get a marginal probability is due to something called the sum rule. Those are two versions of the sum rule written out. The first case is for a probability distribution involving two possible outcomes. The probability of occurrence of outcome B (event A) is given by the formula P(B) = P(A∩B). The probability of non-occurrence of outcome B (event A) is given by the formula P(not B) = P(A∩not B). The joint probability of occurrence and non-occurrence of both events A and B is given by the formula P(A∩B∩not B) = P(A∩B) +P(not B). Similarly, if we consider the joint probability of a series of events, n events, we can add together their individual probabilities to find the marginal probability of A. Therefore, it's exactly the same principle. Then we will discuss conditional probability. Conditional probability is defined as the probability that a statement is true given that some other statement is true with certainty. It is the likelihood that a fact is true given another fact. All items to the right of the dividing line are considered true with certainty. The notation P(A|B) is read: "the probability of A given B." In other words, if B is true, what is the probability of A? So if I throw a six-sided die and it comes up odd, what is the conditional probability that the number on top is a 3? Three odd rolls, one, three and five, yield a conditional probability of one-third. If I throw a three with certainty, what is the probability that my throw is odd? In this case, the probability is one. It appears odd with certainty if it is three with certainty. When we are working with conditional probabilities, we are dealing with dependent rather than independent variables. The general formula for calculating conditional probabilities is that we take the relevant outcomes, the ones that meet our definition of A, and divide them by the total outcomes in our universe. On the other hand, our results have been diminished because B must be true. So, in the example of the die I just gave you, we are reducing our options from six possibilities—one, two, three, four, five and six—to the odd possibilities: one, three and five. So there is one outcome of three and three outcomes of five. My probability is one-third. The conditional probability of throwing a three if I know that the die is odd is equal to the joint probability of throwing a three and the die being odd divided by the conditional probability of the die being odd. We now want to relate our ideas of joint probability, marginal probability, and conditional probability. For that one, we need the product rule. The product rule tells us that the conditional probability of A given that B is true with certainty is equal to the joint probability that both A and B are true divided by the marginal probability that B is true. So it's clear that we don't assume the truth of B when it's to the right of the magic line. This statement is what makes it true, as there is no need to equal zero anywhere else in the equation. The product rule allows us to develop a new definition of independence. Our previous definition is that the joint distribution is equal to the product distribution, okay? In order to arrive at our new definition, we divide both sides by the probability of B. We then have the joint probability of A and B divided by probability of B; and we have the probability of A given B divided by the probability of B assuming that B is not 0. We can derive the product rule by saying that this term is equal to the probability of A given B. And that is equal to the probability of A. Our intuition about what this means is that knowing that B is true tells us nothing about the probability of A. Therefore, A and B must be independent. The converse is true as well. When the conditional probability of A given B does not equal to probability of A, it means they are dependent. Two distributions are either independent or dependent; there is no middle ground.