Probability Theory Some Proofs from Chapter 2

Some Proofs from Chapter 2 2.14 A proof by induction proceeds logically by (i) Prove the statement true for some base case (typically 1 or 2), then (ii)-(iii): Assume that if the statement is true for case n, then it must be true for n + 1. Notice the logic here- Once proven for n = 2, if we prove that the transition from n to n + 1 is always true, then the statement is true for n = 3. If it is true for n = 3, then it must also be true for n = 4, and so on. The statement we wish to prove by induction is: P (E1 ∪ E2 ∪ . . . ∪ En ) ≤ P (E1 ) + P (E2 ) + . . . + P (En ) First we’ll prove it for a pair of sets (it is trivially true for only one set): P (E1 ∪ E2 ) = P (E1 ) + P (E2 ) − P (E1 ∩ E2 ) ≤ P (E1 ) + P (E2 ) We might note that we get equality only if the sets are mutually exclusive (m.e.) Next we prove the transition: Assuming the statement is true for n sets, it must be true for n + 1 sets. That means that, for the transition, we assume that: P (E1 ∪ E2 ∪ . . . ∪ En ) ≤ P (E1 ) + P (E2 ) + . . . + P (En ) And from this, we must prove that: P (E1 ∪ E2 ∪ . . . ∪ En ∪ En+1 ) ≤ P (E1 ) + P (E2 ) + . . . + P (En ) + P (En+1 ) So we start with the left-hand side of the expression. We will then group the sets in a suggestive manner: P (E1 ∪ E2 ∪ . . . ∪ En ∪ En+1 ) = P ((E1 ∪ E2 ∪ . . . ∪ En ) ∪ En+1 ) = P (E1 ∪ E2 ∪ . . . ∪ En ) + P (En+1 ) − P ((E1 ∪ E2 ∪ . . . ∪ En ) ∩ En+1 ) ≤ P (E1 ∪ E2 ∪ . . . ∪ En ) + P (En+1 ) ≤ P (E1 ) + P (E2 ) + . . . + P (En ) + P (En+1 ) 2.22 Show that if A, B are independent then • A0 and B are independent. We need to show that P (A0 ∩ B) = P (A0 )P (B). Working backwards, let’s see if we can find the right relationship: P (A0 ∩ B) = P (A0 )P (B) = (1 − P (A))P (B) = P (B) − P (A)P (B) = P (B) − P (A ∩ B) What we want Convert A0 to A Independence There it is- Here is the correct direction now: P (A0 ∩ B) + P (A ∩ B) = P (B) P (A0 ∩ B) = P (B) − P (A ∩ B) = P (B) − P (A)P (B) = (1 − P (A))P (B) = P (A0 )P (B) M.E. sets Indep of A, B Therefore, A0 and B are independent. • A0 and B 0 are independent. In this case, show that P (A0 ∩ B 0 ) = P (A0 )P (B 0 ) We might be able to do this straight off: P (A0 ∩ B 0 ) = P ((A ∪ B)0 ) = 1 − P (A ∪ B) = 1 − (P (A) + P (B) − P (A ∩ B) = 1 − P (A) − P (B) + P (A)P (B) = (1 − P (A)) − P (B)(1 − P (A)) = P (A0 )P (B 0 ) M.E. sets Indep of A, B 2.27 If A, B, C are independent, show that: (a) A and B ∩ C are also independent. SOLUTION: First you might write down what it is we need to show. In this case, by the definition of independence, we need: P (A ∩ (B ∩ C)) = P (A)P (B ∩ C) We notice that, from independence of all sets, P (B ∩ C) = P (B)P (C). I think we can prove this: P (A ∩ (B ∩ C)) = P (A ∩ B ∩ C) = P (A)P (B)P (C) = P (A)P (B ∩ C) Standard set theory 3-way independence 2-way indep from the full indep Conclusion: We have shown that A and B ∩ C are independent, if A, B, C are independent. (b) A and B ∪ C are independent. It is easiest to prove this by using two theorems• If A, B are independent, so are A0 and B 0 (Proved in class). • Look at part (a). We are done if we can show that A0 and (B ∪ C)0 are independent, from our first theorem. We see that: (B ∪ C)0 = B 0 ∩ C 0 . Notice that, by part (a), that if A0 , B 0 and C 0 are independent, then so are A0 and B 0 ∩ C 0 . 2.32 Prove Theorem 2.12: If the events B1 , B2 , . . . , Bk constitute a partition of the sample space S and P (Bi ) 6= 0 for i = 1, 2, . . . , k, then for any event A in S we can write: P (A) = k X P (Bi )P (A|Bi ) i=1 (Note the word partition, defined in a footnote on page 9, means that the Bi ’s are mutually exclusive and the union is the entire sample space. Try to draw a Venn diagram of the situation). For any event A, since the sets Bi form a partition, we can write A as a union of mutually exclusive sets: A = (A ∩ B1 ) ∪ (A ∩ B2 ) ∪ . . . (A ∩ Bk ) so that the probability becomes a sum: P (A) = P (A ∩ B1 ) + P (A ∩ B2 ) + . . . P (A ∩ Bk ) Each of these can be written as a conditional probability as long as the probability of each Bi is not zero. We check what we are trying to prove, and we notice that: P (A ∩ Bi ) = P (Bi )P (A|Bi ) so that the statement is proven by rewriting our previous sum: P (A) = P (B1 )P (A|B1 ) + P (B2 )P (A|B2 ) + . . . P (Bk )P (A|Bk ) 1. Extra Practice: Prove that, if A and B are independent, then so are A and B 0 . (Try to work it out before you look at the solution below) SOLUTION: We are given that A, B are independent. This means that we are given: P (A ∩ B) = P (A)P (B) We need to show that this implies: P (A ∩ B 0 ) = P (A)P (B 0 ) We might work backwards a bit to see where it leads us: P (A ∩ B 0 ) = P (A)P (B 0 ) = P (A)(1 − P (B)) = P (A) − P (A)P (B) Hmmm... This might be helpful. Notice that our last equality implies that: P (A) = P (A)P (B) + P (A ∩ B 0 ) but by independence, we can substitute P (A ∩ B) for P (A)P (B): P (A) = P (A)P (B) + P (A ∩ B 0 ) = P (A ∩ B) + P (A ∩ B 0 ) This would be true as long as we can write A = (A ∩ B) ∪ (A ∩ B 0 ) and (A ∩ B), (A ∩ B 0 ) are mutually exclusive (which are both true- Look at the Venn Diagram, or Exercise 2.4, p. 30). There is our proof- Now we’ll go in the right direction (backwards): We know that A ∩ B and A ∩ B 0 are mutually exclusive sets, and A = (A ∩ B) ∪ (A ∩ B 0 ) Therefore, P (A) = P (A ∩ B) + P (A ∩ B 0 ) By the independence of A, B: P (A) = P (A ∩ B) + P (A ∩ B 0 ) = P (A)P (B) + P (A ∩ B 0 ) Work through the algebra: P (A ∩ B 0 ) = P (A) − P (A)P (B) = P (A)(1 − P (B)) = P (A)P (B 0 ) Therefore, A and B 0 are also independent. 2. (Additional Exercise:) Show by means of numerical examples that P (B|A) + P (B|A0 ) may or may not be equal to one. Additionally, try to prove that: P (A|B)+P (A0 |B) = 1 SOLUTION to the proof: Notice that by working backwards slightly, P (A|B) + P (A0 |B) = P (A ∩ B) P (A0 ∩ B) P (A ∩ B) + P (A0 ∩ B) + = P (B) P (B) P (B) Do you see that the numerator is actually P (B)? Here is the direct proof: We can write B as the union of mutually exclusive sets: B = (B ∩ A) ∪ (B ∩ A0 ) = (A ∩ B) ∪ (A0 ∩ B) Therefore, 1= P (B) P (A ∩ B) + P (A0 ∩ B) = = P (A|B) + P (A0 |B) P (B) P (B)