6.041SC Spring 2009 Quiz 1 Solutions

Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) Question 1 Multiple Choice Questions: CLEARLY circle the appropriate choice. Scratch paper is available if needed, though NO partial credit will be given for the Multiple Choice. Each multiple choice question is worth 4 points. a. Which of the following statements is NOT true? (i) If A ⊂ B, then P(A) ≤ P(B). (ii) If P(B) > 0, then P(A|B) ≥ P(A). (iii) P(A ∩ B) ≥ P(A) + P(B) − 1. (iv) P(A ∩ B c ) = P(A ∪ B) − P(B). Solution: A counterexample: if we have two events A, B such that P (B) > 0 and P (A) > 0, but A ∩ B = φ, then P (A|B) = 0, but P (A) > P (A|B). It’s easy to come up with examples like this: for example, take any sample space with event A such that P (A) > 0, and P (Ac > 0), it follows that P (A|Ac ) = 0, but P (A) > 0. b. We throw n identical balls into m urns at random, where each urn is equally likely and each throw is independent of any other throw. What is the probability that the i-th urn is empty? � �n 1 (i) 1 − m � �m (ii) 1 − n1 � � �m m � (iii) 1 − n1 n � � n � 1 �n (iv) m m Solution: The probability of the jth ball going into the ith urn is 1/m. Hence, the probability of the jth ball not going into the ith urn is (1 − 1/m). Since all throws are independent from one another, we can multiply � these � probabilities: the probability of all n balls not going into the 1 n ith urn, i.e. it is empty, is 1 − m . c. We toss two fair coins simultaneously and independently. If the outcomes of the two coin tosses are the same, we win; otherwise, we lose. Let A be the event that the first coin comes up heads, B be the event that the second coin comes up heads, and C be the event that we win. Which of the following statements is true? (i) Events A and B are not independent. (ii) Events A and C are independent. (iii) Events A and B are conditionally independent given C. (iv) The probability of winning is 3/4. Solution: The sample space in this case is Ω = {(H, H), (H, T ), (T, H), (T, T )}. The prob­ ability law is a uniform distribution over this space. We have A = {(H, H), (H, T )}, B = {(H, H), (T, H)}, and C = {(H, H), (T, T )}. By the discrete uniform law, P (A) = P (B) = 1 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) P (C) = 1/2. We also have P (A ∩ C) = 1/4, hence P (A ∩ C) = P (A)P (C), and the two events are independent. Intuitively, knowing that you won adds no information about whether your coin turned up heads or not: stating this formally, we have P (A|C) = P (A). d. For a biased coin, the probability of “heads” is 1/3. Let H be the number of heads in five independent coin tosses. What is the probability P(first toss is a head | H = 1 or H = 5)? (i) 1 2 4 ( ) 3 3 5 31 ( 32 )4 +( 31 )5 (ii) 1 2 4 ( ) 3 3 1 2 4 ( ) +( 13 )5 3 3 (iii) (iv) 1 2 4 ( ) +( 13 )5 3 3 1 2 4 5 3 ( 3 ) +( 13 )5 1 5 Solution: Let A be the event that the first toss is a head. P (A|{H = 1} or {H = 5}) = = = = P (A ∩ ({H = 1} ∪ {H = 5})) P ({H = 1} ∪ {H = 5}) P ((A ∩ {H = 1}) ∪ (A ∩ {H = 5}) P ({H = 1} ∪ {H = 5}) P ({H = 5}) + P (A ∩ {H = 1}) P ({H = 1}) + P ({H = 5}) (1/3)5 + (1/3)(2/3)4 �5� �5� . 4 5 1 (1/3)(2/3) + 5 (1/3) e. A well-shuffled deck of 52 cards is dealt evenly to two players (26 cards each). What is the probability that player 1 gets all the aces? 0 @ (i) 0 @ 0 4@ (ii) 0 @ (iii) (iv) 48 22 1 52 26 1 48 22 52 26 A 1 A 1 4!@ A @ 52 26 =4× 26×25×24×23 52×51×50×49 A 1 0 26×25×24×23 52×51×50×49 A 48! 52! 22! 26! 0 48 22 = 1 = 4! × 26×25×24×23 52×51×50×49 A 2 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) Solution: Let A be the event that player 1 gets all aces. By the disrete uniform law, P (A) = |A| . |Ω| (1) � � |Ω| = 52 26 is the number of hands (26 cards from 52) player 1 can have. Additionally, once we have given player 1 all aces, then they must be given an additional 22 cards from the remaining 48 cards in the deck. Hence, � � 48 � P (A) = �22 52 26 f. Suppose X, Y and Z are three independent discrete random variables. Then, X and Y + Z are (i) always (ii) sometimes (iii) never independent. Solution: Since X is independent of Y and Z, X is independent of g(Y, Z) for any function g(Y, Z), including g(Y, Z) = Y + Z (see page 114 of the book). g. To obtain a driving licence, Mina needs to pass her driving test. Every time Mina takes a driving test, with probability 1/2, she will clear the test independent of her past. Mina failed her first test. Given this, let Y be the additional number of tests Mina takes before obtaining a licence. Then, (i) E[Y ] = 1. (ii) E[Y ] = 2. (iii) E[Y ] = 0. Solution: Y is defined as the number of additional tests Mina takes, so this is independent of the fact that she failed her first test. Y is a geometric RV with p = 1/2. Hence, E[Y ] = 1/p = 2. h. Consider two random variables X and Y , each taking values in {1, 2, 3}. Let their joint PMF be such that for any 1 ≤ x, y ≤ 3, PX,Y (x, y) = 0 if (x, y) ∈ {(1, 3), (2, 1), (3, 2)}, and PX,Y (x, y) > 0 if (x, y) ∈ {(1, 1), (1, 2), (2, 2), (2, 3), (3, 1), (3.3)}. Then, (i) X and Y can be independent or dependent depending upon the values of PX,Y (x, y) for (x, y) ∈ {(1, 1), (1, 2), (2, 2), (2, 3), (3, 1), (3.3)}. (ii) X and Y are always independent. (iii) X and Y can never be independent. Solution: If, for example, we are given information that X = 1, we know that Y can never take value 3. However, without this information about X the probability pY (3) is strictly positive and so pY |X (y|x) �= pY (y), for x = 1 and y = 3, i.e. X and Y can never be independent. 3 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) i. Suppose you play a matching coins game with your friend as follows. Both you and your friend each have your own coin. Each time, the two of you reveal a side (i.e. H or T) of your coin to each other simultaneously. If the sides match, you WIN $1 from your friend and if sides do not match then you lose $1 to your friend. Your friend has a complicated (unknown) strategy in selecting the sides over time. You decide to go with the following simple strategy. Every time, you will toss your unbiased coin independently of everything else, and you will reveal its outcome to your friend (of course, your friend does not know the outcome of your random toss until you reveal it). Then, (i) On average, you will lose money to your smart friend. (ii) On average, you will neither lose nor win. That is, your average gain/loss is 0. (iii) On average, you will make money from your friend. Solution: Let Xi be a random variable denoting your winnings at the i’th round of the game, i.e., Xi = 1 if you win, Xi = −1 if you lose. At each round your friend chooses either heads or tails, using some strategy that you don’t know about. The key property is that for any choice that you friend makes, we have pXi (1) = pXi (−1) = 0.5: i.e., we always have a 0.5 probability that our coin toss will match the choice made by our friend. It can be verified that E[Xi ] = 0, and hence your average gain/loss is 0. j. Let X� i , 1 ≤ i ≤ 4 be independent Bernoulli random variables each with mean p = 0.1. Let X = 4i=1 Xi . Then, (i) E[X1 |X = 2] = 0.1. (ii) E[X1 |X = 2] = 0.5. (iii) E[X1 |X = 2] = 0.25. Solution: We have P (X1 = 1|X = 2) = 0.5, because P (X1 = 1|X = 2) = = = P (X1 = 1 ∩ X = 2) P (X = 2) �3� p × 1 p(1 − p)2 �4� p2 (1 − p)2 �3�2 �14� = 0.5 2 �4� (Note that 2 p2 (1 − p)2 is the probability of seeing 2 heads out of 4 tosses, and the probability of seeing 1 head in the last 3 tosses.) �3� 1 p(1 − p)2 is Hence, E[X1 |X = 2] = 1 × P (X1 = 1|X = 2) + 0 × P (X1 = 0|X = 2) = 0.5 4 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) Question 2: Alice and Bob both need to buy a bicycle. The bike store has a stock of four green, three yellow, and two red bikes. Alice randomly picks one of the bikes and buys it. Immediately after, Bob does the same. The sale price of the green, yellow, and red bikes are $300, $200 and $100, respectively. Let A be the event that Alice bought a green bike, and B be the event that Bob bought a green bike. a. (5 points) What is P(A)? What is P(A|B)? Solution: We have P(A) = 4/9 (4 green bikes out of 9), and P(A|B) = 3/8 (since we know that Bob has a green bike, Alice can have one of 3 green bikes out of the remaining 8). b. (2 points) Are A and B independent events? Justify your answer. Solution: Since P(A) = � P(A|B), the events are not independent. Informally, since there is a fixed quantity of green bikes, if Alice buys one, then the chances that Bob buys one too are slightly decreased. c. (5 points) What is the probability that at least one of them bought a green bike? Solution: The requested probability is P(A ∪ B). We have P(A ∪ B) = P(A) + P(B) − P(A ∩ B) = P(A) + P(B) − P(A|B) · P(B) 13 4 4 3 4 = + − · = = 0.722. 18 9 9 8 9 5 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) d. (5 points) What is the probability that Alice and Bob bought bicycles of different colors? Solution: Let’s compute first the probability that Alice and Bob bought bikes of the same color. We have 4 3 3 2 2 1 P({Y, Y }) = · , P({G, G}) = · , P({R, R}) = · . 9 8 9 8 9 8 Therefore, the probability of buying bikes of different color is � � 12 6 2 13 P(different color) = 1 − P(same color) = 1 − + + = = 0.722. 72 72 72 18 e. (5 points) Given that Bob bought a green bike, what is the expected value of the amount of money spent by Alice? Solution: If Bob bought a green bike, then the conditional probabilities of Alice buying a green, yellow, or red bike are 83 , 83 and 28 , respectively. The expected amount of money spent by Alice is therefore 3 3 2 $300 · + $200 · + $100 · = $212.50. 8 8 8 f. (5 points) Let G be the number of green bikes that remain in the store after Alice and Bob’s visit. Compute P(B|G = 3). Solution: If G = 3, then exactly one green bike was bought. By symmetry, there is equal chance that Alice or Bob bought it, thus P(B|G = 3) = 12 . Alternatively, define A \ B as the elements of A that are not in B. We have: P(B|G = 3) = P(B|{A \ B} ∪ {B \ A}) P(B \ A) 20/72 1 = = = . 40/72 2 P({A \ B} ∪ {B \ A}) 6 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) Question 3: Magic Games Inc. is a store that sells all sorts of fun games. One of its popular products is its magic 4-sided dice. The dice come in pairs; each die can be fair or crooked, and the dice in any pair can function independently or, in some cases, can have magnets inside them that cause them to behave in unpredictable ways when rolled together. Xavier and Yvonne together buy a pair of dice from this store. Each of them picks a die in the pair; one of them then rolls the two dice together. Let X be the outcome of Xavier’s die and Y the outcome of Yvonne’s die. The joint PMF of X and Y , pX,Y (x, y), is given by the following figure: Y 4 1 20 1 20 1 20 2 20 3 1 20 1 20 2 20 1 20 2 1 20 2 20 1 20 1 20 1 2 20 1 20 1 20 1 20 1 2 3 4 X (a) (2 points) Find the PMF of the outcome of Xavier’s die, pX (x). Solution: pX (x) = 4 � � pX,Y (x, y) = 1 4 0 y=1 x = 1, 2, 3, 4 o.w. (b) (2 points) Find the PMF of the outcome of Yvonne’s die, pY (y). Solution: pY (y) = 4 � � pX,Y (x, y) = x=1 1 4 0 y = 1, 2, 3, 4 o.w. (c) (2 points) Are X and Y independent? Solution: No. One of many counter examples: pX (x) does not equal pX|Y (x|2). 7 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) Zach and Wendy are intrigued by Xavier and Yvonne’s dice. They visit the store and buy a pair of dice of their own. Again, each of them picks a die in the pair; one of them then rolls the two dice together. Let Z be the outcome of Zach’s die and W the outcome of Wendy’s die. The joint PMF of Z and W , pZ,W (z, w), is given by the following figure: W 4 2 24 1 24 2 24 1 24 3 2 24 1 24 2 24 1 24 2 2 24 1 24 2 24 1 24 1 2 24 1 24 2 24 1 24 1 2 3 4 Z The store also sells a variety of magic coins, some fair and some crooked. Alice buys a coin that on each toss comes up heads with probability 3/4. (d) (5 points) Wondering whether to buy some dice as well, Alice decides to try out her friends’ dice first. She does the following. First, she tosses her coin. If the coin comes up heads, she borrows Xavier and Yvonne’s dice pair and rolls the two dice. If the coin comes up tails, she borrows Zach and Wendy’s dice pair and rolls those instead. What is the probability that she rolls a double, i.e., that both dice in the pair she rolls show the same number? Solution: Let event D be the set of all doubles, and let event A be the event that Alice’s coin toss results in heads. Using the law of total probability: P (D) = P (D|A)P (A) + P (D|Ac )P (Ac ) 3 4 1 1 = × + × 4 10 4 4 58 = = .3625 160 8 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) (e) (5 points) Alice has still not made up her mind about the dice. She tries another experiment. First, she tosses her coin. If the coin comes up heads, she takes Xavier and Yvonne’s dice pair and rolls the dice repeatedly until she gets a double; if the coin comes up tails, she does the same with Zach and Wendy’s dice. What is the expected number of times she will need to roll the dice pair she chooses? (Assume that if a given pair of dice is rolled repeatedly, the outcomes of the different rolls are independent.) Solution: Let random variable N be the number of rolls until doubles is rolled. The distribution on N condition on the set of dice being rolled is a geometric random variable. Using the total expectation theorem, the expected value of N is: E[N ] = E[N |A]P (A) + E[N |Ac ]P (Ac ) 3 1 1 1 = × 4 + × 1 4 4 10 4 23 = = 2.875 8 (f) (5 points) Alice is bored with the dice and decides to experiment with her coin instead. She tosses the coin until she has seen a total of 11 heads. Let R be the number of tails she sees. Find E[R]. (Assume independent tosses.) Solution: The time T until Alice sees a total of 11 heads is the sum of 11 independent and identically distributed geometric random variables with parameter p = 34 . Random variable R, the number of tails she sees, is T − 11. Thus: E[R] = E[T ] − 11 1 = 11 × 3 − 11 4 = 11 = .9167 12 9 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (Quiz 1 Solutions| Spring 2009) (g) (5 points) Alice tries another experiment with her coin. Let A be the event that the second head she sees occurs on the 7th coin toss, and let S be the position of the first head. Find the conditional PMF of S given the event A, pS|A (s). to be Solution: The probability of event A can be found by choosing one of the first 6 outcomes �6� a head, the others tails, and then the outcome of the 7th toss to be head, which is 1 (1 − p)5 p2 , where p = 34 . The intersection of S = s with event A, P (S = s ∩ A), is an event with probability (1 − p)5 p2 for all values of s (s = 1, . . . , 6). Consequently, pS|A (s) = P (S=s∩A) is a uniform P (A) distribution over the range of s (s = 1, . . . , 6). � 1 6 s = 1, . . . , 6 pS|A (s) = 0 o.w. (h) (5 points) Alice’s friend Bob buys a coin from the same store that turns out to be fair, i.e., that on any toss comes up heads with probability 1/2. He tosses the coin repeatedly until he has seen either a total of 11 heads or a total of 11 tails. Let U be the number of times he will need to toss the coin. Find the PMF of U , pU (u). (Assume independent tosses.) Solution: Bob must toss a coin at least 11 times and at most 21 times in order to have either 11 heads or 11 tails. The intersection �u−1�of Bob requiring u tosses and 11 of those tosses being heads, is the sum of probability the 10 sequences that conclude with a head and have a total of 11 � �u heads. The probability of each of those sequences is 12 . If we consider any sequence in Bob’s experiment with u tosses, since the coin is fair, that sequence is equally likely to have 11 heads and u − 11 tails or 11 tails and u − 11 heads. Consequently, the intersection of Bob requiring u tosses and 11 of those tosses being tails is identical to the probability that the sequence had 11 heads. Summing these two mutually exclusive probabilities which total pU (u): � �u−1� � 1 �u u = 11, . . . , 21 2 10 2 pU (u) = 0 o.w. 10