6.041SC Probabilistic Systems Analysis, Problem Set 5 Solutions

Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis Problem Set 5: Solutions 1. (a) Because of the required normalization property of any joint PDF, � � � � 2 �� 2 � 2 2 23 13 2 2 + = a 1= ax dy dx = ax(2 − x) dx = a 2 − 1 − 3 3 3 x=1 y=x x=1 so a = 3/2. (b) For 1 ≤ y ≤ 2, fY (y) = � y a 2 3 (y − 1) = (y 2 − 1), 2 4 ax dx = 1 and fY (y) = 0 otherwise. (c) First notice that for 1 ≤ x ≤ 3/2, fX|Y (x | 3/2) = fX,Y (x, 3/2) = fY (3/2) 3 4 (3/2)x 8x �� � �= . 3 2 5 2 − 1 2 Therefore, E[1/X | Y = 3/2] = � 3/2 1 1 8x dx = 4/5. x 5 2. (a) By definition fX,Y (x, y) = fX (x)fY |X (y | x). fX (x) = ax as shown in the graph. We have that � 40 1 = ax dx = 800a. 0 So fX (x) = x/800. From the problem statement fY |X (y | x) = fX,Y (x, y) = � 1 2x for y ∈ [0, 2x]. Therefore, 1/1600, if 0 ≤ x ≤ 4 and 0 < y < 2x, 0, otherwise. (b) Paul makes a positive profit if Y > X. This occurs with probability P(Y > X) = � � fX,Y (x, y) dy dx = y>x � 0 40 � 2x x 1 1 dy dx = . 1600 2 We could have also arrived at this answer by realizing that for each possible value of X, there is a 1/2 probability that Y > X. (c) The joint density function satisfies fX,Z (x, z) = fX (x) fZ|X (z|x). Since Z is conditionally uniformly distributed given X, fZ|X (z | x) = 21x for −x ≤ z ≤ x. Therefore, fX,Z (x, z) = 1/1600 for 0 ≤ x ≤ 40 and −x ≤ z ≤ x. The marginal density fz (z) is calculated as fZ (z) = � x fX,Z (x, z) dx = � 40 x=|z| 1 dx = 1600 � 40−|z| 1600 , 0, if |z| < 40, otherwise. Page 1 of 7 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis 3. (a) In order for X and Y to be independent, any observation of X should not give any information on Y . If X is observed to be equal to 0, then Y must be 0. In other words, fY |{X=0} (y | 0) = 6 fY (y). Therefore, X and Y are not independent. 1.5 1 fX (x)  if 0 ≤ x ≤ 1,  x/2, (b) fX (x) = −3x/2 + 3, if 1 < x ≤ 2,  0, otherwise. 0.5 0 0 0.2 0.4 0.6 0.8 0 0.1 0.2 0.3 0.4 0 0.2 0.4 0.6 0.8 1 x 1.2 1.4 1.6 1.8 2 2 1.8 1.6 fY |X (y | 0.5) = fY |X (y | 0.5) 1.4 � 2, if 0 ≤ y ≤ 1/2, 0, otherwise. 1.2 1 0.8 0.6 0.4 0.2 0 0.5 0.6 0.7 0.8 0.9 1 1 1.2 1.4 1.6 1.8 2 y 1.5 fX |Y (x | 0.5)   1/2, if 1/2 ≤ x ≤ 1, 3/2, if 1 < x ≤ 3/2, fX|Y (x | 0.5) =  0, otherwise. 1 0.5 0 x (c) The event A leaves us with a right triangle with a constant height. The conditional PDF is then 1/area = 8. The conditional expectation yields: E[R | A] = E[XY | A] � 0.5 � 0.5 = 8xy dx dy 0 y = 1/16. (d) The CDF of W is FW (w) = P(W ≤ w) = P(Y − X ≤ w) = P(Y ≤ X + w). P(Y ≤ X + w) can be computed by integrating the area below the line Y = X + w for all possible values of w. The lines Y = X + w are shown below for w = 0, w = −1/2, w = −1 and w = −3/2. The probabilities of interest can be calculated by taking advantage of the uniform PDF over the two triangles. Remember to multiply the areas by the appropriate joint density fX,Y (x, y)! Take note that there are 4 regions of interest: w < −2, −2 ≤ w ≤ −1, −1 < w ≤ 0 and w > 0. Page 2 of 7 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis y=2−x w=0 1.2 1 y −w y=x 0.8 w ∈ (−1, 0) −w 2 w = −1 0.6 w ∈ (−2, −1) 0.4 1+w 0.2 2+w 2 0 1+w 2+w −0.2 0 0.2 0.4 0.6 0.8 1 x 1.2 1.4 1.6 1.8 The CDF of W is    0,  3/2 · 1/2(2 + w)2 /2, FW (w) = 1/2 · 1/2(1 + w)2 + 3/2 · (1/2 · 1 · 1 − 1/2(−w/2 · −w)),    1,  0,    3/8 · (2 + w)2 , = 1/8 · (−w2 + 4w + 8),    1, if if if if 2 if if if if w < −2, −2 ≤ w ≤ −1, −1 < w ≤ 0, w>0 w < −2, −2 ≤ w ≤ −1, −1 < w ≤ 0, w > 0. As a sanity check, FW (−∞) = 0 and FW (+∞) = 1. Also, FW (w) is continuous at w = −2 and at w = −1. 4. (a) If the transmitter sends the 0 symbol, the received signal is a normal random variable with a mean of −2 and a variance of 4. In other words, fY |X (y | −2) = N (−2, 4). Also, fY |X (y | 2) = N (2, 4) These conditional pdfs are shown in the graph below. Page 3 of 7 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis 0.25 fY |X (y | 2) fY |X (y | −2) 0.2 P(error | X = −2) P(error | X = 2) 0.15 0.1 0.05 0 −8 −6 −4 −2 0 2 4 6 8 The probability of error can be found using the total probability theorem. P(error) = P(error | X = −2)P(X = −2) + P(error | X = 2)P(X = 2) 1 = (P(Y ≥ 0 | X = −2) + P(Y < 0 | X = 2)) 2 1 = (P(N ≥ 2 | X = −2) + P(N < −2 | X = 2)) 2 1 = (P(N ≥ 2) + P(N < −2)) 2 1 N −0 2−0 N −0 −2 − 0 = (P( ≥ ) + P( < )) 2 2 2 2 2 1 = ((1 − Φ(1)) + (1 − Φ(1))) 2 = 0.1587. (b) With 3 components, the probability of error given an obervation of X is the probability of decoding 2 or 3 of the components incorrectly. For each component, the probability of error is 0.1587. Therefore, � � 3 P(error | sent 0) = (0.1587)2 (1 − 0.1587) + (0.1587)3 2 = 0.0676. By symmetry, P(error | sent 1) = P(error | sent 0). Therefore, P(error) = P(error | sent 0)P(sent 0) + P(error | sent 1)P(sent 1) = 0.0676. 5. (a) There are many ways to show that X and Y are not independent. One of the most intuitive arguments is that knowing the value of X limits the range of Y , and vice versa. For instance, if it is known in a particular trial that X ≥ 1/2, the value of Y in that trial cannot be smaller Page 4 of 7 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis than 1/2. Another way to prove that the two are not independent is to calculate the product of their expectations, and show that this is not equal to E[XY ]. (b) Applying the definition of a marginal PDF, for 0 ≤ x ≤ 1, fX (x) = = � � fX,Y (x, y) dy y x+1 1 dy x = 1; for 0 ≤ y ≤ 1, fY (y) = � fX,Y (x, y) dx �xy = 1 dx 0 = y; and for 1 ≤ y ≤ 2, fY (y) = � fX,Y (x, y) dx x = � 1 1 dx y−1 1 1 0.8 0.8 fY (y) fX (x) = 2 − y. 0.6 0.6 0.4 0.4 0.2 0.2 0 0 0 0.2 0.4 x 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 y 1.2 1.4 1.6 1.8 2 (c) By linearity of expectation, the expected value of a sum is the sum of the expected values. By inspection, E[X] = 1/2 and E[Y ] = 1. Thus, E[X + Y ] = E[X] + E[Y ] = 3/2. Page 5 of 7 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (d) The variance of X + Y is E[(X + Y )2 ] − E[X + Y ]2 = E[X 2 ] + 2E[XY ] + E[Y 2 ] − (E[X + Y ])2 . (1) In part (c), E[X+Y ] was computed, so only the other three expressions need to be calculated. First, the expected value of X 2 : � 1 � 1 � x+1 2 2 E[X ] = x 1 dy dx = x2 dx = 1/3. x 0 0 2 Also, the expected value of Y is � 1 � x+1 � 1 2 2 E[Y ] = y dy dx = (3x2 + 3x + 1)/3 dx = 7/6. x 0 0 Finally, the expected value of XY is � 1 � x+1 E[XY ] = x y dy dx 0 x � 1 = (2x2 + x)/2 dx dy = 7/12. 0 Substituting these into (1), we get var(X + Y ) = 1/3 + 7/6 + 7/6 − 9/4 = 5/12. *Alternative (shortcut) solution to parts (c) and (d)* Given any value of X (in ([0,1]), we observe that Y − X takes values between 0 and 1, and is uniformly distributed. Since the conditional distribution of Y − X is the same for every value of X in [0,1], we see that Y − X independent of X. Thus: (a) X is uniform, and (b) Y = X + U , where U is also uniform and independent of X. It follows that E[X + Y ] = E[2X + U ] = 3/2. Furthermore, var(X + Y ) = 4 var(X) + var(U ) = 5/12. 6. (a) Let A be the event that the first coin toss resulted in heads. To calculate the probability P(A), we use the continuous version of the total probability theorem: � 1 � 1 P(A) = P(A | P = p)fP (p) dp = p(1 + sin(2πp)) dp, 0 0 which after some calculation yields P(A) = π−1 . 2π (b) Using Bayes rule, P(A | P = p)fP (p) P(A)   2πp(1 + sin(2πp)) , if 0 ≤ p ≤ 1, = π−1  0, otherwise. fP |A (p) = Page 6 of 7 Massachusetts Institute of Technology Department of Electrical Engineering & Computer Science 6.041/6.431: Probabilistic Systems Analysis (c) Let B be the event that the second toss resulted in heads. We have P(B | A) = = = � � 1 P(B | P = p, A)fP |A (p) dp 0 1 P(B | P = p)fP |A (p) dp � 1 2π p2 (1 + sin(2πp)) dp. π−1 0 0 After some calculation, this yields P(B | A) = 2π 2π − 3 2π − 3 · = ≈ 0.5110. π−1 6π 3π − 3 G1† . Let a = (cos θ, sin θ) and b = (bx , by ). We will show that no point of R lies outside C if and only if |b| ≤ | sin θ|, and |a| ≤ | cos θ| (2) The other two vertices of R are (cos θ, by ) and (bx , sin θ). If |bx | ≤ | cos θ| and |by | ≤ | sin θ|, then each vertex (x, y) of R satisfies x2 + y 2 ≤ cos2 θ + sin2 θ = 1 and no points of R can lie outside of C. Conversely if no points of R lie outside C, then applying this to the two vertices other than a and b, we find cos2 θ + b2 ≤ 1, a2 + sin2 θ ≤ 1. and which is equivalent to 2. These conditions imply that (bx , by ) lies inside or on C, so for any given θ, the probability that the random point b = (bx , by ) satisfies (2) is 2| cos θ| · 2| sin θ| 2 = | sin(2θ)| π π and the overall probability is 1 2π � 2π 0 2 4 | sin(2θ)|dθ = 2 π π � 0 π/2 sin(2θ)dθ = 4 π2 . † Required for 6.431; optional for 6.041 Page 7 of 7