18.443 Exam 1 with Solutions

18.443 Exam 1 Spring 2015 Statistics for Applications 3/5/2015 1. Log Normal Distribution: A random variable X follows a Lognormal(θ, σ 2 ) distribution if Y = ln(X) follows a N ormal(θ, σ 2 ) distribution. For the normal random variable Y = ln(X) • The probability density function of Y is 1 (y − θ)2 − 1 e 2 σ 2 , −∞ < y < ∞. f (y | µ, σ 2 ) = √ 2 2πσ • The moment-generating function of Y is 1 tθ + σ 2 t2 2 tY 2 MY (t) = E[e | θ, σ ] = e (a). Compute the first two moments of a random variable X ∼ Lognormal(θ, σ 2 ). µ1 = E[X | θ, σ 2 ] and µ2 = E[X 2 | θ] Hint: Note that X = eY and X 2 = e2Y where Y ∼ N (θ, σ 2 ) and use the moment-generating function of Y . (b). Suppose that X1 , . . . , Xn is an i.i.d. sample from the Lognormal(θ, σ 2 ) distribution of size n. Find the method of moments estimates of θ and σ2. Hint: evaluate µ2 /µ12 and find a method-of-moments estimate for σ 2 first. (c). For the log-normal random variable X = eY , where Y ∼ N ormal(θ, σ 2 ), prove that the probability density of X is 1 (ln(x) − θ)2 − 1 σ2 f (x | θ, σ 2 ) = √ ( )e 2 , 2πσ 2 x 1 0 < x < ∞. (d). Suppose that X1 , . . . , Xn is an i.i.d. sample from the Lognormal(θ, σ 2 ) distribution of size n. Find the mle for θ assuming that σ 2 is known to equal σ02 . (e). Find the asymptotic variance of the mle for θ in (d). Solution: 1 (a). µ1 = E[X] = E[eY ] = MY (1) = eθ+σ 2 /2 µ2 = E[X 2 ] = E[e2Y ] = MY (2) = e2θ+2σ 2 (b). First, note that: µ2 /(µ21 ) = eσ 2 It follows that a method-of-moments estimate for σ 2 is σ̂ 2 = ln(µ̂2 /µ̂21 ) where 1 n µ̂2 = n1 Substituting σ̂ 2 µ̂1 = n i=1 Xi n 2 i=1 Xi for σ 2 in the formula for µ1 we get θ+σ̂ 2 /2 µ̂1 = e =⇒ θ̂ = ln(µ̂1 ) − σ̂ 2 /2 (c). Consider the transformation X = eY . which has the inverse: y = ln(x) and dy/dx = 1/x. It follows that fX (x) = fY (ln(x))|dy/dx| = 1 1 − 2σ2 (ln(x)−θ)2 √ 1 e 2πσ 2 x (d). The log of the density function for single realizations x is ln[f (x | θ, σ02 ] = − 21 ln(2πσ02 ) − ln(x) − 1 (ln(x)−θ)2 2 σ02 For a sample x1 , . . . , xn , the likelihood function is n 2 £(θ) = i=1 ln[f (xi | θ, σ0 ] n 1 = − 2σ2 i=1 (ln(xi ) − θ)2 + (terms not depending on θ) 0 £(θ) is minimized by θ̂ = Yi = ln(Xi ) values. 1 n n i=1 ln(xi ) – the mle from the sample of (e). The asymptotic variance satisfies 2 E[− d dθJ(θ) ≈ 1/V ar(θ̂) 2 ] Since d2 J(θ) dθ2 = n σ02 is constant V ar(θ̂) ≈ σ02 /n ˆ This asymptotic variance is in fact the actual variance of θ. 2 2. The Pareto distribution is used in economics to model values exceed­ ing a threshhold (e.g., liability losses greater than $100 million for a consumer products company). For a fixed, known threshhold value of x0 > 0, the density function is f (x | x0 , θ) = θxθ0 x−θ−1 , x ≥ x0 , and θ > 1. Note that the cumulative distribution function of X is ( o−θ x P (X ≤ x) = FX (x) = 1 − . x0 (a). Find the method-of-moments estimate of θ. (b). Find the mle of θ. (c). Find the asymptotic variance of the mle. (d). What is the large-sample asymptotic distribution of the mle? Solution: (a) Compute the first moment of a Pareto random variable X : J∞ µ1 = x0 xf (x | x0 , θ)dx J∞ = x0 x × θxθ0 x−θ−1 dx J∞ = θxθ0 x0 x−θ dx −(θ−1) 1 = θxθ0(( θ−1 )xo 0 θ = x0 θ−1 Solving µ1 = µ̂1 = x for θ gives: θˆ = x x−x0 (b). For a single observation X = x, we can write log[f (x | θ)] = ln(θ) + θ ln(x0 ) − (θ − 1) ln(x) ∂ log[f (x|θ) ] = 1θ + ln(x0 ) − ln(x) ∂θ 2 ∂ log[f (x|θ) ] = − θ12 ∂θ2 The mle for θ solves ∂ Pn 0 = ∂J(θ) = P ∂θ ∂θ ( i=1 ln[f (xi | θ)]) n 1 = − ln(xi )] 1 [ θ + ln(x0 ) P n = θ + n ln(x0 ) − n1 ln(xi ) P n =⇒ θ̂ = = [ n1 n1 ln(xi /x0 )]−1 n ln(xi )−n ln(x0 ) 1 (c). The asymptotic variance of θ̂ is V ar(θ̂) ≈ 1 nI(θ) = θ2 n 3 Because I(θ) = E[− ∂ 2 ln[f (x|θ)] ] ∂θ2 = 1 θ2 (d) The asymptotic distribution of θ̂ is √ D 1 n(θ̂ − θ) −−→ N (0, I(θ) ) = N (0, θ2 ) or D 2 θ̂ −−→ N (θ, θn ) 4 3. Distributions derived from Normal random variables. Consider two independent random samples from two normal distributions: • X1 , . . . , Xn are n i.i.d. N ormal(µ1 , σ12 ) random variables. • Y1 , . . . , Ym are m i.i.d. N ormal(µ2 , σ22 ) random variables. (a). If µ1 = µ2 = 0, find two statistics T1 (X1 , . . . , Xn , Y1 , . . . , Ym ) T2 (X1 , . . . , Xn , Y1 , . . . , Ym ) each of which is a t random variable and which are statistically inde­ pendent. Explain in detail why your answers have a t distribution and why they are independent. (b). If σ12 = σ22 > 0, define a statistic T3 (X1 , . . . , Xn , Y1 , . . . , Ym ) which has an F distribution. An F distribution is determined by the numerator and denominator degrees of freedom. State the degrees of freedom for your statistic T3 . (c). For your answer in (b), define the statistic T4 (X1 , . . . , Xn , Y1 , . . . , Ym ) = 1 T3 (X1 , . . . , Xn , Y1 , . . . , Ym ) What is the distribution of T4 under the conditions of (b)? Pn 2 = 1 2 2 (d). Suppose that σ12 = σ22 . If SX i=1 (Xi − X) , and SY = n−1 P m 1 2 i=1 (Yi − Y ) , are the sample variances of the two samples, show m−1 how to use the F distribution to find P (SX 2 /SY2 > c). (e). Repeat question (d) if it is known that σ12 = 2σ22 . Solution: (a). Consider √ T1 = √nX2 T2 = SX √ mY √ 2 SY where 5 X = 2 SX Y = = 2 SX = 1 n Pn 1 Xi 1 Pn 2 1 (Xi − X) n−1 1 Pm 1 Yi m 1 Pm 2 1 (Yi − Y ) m−1 ∼ N (µ1 , σ12 /n) σ2 1 ∼ ( n−1 ) × χ2n−1 ∼ N (µ2 , σ22 /n) σ2 2 ∼ ( m−1 ) × χ2m−1 2 are independent, and Y and We know from theory that X and SX SY2 are independent, and all 4 are mutually independent because they depend on independent samples. For µ1 = 0, we can write √ 1 T1 = √nX/σ ∼ tn−1 2 2 SX /σ1 a t distribution with (m−1) degrees of freedom, because the numerator is i N (0, 1) random variable independent of the denominator which is χ2m−1 /(m − 1). And for µ2 = 0, we can write √ T2 = √mY2 /σ22 ∼ tm−1 SY /σ2 a t distribution with (n−1) degrees of freedom, because the numerator is i N (0, 1) random variable independent of the denominator which is χ2n−1 /(n − 1). (b). For σ12 = σ22 consider the statistic: T3 = = 2 SX SY2 2 /σ 2 SX 1 SY2 /σ22 The numerator is a χ2n−1 random variable divided by its degrees of freedom (n − 1) and the denominator is an independent χ2m−1 random variable divided by its degrees of freedom (m − 1). By definition the distribution of such a ratio is an F distribution with (n−1) and (m−1) degrees of freedom in the numerator/denominator. (c). The inverse of an F random variable is also an F random variable – the degrees of freedom for numerator and denominator reverse. (d). In general we know: 2 (n−1)SX 2 σ1 (m−1)SY2 σ22 ∼ χ2n−1 ∼ χ2m−1 which are independent. 6 So, we can develop the expression: S2 2 /σ 2 (n − 1)SX (n − 1)/σ12 1 > × c) (m − 1)SY2 /σ22 (m − 1)σ22 2 σ (n−1) = P (F(n−1),(m−1) > (m−1) × ( σ22 ) × c) P ( SX2 > c) = P ( Y 1 The answer is the upper-tail probability of an F distribution with (n−1), (m−1) degrees of freedom, equal to the probability of exceeding σ2 (n−1) ( (m−1) × ( σ22 ) × c) 1 For (d), use σ22 σ12 = 1 and for (e) use 7 σ22 σ12 = 1/2 4. Hardy-Weinberg (Multinomial) Model of Gene Frequencies For a certain population, gene frequencies are in equilibrium: the geno­ types AA, Aa, and aa occur with probabilities (1 − θ)2 , 2θ(1 − θ), and θ2 . A random sample of 50 people from the population yielded the following data: Genotype Type AA Aa aa 35 10 5 The table counts can be modeled as the multinomial distribution: (X1 , X2 , X3 ) ∼ M ultinomial(n = 50, p = ((1 − θ)2 , 2θ(1 − θ), θ2 ). (a). Find the mle of θ (b). Find the asymptotic variance of the mle. (c). What is the large sample asymptotic distribution of the mle? (d). Find an approximate 90% confidence interval for θ. To construct the interval you may use the follow table of cumulative probabilities for a standard normal N (0, 1) random variable Z P (Z < z) 0.99 0.975 0.950 0.90 z 2.326 1.960 1.645 1.182 (e). Using the mle θ̂ in (a), 1000 samples from the M ultinomial(n = 50, p = ((1 − θ̂)2 , 2θ̂(1 − θ̂), θ̂2 )) distribution were randomly generated, and mle estimates were com­ puted for each sample: θ̂j∗ , j = 1, . . . , 1000. For the true parameter θ0 , the sampling distribution of Δ = θ̂ − θ0 is ˆ The 50-th largest value of Δ̃ was ˜ = θ̂∗ − θ. approximated by that of Δ +0.065 and the 50-th smallest value was −0.067. Use this information and the estimate in (a) to construct a (para­ metric) bootstrap confidence interval for the true θ0 . What is the confidence level of the interval? (If you do not have an answer to part (a), assume the mle θ̂ = 0.25). Solution: (a). Find the mle of θ 8 • (X1 , X2 , X3 ) ∼ M ultinomial(n, p = ((1 − θ)2 , 2θ(1 − θ), θ2 )) • Log Likelihood for θ £(θ) = log(f (x1 , x2 , x3 | p1 (θ), p2 (θ), p3 (θ))) = log( x1 !xn!2 !x3 ! p1 (θ)x1 p2 (θ)x2 p3 (θ)x3 ) = x1 log((1 − θ)2 ) + x2 log(2θ(1 − θ)) +x3 log(θ2 ) + (non-θ terms) = (2x1 + x2 )log(1 − θ) + (2x3 + x2 )log(θ) + (non-θ terms) • First Differential of log likelihood: (2x1 + x2 ) (2x3 + x2 ) £" (θ) = − + 1−θ θ =⇒ θˆ = 2x3 + x2 2x3 + x2 2(5) + 10 = = = 0.2 2n 2(50) 2x1 + 2x2 + 2x3 (b). Find the asymptotic variance of the mle. 1 E[−£"" (θ)] • Second Differential of log likelihood: d (2x1 + x2 ) (2x3 + x2 ) £"" (θ) = [− + ] dθ 1−θ θ • V ar(θ̂) −→ (2x1 + x2 ) (2x3 + x2 ) − (1 − θ)2 θ2 • Each of the Xi are Binomial(n, pi (θ)) so E[X1 ] = np1 (θ) = n(1 − θ)2 E[X2 ] = np2 (θ) = n2θ(1 − θ) E[X3 ] = np3 (θ) = nθ2 2n • E[−£"" (θ)] = θ(1 − θ) = − • σ̂θ̂2 = θ̂(1 − θ̂) 0.8(1 − 0.8) = = 0.16/100 = (.4/10)2 = (.04)2 2n 2 × 50 θ) (c) The asymptotic distribution of θ̂ is N (θ, θ(1− 2n ) (d) An approximate 90% confidence interval for θ is given by i i {θ : θ̂ − z(α/2) V ar(θ̂) < θ < θ̂ + z(α/2) V ar(θ̂)} i where α = 1 − 0.90 and z(.05) = 1.645, and V ar(θ̂) ≈ (.04). 9 So the approximate 90% confidence interval is: {θ : 0.20 − .06580 < θ < 0.20 + .06580} (e). For the bootstrap distribution of the errors Δ = θ̂ − θ0 , (where θ0 is the true value), the approximate 5% and 95% quantiles are δ = −0.067 and δ = 0.065. The approximate 90% confidence interval is {θ : θ̂ − δ < θ < θ̂ − δ} = [0.2 − 0.065, 0.2 + 0.067] 10