Questions and Answers #7 Confidence Intervals

Questions and Answers Sheet 7 Confidence Intervals Question #1 Confidence Interval for the Difference between Two Binomial Proportions I have this stats problem where I am to create a 98% confidence interval for the difference between proportions. 𝑋1 = 21, 𝑛1 = 90 and 𝑋2 = 39, 𝑛2 = 70 . I did a 2propzinterval test and my confidence interval came out as (−0.497, −0.151) . The question is: A marketing manager claims that the proportion of customers who purchased pretzels did not change after the app campaign. Does the confidence interval contradict this claim? How do I know if this confidence interval contradicts the claim? Thank you for your help! Answer: Not familiar with the intended technique. But I think the following method is good and it works without the Gaussian approximation. Assume 𝑥1 ∼ 𝐵(𝑛1 , 𝑝1 ) and 𝑥2 ∼ 𝐵(𝑛2 , 𝑝2 ) have the same probability 𝑝1 = 𝑝2 = 𝑝 and are independent. Then we have 𝑥1 + 𝑥2 ∼ 𝐵(𝑛1 + 𝑛2 , 𝑝) , and therefore the most likely value for p is, 𝑝 = 𝑥1 +𝑥2 𝑛1 +𝑛2 3 = . Knowing p makes the hypothesis test much 8 easier. If the 𝑛1 = 90 cases are randomly selected from 𝑛1 + 𝑛2 = 160 , the probability to have 𝑥1 successful cases is given by ( 𝑝(𝑥1 |𝑥1 + 𝑥2 = 60) = 60 ) 𝑥1 (100 9 )0−𝑥1 160 ( 90 ) . Numerics shows that the 0.9864 confidence range is 𝑥1 ∈ [27,41] . The biggest confidence range that does not includes, 𝑥1 = 21, is 𝑥1 ∈ [22,46] , which has a confidence level of 0.999965. Thus the chance for x1 (= 21) [22, 46] is only 3.5 × 10−5 if 𝑝1 = 𝑝2 . This gives a statistically strong evidence for 𝑝1 ≠ 𝑝2 . Question #2 Confidence interval of the parameter of exp and normal distribution from MLE? I have a sample 𝑋1 , 𝑋2 , … , 𝑋𝑛 1. If the sample is from exponential distribution, I want to use MLE to estimate the parameter 𝛽 . I know the result that 𝛽̂ = 𝑋1 + 𝑋2 + ⋯ + 𝑋𝑛 𝑛 But how can I calculate the 95% confidence interval of 𝛽 ? 2. If the sample is from normal distribution, I want to use MLE to estimate the parameter 𝜇 and 𝜎. I also know the result that 𝜇̂ = 𝑋1 +𝑋2 +⋯+𝑋𝑛 𝑛 , 𝜎̂ = [ 𝑛−1 𝑛 1/2 ⋅ 𝑆 2 (𝑛)] But how can I calculate the 95% confidence interval of 𝜇 and 𝜎? Step1 to calculate confidence interval, ˆ  z 1−  (ˆ)  n 2 Step 2 to calculate confidence interval 2 𝑑 𝛿(𝜃̂ ) = −𝑛 (𝐸 [ 2 ln ℒ(𝜃)]) −1 𝑑𝜃 I know the equation, but I am still confuzed how to calculate (𝐸 [ 𝑑2 𝑑𝜃 2 ln ℒ(𝜃)]) Answer: The term you cannot calculate is (essentially) the Fischer information: 𝐼(𝜃) = −𝐸 [ 𝑑2 𝑑𝜃 2 ln ℒ(𝜃)] −1 Then 𝛿(𝜃) = 𝑛(𝐼(𝜃)) n ( ) =   e − x j . So, to calculate 𝐼(𝜃) for the exponential: =  ne − n xj j =1 j =1 and therefore ln ( ) = ln( n e − n xj j =1 n ) = n ln( ) −   x j j =1 which you can differentiate twice with respect to 𝜃 : d ln( ( )) d = n  n −  xj j =1 2 d n ln( ( )) = − 2 2 d  So,  d2  n I ( ) = − E  2 ln ( )  = 2  d   and therefore 𝛿(𝜃) = 𝑛(𝑛/𝜃 2 )−1 = 𝜃 2 . To make calculations simpler use that If 𝑋𝑗 are i.i.d. for 𝑗 = 1,2, … , 𝑛 you can take the 𝐼(𝜃) for a single observation 𝑋𝑗 and obtain the Fisher information for X with 𝑛𝐼(𝜃) . For the normal distribution 𝑁(𝜇, 𝜃 2 ) you should find: 𝐼(𝜃) = 𝑛/(2𝜃 2 ) (single parameter) and if you estimate both parameters, i.e. 𝛮(𝜇, 𝜎 2 ) with 𝜃 = [𝜇, 𝜎 2 ]𝛵 then 𝐼(𝜃) is a matrix:  1  2 2 I (?) =   0   0   1  2 4  Question #3 Confidence interval when both 𝜇 and 𝜎 2 are unknown I have the following problem in my problem book: Let 𝑋 ∼ 𝑁(𝜇, 𝜎 2 ) where both 𝜇 and 𝜎 2 are unknown. I have to find a confidence interval for the mean. What I have so far: I know that when 𝜎 2 is unknown I can use t-distribution for finding a confidence interval, i.e.: ∗ 𝑥̅ ± 𝑡𝑛−1 𝑠 √𝑛 , ∗ where 𝑡𝑛−1 is a t-distribution with 𝑛 − 1 degrees of freedom and 𝑥̅ is the sample mean from a sample of size n. The whole problem seems kind of vague to me. Is this enough for describing the confidence interval? I don't know the sample mean 𝑥̅ so is it okay to describe the solution in such a way? Answer: When the standard deviation is unknown but the distribution is Gaussian, the trick is to use the fact that the reduced variable t := x − s2 n where 𝑠 2 is the corrected variance follows a Student's distribution (of 𝑛 − 1 dof). Knowing the probability of this variable to be in a certain range, you have your confidence interval for 𝜇 . Question #4 Confidence interval; exponential distribution (normal or student approximation?) Let's say we have got a sample of size n from an exponential distribution with an unknown mean 𝜆 . We want to construct a confidence interval and so we can compare this: 𝑋̅ −𝜆 2 √𝑆 𝑛 to a student t-distribution with 𝑛 − 1 degrees of freedom. However, as in the case of the exponential distribution, we know that 𝑉𝑎𝑟[𝑋] = (𝐸[𝑋])2 , so rather than introducing an estimator for variance, we can simply use one estimator, i.e: 𝑋̅−𝜆 ̅̅̅̅ 2 √𝑋 . 𝑛 And now the question is: Do we compare this statistic with normal distribution or again with student t-distribution? Answer: I do not recommend to use the Student approximation. Instead, it is better to observe that X1 + (n, ) with 𝜃 = 1/𝜆. + Xn Therefore, 2  ( X1 + + Xn) 1 (n, ) =  22n . Hence you can construct the required interval through the 2 2 quantiles of 𝜒2𝑛 . Question #5 Confidence Intervals for 𝛽1 and 𝛽2 'm having a bit of trouble with the question below. Using this Data -> Data Question: Establish 95% confidence intervals for 𝛽2 and 𝛽3 . I know I use the formula: 𝛽2 ± (t-value)(standard error) How do I find the standard error using the data given in the pic? Answer: I assume that you want to construct two different CIs (for each parameter) and not simultaneously for the vector (𝛽2, 𝛽3). As such, you can use the result var(𝛽̂ ) = 𝜎 2 (𝑋 ′ 𝑋)−1 . When instead of 𝜎 2 you can plug-in its estimator, namely ̂2 = 𝜎 1 10−3 2 ∑10 𝑖=1 𝑒𝑖 . And you should invert the following matrix  N  X X =   X 2t   X 3t  X X X X 2t 2 2t 3t X X X X   2 t 3t  , 2  3t  3t 3t ̂2 (𝑋 ′ 𝑋)−1 then, var(𝛽̂𝑖 ) = 𝜎 𝑖𝑖 Question #6 Confidence Intervals using Normal Distribution with variance known From a sample of 32 observations from a population with 𝜎 2 = 24.602 , we obtained 𝑥 = 79.47 . Construct a 99% confidence interval for 𝜇 . (a) 79.47 ± 1.4424 (b) 79.47 ± 1.7186 (c) 79.47 ± 2.0430 (d) 79.47 ± 2.2622 Answer: Respectively, answers (A) through (D) correspond to symmetric two-sided 90%, 95%, 98%, and 99% confidence intervals. The critical values for each are: a) 1.64485 b) 1.95996 c) 2.32635 d) 2.57583 with standard errors being √24.602/32 ≈ 0.87682 . Thus the respective margins of error are (±) . (A) 1.44224, (B) 1.71853, (C) 2.03979, (D) 2.25854. Question #7 Confidence Itervals; 𝑍𝛼 and 𝑍𝛼/2 I'm confused about what exactly 𝑍𝛼 is, does there exist a formula for it in terms of 𝛼? IF so, is there also one for 𝑍𝛼 ? 2 Answer: Commonly, z  refers to the point (number) such that Pr(𝑍 ≥ 𝑧𝛽 ) = . Here Z is the standard normal distribution. So for instance 𝑧𝛽 , where 𝛽 = 0.005 , refers to the point z such that Pr(𝑍 ≥ 𝑧) = 0.005 . It turns out that in this case 𝑧𝛽 ≈ 2.57 . So the probability that a random variable with standard normal distribution is bigger than 2.57 is 0.005. Thus the area under the standard normal curve in the right tail past 2.57 is 0.005. This kind of information is needed when we calculate confidence intervals when the normal distibution provides a reasonable fit, and also in hypothesis testing in similar situations. There is no nice formula for 𝑧𝛽 in terms of 𝛽. But many pieces of software, including standard spreadsheets, will compute it for you. There are also online calculators that will do it. Before the computers everywhere era, people used tables of the standard normal to locate 𝑧𝛽 for the \beta they were interested in, typically numbers like 𝛽 = 0.05, 0.025 , or 0.01, and so on. Anyone who has done statistics probably remembers forever that 𝑧0.025 ≈ 1.96. Question #8 Constructing a confidence interval for population variance Given the following data set, how does one construct a 99% confidence interval for the population variance? I should mention now I don't have a very good understanding of when to divide by n and when to divide by (𝑛 − 1) . 22.2 24.7 20.9 26.0 27.0 24.8 26.5 23.8 25.6 23.9 I've already tried a couple approaches, none seem to give the correct answers. The first was to find the mean of the samples(sum divided by number of samples), giving ∑𝑋 𝑋̅ = 𝑛 = 24.54 𝑛 Then obtaining the variance of the samples as 𝑠2 = ∑(𝑋𝑛−𝑋̅)2 𝑛 = 3.2924 Given that there are 9 degrees of freedom, and I'm looking for 99% confidence for a two-sided interval, I used a 𝜒 2 table to find values of 𝜒 2 of 1.73 and 23.59. Then the following values using the following formula. (𝑛−1)𝑠 2 𝜒12 ≤ 𝜎2 ≤ (𝑛−1)𝑠 2 𝜒22 1.256 … ≤ 𝜎 2 ≤ 17.128 … Upon seeing that this was wrong, I tried to obtain mean and variance by dividing by (𝑛 − 1) instead of n. This gave. 𝑋̅ = 11.919 … 𝑠 2 = 3.292 … Then, using the same 𝜒 2 values as above, I got the following for a confidence interval 4.547 … ≤ 𝜎 2 ≤ 62.002 … This was also wrong. I cannot tell why these would both be wrong, whether I should have divided by n or (𝑛 − 1) , and how I would obtain the correct answer. Does anyone know where I went wrong here? Answer: 1 First, let's get the notation and definitions right; The sample mean 𝑋̅ = ∑𝑛𝑖=1 𝑋𝑖 . If the population mean 𝜇 𝑛 is unknown and estimated by 𝑋̅ , then the population variance 𝜎 2 is estimated by the sample variance 1 ∑𝑛𝑖=1(𝑋𝑖 − 𝑋̅)2 . . Then 𝑆2 = 𝑛−1 (𝑛−1)𝑆 2 𝜎2 = ̅ 2 ∑𝑛 𝑖−1(𝑋𝑖 −𝑋) 𝜎2 ∼ 𝘊𝘩𝘪𝘴𝘲(𝑑𝑓 = 𝑛 − 1). For your dataset the statistics are: 𝑥 = 𝑐(22.2,24.7,20.9,26.0,27.0,24.8,26.5,23.8,25.6,23.9) 𝑛 = 𝑙𝑒𝑛𝑔𝑡ℎ(𝑥); 𝑎 = 𝑚𝑒𝑎𝑛(𝑥); 𝑠 = 𝑠𝑑(𝑥) 𝑛; 𝑎; 𝑠 10 sample size 24.54 sample mean 1.912648 sample SD Then 95% confidence interval for the population variance \sigma^{2} is obtained as ((𝑛 − 1)𝑆 2 /𝑈, (𝑛 − 1)𝑆 2 /𝐿), where L and U cut 2.5% of the probability from the lower and upper tails, respectively, of Chisq(n-1) . Computations of CIs for 𝜎 2 and 𝜎 in R statistical software follow: 𝑈𝐿 = 𝑞𝑐ℎ𝑖𝑠𝑞(𝑐(. 975, .025), 𝑛 − 1); 𝑈𝐿 ## 19.022768 2.700389 𝐶𝐼 = (𝑛 − 1) ⋅ 𝑠 2 /𝑈𝐿; 𝐶𝐼 ## 1.730768 12.192315 95% CI for pop var √𝐶𝐼 ## 1.315587 3.491750 95% CI for pop SD Notice that 𝑆 = 1.913 is contained in the CI for 𝜎 as it must be, but that S is not at the center of the CI, because the chi-squared distribution is skewed. I assume you can use the appropriate quantiles of 𝐶ℎ𝑖𝑠𝑞(9) to get 99% confidence intervals. Addendum per Comments for 99% CIs: Of course, 99% confidence intervals have to be longer than 95% CIs. 𝑈𝐿 = 𝑞𝑐ℎ𝑖𝑠𝑞(𝑐(. 995, .005), 𝑛 − 1); 𝑈𝐿 ## 23.589351 1.734933 # same as you showed in your question 𝐶𝐼 = (𝑛 − 1) ⋅ 𝑠 2 /𝑈𝐿; 𝐶𝐼 ## 1.395715 18.977103 # using correct numerator, this is different √𝐶𝐼 ## 1.181404 4.356272 Question #9 Constructing confidence Interval (standard normal) I am asked to construct 95 percent confidence interval of the variance, sigma squared using this: ∑𝑛 𝑗=1(|𝑌𝑗 |/𝜎|−𝐸|𝑍 | ) √𝑛𝑉𝑎𝑟(|𝑍|) ? I know the expression given above is standard normal with mean, 0 and variance 1. Note I have I have calculated 𝐸|𝑧| (the expectation of a standard normal) to be √2/𝑝𝑖 and 𝑉𝑎𝑟|𝑧| to be 1 − 2/𝑝𝑖 . Note 𝑌1 … 𝑌𝑛 is a sample from N(0,sigma squared). 𝑇 𝑗 Note: ∑𝑛𝑗=1 | | are pivots, meaning is a pivot for variance, distribution does not depend on variance. 𝜎 PROBLEM: The problem I'm having in constructing the confidence interval is isolating sigma squared (variance) from the expression given at the top to get an interval for variance. Please provide help, suggestions. Thank you Answer: Explanation: You have ∑𝑛 2 𝑗=1|𝑌𝑗 | −𝑛√ 𝜎 𝜋 2 𝜋 ∼ 𝑁(0,1). √𝑛(1− ) Put this fraction between critical values ±1.96 for standard normal distribution and solve with respect to 𝜎 the inequality: −1.96 ≤ ∑𝑛𝑗=1|𝑌𝑗 | 2 − 𝑛√ 𝜎 𝜋 √𝑛 (1 − 2 ) 𝜋 ≤ 1.96.. For n sufficently large this inequality leads to two-sided inequality for 𝜎 . Then square all parts. Question #10 Deriving the confidence interval 𝑃(−Φ𝛼−1 < 𝑋 < Φ𝛼−1 ) = 1 − 2 Let P ~ (0,1) P(−−1  X  −1 ) P( X  −1 ) − P( X  −−1 ) (−1 )) − (−−1 )) (−1 )) − (1 − (−1 )))  −1 +  −1 (1 − 2 ) (1) P( A  X  B) = P( X  B ) − P ( X  A) (2) P( X  A) = ( A) (3)  ( − x ) = 1 −  ( x) (4) −1 f ( f ) = id (5) (6) Answer: Note that for 𝛼 < 0.5 , you get 𝜙 −1 (𝑎) < 0. Hence, 1 − 2𝛼 ≥ 0 so for sure 𝛼 ≤ 0.5 . Then, 𝑃(𝜙 −1 (𝛼) < 𝑋 < −𝜙 −1 (𝛼)) = 1 − 𝜙(𝜙 −1 (𝛼)) − 𝜙(𝜙 −1 (𝛼)) = 1 − 2 Question #11 Estimate of Proportion An airline is interested in determining the proportion of its customers who are flying for reasons of business. If they want to be 90 percent certain that their estimate will be correct to within two percent, how large a random sample should they select? This involves using the Z-distribution to find the 90% confidence interval of the actual value of the proportion. But for this to be done, there needs to be an estimate of the proportion, which is not given. How do we find this out?ie (p) ⋅ (1 − 𝑝) ⋅ 𝑧𝑎/2𝑛 = 0.0004 . How do we find the value of p? Answer: Take the most "pessimistic" estimate of 𝑝(1 − 𝑝) , that is, the largest possible variance. Note that the 1 1 maximum value of 𝑝(1 − 𝑝) is , attained at 𝑝 = . 4 2 The standard deviation of the sample mean is therefore ≤ 1/2 √𝑛 . So we use 1/2 √𝑛 . 2 Remarks: 1. There may be a mistake, or at least a typo, in your equation. Probably 𝑧𝛼/2 is intended. 2. After the experiment has been performed, if 𝑝̂ turns out to be not far from 0 or not far from 1, one may conclude that one has obtained a better estimate than the one planned for. However, 𝑝(1 − 𝑝) stays 1 1 remarkably close to for p not too far from . 4 2 Question #12 Find a 95 confidence interval for 𝜃 based on inverting the test statistic statistic 𝜃̂ . For our data we have 𝑌𝑖 ∼ 𝑁(𝜃𝑥𝑖 , 1) for 𝑖 = 1, … , 𝑛. Therefore it can be proven that the MLE for \theta is given by ∑𝑥 𝑌 𝜃̂ = ∑ 𝑖 2 𝑖 𝑥𝑖 To find the confidence interval I should invert the test statistic 𝜃̂ . The most powerful unbiased size 𝛼 = 0,05 test for testing 𝐻0 : 𝜇 = 𝜇0 vs. 𝐻1 : 𝜇 ≠ 𝜇0 where 𝑋1 , … , 𝑋𝑛 ~ iid 𝑛(𝜇, 𝜎 2 ) has acceptance region 𝐴(𝜇0 ) = {𝐱: |𝑥̅ − 𝜇0 | ≤ 1,96𝜎/√𝑛}. Substituting my problem (I think) we get that the most powerful unbiased size 𝛼 = 0,05 test for testing 𝐻0 : 𝜃 = 𝜃̂ vs. 𝐻1 : 𝜃 ≠ 𝜃̂ has acceptance region 𝐴(𝜃̂) = {𝐲: |𝑦̅ − 𝜃̂| ≤ 1,96/√𝑛} √𝑛𝑦̅−1,96 √𝑛𝑦̅+1,96 or equivalently, 𝐴(𝜃̂) = {𝐲: ≤ 𝜃̂ ≤ } √𝑛𝑥𝑖 √𝑛𝑥𝑖 Substituting 𝜃̂ = ∑ 𝑥𝑖 𝑌𝑖/ ∑ 𝑥𝑖2 we obtain 𝛴𝑥 𝑌𝑖 √𝑛𝑦̅−1,96 √𝑛𝑦̅+1,96 𝐴(𝜃̂) = {𝐲: ≤ 𝑖2 ≤ } √𝑛𝑥𝑖 𝛴𝑥𝑖 √𝑛𝑥𝑖 This means that my 1 − 0,05 = 0,95(95%) confindence interval is defined to be 𝐶(𝑦) = {𝜃̂ : 𝑦 ∈ 𝐴(𝜃̂)} But I can't seem to find anything concrete and I feel that I've made mistakes somewhere. What to do? Answer: ∑ 𝑥𝑌 You have 𝜃̂ = ∑𝑖 𝑖 2 𝑖. 𝑖 𝑥𝑖 Although non-linear in (𝑥1 , … , 𝑥𝑛 ), this is linear in (𝑌1 , … , 𝑌𝑛 ) , and that's why it's a "linear" model (it's not because one is fitting a straight line; if that were true then least-squares fitting of polynomials would be considered non-linear regression, but it's linear regression). So you have a linear combination of independent normally distributed random variables, where the coefficients are constant (i.e. not random). Being a linear combination of independent normally distributed random variables with constant coefficients makes its distribution easy to find:  ( x E(Y ))  (x var(Y )) ˆ  ~ N( , ) ( x ) x i 2 i i i i i 2 i 2 2 i i = N ( , i 1 )  xi2 i since E(Yi ) = xi and var(Yi ) = 1. So Pr( − A 1  xi2  ˆ   + A i 1  xi2 ) = 0.95 i where A is such that Pr(−𝐴 < 𝑍 < 𝐴) = 0.95 . Consequently Pr(ˆ − A 1 x 2 i    ˆ + A i 1 x 2 i ) = 0.95 i and that is your confidence interval. Question #13 Find CI for mean of linear regression with variance unknown For the simple linear regression model: 𝑌𝑖 = 𝛽 ∗ 𝑋𝑖 + 𝜖𝑖 I want to find CI for 𝛽𝑥 which is 𝐸(𝑌𝑖 ) when 𝑥𝑖 = 𝑥 . 2 2 2 𝜎 𝑋 𝜎 I find that 𝛽̂ ∼ 𝑁 (𝛽, ∑(𝑋 2 ) , so the distribution of 𝛽 x is 𝑁 (𝛽𝑥, ∑(𝑋 2 ) . If the variance is known I can use 𝑖 ) 𝑃 (−𝑧𝛼/2 ≤ ̂ 𝑥−𝛽𝑥 𝛽 𝑥𝜎/ ∑(𝑥𝑖2 ) 𝑖) ≤ 𝑧𝛼/2 ) . If the variance is unknown, what unbiased estimator should I use for 𝜎 2 ? Is it the sample variance? What is it in this case? Answer: Suppose number of predictors is p and number of observations is N. Let X be 𝑁 × 𝑝 matrix, and the specific x be 1 × 𝑝 . Then 𝑥𝛽 − 𝑥𝛽̂ ∼ 𝑡𝑁−𝑝 ̂ 𝜎√𝑥(𝑋 𝑇 𝑋)−1 𝑥 ̂2 = where 𝜎 𝑅𝑆𝑆 𝑁−𝑝 = ̂ )𝑇 (𝑌−𝑋𝛽 ̂) (𝑌−𝑋𝛽 𝑁−𝑝 . In the above t stands for t distribution. For simple regression 𝑝 = 1 , so 𝜎 2 turns out to be sample variance. However, you need to use t distribution, not Gaussian. Question #14 Finding upper confidence limit without mean and sd So the question I'm trying to answer looks like this: We are interested in p, the population proportion of all people who are currently happy with their cell phone plans. In a small study done in 2012, it was found that in a sample of 150 people, there were 90 who were happy with their cell phone plans. Find the upper confidence limit of an 82% confidence interval for p. I know the formula is estimate ± (critical value)(standard error) Answer: The mean of a distribution of sample proportions is equal to the population proportion (p). Thus, the standard deviation of this sample proportion can be found by using the standard error. This formula is √(𝑝 ∗ (1 − 𝑝)/𝑛).. You can solve for the standard error now. You have the population proportion estimate which is just 90/150 or .6, can you handle it from there?? The critical value can be found by using a table. In addition, you might need to use a "continuity correction" because we need to correct for the fact that we are approximating a discrete distribution (the sampling distribution of p) with a continuous distribution (the normal distribution)... This continuity correction is given by 0.5/𝑁, thus your final equation for the upper limit will look something like: 0.60 + (critical value)(. 04) + 0.5/150 Alternatively, The lower limit would simply be: 0.60 − (critical value)(0.04) − 0.5/150 Question #15 For fixed k, the variables (2𝑘 + 1)𝑓̂𝑘 (𝜆) are asymptotically distributed according the Gamma distribution with shape parameter 2𝑘 + 1 and mean (2𝑘 + 1)𝑓𝑋 (𝜆) . It turns out that this suggests a confidence interval of the form (4𝑘+2)𝑓̂𝑘 (𝜆) (4𝑘+2)𝑓̂𝑘 (𝜆) , 2 ) 2 𝜒4𝑘+2,1−𝛼 𝜒4𝑘+2,𝛼 ( 2 where 𝜒𝑘,𝛼 the 𝛼 -quantile of the chi-square distribution with k degrees of freedom. To show that this interval is asymptotically of level 1 − 2𝛼 Answer: Step 1 The mean of Γ(𝛼, 𝛽) equals to 𝛼 𝛽 = 2𝑘+1 𝛽 = (2𝑘 + 1)𝑓𝑋 (𝜆) ⇒ 𝛽= 1 𝑓𝑋 (𝜆) . Rewrite your statement as d (2k + 1) fˆk ( ) ⎯⎯ →(2k + 1, 1 ). f X ( ) Use the PDF of Γ(𝛼, 𝛽) 𝑓𝑌 (𝑥) = 𝛽𝛼 Γ(𝛼) 𝑥 𝛼−1 𝑒 −𝛽𝑥 , 𝑥 > 0 and check that the multiplying 𝑌 ∼ Γ(𝛼, 𝛽) by 𝑐 > 0 results in changing 𝛽 to 𝛽 𝑐 𝛽 only: 𝑐𝑌 ∼ Γ (𝛼, ) 𝑐 So, in order to get asymptotically chi-squared distribution 1 2 𝜒4𝑘+2 = Γ (2𝑘 + 1, ), 2 divide original estimator (2𝑘 + 1)𝑓̂𝑘 (𝜆)𝑏𝑦𝑓𝑋 (𝜆) and multiply by 2: 2(2k + 1) fˆk ( ) d 1 ⎯⎯ →(2k + 1, ) =  42k + 2 . f X ( ) 2 ( (4k + 2) fˆk ( )  2 4 k + 2,1− = (  42k + 2,   f X ( )  (4k + 2) fˆk ( )  42k + 2, )= (4k + 2) fˆk ( )   42k + 2,1− ) → 1 − 2 f X ( )  d  42k +2 Question #16 Given an sample of data set X which is normal-distributed to (  = 240,  ) , I want to find the 95% confidence interval of max(𝜇 − 200,0) . As the max(𝜇 − 200,0) cut the normal-distribution curve at the point of 200, the sample of max(𝜇 − 200,0) is not more normal distributed. Therefore what is a reasonable 95% confidence interval? Answer: Step 1 Your distribution contains a 𝛿 -function at 𝑥 = 0 of probability mass equal to the probability mass of your normal distribution less than the value x=200 . (This will involve an error function.) The remaining probability mass will be due to the portion of your Gaussian to the right of 𝑥 = 200 . As such, your 95% confidence interval will extend from 𝑥 = 0 up to the value x for which 5\% of the total probability mass (or 10\% of the truncated Gaussian) is above x. Clearly, this is expressed as an error function. Question #17 Given 𝑋1 , … , 𝑋100 and 𝑌1 , … , 𝑌50 which are independent random samples from identical distribution 𝑁(𝜇, 1) . Each of two statisticians is trying to build confidence interval for 𝜇 on a confidence level of 0.8 1 − 𝛼 = 0.8 . You have to find probability that those intervals are disjoint. I know i have to find 𝑃 (|𝑋 − 𝑌| > 1⋅1.28 √50 + 1⋅1.28 √100 ) which is 1 − 𝑃(−0.309 ≤ 𝑋 − 𝑌 ≤ 0.309) Question is what should i do next how to calculate this probability? Answer: Step 1 One interval has endpoints 𝑋̅ ± 1.28 𝑌̅ ± 1.28 1 √50 1 √100 and the other . The intervals are disjoint if X + 1.28 That happens if Y − X  1.28( | X − Y | 1.28( 1 1 1 1  Y − 1.28 orY + 1.28  X − 1.28 . 100 50 50 100 1 1 1 1 + ) or X − Y  1.28( + ) . In other words 100 50 100 50 1 1 + ). 100 50 This is equivalent to | X −Y |  1.28. 1 1 + 100 50 You have X − Y ~ N (0, 1 1 3 + ) = N (0, ) , so that 100 50 100 X −Y ~ N (0,1). 3 /100 standard deviation( X −Y 1 1 + 100 50 standard deviation( X − Y ) 1 1 + 100 50 )= 3 100 [10 pt ] = = √3 1+√2 1 1 + 100 50 = √6 − √3 So, Pr( | X −Y |  1.28) 1 1 + 100 50 = Pr(. | X −Y | 1.28 / ( 6 − 3)  ) 1 1 6− 3 + 100 50 [10 pt ] = Pr(| Z | 1.28 ) where Z ~ N (0,1) 6− 3 If the variance were unknown but the variances of the two populations were equal then we would be working with a t-distribution. Question #18 Given you have an independent random sample 𝑋1 , 𝑋2 , … , 𝑋𝑛 of a Bernoulli random variable with parameter p, estimate the variance of the maximum likelihood estimator of p using the Cramer-Rao lower bound for the variance So, with large enough sample size, I know the population mean of the estimator 𝑃̂ will be p, and the variance will be: Var[ Pˆ ] = 1 nE[(( / p) ln f x ( X ))2 ] Now I'm having some trouble calculating the variance of 𝑃̂ , this is what I have so far: since the probability function of 𝑋̅ is binomial, we have: n n  n  Xi n − X i   * p i=1 *(1 − p) i=1 fx ( X ) =  n   Xi     i =1   n  n n  ) + X ln( p) + (n − X ) ln(1 − p) so: ln f X ( X ) = ln( n  i i   Xi   i =1   i =1  i =1  and: ∑𝑛 𝑖=1 𝑋𝑖 𝑝 𝜕 ln 𝑓𝑋 (𝑋) − = 𝜕𝑝 (𝑛−∑𝑛 𝑖=1 𝑋𝑖 ) (1−𝑝) = 𝑛𝑋̅ (𝑛 − 𝑛𝑋̅) − (1 − 𝑝) 𝑝 and: ( 𝜕𝑙𝑛 𝑓𝑋 (𝑋) 2 ) = 𝜕𝑝 2 𝑛𝑋̅ (𝑛 − 𝑛𝑋̅) ( − ) = (1 − 𝑝) 𝑝 𝑛2 𝑝2 −2𝑛2 𝑝𝑋̅ +𝑛2 ̅̅̅̅ 𝑋2 𝑝2 (1−𝑝)2 2 ̅̅̅̅2 ] = 𝜇 2 + 𝜎 , and for a Bernoulli random variable 𝐸[𝑋] = 𝜇 = 𝑝 = 𝐸[𝑋̅]𝑎𝑛𝑑 𝑉𝑎𝑟[𝑋] = 𝜎 2 = since 𝐸[𝑋 𝑛 𝑝(1 − 𝑝): E[(  ln f X ( X ) 2 ) ] p n 2 p 2 − 2n 2 pE[ X ] + n 2 E[ X 2 ] p 2 (1 − p) 2 p(1 − p) n 2 p 2 − 2n 2 p 2 + n 2 ( p 2 + ) n = p 2 (1 − p) 2 np(1 − p) = 2 p (1 − p) 2 n = p(1 − p) = Therefore, Var[ Pˆ ] = 1 = nE[(( / p) ln f x ( X )) 2 ] n 1 p(1 − p) = n n2 p(1 − p) However, I believe the true value I should have come up with is 𝑝(1−𝑝) 𝑛 . Answer: You seem to have confused the formulas for the lower bound in the case where the sample is IID, and the more general case of not necessarily identically distributed variables. The correct formula should be (in the unbiased case) Var[ pˆ ]  E[(  [log f X ( x | p)]) 2 ]−1 , p when the joint density is over the sample x = ( x1 , , xn ) . Note there is no factor of n in front. But when the sample is IID, this can be written Var[ pˆ ]  n E[(  [log f X ( x | p)]) 2 ]−1 , p where the density (notice no boldface!) is the univariate density for a single observation from the sample. Question #19 How can I find the Margin of Error E The professor hasn't gone over this section but I want to understand it so that I won't confused in class. I want to learn so that I can do the other problems. So this is the problem, If anyone could teach me so I could take note I'd deeply appreciate it. Thank you for your time. Question: Assume that a random sample is used to estimate a population proportion p. Find the margin of error E that corresponds to the given statistics and confidence level. 𝑛 = 550 , x equals 330, 90 % confidence I know how to obtain the 𝛼/2 to find the confidence. I just went over the Binomial Distribution and Limit Theorem, which understand now this is next. If anyone would care to spent their time explaining I'd be forever grateful, thank you again for your time. Answer: Explanation: Confidence intervals are characterized by two things: sample means and standard errors. The sample mean gives you the center of the interval, and the standard error helps define the radius. We shall write 𝑥̅ ± 𝑧𝛼/2 𝜎𝑥̅ for the confidence interval. Remember in this context we're talking about proportions, so we need the material from the previous question you asked, of which I will use freely. Recall that the standard error is 𝜎𝑥̅ = 𝜎 √𝑛 , where n is the sample size and 𝜎 is the standard deviation of the associated distribution. As we're using a binomial distribution, x = npq n = pq Now that makes things a great deal easier. The second part of the radius relies on this 𝑧𝛼/2 , the z-score associated with a particular alpha-level. What is this alpha-level? Take your confidence level and subtract from one, done. For a 90% confidence level, the alpha-level is 0.1, so we must search in a normal distribution table for a z-score whose left tail area is 0.05. I'll leave it to you to see that this is approximately 1.645 (one of the magic numbers to remember). 330 220 The radius is then 1.645 ⋅ √ 550 550 , which, according to some texts I've seen, but this may vary on the source, is the margin of error. Question #20 How to keep the information of the confidence interval in the predictive step? Suppose I have a sample 𝑋1 , 𝑋2 , … 𝑋𝑛 . Each 𝑋𝑖 comes from a poisson distribution 𝑋𝑖 ∼ 𝑃𝑜(𝜆) . The unknown paremeter 𝜆 is inferred using the next confidence interval: 𝜆± = 𝑋 ± 𝑡 ∗ 𝑋 √𝑛 where 𝜆+ is the upper bound and 𝜆− is the lower bound, t∗ is the confidence expressed over t-student distribution. I have two questions. The second question is actually the question I ask, the first is only to check my reasoning is right. First: Taking into account the data come from Poisson distribution and taking into account the Central Limit Theorem, is it right the inference a did?It means: is it right to use the t-student approach? Or is it another better way to do? Second: (That's the main question): As I don't know the 𝜆 parameter, I inferred it using confident intervals. Now I want to carry out some predictions of future events and calculate the associated probabilities, it means I wanto to know: 𝑝𝑟𝑜𝑏{𝑋𝑛+1 = 𝐾} . To do this: What number in the interval [𝜆− , 𝜆+ ] I should take? Obviously, maximize the likelihood is the best option, so I would take 𝑋 . So the question is: Is there a way to take into account the uncertainty in the probabilities of the prediction? or: Is there a way to keep the knowledge of the confident interval in the probabilities of the new prediction? or: At the moment in that you use an only number(point estimation), how to keep the uncertainty information? To be clearer take into account these two examples. The same process but with two different interval confidence: 1) [𝜆− , 𝜆+ ] = [6,8] 𝑋 = 7 → 𝑋𝑛+1 ∼ 𝑃𝑜(𝜆 = 𝑋) → 𝑃𝑟𝑜𝑏{𝑋𝑛+1 = 6} = 0.449 2) [𝜆− , 𝜆+ ] = [1,13] 𝑋 = 7 → 𝑋𝑛+1 ∼ 𝑃𝑜(𝜆 = 𝑋) → 𝑃𝑟𝑜𝑏{𝑋𝑛+1 = 6} = 0.449 The first contain less uncertainty than the second, so how could I maintain the information of confident interval? I hope I was clear in the question. Answer: The fundamental question seems to be how to find a confidence interval for 𝜆 given n independent counts 𝑋𝑖 from 𝘗𝘰𝘪𝘴(𝜆) . In that case 𝑇 = ∑𝑛𝑖=1 𝑋𝑖 ∼ 𝘗𝘰𝘪𝘴(𝑛𝜆) . I don't understand your interval because the margin of error seems to use an estimate of variance (divided by √𝑛) rather of the standard deviation. And I don't see how a t distribution is appropriate. You can get a good confidence interval for 𝑛 𝜆 from 𝑇 + 2 ± 1.96√𝑇 + 1. Then divide the endpoints by n to get a confidence interval for 𝜆 . Another good way to get an interval estimate for 𝜆 is to use a Bayesian procedure. You can use a relatively noninformative gamma prior if you have no useful prior information. If you are trying to use the estimate of 𝜆 to predict future behavior of the same process, then a Bayesian approach may be preferable.