15.075 Notes, Inferences for Two Samples

Chapter 8 : Inferences for Two Samples In previous chapters, we had only one sample and we wanted to see whether its mean or variance might be above or below a certain value. In Chapter 8 we compare statistics from 2 populations, and we want to know whether one mean is larger than another, whether the means are different, etc. The techniques of this chapter are very useful for comparative studies. Independent Samples Design: There are a few different ways we can do an experiment. In an independent samples design, we have an independent sample from each population. The data from the two groups are independent. Sample 1: Sample 2: x1 , . . . xn1 y1 , . . . yn2 Here n1 does not need to equal n2 , that is, the samples can be different sizes. The xi ’s and yi ’s are all statistically independent. The difference is that the yi ’s receive the treatment and the xi ’s do not. For example, the xi ’s and yi ’s are student grades. The first group is the control group, and the second group was taught by a different method. Note: How might you compare 2 independent samples graphically? Let’s say you wanted to find out if one sample had generally larger values than the other for instance? Matched Pairs Design: In the matched pairs design, the observations from each sample are paired. An example is that the xi ’s are the student scores before a training program, and the yi ’s are the scores of the same students after the training program. pair: 1, 2, . . . , n Sample 1: x1 , x2 , . . . , xn Sample 2: y1 , y2 , . . . , yn Here the ith observation in the first group is similar in some way to the ith observation in the second group. The way in which they are similar is called 1 the blocking factor. Note: How might you consider matched pairs graphically? 8.3 Comparing means of 2 populations We will test whether the mean of one of the populations is different than the other by a difference of δ0 . 1) Independent Samples Design for Large Samples (n1 , n2 > 30): Sample 1: x1 , . . . , xn1 from a population with unknown µ1 and σ12 with sample mean x̄ and sample variance s21 . Sample 2: y1 , . . . , yn2 from a population with unknown µ2 and σ22 with sample mean ȳ and sample variance s22 . We are testing: H0 : µ1 − µ2 = δ0 H1 : µ1 − µ2 = δ0 So, if we want to test whether or not the means are different, we set δ0 = 0. The main idea is that since n1 and n2 are both large, we are going to use the ¯ − Ȳ is approximately central limit theorem to say that the distribution of X normal. We can calculate: E(X̄ − Ȳ ) = µ1 − µ2 ¯ + Var(−Y¯ ) + 2Cov(X, ¯ −Y¯ ) Var(X̄ − Ȳ ) = Var(X) = Var(X̄) + (−1)2 Var(Ȳ ) + 0 σ12 σ22 = + . n1 n2 From the CLT we know that Z is approximately N (0, 1) where: Z= X̄ − Ȳ − (µ1 − µ2 ) σ12 /n1 + σ22 /n2 2 . This means we can do an α-level z-test for comparing the means using the test statistic: x̄ − ȳ − δ0 z=p 2 , s1 /n1 + s22 /n2 where remember that for large samples s1 ≈ σ1 and s2 ≈ σ2 . To summarize, the statistic above is for conducting a 2-sample indepen­ dent samples design z-test where both samples are large, and the goal is to compare the means. That’s the basic idea, and the front page of your book has the summary written out for you to carry out the test. Basically if z is too big or too small, you’ll reject H0 . Example 1 2) Independent Samples Design for Small Samples (n1 , n2 ≤ 30): We could create a z-test using rv Z if the populations are normal and if we know the population variances. But in most cases, we don’t know this. In that case, we can’t use the z-test since we have no variances and we also can’t use the CLT to claim that Z defined above is approximately N (0, 1). We’ll have to assume the populations are normal and use the t-test. Sample 1: x1 , . . . , xn1 ∼ N (µ1 , σ12 ) where µ1 and σ12 are unknown. Sample 2: y1 , . . . , yn2 ∼ N (µ2 , σ22 ) where µ2 and σ22 are unknown. Case 2a: σ12 = σ22 . (You have to know this somehow ahead of time to use this test. Or you can check this assumption using an F-test that I’ll show you in Chapter 8.4. It is also assumed that you don’t necessarily know σ1 or σ2 in advance.) Let σ 2 := σ12 = σ22 . We are testing: H0 : µ1 − µ2 = δ0 H1 : µ1 − µ2 = δ0 I need to do some calculations to derive the test statistic. We need to know that: E(X̄ − Ȳ ) = µ1 − µ2 . 3 It’s going to be a kind of t-test, so I’ll need an estimator for the variance. To estimate the variance we could use either s21 or s22 but instead we use a combination so we get a better estimator: 2 s = − x̄)2 + i (yi − ȳ)2 = “pooled variance.” (n1 − 1) + (n2 − 1) i (xi It turns out (with some work required) that the following rv T has a tdistribution with d.f. (n1 − 1) + (n2 − 1) which equals n1 + n2 − 2: T = X̄ − Ȳ − (µ1 − µ0 ) 1 n1 s + . 1 n2 We can use the test statistic: t= x̄ − ȳ − δ0 1 n1 s + 1 n2 (where s is the square root of the pooled variance above) for a 2 sample t-test to compare the means for an independent samples design experiment where the samples have equal variance, d.f. n1 + n2 − 2. Example 2 Case 2b: σ12 = σ22 and you don’t know either of them. In this case, it is tempting to use the distribution of: T = X̄ − Ȳ − (µ1 − µ2 ) s21 n1 + s22 n2 , but T does not have a t-distribution. However, its distribution can be ap­ proximated by the t-distribution with d.f. ν where ν is computed according to the “Welch-Satterthwaite method: ν= - s2 1 n1 s21 s22 + n1 n2 -2 - n1 −1 4 + 2 s2 2 n2 -2 , n2 −1 where fractions are truncated to the nearest integer. So to test: H0 : µ1 − µ2 = δ0 H1 : µ 1 − µ 2 = δ 0 , compute test statistic: x̄ − ȳ − δ0 t= q 2 s1 s22 + n2 n1 and comparing to tν,α/2 (2-sided) or tν,α (1-sided) gives the approximate solu­ tion, using ν computed according to the Welch-S method. We can certainly also compute pvalues and confidence intervals, which are provided in the ta­ ble in the front of the book. The Welch-S method really makes a difference when: 1. s1 and s2 are very different 2. n1 and n2 are very different. 3) Matched Pairs Design: Given n pairs: pair: 1, 2, . . . , n Sample 1: x1 , x2 , . . . , xn Sample 2: y1 , y2 , . . . , yn Assume Xi ∼ N (µ1 , σ12 ) and Yi ∼ N (µ2 , σ22 ) but Xi and Yi are not indepen­ dent, they are correlated. The pairs themselves are mutually independent (e.g. patients’ temp before taking tylenol, patients’ temp after tylenol). Let ρ := corr(Xi , Yi ) (it’s the same for all i). Define Di = Xi − Yi . It turns out that the Di ’s are independent normal rv’s with: µD = E(Di ) = E(Xi − Yi ) σD 2 = Var(Di ) = Var(Xi − Yi ) = Var(Xi ) + Var(−Yi ) − 2Cov(Xi , Yi ) = σ12 + (−1)2 σ22 − 2ρσ1 σ2 . 5 Since ρ > 0 when the pairs are matched, the variance we computed is smaller than that of the independent samples case. Now we can actually reduce the whole thing to the single sample setting. Let di = xi − yi . To test: H0 : µD = δ0 H1 : µD = δ0 , P 2 . . , Dn ∼ N (µD , σD ). We calculated d¯ = n1 i di and we also We assume D1 , .q P 1 ¯2 calculated sd = n−1 i (di − d) . The test statistic is just: t= x̄ − ȳ − δ0 √ , sd / n and we perform a t-test (this is called a “paired” t-test). • Reject H0 when |t| > tn−1,α2 • Reject H0 when pvalue= 2P (Tn−1 ≥ |t|) < α � � s s d d ¯ ¯ • Reject H0 when δ ∈ / CI, where CI is δ ∈ d − tn−1,α/2 √n , d + tn−1,α/2 √n . We can adapt the power and sample size determinations from Chapter 7 even though the variables aren’t normal to get approximate values. Pinning down H1 : µD = δ1 , δ1 δ1 √ √ π(δ1 ) ≈ Φ −zα/2 + + Φ −zα/2 − . σD / n σD / n The following sample size calculation gives the sample size needed for a matched pairs test with α-risk α and power 1 − β to detect a difference in means of δ1 : (zα/2 + zβ )σD 2 n= δ1 (you can replace σD by sd if the sample size is large enough in both of these formulas). 6 8.4 Comparing the Variances of 2 Populations: The F-test for independent samples design (heavily requires normality) compares the variance of two populations where we have samples: Sample 1: x1 , . . . , xn1 ∼ N (µ1 , σ12 ) Sample 2: y1 , . . . , yn2 ∼ N (µ2 , σ22 ) To compare the population variances, we consider σ12 /σ22 , estimated by s21 /s22 . We learned in Chapter 5 that the rv below has an F-distribution with d.f.’s n1 − 1 and n2 − 1: S 2 /σ 2 F = 12 12 . S2 /σ2 So if we want to test: H0 : σ12 = σ22 H1 : σ12 = σ22 we compute the test statistic s21 F = 2 s2 and since the upper and lower α/2 critical points of the F-distribution are fn1 −1,n2 −1,1−α/2 and fn1 −1,n2 −1,α/2 , then we: • reject H0 when F < fn1 −1,n2 −1,1−α/2 or F > fn1 −1,n2 −1,α/2 . • reject H0 when P (F < fn1 −1,n2 −1,1−α/2 ) < α/2 or P (F > fn1 −1,n2 −1,α/2 ) > α/2. / CI. • reject H0 when F ∈ Let’s derive the CI: fn1 −1,n2 −1,1−α/2 s21 /σ12 ≤ 2 2 ≤ fn1 −1,n2 −1,α/2 . s2 /σ2 We need to solve for σ12 /σ22 . Just rewriting to make the notation simpler, s21 /σ12 f− ≤ 2 2 ≤ f+ . s2 /σ2 7 Let’s do the left equation first. Solving for the ratio σ12 , σ22 s21 σ22 f− ≤ 2 2 s2 σ1 σ12 s21 1 ≤ 2 . σ22 s2 f − Then for the right equation, we’ll have: σ12 s21 1 ≥ 2 . σ22 s2 f + Putting it together: 1 s21 σ12 s21 1 ≤ 2≤ 2 . f+ s22 σ2 s2 f− So we’ll reject H0 when:   1 s21 1 s21 1∈ / , . fn1 −1,n2 −1,α/2 s22 fn1 −1,n2 −1,1−α/2 s22 (The 1-sided tests can be derived similarly). 8