Hypothesis Tests: Two Related Samples

Hypothesis Tests: Two Related Samples Major Points Related samples? Matched Samples? „ Difference scores? „ An example „ t tests on difference scores „ Advantages and disadvantages „ Effect size „ 2 Review: Hypothesis Testing 1. 2. 3. 4. 5. 6. 7. State Null Hypothesis Alternative Hypothesis Decide on α (usually .05) Decide on type of test (distribution; z, t, etc.) Find critical value & state decision rule Calculate test Apply decision rule 3 1 Related/Dependent Samples Samples can be related for 2 basic reasons „ First, they are the same people in both samples „ „ This is usually called either repeated measures or within subjects design 4 Related/Dependent Samples Samples can be related for 2 basic reasons „ Second, individuals in the two sample are so similar they are essentially the same person „ „ Often called a matched-pairs design 5 Related/Dependent Samples Repeated Measures „ The same participants give us data on two measures „ „ „ e.g. Before and After treatment IQ levels before IQPLUS, IQ levels after IQPLUS 6 2 Related/Dependent Samples Matched-Pairs Design „ Two-separate groups of participants; but each individual in sample 1 is matched (on aspects other than DV) with an individual in sample 2 „ Sample #1 S1 S2 S3 S4 S5 … SN Matched Matched Matched Matched Matched Matched Sample #2 S1 S2 S3 S4 S5 … SN 7 Related/Dependent Samples With dependent samples, someone high on one measure is probably high on other. „ Scores in the two samples are highly correlated „ Since they are correlated cannot treat them as independent (next chapter) „ However the scores can be manipulated (e.g. find the differences between scores) „ 8 Difference Scores „ Calculate difference between first and second score „ „ e. g. Difference = Before - After Base subsequent analysis on difference scores „ Ignoring Before and After data 9 3 An Example „ Therapy for rape victims „ Foa, Rothbaum, Riggs, & Murdock (1991) One group received Supportive Counseling „ Measured post-traumatic stress disorder symptoms before and after therapy „ 10 Hypotheses? H0: µsymptoms/before ≤ µsymptoms/after „ H1: µsymptoms/before > µsymptoms/after OR „ H0: µsymptoms/before - µsymptoms/after ≤ 0 „ H1: µsymptoms/before - µsymptoms/after > 0 OR „ H0: µ(symptoms/before - symptoms/after) ≤ 0 „ H1: µ(symptoms/before - symptoms/after) > 0 „ 11 Supportive Therapy for PTSD Person 1 2 3 4 5 6 7 8 9 Mean SD Pre Treatment 21 24 21 26 32 27 21 25 18 23.889 4.197 Post Treatment 15 15 17 20 17 20 8 19 10 15.667 4.243 12 4 Supportive Therapy for PTSD We want to compare the means to see if the mean after is significantly larger than the mean before „ However, we can’t perform the test this way (reasons I’ll explain in the next chapter) „ Since scores in the 2 conditions come from the same people we can use that to our advantage (subtract post from pre) „ 13 Calculating a difference score Person 1 2 3 4 5 6 7 8 9 Mean SD Pre Treatment 21 24 21 26 32 27 21 25 18 23.889 4.197 Post Treatment 15 15 17 20 17 20 8 19 10 15.667 4.243 Difference (Pre - Post) 6 9 4 6 15 7 13 6 8 8.222 3.598 14 Supportive Therapy for PTSD Person 1 2 3 4 5 6 7 8 9 Mean SD Difference (Pre - Post) 6 9 4 6 15 7 13 6 8 8.222 3.598 We now have a single sample problem identical to chapter 12. These are change scores for each person. 15 5 Results The Supportive Counseling group decreased number of symptoms „ Was this enough of a change to be significant? „ Before and After scores are not independent. „ See raw data (subjects high stayed high, etc.) „ r = .637 (obviously dependent samples) „ 16 Results „ If no change, mean of differences should be zero So, test the obtained mean of difference scores (we’ll call D) against µ = 0. „ Then, use same test as in Chapter 12. „ „ We don’t know σ, so use s and solve for t 17 tD test D and s D are the mean and standard deviation of the difference scores. Standard Error of D is s D = tD = „ sD n D − µ 8.222 − 0 8.222 = = = 6.86 3.598 sD 1.199 9 df = n - 1 = 9 - 1 = 8 18 6 t test 8 df, α = .05, 1-tailed → tcrit = 1.860 We calculated t = 6.86 „ Since 6.86 > 1.860, reject H0 „ Conclude that the mean number of symptoms after therapy was less than mean number before therapy. „ Supportive counseling seems to help reduce symptoms „ „ 19 SPSS Printout Paired Samples Statistics Pair 1 PRE POST Mean 23.89 15.67 N 9 9 Std. Deviation 4.197 4.243 Std. Error Mean 1.399 1.414 Paired Samples Correlations N Pair 1 PRE & POST 9 Correlation .637 Sig. .065 Paired Samples Test Paired Differences Pair 1 PRE - POST Mean 8.22 Std. Deviation 3.598 Std. Error Mean 1.199 95% Confidence Interval of the Difference Lower Upper 5.46 10.99 t 6.856 df 8 Sig. (2-tailed) .000 20 Related/Dependent Samples „ Advantages Eliminate subject-to-subject variability Control for extraneous variables „ Need fewer subjects „ „ „ Disadvantages Order effects „ Carry-over effects „ Subjects no longer naive „ Change may just be a function of time „ Sometimes not logically possible „ 21 7 Effect Size Again „ We could simply report the difference in means. „ „ „ Difference = 8.22 But the units of measurement have no particular meaning to us—Is 8.22 large? We could “scale” the difference by the size of the standard deviation. 22 Effect Size d= = µ1 − µ2 µ Before − µ After = σ σ Before 23.89 − 15.67 8.22 = = 1.96 4.20 4.20 Note: This effect size d is not the same thing as D (difference) It’s called d here because it is in reference to Cohen’s d 23 Effect Size The difference is approximately 2 standard deviations, which is very large. „ Why use standard deviation of Before scores? „ Notice that we substituted statistics for parameters. „ 24 8