Hypothesis Testing - Study Guide Part 2

Step 3: Comparing zobs with zcrit ______________________________________________ The shaded region is referred to as the rejection region ______________________________________________ If zobs falls in the shaded region we reject the null hypothesis. Interpretation: the experimental manipulation affected the outcome of the experiment Step 3: Comparing zobs with zcrit ______________________________________________ If zobs does not fall in the shaded region we fail to reject the null hypothesis. Interpretation: the experimental manipulation did not affect the outcome of the experiment Question about terminology ______________________________________________ Why do we ‘fail to reject the null’ instead of ‘accepting the null’? BO 1) Statisticians are conservative Professor Hobbes and his suspected cheaters: HO: The two students DID NOT cheat. HA: The two students DID cheat. If the evidence of cheating was suspicious, but not strong enough for a formal accusation, would Hobbes:  Accept the null?  Fail to reject the null? 2) Statisticians are cautious Is the earth flat? 3) Mathematics Jake’s example: Ho: µ = 100 Ha: µ ≠ 100 What is the probability that µ = 100? ______________________________________________ BO = Beatable offense Statistical significance ______________________________________________ The goal of hypothesis testing is not to make a decision about the null hypothesis. We use the decision regarding the null to make a statistical inference about the effect of our experimental manipulation. Thus, statisticians communicate their results in terms of statistical significance of their experimental manipulation. ______________________________________________ Example: If α = .05, Jake would reject the null hypothesis, but he would report that moving the napkin dispensers significantly decreased napkin consumption. If α = .01, Jake would fail to reject the null hypothesis, but he would report that moving the napkin dispensers did not significantly decrease napkin consumption. Put another way, we do not have enough evidence to conclude that moving the napkin dispensers significantly decreased consumption. Hypothesis testing errors ______________________________________________ Reality H0 is True H0 is False Reject H0 TYPE I ERROR () Correct Rejection Fail to Reject H0 Correct FTR TYPE II ERROR () Result of Test Type I Error: rejecting the null even though (in reality) it is true; P (Type I error) = α. Type II Error: failing to reject the null even though (in reality) it is false. P (Type II error) = β Questions: 1) Which error type concerns statisticians more? Why? 2) So, why not just set α to be as small as possible? 3) Why did statisticians recycle α as the symbol for the Type I error rate? The mysteries of α ______________________________________________ How can it be that α simultaneously serves as both  the critical probability for rejecting the null hypothesis (which sets zcrit, the boundary of the rejection region), and  the Type I Error rate? Critical probability If α = .05, we will reject the null hypothesis if the probability of observing our sample mean is less than .05. Why is the Type I error rate = .05 in this scenario? We can only make a Type I error if the null is true. Q: What has to happen for us to reject the null even if it is true? A: Our sample mean has to fall in the rejection region. Q: What is the probability that our sample mean will fall in the RR if the null is true? A: α An analogy: ‘Ping Pong Volcano’ ______________________________________________ Imagine you are on the midway at the Eureka County Fair. Biff asks you for $1 to play Ping Pong Volcano. Here is how the game works:  100 ping pong balls are placed in a giant volcano.  95 of the balls are yellow  5 are red.  The player presses a button which causes one ping pong ball to ‘erupt’ from the volcano.  Yellow ball: Win a stuffed animal  Red ball: Get the stuffing beaten out of you by some rough-and-tumble carnie people Let’s assume Ho: I will win a stuffed animal. Ha: I will get the stuffing beaten out of me. 95%: the outcome will be consistent with the null. 5%: the outcome will be inconsistent with the null Linking α and Ping Pong Volcano ______________________________________________ Ping Pong Volcano Randomly select one ball from the volcano 95 yellow balls 5 red balls 5% chance of losing the game Hypothesis Testing Randomly select one sample from the sampling distribution 95% of samples 5% of samples 5% chance of selecting a ‘red ball’ sample Last question: You are omniscient, so you know moving the napkin dispensers has no effect? What is the probability that he will collect a sample of data that will lead him to reject the null (commit a Type I error)? The basic steps for conducting a hypothesis test ______________________________________________ 1) Specify the NULL hypothesis (HO) o Usually µ = some value 2) Specify the ALTERNATIVE hypothesis (HA) o Usually µ ≠ some value 3) Designate the rejection region by selecting . o Must do this BEFORE data collection! 4) Determine the critical value of your test statistic o Find the z-score such that /2 lies in each tail 5) Use sample statistics to calculate test statistic. o zobs = M   0 / n 6) Compare zobs with zcrit: o If test statistic falls in the rejection region, we reject the null. o If test statistic does not falls in the rejection region, we fail to reject the null. 7) Interpret your decision regarding the null o What do your data imply regarding the question that motivated your experiment? Hypothesis Testing: Self-scheduled Assignments ______________________________________________ To eliminate complaints about final exam conflicts, Professor Hobbes allowed his students to self-schedule their final exams for any day of finals week. He was curious about whether self-scheduling would affect student performance. Hobbes compared exams scores from this semester with what he has come to expect from his many years of teaching Intro Philosophy: µ = 85, σ = 16. He chose a sample of 100 students from his class and obtained a mean exam score of 81. Conduct a hypothesis test (α = .05) to determine whether or not self-scheduling influenced exam performance. Step 1: H0: µ = 85 Step 2: HA: µ ≠ 85 Steps 3 and 4:  = .05: zcrit = 1.96. Step 5: zobs = M  0 / n = 81  85 16 / 100 =  4.0 1.6 = -2.5 Step 6: Zobs falls in the rejection region. The notation for reporting the results of a hypothesis test is: z = -2.5, p < .05. Step 7: What does this tell us about the effect of selfscheduled exams on performance? What if α =.01? Edison Light Bulbs ______________________________________________ Taking this class has made you intellectually curious. I’m serious! So, you decide to see whether Edison light bulbs last as long as the package says they do; according to the package: µ = 1200 hr and σ = 180 hr. Along with your physics major roommate, you construct a bank of 100 light bulbs and watch them round the clock to see when they burn out. The mean of your sample is 1170 hours. Does this constitute evidence of consumer fraud by Edison light bulbs if α = .05? Edison Light Bulbs ______________________________________________ Taking this class has made you intellectually curious. I’m serious! So, you decide to see whether Edison light bulbs last as long as the package says they do; according to the package: µ = 1200 hr and σ = 180 hr. Along with your physics major roommate, you construct a bank of 100 light bulbs and watch them round the clock to see when they burn out. The mean of your sample is 1170 hours. Does this constitute evidence of consumer fraud by Edison light bulbs if α = .05? Step 1: H0: µ = 1200 Step 2: HA: µ ≠ 1200 Step 3:  = .05, so… Step 4: … zcrit = 1.96. Step 5: zobs = M  = 0 / n = 1170  1200 180 / 100  30 18 = -1.67 Step 6: Zobs does not fall in the rejection region. z = -1.67, p > .05. Step 7: What does this tell us about the life expectancy of Edison Light bulbs? What if α =.10? Obrecht, Chapman, & Gelman (2007) ___________________________________________ How good are people at using ‘Amazon information’? Previous research is somewhat mixed:  Not so good: Kahneman & Tversky (1972)  Not so bad: Nisbett, et al. (1983) Current experiment Will people use statistical information effectively in a forced-choice preference task?  Sample size  Mean difference  Standard deviation o hard because of language and inversion Results: People paid attention to all three, but gave VERY little weight to sample size or SD Interpretation: 1. SD, and sample size information are harder to use because they require mathematical representations. 2. Intuitive understanding of standard deviation: ignore the crackpots. Big Y Light Bulbs ______________________________________ We sample 100 Big Y light bulbs to determine if they last as long as the package claims: 1200 hours. M = σ = 1170 150 What is the probability that we get this sample from a population with a mean of 1200? z M  0 / n z  1170  1200 150 / 100 z = -2 If we go to the z table, we find that p(z  2) = .0228. What if M = 1170 and s = 300? What if M = 1140 and s = 300? What if M = 1140, s = 300 and n = 225? The inference we draw on the basis of a hypothesis test depends on M, SE and n.