Types of Error

Types of Error „ There are two possible types of error in significance testing: Math 140 Introductory Statistics Null hypothesis is actually Your decision True False Don’t Reject H0 Correct Type II error Reject H0 Type I error Correct Type I error (When Test Statistic is Large) Type II error (When Test Statistic is Small) „ If the test statistic is large in absolute value (like Miguel and „ If the test statistic is small in absolute value (like Jenny and Kevin’s example), then the possible explanations for this are: „ „ „ 1. The null hypothesis is true and a rare event occurred. That is, it was just bad luck that resulted in being so far from p0. 2. The null hypothesis isn’t true, and that’s why the sample proportion was so far from p0. 3. The sampling process was biased in some way, and so the sample value is itself suspicious. „ If the last explanation is ruled out, then the usual decision is to reject the null hypothesis H0. However, you may be making a Type I error—rejecting H0 even though H0 is actually true. Maya’s sample), then the possible explanations for this are: „ „ „ 1. The null hypothesis is true, and you got just about what you would expect in the sample. 2. The null hypothesis isn’t true, and it was just by chance that turned out to be close to p0. 3. The sampling process was biased in some way, and so the sample value is itself suspicious. „ If the last explanation is ruled out, then the usual decision is to not reject the null hypothesis H0. However, you may be making a Type II error—not rejecting H0 even though H0 is actually false. 1 Minimizing the Error P-Values „ Type I Error. Null hypothesis is true but you reject it. „ Instead of just reporting that you either have or have If the null hypothesis is true, then the probability of making a Type I error is equal to the significance level of the test. To decrease the probability of a Type I error, decrease the significance level. Changing the sample size has no effect on the probability of a Type I error. „ Type II Error. Null hypothesis is false and you fail to reject it. To decrease the probability of making a Type II error, you can take a larger sample n or you can increase the significance level α. (But if you do the last option you will increase the probability of a Type I error) not rejected the null hypothesis, it has become common practice also to report a P-value. „ The P-value for a test is the probability of seeing a result from a random sample that is as extreme as or more extreme than the one you got from your random sample if the null hypothesis is true. (The P-value for a test is a conditional probability) Example One-Tailed Tests of Significance „ Suppose that 22 students out of a random sample of „ When testing the effectiveness of a new drug, the investigator 40 students carry a backpack to school. Follow steps a–d to test the claim that exactly half of the students in the school carry backpacks to class. „ „ „ „ a. Name the test and check the conditions needed for it. b. State the hypotheses in words and symbols. c. Calculate the value of the test statistic. Calculate the P-value for the test. Use this P-value in a sentence that explains what it represents. d. What is your conclusion? Explain in the context of this problem. must establish that the new drug has a better cure rate than the older treatment (or that there are fewer side effects). He or she isn’t interested in simply rejecting the null hypothesis that the new drug has the same cure rate as the older treatment. He or she needs to know if it is better. In such situations, the alternate hypothesis should state that the new drug cures a larger proportion of people than does the older treatment. „ This is called a one-tailed test of significance. Tests of significance can be one-tailed if the investigator has an indication of which way any change from the standard should go. This must be decided before looking at the data. 2 Alternate Hypothesis Example: One-Sided Test of Significance „ When testing a proportion, the „ The editors of a magazine have noticed that alternate hypothesis can take one of three forms. „ Ha : The percentage of successes p in the population from which the sample came is not equal to p0. „ Ha : The percentage of successes p in the population from which the sample came is greater than p0. „ Ha : The percentage of successes p in the population from which the sample came is less than p0. Ha : p ≠ p0 Ha : p > p0 Ha : p < p0 Example: One-Sided Test of Significance 2. Give the name of the test and check the conditions for its use. Name: Significance test for a proportion. 2. State the hypotheses, defining any symbols. „ n = 1027, p0 = 0.5 (the standard, if having a succesful life is not affected by having good friends or not) „ „ „ The sample is a simple random sample (it says in the problem), from a binomial population (a person either agrees or not that a successful life depends on good friends) . np0 = (1027)(0.5) = 513.5 > 10 n(1 – p0) = (1027)(1– 0.5) = 513.5 > 10 Total number of adults > 10 (1027) = 10270 „ H0 : The proportion of people p who believe that a successful life depends on good friends is equal to p0 = 0.5. (p = p0 ) Ha : The proportion of people p who believe that a successful life depends on good friends is greater than p0 = 0.5 (p > p0 ) people seem to believe that a successful life depends on having good friends. They would like to have a story about this and use a headline such as “Most adults believe friends are important for success.” So they commissioned a survey to ask a random sample of adults whether a successful life depends on having good friends. In a random sample of 1027 adults, 53% said yes. Should the editors go ahead and use their headline? Example: One-Sided Test of Significance 3. Compute the test statistic z and find The test statistic is z= the P-value 0.53 − 0.5 = 1.92 0.5(1 − 0.5) 1027 And then the P-value is the probability that we get a sample with proportion greater than this test statistic (1.92) (Because we have a one-sided significance test). Thus P-value = normalcdf(1.92, 999999) = 0.0274288 3 Example: One-Sided Test of Significance 4. Write a conclusion. „ Since the P-value equals 0.0274 and this is less than α = 0.05 = 5%. Then we should reject the null hypothesis. „ If the percentage of all adults who believe a successful life depends on having good friends is 50% or less, then the probability of getting a sample proportion of 53% or more is only .0274. Since this value is too small this gives strong evidence that the true percentage is greater than 50%. The editors should feel free to run the headline. 4