Statistics: Type I and Type II Errors

STATISTICS Type I (α) and Type II (β) Errors and Power (1-β) Type I Error (False Positive) • Alpha (α) is the probability that the test will lead to the rejection of the hypothesis tested when that hypothesis is true. – Hypothesis: The medical device results in an improved outcome. α=0.05 means that there is only a 5% probability that this is wrong; i.e., low chance of a false positive. In other words, we are 95% sure that the new device is better than the control. – There could be dire consequences if we are wrong because the new device would be used even though it is no better than the control. Therefore, we have to be confident that we are right (i.e., a 95% probability that the new device is better than the control). Type II Error (False Negative) • Beta (β) is the probability that the test will reject the hypothesis tested when a specific alternative hypothesis is true; 1-β is the “power.” – Hypothesis: The medical device results in no improvement in outcome. β=0.2 means that there is only a 20% probability that the new device is shown by the study to be the same as the control, when it is actually better; i.e., a 20% chance of a false negative. – Not as much a problem if we are wrong in saying that the new device is not better than control, because the new device would just not be used. Therefore we can accept only an 80% probability (i.e., power) that there is no difference between the device and control. In meeting the criterion of α≤0.05 it is shown that there is a high probability (95%) that the new device is better than the control. The study should have a sufficient number of patients in each group (i.e., a sufficient power; 80% is enough) such that if there were no difference it would have been detected. See: http://calculators.stat.ucla.edu/