Nurs 211 - Stats Lecture 3 Notes

Lecture 3 - Sept 25 Sample, hypothesis, error and bias Sample vs population recap Sample size - Larger the sample size the more precise because the less variability you’ll have in that sample - Too small a sample will be unable to detect a difference - Too large a sample may find differences that don’t exist and also waste time and resources - Maybe unethical, why? - Calculate sample size needed using: - Significance level (a) - usually 0.5 (related to type 1 error) - Smallest effect size interested in (survival after chemo) - Standard deviation (from pilot or historical data) - Power - probability of finding a significant difference (it’s usually 0.8 as a minimum) (related to type 2 error) Determining sample size - Sample size is mostly determined by: - The size of the effect you want to see = Cohen’s d (standardized measure of difference) (with boundaries) (basically get a value and it’s split between small, medium and large) - Small: <0.3 (or -0.3) - Medium: 0.3 to 0.5 (or -0.3 to -0.5) - Large: >0,5 (or -0.5) - Sample size and effect size are inversely related - The smaller the effect size (difference between group means), the larger the sample that is required to see it - Can be a mean difference Importance of sample homogeneity - Sample needs to be homogenous - or similar - If heterogenous or too different, you can not make inferences because too much noise But - If too homogeneous may limit generalizability - In research, the main technique for this is… - Inclusion criteria - Exclusion criteria - Too many ages could mean that age influences the outcome and not the intervention (pill, wound size etc) Important - sample size and normal distribution - Remember central limit theorem - The more people there are in a sample, the more the mean of the sample will “look like” the wider population - The more chance that the variables of interest will be normally distributed The variance decreases - The less variance means a normal distribution - As the sample size increases, just about any distribution will tend to become normally distributed Sampling error or bias? - No matter which sampling method you use, there will always be sampling error due to chance alone - Sampling error (random things that have an impact on results) and bias (something in the system or your interpretation of data) are NOT the same thing - Bias is a systematic error made when using a random sampling approach that results in a nonrandom sample. Examples: - Use of phone book to select a sample - Use of a systematic sampling with a population list that is ranked in some way Many types of bias (selection bias, participation bias, measurement bias, performance bias, reporting bias) Hypothesis testing - Hypothesis follows a decision-making process for evaluating claims about a population Testing a “guess” about the sample - Can we prove or disprove the idea - Allows us to use sample data to test a claim about a population, such as testing whether a population mean equals some number - For example: collecting data to see if there is a difference between two or more groups - Control group - not given an intervention - Experimental group - given an intervention Types of hypotheses - Null hypothesis (H0) - a claim that there is no difference between the population parameter and the hypothetical value (no difference between the medication and the placebo) - Alternative or research hypothesis (Ha or H1) - a claim that disagrees with the null hypothesis (there is a difference) There can be: - Non directional hypothesis - direction of difference not stated (two tailed test) (better or worse quality of life after intervention) - Directional hypothesis - direction of difference stated (one tailed test) (not the best because it does not represent the entire population, it can skew the data, it might give bias because you’re not looking at people with a bad quality of life and you're only interested in those whose quality of life improved after intervention) Hypothesis testing involves recognition of error - Two types of errors - Type I Error - you find there was a difference when there really was no difference = false positive (positive we found a result but we actually didn’t) (more likely to happen because you are more likely to get your paper published if you find a difference) - Type II Error - you find there was no difference but actually there was a difference = false negative (think we found no result but we actually did) Type I error (false positive) - Believe there is a genuine effect in population, based on study results, when in fact there is no effect - Like believing someone is guilty when in fact they are innocent - The result could have occurred due to chance - Level of significance typically set at the 0.05 level (risk of Type I error or “alpha”) Means willing to accept a 5% chance your results are due to random error (the level is something you set before you do the study but it is generally 0.05) - Ways of expressing this - P<0.05 (p-value - find after the study) - 5% - 5 chances in 100 - 1 chance in 20 Type II error (false negative) - Believe there is no effect when in fact there is an effect or difference Like believing someone is innocent when in fact they are guilty - Function of sample size - smaller = more chance type 2 error - Called the β (beta) level - Normally set at 0.2 - Means willing to accept 20% chance that there really is a difference and we don’t find the difference - 20% is a compromise to get achievable sample size - Type 2 errors occur due to - Errors in experiment design - Sampling errors - Analysis errors - As type 1 error goes up, type 2 error goes down and vice versa Fundamental outcomes in hypothesis tests Truth H0is true H0is false We reject H0 Type I error (a) Power (1-β) We don’t reject H0 Confidence interval (1-a) Type II error (β) Consequence of Type I and Type II eros - RCT of new drug to treat cancer - Null hypothesis - no difference in survival between two groups (treatment and control) Type 1 Error (false positive) Type 2 Error (false negative) Reject null Wrongly conclude that treatment group has better survival Conclude no difference in survival between groups Consequence Patients subjected to ineffective treatment, costs, morbidity and mortality Missed opportunity to treat cancer Statistical Significance - The measurement of the statistical significance (it’s a p value) defines whether the null hypothesis is assumed to be accepted or rejected - P value > 0.05 no difference - Related to type I error - Most common value = 0.05 (but they set it at sometimes 0.01) - So 95% or 99% (bigger sample size) confidence we have found the true result Important point - If statistically significant, we have failed to reject the null hypothesis (we have found a difference) - So statistical significance usually would be a p<0.05 Statistical Power - Power is the probability of the researcher finding a difference between groups if one truly exists - Related to making a Type 2 error - Power is the chance of finding a true difference between the results if one exists - want to be sure you're not detecting a false positive result power calculated as 1 - β - β is usually 0.2 so standard value for power is 0.80 - This means that there is an 80% chance of finding a statistically significant result if one really exists Summary - two types of error - Alpha - a - Probability of Type I Error - Defined as - probability of rejecting the null hypothesis (H0) when H0is true Beta - β - Probability of Type II Error - Defined as - probability of failing to reject H0 when H0is false True False Reject H0 Error Correct decision Fail to reject H0 Correct decision error Remember - The more the confidence level, the less likely you will make a Type I error The higher the confidence, the more rigorous the test (95%, 99%) - Increasing the a increases the chance of making a Type I error but decreases the chance of making a Type II error - As power increases the risk of making a Type II error decreases - Power can be thought of as the “sensitivity”, “precision” , “probability of detection” Summary - Type 1 and Type 2 errors - Error is due to chance, bias, study design or selection - As Type 1 error decreases, Type 2 error increases (they are a balance/equilibrium) Use chance of type 1 (a) and 2 (β) to calculate and are also used to calculate Sample size - Power of the test (probability of finding an effect if one exists) - Effect size Confidence interval - How confident we are that it is a meaningful difference, Estimate confidence based on 2 standard deviations - Width of the confidence interval depends of two things - The variation within the population of interest - The size of the sample - How flat the distribution is. The bigger the sample size the smaller the distribution - How it might be reported - 34 (95% CI 20-40) (obviously the numbers are different) What do CIs tell you about the sample - Narrow CI means either - Large sample - Very little variation in the sample - Wide CI mean either - Small sample - Large variation in the sample