Chapter 9-10: Gathering Data

Chapter 9-10: Gathering Data The 3 big Ideas of Sampling Examine a part of a whole Randomize It’s a Sample Size From a population (an entire group of individuals), draw a sample (a smaller group of individuals selected from a population and should represent it. Sample Surveys : designed to ask question so of a small group of people in the hope of learning something representational of the entire population. However, we must be aware of Bias : When results tend to over/under emphasize some characteristics of the population There is no way to fix/salvage useful information from biased results. Over/Under-estimates the parameters of interest This can be avoided by selecting individuals for the sample at random Sources of Bias Selection Bias (Undercoverage) When some portion of the population is not sampled at all or has a smaller representation in the sample than it has in the population. Usually the people that are not covered differ from the rest of the population, so bias exists. Ex: a sample survey of households will miss not only homeless people but prison inmates. Ex : an opinion poll conducted by telephone will miss the households without residential phones. Response Bias : refers to anything in the survey design that influences the responses. Ex : respondents may lie, especially if asked about illegal or unpopular behavior. Ex : Smoking is very unhealthy, do you like smoking? Nonresponse bias : A common and serious potential source of bias for most surveys. occurs when responses are not obtained from individuals selected for the study. Ex: Have you ever refused to take a survey? Exercises Ex. Name and describe the kind of bias that might be present if the statistics teacher decides that instead of randomly selecting students to survey on how they feel about the course she just asks students to volunteer for the survey. Nonresponse bias —the bias would probably be towards those students who say they enjoy the course. Ex. A chemistry professor who teaches a large lecture class surveys the students who attend his class on how he can make the class more interesting to get more students to attend. This survey method suffers from A. nonresponse bias B. response bias C. undercoverage D. none of the above Ex. We propose 2 designs to test the effectiveness of the new medication: Design 1: In order to test the effectiveness of a new medication, a random sample of individuals were chosen from a particular population. The drug was administered to the individuals the instance they began to experience a pain. After a fixed period of time elapsed they were asked to rate the effectiveness of the drug (in terms of pain reduction) on 10 point scale, 10 being no pain to 1 meaning the drug was ineffective. The results showed the average rating was 6.6. The study was repeated. Design 2: This time each individual was randomly assigned to one of two treatment groups. Ex. In order to determine the proportion of the voting population that supports a new government policy, a local news organization carried out a survey. The results showed that only 32% of the people answering the survey support the policy. Consider where they obtained their evidence (data) and answer these questions: Can these results be generalized to the entire voting population? Comment on any problems with the ways in which the data were collected. i. An online survey was conducted. They asked individuals to log onto their website and offer their opinion. No random selection, voluntary response - we can’t generalize this to any population. Selection/Undercoverage bias: People without a computer (or tv) can’t respond! ii. They randomly selected individuals at the local mall and asked for their opinion. Even though a random sample was conducted, it was not a random sample from the population of interest. We could perhaps generalize the result to the population of shoppers at the local mall. We should also take into account the time the survey was conducted. Selection/Undercoverage Bias: You have to be at the mall to respond. Non-response Bias: People refuse to respond. iii. They randomly selected phone numbers and called people. By far the best method, but still… Selection/Undercoverage Bias: You need a phone! (Not really an issue anymore) Non-response Bias: People hang up! Ex. Example (Textbook, pg 325, # 8): A question posted on the Lycos Website on 18 June 2000 asked visitors to the site to say whether they thought that marijuana should be legally available for medicinal purposes. Population – all U.S. adult Parameter – proportion that feels marijuana should be legalized for medicinal purpose Sample – those visiting the web site who responde Method – voluntary response (no randomization employed Bias – nonresponse bias; those who visit the website and respond may be predisposed to a particular answer. What matters is the Sample size, # of individuals in the sample, not the size of the population Why not sample an entire population rather than just using a sample? This is a Census : a special sample that includes everyone (the entire population). Cons: Expensive Time Complexity Undercount Population constantly changes Parameter (aka population parameter): Is part of a model for a population (It’s often unknown). Estimated using statistics Ex. mean, standard deviation Statistic (aka sample statistic): A summary found from data in a sample. Are known once the data is observed When the statistics we compute from the sample accurately reflect the corresponding parameter, then the sample is said to be representative There are 2 types of conclusions Causal Inference : the difference in the responses caused by the difference in treatments when comparing results from 2 treatments group We can only use this when we have random allocation. We cannot make causal inferences from observational studies. When there is no random allocation, the difference in responses could have been caused by other lurking variables Lurking variables are variables related to both group membership and to the response. There are other variables that could explain the result Ex . After many dogs and cats suffered health problems caused by contaminated foods, a researcher is trying to find out whether a newly formulated pet food is safe. Our experiment will feed some dogs the new food and some cats a food known to be safe, and a veterinarian will check the response. Why would it be a bad design to feed the test food to some dogs and the safe food to cats? This would create confounding. We would not be able to tell whether any differences in animals’ health were attributable to the food they had eaten or to the differences in how the two species responded. Ex . For children between the ages of 6 and 10, the larger their shoe size the better they do on a particular math test. Does this mean that larger feet causes one to do better on the test? Probably not. Age is a reasonable confounding variable that might be a more reasonable explanatory variable for the test performance. Age might also be used to explain shoe size. Population inference :R esults from the sample that can be generalized to an entire population made when when we have random sampling (randomly select individuals in the samples from the population) Randomizing protects us from the influences of all features of our population by making sure that , on average, the sample looks like the rest of the population, or else biased. In the case, conclusions should be restricted to sample, and NOT generalized to the population Types of Random Sampling Simple Random Sampling (SRS) (of size n) each sample of size n in the population has the same chance of being selected Ex . drawing from a hat with all the names in a population Samples drawn at random generally differ from one another Each draw random #s selects different people for our sample These differences lead to different values for the variables we measure We call these sample-to-sample differences sampling variability sampling variability Ex . Local school district randomly tests high school students for ADD. There are 3 high schools in the district 9-12. The school board pools all of the students together and randomly samples 250 students. Is this simple random testing? A. Yes, because the students were chosen at random. B. Yes, because each student is equally likely to be chosen. C. Yes, because they could have chosen any sample of 250 students from throughout the district. D. No, because we can’t guarantee that there are students from each school in the sample. E. No, because we can’t guarantee that there are students from each grade in the sample. Stratified Random Sampling The population is first divided into homogenous groups, called strata , then take SRS within each stratum before the results are combined This method reduces bias and variability of the results Ex. Suppose you want to estimate the proportion of Canadians that support federal party X based on an appropriate representation from each province. You could break up the population by province and select a SRS from each province Ex . Suppose the state decides to randomly test high school wrestlers for steroid use. There are 16 teams in the league, and each team has 20 wrestlers. State investigators plan to test 32 of these athletes by randomly choosing two wrestlers from each team. Is this a simple random sample? A. Yes, because the wrestlers were chosen at random. B. Yes, because each wrestler is equally likely to be chosen. C. Yes, because stratified samples are a type of simple random sample. D. No, because not all possible groups of 32 wrestlers could have been the sample. E. No, because a random sample of teams was not first chosen. Ex . The January 2005 Gallup Youth Survey telephoned a random sample of 1,028 U.S. teens aged 13-17 and asked these teens to name their favorite movie from 2004. Napoleon Dynamite had the highest percentage with 8% of teens ranking it as their favorite movie. Which is true? I. The population of interest is U.S. teens aged 13-17. II. 8% is a statistic and not the actual percentage of all U.S. teens who would rank this movie as their favorite. III. This sampling design should provide a reasonably accurate estimate of the actual percentage of all U.S. teens who would rank this movie as their favorite. A. I only B. II only C. III only D. I II, and III Systematic Random Samples Start the systematic selection from the randomly selected individual, then sample every kth person When there is no reason to believe that the order of the list could be associated in any way with the responses sought, systematic sampling can give a representative sample. Systematic sampling can be much less expensive than true random sampling. Ex : Suppose you want to estimate the proportion of individuals that supports federal party X at your area. You can set up a booth at your area and ask every 50th person your question. Cluster Random Sampling Splitting the population into similar groups (or clusters), select one or a few clusters at random and perform a census within each of them. This sampling design is called cluster sampling. If each cluster fairly represents the full population, cluster sampling will give us an unbiased sample. Cluster sampling is not the same as stratified sampling. Exercises Ex . A statistics teacher wants to know how her students feel about an introductory statistics course. She decides to administer a survey to a random sample of students taking the course. She has several sampling plans to choose from. Name the sampling strategy in each. Ex . You want to determine the proportion of university students that have “jobs” while attending school. 1) You have a complete list of students. SRS (Simple random sample) 2) You have a complete list of students and want to make sure each faculty is appropriately represented. Stratified Random Sample . Select a SRS from each faculty. 3) You do not have a complete list of students, but believe that at any given time the group of students in each classroom across campus all individually form a representative sample of the entire student body. Cluster Sample . Select a SRS of classrooms, then go to each selected classroom and sample everyone. 4) You do not have a complete list of students. You believe that students walking in front of the Registrar’s building throughout the day are a good representation of the entire student body. Because of the massive number of students walking past your booth you can’t sample everyone. Systematic Random Sample . Sample every 20th person that walks by. Study Designs There are 2 main types of study designs Observational Studies Ex. i nvestigator observes individuals and measures variables of interest but does NOT attempt to influence the responses. Study impact of distance of home to power plant on cancer incidence. Since the researchers did not assign households to live at a certain home (ie. He didn’t influence the distance) and simply observed the households randomly, it was an observational study. Because researchers in this example first identified the distance of the subjects’ home to the power plant and then collected data on whether they had cancer, this was a retrospective study. Useful when one outcome is rare Based on historical data, so they can have errors. Had the researchers identified subjects in advance and collected data as events unfolded, the study would have been a prospective study. Observe explanatory and response variable More costly than a retrospective study, but better data - Observational studies are valuable for discovering trends and possible relationships. - However, it is not possible for observational studies to demonstrate a causal relationship. Ex. More dogs are being diagnosed with thyroid problems than have been diagnosed in the past. A researcher identified 50 puppies not already diagnosed with a thyroid problem and followed the dogs for several years to see if any developed thyroid problems. This is a(n) A. Randomized experiment B. Survey C. Prospective study D. Retrospective study Randomized, Comparative Experiments An experiment is a study design that allows us to prove a cause-and-effect relationship. - An experiment: Manipulates factor levels to create treatments. Randomly assigns subjects to these treatment levels. Compares the responses of the subject groups across treatment levels. Ex : Study the impact of exercise on blood pressure. We can enroll assign exercise schedules to individuals participating in the study. More Examples According to a recent national article, the female average wage grew faster than the male average wage at ABC Bank. These results are based on a random sample of females and males working at ABC Bank across Canada in 1971.