Surveys and Observations - Describing What People Do Part 2

 Sampling practice o Imagine that you are planning to estimate the price of the average book at your college book store…  What is the sample in this study, and what is the population?  The sample is 200 books, the population is 13,000 books.  How might you collect a simple random sample of books?  How might you collect a stratified random sample?  How might you collect a convenience sample?  Print all the titles of books and randomly pick from a hat   Pick whatever is infront of you. ( recording price of first 200 books)  How might you collect a systematic sample?   Pick every third books How might you collect a cluster sample of 200 books?  Make a list of all of display shelfs, and select 4 of the 20 book shelfs, then randomly select 4 of the 20 book shelfs at random, and record the price of all the books on the 4 shelfs selected until you hit 200.  Key questions in activity disaster preparedness : o A) sampling technique is biased, the population we can generalize to this poll is CNN readers. Sampling technique is more important than sample size  Interrogating external validity: what matters most about a sample o When a representative sample is not the top priority  External validity is essential for some frequency claims but not all   Causal claims least important External validity might not be a top priority with association and causal claims  Need to consider the specific claim at hand and whether having a biased sample really matters. o Larger samples are not more representative  Size of sample is less important than sampling technique  Larger sample size, smaller the margin of error, smaller margin of error the sample more accurately represents the population.   How was the sample collected, not how many Is a representative sample rather than a biased sample essential ? yes or no o A political poll predicting who will win a governors race  Representative, because you don’t want biased opinion o Estimating a school achievement  Biased, Chapter 7 book notes:  7.1 Generalizability: does the sample represent the population? o True:   random assignment is only used in experimental designs convenience sampling technique is the most common in behavioral research o false:  self selection is a form of unbiased sampling  combining sampling techniques increases bias o random assignment does not make a sample unbiased, it increases the internal validity of the research but does not affect the external validity. o When random sampling occurs, each member of a population has an equal chance of being selected. Cluster sampling is when people in subsets of the population are randomly sampled. Oversampling is often used to select, in an unbiased way, small subsets of the population that are not large enough in number to be accurately measured. All three of these techniques are representative sampling techniques. o Unbiased:  Systematic sampling  Stratified sampling  Probability sampling o Biased   Convenience  Snowball sampling  Purposeful sampling Clicker: Which of the following is NOT an example of probability sampling o Convenience samples  Interrogating external validity: what matters most? o True  External validity is not always the highest priority  A random sample of 1,000 people is enough to generalize to an entire population o False  If a finding does not generalize to other populations the finding is invalid  A larger sample means that it will be more representative o What type of claim is external validity especially important for?  Frequency claim o What would be the best way to increase a samples external validity  Use a representative sample o When purposeful sampling occurs, a particular group is targerted and recruited in a nonrandom way. When those who participate help recruit other they know to participate its called snowball sampling. When participants are recruited from a group of people who are readily available to the researcher, convenience sampling occurs. All three of these techniques are unrepresentative. o When random sampling occurs, each member of a population has an equal chance of being selected. Cluster sampling is when people in subsets of the population are randomly sampled. Oversampling is often used to select, in an unbiased way, small subsets of the population that are not large enough in number to be accurately measured. All three of these techniques are representative sampling techniques. o If a study’s sample is not generalizable to the rest of its population, we would say the study has poor external validity. o The population is the entire group of people that a researcher is interested in, while a sample is a smaller subset of that entire group. A census occurs when a researcher investigates an entire group of people  Chapter 7 on sampling emphasizes ____ validity o External  Which of the following is an example of probability sampling technique o Systematic sampling  Which of the following is true regarding sampling and representativeness? o sample size is more important than sampling method Chapter 8: simple bivariate correlations.  Introduction to bivariate correlations o Describing associations between two quantitative variables o Describing association with categorical data  Bivariate correlations o Associations between two measured variables  Correlational studies can include both quantitative and categorical variables but they are graphed differently.  Association claim: 2 measured variables  Interrogating association claims (two most important) o Construct validity  How well was each variable measured?  Was it measured reliability and with validity? o Statistical validity (5 questions)  How well do the data support the conclusion?  How strong is the relationship?  Is it statistically significant?  Are there outliers  Is the relationship curvilinear? o Internal validity ( not usually relevant)  Can we make a causal inference from an association?  Can we make a causal inference?  1) covariance o An association establishes that A  B  2) temporal precedence ( directionality problem) o Do we know which one came first in time?  3) internal validity o 3rd variable an internal validity problem  Subgroups can sometimes explain the relationship  o 3rd variable NOT an internal validity problem  But subgroups are not always a problem. o External validity (also taken into consideration)  To whom can the association be generalized?  How did they get their sample? o Sampling technique is more important than sample size o Random sampling is the most generalizable but not always practical to use  Does external validity really matter in an association claim? o Need to always look at all 4 validates together as a set   Moderators address external validity? Moderators o (extra variable, depending on level changes relationship between two variables of interest) o Moderators and external validity  The association may not generalize from one situation to others. o Difference between moderators and sub groups  Moderator: Whether the association is DIFFERENT within levels of an extra variable  Subgroups: is the SAME within subgroups o Difference between moderators and third variables. o  Clicker: o When an association differs within the levels of a third variable this is a…  Moderator CHAPTER 8 BOOK NOTES  Bivariate correlation is exactly 2 measured variables.  The variables in bivariate correlational study can be either quantitative or categorical. o If both variables are quantitative, the data is usually o in a scatterplot. If one variable is categorical, the data are usually depicted in a bar graph.  Association claim state that two variables are linked, but they do not state that one causes the other.  Association claims are supported by correlational research, in which both variables are measured.  Her data show a .35 correlation. Maggie find that the correlation is much strong for men than for women Moderator  Interrogating an association claim o Very important: construct validity and statistical validity o Somewhat important: external validity o Least important: internal validity  Three types of associations o Positive o Negative o Zero  Cohens guidelines for evaluating strength of association (based on r) o .10 or -.10 small, or weak o .30 or -.30- medium or moderate o .50 or -.50 large or strong  Describing association with categorical data o The nature of the association can be described with scatterplots and the correlation coefÏcient r. o For the association between marital satisfaction and online dating the dating variable is categorical; its values fall in either one category or another. o Graphing associations when one variable is categorical:  When both variables in an association are measured on quantitively scales, a scatterplot is usually the best way to represent the data.  Analyzing associations when one variable is categorical o When at least one of the variables in an association claim is categorical, researchers may use different statistics to analyze the data. They occasionally use r, it is more common to test whether the difference between means (group averages) is statistically significant, usually by using a statistic called the t test, or other statistical tests.  A study with all measured variable is correlational o Associations claims can be supported by either scatterplots or bar graphs, using a variety of r or t tests. o When the method of the study measured both variables, the study is correlational, and therefor it can support an association claim.  Interrogating association claims o The two most important validities to interrogate are construct validity and statistical validity.  Construct validity: how well was each variable measured?  Statistical validity: how well do the data support the conclusion?  What is the effect size?  Is the correlation statistically significant?  Could the outliers be affecting this association? o Outliers matter more when the sample is small   Is there restriction of range? Chapter 8 clicker quiz: o When interrogating the statistical validity of Dr. Oswalds research, we need to examine..  The effect size of the correlation coefÏcients, AND the statistical significance (p value) o To interrogate the construct validity of Dr. O’s research you would,  Examine the reliability and the validity of the variables o If one of the peer reviewers is concerned about the external validity of her study, which of the following is the most important aspect of Dr. O’s study to consider  The sampling technique Chapter 9: multivariate correlational research  Review of the three causal criteria o Covariance  As A changes B changes, high levels go with high levels/ low levels go with low levels  Are the variables related or associated? o Temporal precedence  A comes first in time before B o Internal validity  There are no possible alternative explanations for the change in B; A is the only thing that has changed   Is there a third variable Review and overview o Bivariate correlations show covariance but NOT temporal precedence  Not sure which variable came first  Solution: cross lag panel designs (longitudinal study) o And not internal validity no control for third variables   Solution: multiple regression How to establish temporal precedence with longitudinal designs o Interpreting results from longitudinal designs (3 different kinds)  Cross sectional correlations   Correlation between two variables measured at the SAME TIME. Autocorrelations  Correlation of ONE variable with ITSELF measured at TWO different times o Happiness at time 1 and happiness at time 2. Same variable correlated with itself.  Cross lag correlations  Correlation between an earlier measure of one variable and a later measure of another variable o Variable A at time 1, variable B at time 2 o Longitudinal studies and the three criteria for causation o Why not just do an experiment? Dr farahs research: What type of study is this o Longitudinal design  Write down the LETTER that corresponds to the correlation o What is the cross sectional correlations  A AND B o What is the auto correlations  D AND F o What are the cross lag correlations  C AND D o Which pattern of cross lag correlations would sugg Clicker chapter 9:  The song two out of three aint bad is the theme song for ch. 9 on multivariate correlational research. What are the two criteria for making a causal claim that longitudinal designs meet …… o Covariation and temporal precedence  Which statistical symbol is important in interpreting multiple regression results? o Beta  Which of the following is NOT one of the telltale signs that a researcher has used multiple regression in her research o Controlled for  Controlling for means holding potential third variable constant o As caused by  This one is NOT one of the signs o Correcting for o Taking into account  In a multiple regression design ___ is to IV as __ is to DV o Predictor variable; criterion variable  What are the Two criteria for making a causal claim that multiple regression designs can help meet? o Covariation and internal validity  Aka the third variable problem Review chap 8  Bivariate correlations show covariance  But not temporal precedence- not sure which variable came first o Solution: cross lag panel signs (longitudinal)  And not intern validity – no control for third variables o Solution: multiple regression Ruling out third variables with multiple regression designs  Measuring more than two variables  Regression results indicate if a third variable affects the relationship  Adding more predictors to a regression  Regression in popular press articles  Regression does not establish causation Multiple regression helps with the third variable problem  Test whether a relationship holds true even when a proposed third variable is controlled for.  The researcher is studying 3 or MORE variables and needs to start by deciding which variable is the DEPENDENT VARIABLE aka (criterion variable) researchers most interested in understanding or predicting, ( DV usually in the table title, and top row of the table itself)  The rest of the variables are IV’s aka predictor variables , the predictor variables in multiple regression are used to explain the variance in the criterion variable. Typically listed below your DV.  Beta value for each predictor variable  Beta and r denote direction + or -, also strength and magnitude of a relationship. Positive beta, positive relationship between beta variable and criterion when other variables are controlled for.  One difference between r and beta: o With r Cohen benchmarks effect size standards o With beta, aren’t hard cut off, don’t really know effect size because they change depending on predictor variables Multiple regression example 2: “intelligence in children is correlated with the number of books in the home”  Simple bivariate correlation  Our 2 variables: intelligence in children, # of books in the home o Directionality problem  o Third variables?  Income, how much your parents have you read,  Dv: intelligence  IV: o books in the home  beta= .21 p < .05 o parents SES BETA= .20 p< .05 intelligence in children is correlated with the number of books in the home. Proposed third variable is parents SES?  1) what is the dv? And what are the predictor variables? o Intelligence is the DV, books in home IV  2) which pattern of results would indicate that only the number of books in the home is independently predicts child IQ, when parents SES is held constant o Only the beta for # of books in home would be significant and parents SES would not be significant. Only 1 beta is Signiant, parents SES does not predict IQ.  3) which pattern of results would indicate that only parents SES independently predicts child IQ, and is not attributable to number of books in the home o Only parents SES would be significant beta. Leave second beta alone.  4) which pattern of results would indicate that number of books in the home predicts child IQ, controlling for parents SES AND parents SES predicts child IQ, controlling for number of books in the home? o All significant betas  Clicker: which is true… Sig beta of .21 o All of these