Research Meth In Psychology Chapter 3 Notes

Introduction - You now have the tools to differentiate among the three major claims you will encounter in research journals and in the popular media. - Once you identify the kind of claim a writer is making, you need to ask targeted questions as a critically minded consumer of information. The rest of this chapter will sharpen your ability to evaluate the claims you come across using what might be called "the four big validities": construct validity, external validity, statistical validity, and internal validity. In general, a valid claim is reasonable, accurate, and justifiable. - In psychological research, we do not say that a claim is simply "valid". Instead, psychologists specify which of the validities they are applying. You will learn to pause before you declare that a study is "valid" or "not valid". Instead, you will learn to specify which of the four big validities the study has achieved. - Although the focus for now is on how you can evaluate other people's claims based on the four big validities, you will also use this same framework if you plan to conduct your own research. Depending on whether you plan to test a frequency, an association, or a causal hypothesis, you will need to plan your research carefully, in order to emphasize the validities that are most important for your goals. Interrogating Frequency Claims - To evaluate how well a study supports a frequency claim, you will usually need to ask about two of the big validities: construct validity and external validity. Construct Validity of Frequency Claims - To ask about the construct validity of a frequency claim, the question to consider is how well the researchers measured their variables. Take the claim "2 million U.S. teens depressed" for example. - There are probably dozens of ways to evaluate whether a person is depressed. You could ask trained therapists to clinically interview teenagers and assess which of them are depressed. You could ask study participants to complete a structured, or a self-report questionnaire. - When you ask how well a study measured or manipulated a variable, you are interrogating the construct validity of the operationalization. Construct validity concerns how accurately a researcher has operationalized each variable, be it depression, happiness, debt stress, gender, BMI, or self-esteem. External Validity of Frequency Claims - The second important question to ask about frequency claims concerns generalizability: How did the researchers choose the study's participants, and how well do those participants represent the population they are supposed to represent? - External validity how well the results of the study generalize to, or represent, people and contexts besides those in the study itself. - To claim that half of Americans struggle to stay happy, the researchers in this study needed to ensure that the participants adequately represented all Americans. Interrogating Association Claims Construct Validity of Association Claims - Association claims differ from frequency claims in that they measure two variables instead of just one, and they describe how these variables are related to each other. - For the headline "Heavy Cell Phone Use Tied to Poor Sperm Quality," you should ask how well the researchers measured cell phone use and how well they measured sperm quality. For example, cell phone use could be measured quite accurately using phone bills; a much less accurate measure would be obtained by asking people to remember who much they use their cell phones. - In any study, measurement of variables is a fundamental strength or weakness - and construct validity questions assess how well such measurement was conducted. If you gather information on construct validity and conclude that one of the variables was measured poorly, you would not be able to trust conclusions related to that variable. - If you conclude that the construct validity in the study was excellent, you can have more confidence in the association being reported. External Validity of Association Claims - For the association between cell phone use and sperm quality, you would ask whether the results form this study's participants (perhaps a group of 20 to 30 year old volunteers from Cleveland) would generalize to other people and settings. - You can ask about generalization to other contexts by asking, for example, if sperm quality might also be linked to electronic devices besides cell phones - such as portable video games or music players. Statistical Validity of Association Claims: Avoiding Two Mistaken Conclusions - Psychologists use statistics to describe their data and to estimate the probability that a pattern of results can or cannot be attributed to chance. Statistical validity, also called statistical conclusion validity, is the extent to which those statistical conclusions are accurate and reasonable. - Generally speaking, a study that has good statistical validity has been designed to minimize two kinds of mistakes. First, a study might mistakenly conclude, based on the results from a sample of people, that there is an association between two variables. - Second, a study might mistakenly conclude from a sample that there is no association between two variables (say, use of ADHD medication and later drug abuse) when there really is an association in the full population. Statistical Validity of Association Claims: Strength and Significance - You need to consider some additional questions as you examine the statistical validity of a study. For an association claim, one important question to ask is, How strong is the association? - Some associations-such as the association between height and shoe size - are quite strong. - Though you can predict income form height, your prediction will be less accurate than when you predict height form shoe size. Another question worth interrogating is the statistical significance of a particular association. - Some associations that are reported by researchers might simply be due to chance connections between the two variables - but if an association is statistically significant, it is probably not due to chance. - If the association between abdominal fat and dementia risk is statistically significant, it would mean the there is a low probability that the association is merely a chance result (when there is not true association). - In sum, when you come across an association claim, you should ask about three validates: construct, external, and statistical. You can ask how well the two variables were measured (construct validity). You can ask whether you can generalize the result to a population (external validity). And you can ask about whether the researchers might have made any statistical conclusion errors, as well as evaluating the strength and significance of the association (statistical validity). Interrogating Causal Claims - Unlike an association claim, a causal claim says not only that two variables are simply related but also that one variable cause the other. Instead of using verbs such as is associated with, is related to, and is linked to, causal claims use directional verbs such as affects, leads to, or reduces. Three Rules for Causation - Covariance is the first rule a study must meet in order to establish a causal claim. A study must meet two additional criteria to justify the use of causal language: temporal precedence and internal validity. - To say that one variable has temporal precedence means that it comes first in time, before the other variable. To make the claim "Music lessons enhance IQ" a study must show that the music lessons came first and the gains in IQ came later. - Another criterion, called internal validity, or the third-variance rule, means that a study should be able to rule out alternative explanations for the association. - That is, there could be an internal validity problem - it is the school district, not the music lessons, that causes these children to score higher on IQ tests. In Chapter 2 you read about confounds in personal experience. Such confounds are also examples of internal validity problems. Experiments Can Test Causal Claims - What kind of study can satisfy all three criteria for causal claims? Usually, to support a causal claim, researchers must conduct a well-designed experiment, in which one variable is manipulated and the other is measured. - Experiments are considered the gold standard of psychological research because of their potential to support causal claims. - When psychologists do an experiment, they manipulate the variable they think is the cause and measure the variable they think is the effect. In the context of an experiment, the manipulated variable is called the independent variable, and the measured variable is called the dependent variable. - Remember - to manipulate a variable means to assign participants to be at level or the other. - In an actual study that tested this claim, researcher Glen Schellenburg (2004) of Toronto, Canada, manipulating the music lesson variable by assigning some children to take music lessons (either keyboard lessons or voice lessons), other children to take drama lessons, and still other children to take no lessons. Experiments Provide Temporal Precedence and Internal Validity - In addition, when researchers manipulate a variable, they can control for alternative explanations; that is, they can ensure internal validity. - Instead, Schellenberg used random assignment to ensure that the children in the four groups were as similar as possible. - Only by randomly assigning children to one of the groups could Schellenberg ensure that the children who took music lessons were as similar as possible, in every other way, to those who took drama lessons or no lessons. Random assignment increased internal validity by allowing Schellenberg to control for potential alternative explanations. - Schellenberg's experiment met all three rules of causation - covariance, temporal precedence, and internal validity - he was justified in making a causal claim for his data. His study found that music lessons really do enhance (that is, cause an increase in) IQ.