Minimizing Bias Random Variation (Like tossing a coin, so that the outcome is out of our control) Biased Sample If members of one group are different from the ones of the other group in a systematic way. Results will be off target, because differences in subjects (not just error) are likely to be consequences of preexisting differences in subjects. Often, college students take part in studies but are not representative for adults in general. Subjects are often from Western, Educated, Industrialized, Rich & Democratic societies - not representative for humankind in general. Reliability & Validity Reliability/Replicability About measurement errors Similar result each time Inter-observer: same behavior seen by two independent observers Operational definition defines variables clearly so that it is identifiable & repeatable Validity Valid if it measures/predicts what it's supposed to measure/predict Called face validity if it assesses variable it was supposed to measure Can be a source of bias Criterion (direct index) criterion validity Autism & Communication 1990: Recovery by Douglas Biklen who claimed that autistic persons could type meaningful messages with one finger on a keyboard. Needed help of a "facilitator" (so to hold his hand to calm him down). But experiments showed that it was not their capability to communicate but solely a reaction to movements of the facilitators' hand. Douglas Biklen was deluded by his own expectation (Observer-Expectancy Effect) & the desire to help the autistic community (another strong bias). → In order to avoid that, observers should be kept blind (not know what expected outcome of study is or about conditions) Double-Blind Experiment To also avoid subject expectancy effects, also subjects are kept blind → not informed about aspects of study that could influence their behavior → placebos. Replicating Earlier Studies 2015: Group of psychologists reproduced 100 experiments published in a top psychology journal & only about one third showed similar results. → Reliable/Replicable? Yes, they are. Reasons: Probability of a result occurring solely by chance at about p < 5% Differences in setting & subjects Science is very conservative - only well-established results of studies are published