Managerial Psychology Laboratory Recitation for Survey Design

Survey design & Taking measurements Lets try ... • Make up a question to ask anything you want to know about MIT students (do not use a yes / no question format) • Write down your question and its response scale • Examples ... • Turn to your neighbor and present your questions to each other – Why this question, why this response scale etc. How do people answer questions? 3 cognitive stages in answering questions interpret question • • retrieve info & form judgment map judgment onto response scale "File drawer" view: – Attitudes are stable dispositions that are stored in long-term memory, and you just retrieve them "The constructive" view: – Attitudes are constructed on the fly, computed when necessary from whatever information comes to mind Do people interpret the question as intended? • What brand of soft drink do you typically buy? – – – – – What is the time period involved? (How far back do I go … I bought Gatorade in the summer; Hi C when I was a kid) Do purchases at movies, in restaurants, & sporting events count? Does "you" refer to me or to my household? What is a "soft drink" exactly? • do lemonade, iced tea, fruit punch, & mineral water count? Are Coke and Cherry Coke the same brand? interpret question retrieve info & form judgment map judgment onto response scale Can people retrieve the relevant information? • How many ounces of alcohol did you consume last year? – What would be a better question to ask? interpret question retrieve info & form judgment map judgment onto response scale How do people select responses? • How successful are you? – on a scale from 1 to 10? – • Is 10 the valedictorian of your high school? Ted Turner? Michael Jordan? interpret question retrieve info & form judgment map judgment onto response scale 3 cognitive stages in answering questions To design a good question you MUST be certain that people can 1) Interpret your question as you want them to 2) Access the relevant information 3) Map their response on the scale in a meaningful and consistent (across and between people) way interpret question retrieve info & form judgment map judgment onto response scale Effects of scale formats Effects of scale format How successful have you been in life ? 0 1 2 3 4 5 6 7 8 9 10 Not at all -5 -4 -3 -2 -1 Not at all Very much 0 1 2 3 4 5 Very much Response scale provide cues for the "normal" levels of a behavior How many hours a day do you spend watching TV ? 7% _____ 18% _____ 26% _____ 15% _____ 18% _____ 16% _____ up to .5 .5 to 1 1 to 1.5 1.5 to 2 2 to 2.5 more than 2.5 How many hours a day do you spend watching TV ? 62% _____ 23% _____ up to 2.5 2.5 to 3 8% _____ 3 to 3.5 5% _____ 3.5 to 4 2% _____ 0% _____ 4 to 4.5 more than 4.5 Response scale provide cues for the "normal" levels of a behavior How many times do you floss in a typical day ? 0 1 2 3 4 5 6 7 8 9 10 >10 How many times do you floss in a typical year ? 0 1 2 3 4 5 6 7 8 9 10 >10 How likely are you to develop gum disease? How much would you pay for a bottle of Scope mouth wash? Other examples …. Effects of scales: • In many cases the scales we give people are not ones they are used to think about (even hours of watching TV) • In such cases people use the scales to determine the acceptable range and their subjective fit within the range to determine their response The importance of context 1a. How bad is it to jaywalk? 2. 0 1 2 3 4 5 6 7 8 9 10 not at very all bad bad 2. How bad is it to spray paint the side of a building? 1b. How bad is it to murder your professor? 2. 0 1 2 3 4 5 6 7 8 9 10 not at very all bad bad 2. How bad is it to spray paint the side of a building? 5. 4. 0 1 2 3 4 5 6 7 8 9 10 not at very all bad bad 0 1 2 3 4 5 6 7 8 9 10 not at very all bad bad – – Ease of recall affects liking I Condition 1 Condition 2 List 10 reasons why you love X List 3 reasons why you love X 1. __________ 2. __________ 3. __________ 4. __________ 5. __________ 6. __________ 7. __________ 8. __________ 9. __________ 10. _________ 1. __________ 2. __________ 3. __________ Ease of recall affects liking II Condition 1 Condition 2 List 10 reasons to buy a BMW List 3 reasons to buy a BMW 1. __________ 2. __________ 3. __________ 4. __________ 5. __________ 6. __________ 7. __________ 8. __________ 9. __________ 10. _________ 1. __________ 2. __________ 3. __________ Effects of scales: • The constructive view of preferences: – People do not know but if you ask them they will give you an answer – And they will be sure about it – But ….. • In some cases people”learn” about themselves from their own answers Question Order I 1) How satisfied are you with your life overall? 2) How is the weather? How satisfied are you with your life overall? – – In one of these conditions, was there a strong correlation between current weather (whether it was raining outside or sunny) and people's appraisal of their life as a whole. – In which of these two conditions was the correlation observed. Question Order II 1a) How many dates did you have last month? 1b) How satisfied are you with your life, in general? • 2a) How satisfied are you with your life, in general? 2b) How many dates did you have last month? Response language • Asking questions is a part of a discussion • People make assumptions about the intended meaning • Think about what the question suggests / assumes / implies in the context of a conversation Summary • The way we ask questions can have a large effect on how people understand the questions, interpret them, and ultimately answer them • Be careful!!! Extracting sensitive information Extracting sensitive information I 1) Do you happen to have done X in the last year? 2a) As you know, many people have been doing X these days. Do you happen to have done X? 2b) Do you know anyone who has done X? How about yourself? Extracting sensitive information II Pull a coin from your pocket. If it lands Heads, answer Question 1. If it lands Tails answer Question 2: 1. Is your birthday between Jan 1st and June 1st ? 0.5 Yes (25%) 0.5 No (25%) H (0.5) T (0.5) Yes X (80%) No X (20%) Yes (40%) No (10%) Yes = 65% 2. Have you ever done X? Sensitive information • Much of the information we are interested is sensitive • More so as people get more suspicious! • There are workarounds • In many cases it is recommended to try and estimate the effect of hiding information Lets go back ... • Look back at the question you wrote earlier. Is there anything that you would change? • Even better... • Look back at the question your neighbor wrote earlier. Is there anything that you would change? Measurement summary • The constructive view of answering questions • The effects of scale formats (numbers, range, working, expectation etc.) • Extracting sensitive information