Managerial Psychology Recitation 4: Survey Design (a dash “-“ indicates student responses) Survey Design and taking measurements I hope you have prepared for class today. Do you wake up thinking about measurement in surveys? Does it not make you jump out bed? Today, we will write survey questions. Sit next to someone. Can how a question is phrased influence the response? How could we influence the prediction - if bush or Kerry will be elected? Don’t sit in the last 4 rows, as we agreed Here is what I want you to do: Make up a survey question to ask anything you want to know about MIT students. Do not use a y/n format. Write down your question and the response scale. Only limit: not y/n (waits a couple of minutes.) What is your question and the response scale? -Do you feel you are able to balance your academic and social life? -Strongly disagree Disagree Neutral Agree Strongly agree Turn to your neighbor and present your questions to each other. Tell them why you chose your question and your response scale (late person walks in) We were just discussing things that embarrass us, it is your turn… Lecture 3 steps to answer survey Questions: Interpret the question Retrieve info and form judgment Map judgment onto response scale File Drawer view: Attitudes are stable dispositions stored in long term memory, and you retrieve them Constructive view: have to create info attitudes are constructed on the fly and computed when necessary from whatever information comes to mind Interpretation What brand of soft drink do you usually buy? A simple question to interpret, no? All of your life? Summer and winter? Recently? You? Friends you buy for? Diet Coke and Coke the same brand? Can be interpreted in many ways. Different people think differently about the question = variance Questions about action are the most open to interpretation Retrieve How many ounces of alcohol did you consume last year? hard to answer, not accurate What would be a better question to ask? Mapping How successful in life are you 1-10? What is 10? Is 10 the valedictorian? -Compare to bill gates -happy self is 10 -it is where you set your expectations for yourself -top 10% of population would be a 10 This is a very tricky topic To design a good question, you must be certain people can: 1. Interpret the question in the way you want them to 2. Access the relevant information How much wind have you experienced in the past week? Not a reasonable question, you won’t remember. 3. Map their response onto the scale in a meaningful and consistent way within a person and across people Scale formats How about this (slide) How successful have you been in life? 0 to 10 -5 to 5 -nobody wants to be negative Negative 5 implies failure How about 0? Should we have a 1 to 11 scale? The response scale provides cues for the “normal” levels of behavior. TV (slide) How many hours a day do you spend watching TV? Real data, 2 response scales Difference – almost double in the >2.5 category If you take a bathroom break in the middle, do you subtract that time? If you eat while you are watching TV, does that count too? The scheme is very simple, but not consistent Why the difference? -search to classify activities as tv Use scale to establish a range If I think I am slightly above average, I might choose the second category Example: Psychologists were given a description of a mental patient and asked the likelihood that the mental patient would commit a crime again? They were given a scale of 1%-9% or 9%-100% Both groups answered in the middle of the range. These were professional psychologists! Floss (slide) How many times in a day, 1-10? How many times in a year, 1-10? How likely are you to develop gum disease? How much would you pay for a bottle of scope mouthwash? If you answer on the left end of the scale, do you feel you need to go to the dentist? Does the top or bottom scale affect your answer? -top scale 10x a day not reasonable, ridiculous But the general effect is true for the scales Schwartz example: Same scale, but question: how often do you masturbate? Then asked, how happy is your love life? To change your own interpretation of your love life is profound. You should be familiar with your own love life. They replicated the experiment in Germany, where there are less strict guidelines The surveyor offered her phone # if asked - those who answered that they were unhappy with their love lives asked for the number. Reading into it, construct #s Imagine questions that, after answering them, you are more pro Bush or pro Kerry Surveys can create preferences, not just learn them Actual Bush supporter Question: How would you feel if you learned that McCain had a child by a black prostitute? It was not true, but later a paper showed him with his adopted black child, and people took it as evidence that it was true. Effects of scales (slide) In many cases, it makes people reinterpret their own behavior Importance of context (slide) 1. If jaywalk, rate how bad 2. If kill Professor, rate how bad What do you think would happen to question 2? -compare to question 1 badness What is very bad is already determined by the first one. Scale not only based on the question’s scale, but also previous questions S.O.’s (slide) List 10 reasons you love X Or List 3 reasons you love X Is there a difference? -condition 1 higher, more reasons -condition 2 higher, can’t think of enough to fill list For most 10 is hard, people run out Follow up: how much do you love your S.O.? Condition 1 gave lower answers. The process implies you should be able to come up with 10 reasons Say I wanted a higher evaluation at the end of semester. What could I ask to change the class rating? -list 3 reasons you love this class -compare to a class you know they hate -list 10 reasons you don’t like this class, and they will run out As you can see, there are a lot of possibilities for application Another – if want people to buy BMWs Question 1 or2? Vote. 2 is the answer Constructive view of preferences Many examples suggest people don’t know their preferences Only when you ask does it materialize. If you ask, they give an opinion – if you stop to ask for directions, people will give directions, sometimes even if they don’t know them. Sometimes people learn about themselves from their own answer Question Order (Slide) People think question 2 is correlated with the weather. Why? -think weather, then answer -life more positive if bad weather Q1 actually correlated. Interpretation of the question based on what they think you are asking People think you would not be asking the same question a second time, that you are asking for new information Question Order II (slide) 1st correlated, 2nd uncorrelated Response language People think in terms of discussion Where are you from? -Virginia How is it there, scale of 1-10? If I say 1st, “summer is coming,” you assume I want to visit Extracting sensitive information The more people feel they cannot trust a situation, the less likely they will give sensitive information. Money experiment in hallway at MIT - free money: $1, or $5, or $10, or $50 The bigger the amount, the more suspicious it is, and fewer people go for it. How can we overcome this? Have you done x? … (slide) I don’t mean ecstasy Extracting sensitive Information I 2 made it acceptable 3 reporting other people is easier. If the answer is yes, it is easier for you to admit. Lots of people are sensitive, embarrassed, don’t want admit things Extracting information II Why is this interesting? -could lie -obligated to answer truthfully? Embarrassed to admit I did x But you don’t know what question I answered, who I was. Yes to 1 or 2? Anonymous. Coin flip: Heads, answer y, tails answer truth. This protects the individual response, but gets the general population answer by extracting the information statistically. This way helps people hide, but the population level response is intact We can backtrack the answer Most information we ask for is sensitive So we use workarounds (Passes out survey), Please look and answer as quickly as you can. Fresh Samantha paradox Read the readings, know them well Types of measurement will be on the exam. next wednesday will be a recitation.