Notes on Creating a Survey

Types of questionnaires 1) Mail questionnaire 2) Telephone 3) In-house survey 4) Mall intercept 5) Computer questionnaire What basic information do you want to get? * attitudes and opinions * awareness and knowledge * intentions (to buy, etc.) * behavior (how often, etc.) What segmenting information do you need? * demographic * socioeconomic * ownership-experience * media exposure * personality/psychographics * values Bird's eye view of a questionnaire; 1. Identify yourself 2. Ask interesting, involving questions first 3. Go from general to specific: Funnel analogy 4. Threatening and demographic questions last 5. Thank you PROBLEMS * uninformed respondents; bogus recall, awareness Compare: A What is current value of your home? B Are you aware of the current assessed value? What do you think the current assessed value is? How close to the actual assessed value is this? Moral: give people a chance to say they don't know. * forgetting (omission, telescoping, creation) A Did you buy X in past three months? B When was the product bought most recently (>12,7,4,<3)? White glue: Aspirin Auto tires Record album A 46 68 32 41 B 32 57 24 32 General moral: Don't ask implicit questions, or two questions in one. Did you buy X within the past three months requests two pieces of information. Which of the following statements most closely coincides with your opinion of the Metallic Metals Act? __ It would be a good move on the part of the US. (21.4%) __ It would be a good thing, but should be left to the individual states. (58.6%) __ It is all right for foreign countries, but should not be required here. (15.7%) __ It is of no value at all. (4.3%) __ No opinion. (0.3%) * Willingness to answer correctly A In the past year, how often did you become intoxicated while drinking any kind of alcoholic beverage? (Answer card) B Sometimes people drink a little too much beer, wine, or whiskey so they act diferent from usual. What word do you think we should use to describe people when they get that way, so that you will know what we mean and feel comfortable talking about it? Occasionally, people drink on an empty stomach or drink a little too much and become __________. In the past year, how often did you become ___________ while drinking any kind of alcoholic beverage? (Ratio: 2:1 - 3:1) Randomized response rule *Phrasing A Would you favor a law that would require a person to obtain a police permit before he or she could buy a gun? B Would you favor a law that would require a person to obtain a police permit before he or she could fire a gun? A Do you prefer your hamburgers flame-broiled or fried? B Do you prefer a hamburger that is grilled on a hot stainless-steel grill or cooked by passing the raw meat through an open gas flame? (FB 75%) (FB 53%) * Status quo bias (higher preference for the action that preserves the status quo) A Do you think the US should forbid public speeches against democracy? B Do you think the US should allow public speeches against democracy? (Yes: 54%) (No: 75%) Level of detail (all the things that are logically included are not automatically apparent): In a situation like Vietnam were to develop in another part of the world, do you think the US should or should not intervene? In a situation like Vietnam were to develop in another part of the world, do you think the US should or should not intervene to stop a Communist takeover? +15% This applies even to matters of belief: What are the chances of: (1) Nuclear war between the superpowers sometime in the next five years? What are the chances of: (2) Unrest in Poland triggers a USSR intervention, which then escalates into a nuclear war with the US? Response Format Open ended; good at start of survey; provide with a feel for the quality of subjects later responses; also, courteous to the subject; Multiple-choice; Order bias: Reading-for last alternative; Visual-for first 12345678 87654321 --512 --2250 True-40320 Question sequence: (1) Do you think the US should let Soviet newspaper reporters come in and send back to their papers the news as they see it? (2) Do you think the Soviet Union should let American newspaper reporters come in and send back to their papers the news as they see it? 12: 55% 21: 75% Or: (1) Are you concerned about the poor quality of X? (2) Would you be willing to pay more for a higher quality X? (3) How much would you be willing to pay? Subtraction effect: (1) Do you think that abortion should be allowed if the mother's life is in danger? (2) Should abortion be allowed. Basic things to keep in mind: (1) Everything in the survey communicates something about you to the respondent. (2) Remember that respondent is not an expert. (3) Remember that you are catching him or her in the midst of some other activity; time is probably precious. (4) Many respondent are concerned to be look smart, appear good. (5) Respondents want to be consistent across different questions. Randomized response rule: 1. Toss a coin, note whether it lands Heads or Tails. 2. Please raise your hand if one of the two conditions is satisfied: (a) If your coin landed Heads and if your birthday is in Jan, Feb, March, April, May, or June. (b) If your coin landed Tails, and if on at least one occasion you drove through a red light when no one was looking.