Chapter 15 Questionnaire Design

Chapter 15 Questionnaire Design TRUE/FALSE 1. For a questionnaire to fulfill a researcher’s purposes, the questions must meet the basic criteria of relevance and accuracy. ANS: T 2. A questionnaire is accurate to the extent that no unnecessary information is collected in order to solve the problem under study. ANS: F This describes relevancy. 3. Accuracy means that the information collected from a questionnaire is valid. ANS: T 4. There exists a step-by-step process for ensuring accuracy in the wording of questions in surveys. ANS: F There is no step-by-step procedure for ensuring accuracy in question writing that can be generalized across projects. 5. Question wording and sequence can substantially influence accuracy. ANS: T 6. Fixed-alternative questions give respondents specific limited-alternative responses and ask them to choose the one closest to their own viewpoints. ANS: T 7. Open-ended response questions are frequently used in exploratory research. ANS: T 8. Open-ended response questions are valuable at the beginning of personal interview sessions. ANS: T 9. The cost of administering open-ended questions is much lower than that of administering fixed-alternative questions. ANS: F The cost is much higher for open-ended questions. 10. One advantage of open-ended response questions is the possibility that interviewer bias will affect the results. ANS: F Interviewer bias is a disadvantage. 11. Articulate respondents tend to give shorter answers to open-ended response questions than respondents who are less articulate. ANS: F Articulate individuals tend to give longer answers. 12. Fixed-alternative questions require more interviewer skill to administer than open-ended response questions. ANS: F Fixed-alternative questions require less interviewer skill. 13. A fixed-alternative question may tempt respondents to check an answer that is more prestigious or socially acceptable than the true answer. ANS: T 14. A frequency-determination question allows the respondent to provide multiple answers to a single question. ANS: F This is a checklist question. 15. Surveys used in telephone interviews should be written in a conversational style. ANS: T 16. A loaded question suggests a socially desirable answer or is emotionally charged. ANS: T 17. A counterbiasing technique is used with the expectation that two alternative phrasings of the same question will yield a more accurate total response than will a single phrasing. ANS: F This is a split-ballot technique. 18. A question covering several issues at once is referred to as a leading question. ANS: F This is referred to as a double-barreled question. 19. Two consequences of respondents’ forgetting the exact details of their behavior are called telescoping and squishing. ANS: T 20. Order bias in questionnaire design frequently occurs when broader questions are asked before specific issues are explored. ANS: F Asking specific questions before asking about broader issues is a common cause of order bias. 21. Asking general questions before specific questions in order to obtain unbiased responses is called the anchoring technique. ANS: F This is called the funnel technique. 22. The paging layout format on an Internet questionnaire (going from screen to screen) greatly facilities skip patterns. ANS: T 23. A radio button is a message box that can be used to provide highlighted information. ANS: F A radio button is a circular icon, resembling a button, that activates one response choice and deactivates others when a respondent clicks on it. 24. Variable piping software allows variables, such as answers from previous questions, to be inserted into unfolding questions in an Internet survey. ANS: T 25. A pretest of a questionnaire can help to establish the likely response rate for mail surveys. ANS: T MULTIPLE CHOICE 1. Which of the following refers to the extent that all information collected in a questionnaire addresses a research question that will help the decision maker address the current business problem? a. relevant b. accuracy c. frequency d. counterbalanced ANS: A 2. A questionnaire that collects information that is valid is said to be: a. relevant b. accurate c. counterbalanced d. balanced ANS: B 3. A question that poses some problem or topic and asks respondents to answer in their own words is called a(n): a. sentence completion question b. unbalanced question c. fixed-alternative question d. open-ended question ANS: D 4. "Can you name five brands of tires?" is an example of what type of question? a. fixed-alternative b. pivot c. open-ended response d. filter ANS: C 5. Another term for fixed-alternative questions is _____ questions. a. open-ended response b. closed-ended c. pivot d. concise ANS: B 6. "What is your gender? _____ M a. loaded b. leading c. fixed-alternative d. filter _____ F" is an example of what type of question? ANS: C 7. All of the following are disadvantages of open-ended response questions EXCEPT: a. cost b. interviewer bias c. the information provided by responses is not useful to decision makers d. articulate individuals give a large share of the responses but they may not be representative of the entire population ANS: C 8. Compared to open-ended response questions, fixed-alternative questions: a. require less interviewer skill b. take less time c. are easier for the respondent to answer d. all of the above ANS: D 9. "Do you own a cell phone? _____ Yes a. leading b. simple-dichotomy c. loaded d. open-ended response _____ No" is an example of what type of question? ANS: B 10. What type of question is the following? What time of day did you watch that movie at the Multiplex? _____ Matinee _____ Evening _____ Rush-hour special a. b. c. d. filter pivot open-ended response determinant-choice ANS: D 11. Which of the following scales is a type of fixed-alternative question? a. Likert scale b. Stapel scale c. semantic differential scale d. all of the above ANS: D 12. What type of question is the following? Which of the following services of the United Missouri Bank, besides your personal checking account, do you currently use? (check all that apply) ______ Savings account ______ Car loan ______ Home mortgage loan a. b. c. d. pivot question filter question checklist question counterbalancing question ANS: C 13. Dichotomous or multiple-choice alternatives in fixed-alternative questions should not have overlap among categories, which means the categories should be: a. exhaustive b. mutually exclusive c. positively worded d. balanced ANS: B 14. All of the following are guidelines that help prevent the most common mistakes in designing questions EXCEPT: a. avoid personal questions b. avoid complexity c. avoid leading questions d. avoid double-barreled questions ANS: A 15. "What is your favorite hobby, playing video games, or what?" is an example of what type of question? a. loaded b. pivot c. filter d. leading ANS: D 16. "In light of the current economic crisis, do you agree or disagree that the President of the United States is doing a good job of managing the economy?" is an example of what type of question? a. counterbalancing b. order bias c. double-barreled d. loaded ANS: D 17. When half of the respondents are asked to agree or disagree with the statement: "Foreign cars are better made than cars made in the U.S." while the other half of the respondents are asked to agree or disagree with the statement: "Cars made in the U.S. are better made than foreign cars," this is an example of a: a. pivot question b. filter question c. double-barreled question d. split-ballot technique ANS: D 18. An introductory statement or preamble to a potentially embarrassing question that reduces a respondent’s reluctance to answer by suggesting that certain behavior is not unusual is called a: a. filter question b. lead-in statement c. split-ballot question d. counterbiasing statement ANS: D 19. "Do you agree or disagree with the statement: The Federal Reserve Bank and the large U.S. banks are responsible for the high foreclosures on home mortgages" is an example of what type of question? a. counterbiasing b. open-ended response c. counterbalancing d. double-barreled ANS: D 20. "Should Dillard's keep its excellent department store credit card program? _____ Yes _____ No" is an example of: a. using a pivot question b. making assumptions c. a double-barreled question d. using a checklist ANS: B 21. Asking respondents to remember something without providing any clue is called: a. unaided recall b. aided recall c. recognition d. cognitive retrieval ANS: A 22. Which of the following occurs when respondents believe that past events happened more recently than they actually did? a. telescoping b. squishing c. myopic remembering d. zooming ANS: A 23. When respondents think that recent events took place longer ago than they really did, this is called: a. telescoping b. squishing c. faltering d. misredemption ANS: B 24. Which technique recommends asking general questions before specific questions in order to obtain unbiased responses? a. skip logic technique b. telescoping c. squishing d. funnel technique ANS: D 25. In political elections for candidates who are not well-known by many voters, such as judges, it frequently happens that the candidate who is listed first on the ballot receives the most votes, this is an example of a(n): a. b. c. d. Hawthorne effect self-fulfilling prophecy order bias split-ballot technique ANS: C 26. When a respondent in a personal interview is asked questions about shopping at a specific department store in the shopping mall, and then is asked general questions about shopping at that mall, what type of mistake has been made in questionnaire design? a. filter effect b. order bias c. double-barreled effect d. loaded effect ANS: B 27. The idea that the first concept measured on a questionnaire frequently becomes a comparison point from which subsequent questions are evaluated by the respondents, is known as a(n): a. loaded effect b. filter effect c. pretest effect d. anchoring effect ANS: D 28. A question such as: "Have you ever received a speeding ticket? _____ Yes _____ No," when followed by a question such as: "When did you receive that ticket?" is an example of what type of question? a. double-barreled b. leading c. filter d. anchoring ANS: C 29. A filter question used to determine which version of a second question that will be asked is called a(n): a. anchoring question b. pivot question c. funneling question d. piping question ANS: B 30. Which type of question conserves space on a questionnaire? a. pivot question b. funnel question c. multi-faceted question d. multiple-grid question ANS: D 31. Which Internet questionnaire layout presents the entire questionnaire on one page? a. paging layout b. scrolling layout c. continuous layout d. perpetual layout ANS: B 32. When a respondent to an Internet survey has to click on the “NEXT” button at the bottom of the screen in order to continue to answer the questionnaire, this button is what type of button? a. pull b. push c. radio d. advancement ANS: B 33. A bar at the top of the page in an Internet survey that indicates how much of the survey still needs to be completed is known as what type of bar? a. drop-down b. radio c. push d. status ANS: D 34. Which of the following are common ways of displaying questions on a computer screen? a. drop-down box b. check boxes c. open-ended boxes d. all of the above ANS: D 35. A space-saving device within Internet surveys that allows the respondent to "click here" to see a set of possible choices in a list at that point in the survey is known as a: a. status box b. drop-down box c. radio box d. status box ANS: B 36. Which of the following allows variables, such as answers from previous questions, to be inserted into unfolding questions on Internet surveys? a. variable piping software b. interactive software c. error trapping software d. funneling software ANS: A 37. When a respondent is attempting to subscribe to Southwest Airlines e-mail of flight sales and is presented with a box that says: "You must provide your zip code in order to complete your registration," this is an example of: a. a double-barreled response b. error trapping c. a radio button d. zoning ANS: B 38. A live, real-time support feature that solves problems or answers questions respondents may encounter in completing the questionnaire is called a(n): a. search engine b. interactive help desk c. interactive tabulation d. intervention ANS: B 39. A tabulation of the results of a pretest to help determine whether the questionnaire will meet the objectives of the research is referred to as: a. preliminary tabulation b. primary tabulation c. initial tabulation d. back translation ANS: A 40. When "Out of sight, out of mind" was translated into a foreign language, and then was translated back into English by a language expert in that language to become: "Invisible things are insane," this was an example of: a. the linguistic effect b. a loaded question c. back translation d. a counterbiasing statement ANS: C COMPLETION 1. A(n) ____________________ question allows the respondent to answer the question in a personal interview in his or her own words. ANS: open-ended response 2. A question in which the respondent is given a list of possible responses and is asked to choose one of them is an example of a(n) ____________________ question. ANS: fixed-alternative 3. A fixed-alternative question in which the respondent is asked to check either “ _____ Yes or _____ No” is an example of a(n) ____________________ question. ANS: simple-dichotomy 4. A type of fixed-alternative question in which a respondent is asked to select the category that best represents his gross income for the previous year is an example of a(n) ____________________ question. ANS: determinant-choice 5. A fixed-alternative question which lists ten possible services that could be provided by a local bank and asks respondents to check as many as apply to the question: “Which of the following banking services do you currently use for First City Bank?” is an example of a(n) ____________________ question. ANS: checklist 6. Questions that suggest or imply a certain answer are called ____________________ questions. ANS: leading 7. A question that is written in such a way that it arouses strong emotion in the respondent is called a(n) ____________________ question. ANS: loaded 8. A question that is written in such a way that it really asks two questions at the same time (e.g. two separate ideas separated by "and") is called a(n) ____________________ question. ANS: double-barreled 9. When the respondent's answers to previous questions in a questionnaire create a bias in his or her answers to later questions, this is known as ____________________ bias. ANS: order 10. In a personal interview, asking general questions before asking specific questions in order to create less bias in the responses is known as the ____________________ technique. ANS: funnel 11. A question that "screens out" respondents because they do not qualify to answer a subsequent question (e.g. "Do you own a DVD player?") is called a(n) ____________________ question. ANS: filter 12. A filter question that is used to determine which subsequent question will be asked depending on the respondent's answer to the filter question is called a(n) ____________________ question. ANS: pivot 13. When a customer is selecting a flight at Southwest Airlines Web site by clicking on a circular icon next to the flight of her choice, this is called a(n) ____________________. ANS: radio button 14. In an Internet questionnaire, boxes where respondents can type in their own answers to questions are called ____________________. ANS: open-ended boxes 15. The process of taking a questionnaire that has previously been translated from one language to another and having it translated back again by a second, independent translator is called ____________________. ANS: back translation ESSAY 1. List the key decisions in questionnaire design. ANS: Researchers planning a questionnaire’s design must make several decisions, which typically take place in the following sequence: (1) What should be asked? (2) How should questions be phrased? (3) In what sequence should the questions be arranged? (4) What questionnaire layout will best serve the research objectives? (5) How should the questionnaire be pretested? Does it need to be revised? 2. Compare and contrast open-ended and fixed-alternative questions and give an example of each. ANS: Open-ended response questions pose some problem or topic and ask respondents to answer in their own words. Examples include “What names of local banks can you think of offhand?” or “”How can our stores better server your needs?” Fixed-alternative questions (sometimes called closed questions) give respondents specific limited-alternative responses and ask them to choose the one closest to their own viewpoints. For example: “Do you use Tide laundry detergent?” ____ Yes ____ No While open-ended response questions are most beneficial when the researcher is conducting exploratory research, their cost is substantially higher than that of administering fixed-alternative questions. Fixed-alternative questions require less interviewer skill, take less time, and are easier for the respondent to answer. However, unanticipated alternatives may emerge when respondents believe that closed answers do not adequately reflect their feelings. 3. Name and define the primary types of fixed-alternative questions. ANS: The types of fixed-alternative questions are: (1) Simple-dichotomy (dichotomous) question - requires the respondent to choose one of two alternatives. (2) Determinant-choice question - requires the respondent to choose one--and only one-- response from among several possible alternatives. (3) Frequency-determination question - a determinant-choice question that asks for an answer about the general frequency of occurrence. (4) Checklist question - allows the respondent to provide multiple answers to a single question. 4. Discuss factors to consider when determining the order of questions on a questionnaire. ANS: One factor to consider is order bias, which can result from a particular answer’s position in a set of answers or from the sequencing of questions. Asking specific questions before asking about broader issues is a common cause of order bias. As a result, it is advisable to ask general questions before specific questions to obtain the freest of open-ended responses. This procedure, known as the funnel technique, allows the researcher to understand the respondent’s frame of reference before asking more specific questions about the level of the respondent’s information and the intensity of his or her opinions. With attitude scales, there also may be an anchoring effect, which means the first concept measured tends to become a comparison point from which subsequent evaluations are made. Randomization helps minimize this effect. Filter questions minimize the chance of asking questions that are inapplicable. A pivot question is a filter question used to determine which version of a second question will be asked. 5. Discuss layout issues with respect to Internet questionnaires, the tools used to assist respondents’ navigation through the survey, and the alternative ways of displaying questions. ANS: There are several layout issues to consider. The first decision is whether the questionnaire will appear page by page, with individual question on separate screens (called paging layout), or on a scrolling basis, with the entire questionnaire appearing on a single Web page that the respondent scrolls from top to bottom (called scrolling layout). Whether a Web survey is page-by-page or scrolling format a push button with a label should clearly describe the actions to be taken. Because many Internet surveys offer no visual clues about the number of questions to be asked, it is important to provide a status bar or some other visual indicator of questionnaire length. Four common ways of displaying questions on a computer screen are: (1) radio button - clicking on an alternative response deactivates the first choice and replaces it with the new response. (2) drop-down box - a space-saving device that allows the researcher to provide a list of responses that are hidden from view until they are needed. (3) check boxes - allow respondents to check all that apply. (4) open-ended boxes - boxes in which respondents type their answers to open-ended questions. (5) pop-up boxes - boxes that appear at selected points and contain information or instructions for respondents.