Ordering the Questions in the Questionnaire

From Market Research
Jump to: navigation, search

Questions in a questionnaire are ordered so as to reduce the time taken to complete the questionnaire and to reduce order effects.

Minimizing the time to complete the questionnaire

Screeners

A screener is a question that can be asked at the beginning of a questionnaire with the objective of determining eligibility. For example, if conducting a study on cola drinkers, then a question about cola consumption is asked at the beginning of the questionnaire so that non-cola consumers are not asked the rest of the questionnaire (i.e., are screened out to use the industry jargon).

Skips

A skip is an instruction in a questionnaire which makes respondents skip questions for which their answers are not required. For example, if a person has no children then they should skip a question asking about the ages of their children.

Filters

Options that are not relevant to some respondents should not be shown (i.e., filtered). For example, if a consumers has never heard of a product they should not then be shown that product in a following question asking which products they consume most often.

Multiple response questions and grids

Questions with similar structures should be grouped together. For example, true/false questions can be grouped together as multiple response questions. And, questions which share a response scale can be grouped into grids.

Deleting questions

The shorter the questionaire the better. Consequently, questions should only be included that are believed to be essential.

Minimizing order effects

An order effect occurs when people give answers in questionnaires that are influenced to an extent by the order with which the questions, or options in a question, are presented to respondents.

Knowledge and salience

The simplest type of order effect occurs when earlier questions unintentionally educate a respondent, or, make them think about a particular thing. For example, if a question that asks people if they have heard of Qantas and then, later in the questionnaire asks them to type the names of all the airlines for which they can recall recently seeing TV advertisements, there is a good chance that Qantas will get a higher result in the latter question than otherwise.

Leading questions

Leading questions cause respondents to change their opinions, which can effect either the answer of the question that is leading, or, the answers to following questions. Consider the following from the classic British sitcom Yes Prime Minister:[1]

Sir Humphrey: "You know what happens: nice young lady comes up to you. Obviously you want to create a good impression, you don't want to look a fool, do you? So she starts asking you some questions: Mr. Woolley, are you worried about the number of young people without jobs?"

Bernard Woolley: "Yes"

Sir Humphrey: "Are you worried about the rise in crime among teenagers?"

Bernard Woolley: "Yes"

Sir Humphrey: "Do you think there is a lack of discipline in our Comprehensive schools?"

Bernard Woolley: "Yes"

Sir Humphrey: "Do you think young people welcome some authority and leadership in their lives?"

Bernard Woolley: "Yes"

Sir Humphrey: "Do you think they respond to a challenge?"

Bernard Woolley: "Yes"

Sir Humphrey: "Would you be in favour of reintroducing National Service?" [A requirement that people join the army after leaving school.]

Bernard Woolley: "Oh...well, I suppose I might be."

Sir Humphrey: "Yes or no?"

Bernard Woolley: "Yes"

Sir Humphrey: "Of course you would, Bernard. After all you told me can't say no to that. So they don't mention the first five questions and they publish the last one."

Bernard Woolley: "Is that really what they do?"

Sir Humphrey: "Well, not the reputable [market research firms] ones no, but there aren't many of those. So alternatively the young lady can get the opposite result."

Bernard Woolley: "How?"

Sir Humphrey: "Mr. Woolley, are you worried about the danger of war?"

Bernard Woolley: "Yes"

Sir Humphrey: "Are you worried about the growth of armaments?"

Bernard Woolley: "Yes"

Sir Humphrey: "Do you think there is a danger in giving young people guns and teaching them how to kill?"

Bernard Woolley: "Yes"

Sir Humphrey: "Do you think it is wrong to force people to take up arms against their will?"

Bernard Woolley: "Yes"

Sir Humphrey: "Would you oppose the reintroduction of National Service?"

Bernard Woolley: "Yes"

Sir Humphrey: "There you are, you see Bernard. The perfect balanced sample."

Guiding and educating respondents

The opposite to leading a respondent is to guide them, as occurs in Probing and Laddering. In studies for new products it is routine to progressively introduce more information about the products throughout the questionnaire, so that the respondent is gently introduced to the information and also so that the researcher can understand the incermental impact of the additional information.

Fatigue

Harder questions should generally be asked towards the beginning of the questionnaire, so that if respondents become fatigued while completing the questionnaire, the fatigue occurs after they have completed the more challenging questions. And, the order of questions can be used to maximize the energy of the respondent, mixing boring questions with more interesting questions (e.g., projective questions).

References

  1. “The Ministerial Broadcast”, First airtime BBC: 16 January 1986.