A concept test is used for measuring and understanding the appeal of a new product.
The concept board
Concept testing generally involves creating concept boards, which explain the new product to consumers, giving them enough information so that they can have a meaningful reaction when asked if they will or will not buy it. For example, this concept board was used to test the appeal for Magnum Gold prior to its launch.
Copyright Unilever Australia (used with permission)
The success of a concept test hinges on the quality of the concept’s description. It is not unknown for 20 minutes to be spent educating the respondent about a concept, but in most studies consumers get 10 to 20 seconds (which is likely sufficient for simple concepts).
If a concept board is insufficient to accurately get consumers to evaluate a concept then consumers may also be given an opportunity to trial the product. Most commonly, products will be shipped to consumers’ houses (this is called -in-home placement), or, consumers will go to a central location where the concept testing may also involve some sensory research (research designed to get specific feedback on how to improve the sensory aspects of the food, such as smell, taste and viscosity). When presenting radical new technologies, the process of respondent education is called information acceleration.
The impact of respondent education can be great. One study found that after describing a concept, the proportion of respondents who said they would 'definitely' or 'probably' buy was 59% at $1.59 and 59% at $2.29 (i.e., the price had no effect). However, when the same study was conducted with an in-home use component (i.e., leaving the product with the consumers for a few weeks), price had a large effect, with the stated intention to buy being 64% at the lower price and 48% at the higher price.
If presenting complex concepts to respondents it is advisable to test for comprehension in some way, such as in Q1 of the questionnaire below.
In addition to respondent education, questions should be incentive compatible, and this is harder to achieve. Our concern is not simply one of respondents trying to mislead. There is also the issue of whether respondents will put as much effort in to answering the question as they would in a real-world purchase situation (as if their efforts are different, then their answer in the questionnaire will be unlikely to reflect what they do in reality). Most concept tests use the same structure, although questions are modified and added as required:
Concept testing questionnaire
SHOW CONCEPT BOARD
Please look at the description of the product. Please take as much time as you need to look at it. We will then ask you some questions about the product.
Q1. Is there anything in this description that you do not understand or would need more information about in order to decide whether or not to buy this product? If so, what? OPEN-ENDED
'Q2. What, if anything, do you think you would particularly like about this product? OPEN-ENDED
Q3. What, if anything, do you think you would particularly dislike about this product? OPEN-ENDED
Q4. Which phrase from those below best describes how likely you would be to buy the product for yourself?
I would definitely buy it I would probably buy it I am not sure whether I would buy it or not I would probably not buy it I would definitely not buy it
Q5. On average, how often do you think it would be eaten?
Every day Two or three times a week Once a week Once every two/three weeks Once a month Once every two/three months Once every four/six months Less often than every six months Never
Q6. Compared with similar products how different do you think this product is?
Very different Different A little different Not very different Not at all different
Q7. How well do you think the idea for this particular snack fits what INSERT BRAND NAME means to you?
0 Does not fit at all 1 2 3 4 5 6 7 8 9 10 Fits extremely well
Q8. Do you have any further comments about this product? If so, what? OPEN-ENDED
Qualitative concept testing
Where the concepts are being tested using qualitative research, a similar structure is usually followed, although commonly with a general exploration of needs and wants at the beginning of the study (this can also occur with quantitative studies). Where the purpose of the concept test is to refine concepts, rather than work out which are most appealing, qual is usually more appropriate due to the potential for more depth from the diagnostics.
Typically a concept test will involve multiple separate concepts and concepts are evaluated based on a combination of relative performance against each of the other concepts and internal benchmarks. Usually the key question that is used to evaluate whether the concept is a winner is Q4, which is referred to as the purchase intent question. The Magnum concept shown above was one of 26 concepts tested. The resulting concept scores were:
Large companies typically keep records of previous concept tests and develop benchmarks for new products. Common benchmarks include getting a top two box score of 60%, or a top box score, which is also referred to as a DWB (“Definitely Would Buy”) of 25%. Such benchmarks are rarely valid. This is because people are not able to accurately state their future behavior (how can they know how much they will like an ice cream before they have eaten it) and because many other factors determine the success of a new product, such as distribution, word-of-mouth, and so on. For example, Magnum Gold’s top box score (the proportion of people to say they would 'definitely' buy) was 14% and its top 2 box score was 38%, which are below most commonly-used thresholds, yet it went on to become the top selling ice cream in Australia. And, many other products have passed these thresholds and failed.
Although it is not unusual for companies to take the approach of launching product(s) that 'win' the concept testing, this can be dangerous as consumers have a tendency to give higher ratings to concepts that are similar to successful products already in the market. Consequently, the products with the highest purchase intent often face the stiffest competition if launched. One solution to this is to prioritise concepts based on a combination of purchase intention and uniqueness (e.g., as measured in Q6).
- For a case study see: Bass, Frank M, Kent Gordon, Teresa L Ferguson, and Mary Lou Githens (2001), "DIRECTV: Forecasting diffusion of a new technology prior to product launch," Interfaces, 31 (3), S82-S93.