Many researchers will apply a conventional research approach to a novel concept, and then interpret the results literally. I won’t do that because I’ve worked with innovative products, and I’ve seen how taking the results at face value can be a costly error.
When I worked in the video game industry, I was in charge of research for an innovative game with a unique pricing structure. Before the game came on the market, my company hired an outside research agency ("Agency X") to help us understand how customers would react to our pricing plan. With Agency X's assistance, we conducted focus groups and a survey, showing customers a written description of the game and asking about their interest in it and price expectations. We found that customers were somewhat interested in the game, but they strongly objected to our pricing plan, and almost none were willing to pay for it. Agency X recommended that we substantially drop our price to encourage sales, citing examples of other games that failed for being too expensive. Their recommendation made sense on a superficial level, but my company decided to ignore it, figuring they could always lower the price later if the game didn’t succeed and having confidence in our stellar design team.
That game, at the high price, went on to be one of the most successful and profitable games of all time. Following Agency X's recommendations would have lost us money and could have destroyed the game. In retrospect, the research had clues that hinted at the game’s immense potential, which went ignored. The biggest clue was that an unusually high percentage of customers said they planned on joining a free trial of the game, but they were certain that they would quit when the trial was over. True to their word, when the game was released, they signed up for a free trial. However, instead of leaving, they got hooked on the game's unique style. They realized that they couldn't live without it, so they ended up paying more than they ever expected, much to their own surprise.
Failures occur when researchers assume that customers can be fully trusted to evaluate products that are beyond their current imagination. Taking this experience as a cautionary tale, I always look for the deeper clues, like the high free trial interest numbers, that may be telling a different story about an innovative product.
