If you had a chance would you have tested the Obama idea in a focus group?

Posted in Uncategorized with tags , , , , , on January 27, 2009 by Cristi Popa

While we all hold our breath until someone will tell us why on earth qualitative research still exists, let’s take a look at some really funny clips about qualitative research (focus groups actually) and idea testing (may it be communication or product). The first is a parody of a prehistorically focus group on a “new idea”: fire. You must know it.

The second one is something I accidently came across here. The context is this: in 1984 Apple releases this spot:

It becomes iconic shortly after. 23 years later some clever advertiser does this clip in order to promote some advertising industry event. He makes storyboards with VO after the Apple ad and tests it in a focus group. The results are disastrous as you can see bellow.

Let’s set aside the fact that he was testing the ad with consumers from another generation so there was no chance to decode the ad as someone 25 years ago would do or for that matter let’s also ignore the fact that focus groups were not around 10 thousand years ago. Basically, both of them are saying the same thing: focus groups are not the indicated tools for testing innovative ideas or products. This idea is widely spread especially in advertising agencies. And I totally agree with it. You cannot test in a focus group too innovative stuff. It is only human for respondents to reject novelty because they get scared. Remember Galileo Galilei and the way that focus group ended. Still, how many of the commercials, products and ideas out there are that innovative? Not too many. And here’s a thought: was the Obama idea tested in groups? If so, with what results?


Quit bitching and moaning over qualitative research

Posted in Uncategorized with tags , , on January 26, 2009 by Cristi Popa

It became almost fashionable to criticize qualitative market research. The charges are various and range from the old count of lacking mathematical proof to the newer, hip ones – not thinking out of the box, not keeping up with the real world’s developments, generally not being cool and interesting enough. All these and much more are real to some extent. Still, it is a multibillion business each year, so one would think that there should be something good about qualitative research. But what are those things? So, the question is: why do you still buy it, read it, use it?

Why shouldn’t little boy-moderators play with dolls and little girl moderators with toy cars?

Posted in Uncategorized with tags , , , , on January 19, 2009 by Cristi Popa

13Bias is the Boogie man of qualitative research. It lurks everywhere around us: recruitment, stimuli, but, more importantly, moderation. Moderation can bias the results of research big time. Questions like “You like this, don’t you?”, remarks like “of course by this you mean …” are textbook examples of don’ts. But moderation bias is more insidious than that. It can originate in the tone of voice, dressing style, age, gender, body language etc. So, the little boys and girls moderators are careful to wear adequate clothes (resembling in style and price with those of the respondents), to have a similar phone, similar tastes (drink what they drink, smoke if they smoke, eat if they do), similar type of humour etc. before starting to play. Let’s say that the research agency has an IDI with a 35 y.o. housewife, mother of two, low to medium income, user of medium tier shampoos. Will it send the 25 y.o. little boy, a big fan of punk music and user only of premium shampoos which he “barrows” from his girlfriend? Or the 32 y.o. female that has a child and does her own shampoo shopping?

For the sake of the argument, let’s say that the boy is the only one available. So, the agency sends him. In this case, he will say to the respondent that he’s 28 y.o. so to narrow the gap, wind down his hair style, wear a shirt, borrow a low end Nokia from a colleague, say that he has a little nephew and generally lying his pants off to get the respondent think he’s her best friend. And, in the end, the moderator has to be a respondent’s best friend, so he has to be as similar as possible. But is that enough? Not really, because when getting to the part of usage habits, the respondent gets embarrassed to talk about how she’s getting naked in the tub. And so, there is a last thing the male moderator has to change. Fortunately for all you male moderators out there, it’s no need to. And here is why:

1.     First of all, respondents are usually not retarded. They will find it very odd that you “by accident” are so similar to them. And pretty soon they will start to challenge you.

2.     Moderating is hard enough without having to watch everything you do and say in order not to blow your cover. Giving attention not to reveal your “secret identity” will take a lot of your energy from the actual moderating process.

3.     And, most importantly, if the ideal would be to have a perfect identical moderator for each respondent, then why do we even bother to interview her. The moderator can meditate a little about the discussion topic and directly write the report.

In order to find another trap of this line of thought, let’s go back to the research agency and send the female moderator to do the interview. She is pretty similar to the respondent, as you might remember. So she has no reason to lose any energy by lying. She is practically her best friend. But, wait a minute, she still doesn’t find out what happens in that tub. What is happening? Well, the respondent is acting as a friend. And between such good friends that have so much in common there is no need for too many words. Certain complicity is born. You can spot it in phrases like “you know…when you enter the tub…” and then a smile. And the female moderator would say “well people are different so please tell me from your point of view what happens”. And the respondent replies “well it’s obvious. Aren’t you a woman too?” and so forth. The harder the moderator presses in this point, the more likely it is for the respondent to get annoyed and say anything just to get off the hook. The friendship is gone. And so is the interview.

So, what if we would drop all this similarity ideal and go for alterity? What if we would say to the respondents:  “I am different from you. I am a young man trying to understand how you wash your hair. I have no clue of how you do it, so please explain it to me in great detail. Being such a stranger to the subject will force me to ask you a tone of what may seem stupid questions, but please bear with me and make me understand”. I have tried it and it works. You wouldn’t believe the openness you are treated with after they understand your position. And that is why I know what happens in that tub.

In the end, here is a thought for all the girls and boys moderators: let’s just admit that it is impossible not to bias the data. Being alive means that we constantly interact with everything around us and by doing so we alter those things. There is no way to control all the variables coming into play in any interaction, may it be qualitative research or going to a movie. In fact, all we can do is to be aware of this and try as much as possible to take it into consideration in our analysis.


Using Twitter in qualitative research

Posted in Qualitative research web 2.0 with tags , , , , , on January 14, 2009 by Cristi Popa


One of the main points made when crucifying the focus group as a qualitative research tool is its artificiality, its way of abducting the consumer from its natural context and putting him in a moderation room. As a reaction to this shortcoming marketers resort from time to time to more ethnographical approaches, like IHV’s, AST’s etc. The idea is to get the researcher integrated in the natural context. If for focus groups the ideal is going as deep as possible, for the IHV’s is ear dropping.


In this context comes Web 2.0 with all its networking and socializing potential and short after, the idea of using it as a research tool. Add this to the marketing guru frenzy when everybody is desperate to buy a ticket to some bold, bare foot marketing gods that say the same fundamental stuff but with different wrappings, all of them announcing web 2.0 as the second coming. And so, you get a wave of enthusiasm towards research on YouTube. Twitter, blogs, Face book, Yahoo Messenger, forums, online communities etc. Now everybody can eardrop on everybody. And at this point something very interesting happens: people start thinking that this is a great tool researchers can use to tap consumers’ real feelings, wording etc. As a result, a long list of arguments in favour of stalking online communities raging from low costs to the raw, uncorrupted character of the data emerges.


What almost everybody seems to forget is exactly the background. Yes people are not inhibited by a moderator, and mirror, and camera but they are inhibited by themselves, their peers, the medium and many others. Take Twitter for example. In there you choose your crowd. Friends, people that seem to share common interests or common friends are the usual choices. On top of that, come opinion leaders, the cool people of the day and somewhat more official sources – companies. And all these birdies tweet. But they tweet with a reason – they want to be perceived in some manner (a little sparrow wants to be seen as a hawk), or they want to belong to a certain crowd (the duck trying to stay with the swans), they want to have access to the cool links etc. And all these reasons alter what they’re tweeting about, how they do it, when and with whom. And here comes the clever fox thinking that all that it has to do is to listen in on the tweets and find out where the birdies keep their eggs. So it goes to http://search.twitter.com/ and puts in “eggs”. And it sees the sparrow bragging about her big nest in the mountains, the duck saying it has no time to lay eggs as it’s occupied with facial treatments and thinks “jack pot”. But in fact it does not have a clue.


Coming back to the real world, we have to understand that all these web 2.0 developments offer us new ways of conducting research but do not simplify it to the extent of a Google search. It just provides us with more info sources which we still have to interpret against the context. So, do not think that the further away from the moderation room, the more real it is.