I am sorry for my light blogging lately, but the recent big swell that hit California, a very interesting WOMMA Research Symposium and a bad cold kept me away from my laptop. So, let’s resume my summary of last month’s ESOMAR Panel Research conference.
Some of you must be wondering “what the hell does conditioning mean, and why should I care?” Well, if you are using online panels for market research, you should really care. So what is conditioning? Trixie used Buck’s definition of conditioning “a change in measured consumer behavior due solely to the act of measurementâ€. Let’s take a simple example: if I show you today a trailer for the movie “Dreamgirlsâ€, and ask you tomorrow which movie you are aware of, you will be more likely to be aware of the movie tomorrow than someone who has not seen the trailer. Who knows? You might even go and see the movie because of the survey you just took. Conditioning has probably always been the biggest challenge for panel-based research. This has become even worse in this world of online panelists belonging to tens of different online panels and being submitted to all sorts of stimuli.
A lot of researchers look at “hyperactive respondents†or “professional respondents†as the major challenge for research, but most fail to realize that the underlying issue is actually conditioning. To avoid conditioning, most researchers use what we call “past participation screenersâ€. Those are screening questions that rely on the participant to remember surveys they may have taken over sometimes a very long period of time, and the category of product that was discussed. These past participation screeners are not only questionable because of their declarative nature, but they are often so obviously leading (“have you taken a survey about alcoholic beverages in the past three months†yes/no) that only candid survey takers are screened out when professional survey takers understand that they have to answer “no†to continue. I will probably come back to the question of past participation screeners in the future, but let’s go back to Trixie’s paper.
Trixie showed that conditioning is a complex phenomenon. She showed that conditioning is negligible with a “low frequency†of re-interviewing (unfortunately not made public) sometimes more problematic with a “high frequency†interviewing. But he examples she gave were surprising and illustrated the fact that there may be several opposite forces at play which may neutralize (or not) each other. For example, she described the effect of “learning†(the respondent is now introduced to criteria he has not thought about before) that leads a respondent to know more brands, which can be countered by the effect of “boredom†(respondents are tired of talking about this topic or don’t mention brands because they know that the more brands they mention, the more questions they will have to answer in the questionnaire, etc). Other effects (“stimulus†effect, “interactive†effect etc) were not clearly demonstrated, but using different categories of product, different frequencies or different stimuli may result to different conclusions.
I left the presentation convinced that this is an area that needs more research on research, and probably more cooperation between research providers. If you have conducted any research on that field, I would be happy to hear about your findings.
0 Responses to “The effect of conditioning when re-interviewing, Trixie Cartwright (TNS UK)”