Patrick Glaser from CMOR asked me to participate in a message board about non-response in panel research.
For those of you not familiar with the differences between all US research organizations, CMOR is a non-profit organization, established in 1992 by the ARF, CASRO, MRA and AMA to encourage citizen participation in public opinion and survey research. CMOR’s mission is to improve respondent cooperation, as shows their launch of the “Your opinion counts” label.
According to Patrick, “the purpose of this forum is to discuss whether or not it makes sense to develop outcome rates for panel research, and if so, what they should reflect”. He opened the thread with this question: “(What are) your initial thoughts as to whether measures/formulas for non-response in panels would be meaningful (why/why not)?”
See below my reply:
I would say, yes if the formula includes the “panellization†effort, no if it’s only a response rate calculated out of a managed/scrubbed list of active respondents, which I would call click-through rates.
I agree with Daniel Slotwiner that non-response needs to incorporated at all levels, in particular at the recruitment level. I salute Knowledge Network’s calculation of response rates and their transparency, and I don’t think that 10% is low at all. But if KnowledgePanel’s recruitment methodology makes it possible to include such recruitment metric, KnowledgePanel’s probability panel is an exception in the industry.
The difficulty is I don’t see how to apply a similar definition to non probability online samples. I don’t think that adding the registration path adds much to the click-through rate metric – recruitment is the key. Note that I talk about “non probability online samplesâ€, not “volunteer†or “pull†samples.
Most forms of marketing (online or offline) include some “push†component. Banners, e-mail marketing, even co-registrations are “push†marketing. Online panels, reward programs and communities, which compose the “online sample market†all use “push†recruitment methodologies. Of course, some pushes are more effective and disruptive than others. If some knocks on your door, calls you at home and intercepts you in a mall while you are shopping, those are stronger pushes and easier to measure than exposure to online marketing, but “pull†or “volunteer means something else.
The term “Pull†online marketing can be applied to search engine optimization or marketing. If you get respondents from your study because your polling site is well positioned on Google on search terms such as “paid surveys†(although one could argue that organic searches come from the awareness of such programs, which are consequences of previous “push†campaigns…), you have a “pull†panel. If some online sample sources use this method as part of their marketing plan, this constitutes only a minor fraction of the recruited population.
I would also be careful about associating the term “pull†and the notion of “river†sample. I would tend to believe that, on the contrary, “rivers†intercept willing participants and send them directly to a survey instead of a panel registration page, and are consequently more “pushy†than traditional online recruitment.
You can read the thread here.
0 Responses to “CMOR online discussion: non-response in panel research”