The speakers from Harris Interactive identified four conditions of challenging respondents:
- hyperactive (or often called professional)
- fraudulent
- inattentive
- conditioned
Their study dispelled a few myths about hyperactive respondents, for example that they are not more likely to take the survey in the first two days, that they are more likely to provide information to sensitive questions. In other words, they don’t see in hyperactive respondents any major threats to data validity. The speakers then presented methodologies to identify patterns of fraudulent survey takers and to identify inattentive respondents. They mentioned an intereting multi-indicator approached presented in 2005 by Theo Downes – Le Guin from Doxus that recommends excluding from the data set only respondents with multiple warnings. They showed interesting numbers showing how inattention depends on survey length.
Finally, they added that more research is needed as we need to understand why respondents are inattentive in the first place. In a number of cases, survey design may be to blame, not users. I cannot disagree with this: I have seen surveys that are so poorly designed that a respondent would almost be suspicious to fill it properly!
Renee and Holand ended up by introducing their new challenge of developing a quality index. I haven’t figured out yet what exactly it entails and if it is any different from a fraud score, but I am confident that we will hear more about this in the future.
0 Responses to “ESOMAR Panel Research Conference, Panel and data quality, comparing metrics and assessing claims – Renee Smith and Holand Hofma Brown ”