L D Weiss, Inc.

Product Development Consulting



Larry Weiss


Greenbook article Sept. 2015: Can Researchers Trust Online Access Panel Data?

Larry's comment:

I believe the online research quality problem is less due to multiple panel participation and more to respondent cynicism at long, tedious "fill-in-the-form" interviews chock-a-block with badly worded or overly long and "correctly-worded" prompts, ambiguous or irrelevant answers, forced response when no response is appropriate and the dreaded grid questions. Worse, the respondent is neither given feedback as the in-person interviewer does with engaging head nods and "uh-huhs," nor is the respondent asked for her opinion on whether this interview was enjoyable and captured her opinions accurately and fully. Most researchers don't have the guts to ask it.

My business is developing new products, not research. I need honest and accurate answers to relevant questions. I insist that every interview gives respondents feedback as they go, calculates evaluative scores real-time, shows respondents their score and what it means, asks for their agreement or disagreement and why, and why they rated as they did. And, at the end, asks for their wrap-up evaluation of the interview itself.

A core problem is researchers and clients are stuck in the paper and pencil paradigm for online research. Try moving "back" to the old oral conversation paradigm of in-person interviewing. Respondents hate to "fill out forms" and love when the interview "talks interactively" with them, eliciting their opinions in an engaging way. I know. I've been slammed by respondents when I did it stupidly, and cheered when we got it right. After enough real research on the research with the interviewee, you learn to keep interviews under six minutes, full of feedback, conversationally engaging and pleasant for the respondent.

There is hope.