Sunday, December 06, 2009

Why Email and Incentives Hurt Market Research

Poor practices surrounding the utilization of incentives with online email panels is hurting our industry.

Email panels for purposes of market research have grown immensely over the past decade. These panels are easy to build, have little to no barrier to entry and these mega panels seem to dominate the industry.

The concept is simple: recruit individuals to provide their email address and fill out a comprehensive profiling survey. When a suitable survey is available, panelists are sent an email and have the option to fill out the survey. The panel manager charges for each complete and provides the panelist with some incentive for their time and efforts.

This is the beginning of a dangerous, downward spiral in our industry. A primary weakness of this methodology is its strong tie to email. Today people are bombarded with tons of email from friends, family, special offers, surveys and spam. As a result, the attention span for an email is very subject sensitive.

Massive panels have been developed with hundreds of thousands or millions of members. At one point the size of a panel was an indicator of the quality and reach of the panel. In reality, the number of active panel members is a fraction of the total members.

Poor research practices, mass panels, multiple cross-panel memberships, and the rise of professional survey takers have given surveys the appearance of spam. As a result, response rates and panel cooperation rates have dropped, and the cost of building an effective panel has increased. To try and combat this situation, panel managers are forced to provide incentives for respondents to complete a survey.

The deadly spiral continues. As participation rates drop, the natural reaction is to increase the provided incentive. As the cost to administer surveys and incentives increases (which can potentially account for 80% of total project costs), the opportunities to conduct research become more limited and costly. Due to this economic and opportunistic crunch, many market researchers are forced to run fewer studies but require more information per respondent leading to long cumbersome surveys. The longer the survey, the higher incentive that is required, and the more restrictive a project becomes. Thus a deadly spiral continues where researchers are forced to try and get more information from fewer respondents, ultimately hurting the industry as a whole.

20, 30 and 40 minute surveys are not uncommon, and, in my opinion, are ridiculous and do not belong online. There are always exceptions but this is my general rule. Conduct a 30 minute online study and a million member panel all of a sudden becomes quite small and expensive.

Much of this momentum was created because of the reliance on email as the primary delivery method. Email is on the decline; we have an emerging generation that views email as a medium for forwarded jokes and mobile text as a medium for meaningful communication.

There are solutions and there are opportunities but only in the past few years have we started seeing these explored. If the industry can conquer it's dependency on email and move to "less information from more people" (this means shorter surveys), I suspect participation or success rates will increase as will the opportunity to conduct more research because the cost of conducting each research project will be decreasing.

There is a place for incentives and for online email panels. We just need to be very careful and aware of what's happening.

No comments: