Online Research White Paper

Best Practices For Private Online Panels

Over the past decade, many corporations have set up private online panels or online communities as an economical way to conduct surveys and qualitative projects.

A “private research panel” (sometimes called a “custom panel” or “customer panel” or “proprietary panel”) is created by a company solely for its own use. The company recruits and maintains the online panel and uses the panel exclusively for its own research purposes. While many private online panels are a success, many are deemed failures. The purpose of this article is to outline some guidelines and best practices for private online panels.

Private Panel Versus Online Community

A private online panel is sometimes confused with an online community. Typically, a private online panel is made up of thousands of participants, while an online community tends to be much smaller (i.e., hundreds of members). A private panel tends to have a long life; it can be successfully operated for years. A community, however, is often limited in life (maybe a few months or a year) to minimize the risks of respondent conditioning. Private panel members are rarely permitted to interact with other panel members, but an online community might allow its members to interact with each other. Online communities tend to be less expensive to operate than private panels because the number of participants is smaller, and the same community members are used over and over again. Sampling controls are minimal for communities, but they can be quite rigorous for private panels. As a best practice, it is probably wise to think of communities as qualitative research and to think of private panels as quantitative research (although both can reverse roles at times).

Private Panel Rationale

The growth in the number of private panels is largely a function of a slow-growth world economy and the corresponding budget pressures within large corporations. It is the lure of cost-efficiencies that prompts most companies to consider private panels. Under optimal conditions, a private online panel can reduce the cost of a typical research project by 25% to 35%.

Falling response rates to online commercialaccess panels is also a major reason for the adoption of private panels (where response rates tend to be higher). Another major factor is category incidence. If a company’s product is used by only 1% of the population or less, then a private panel of customers might be the only affordable way to do research. Another consideration is the degree of consumer interest in the product category. High-interest product categories are generally better candidates for online private panels (but even low-interest categories can often benefit from private panels).

Types of Research

Private online panels are not appropriate for all types of research. If you are trying to survey a representative sample of all adults, then the private panel is of limited value since it most likely contains only your customers or those interested in your brand, subsets of the total adult population. Many studies, however, can be conducted among your customers or those interested in your brand; it’s not a perfect sample, but often it is good enough to help you make the correct decision. It is also possible to recruit noncustomers into private panels, but the recruiting costs for noncustomers can be quite high. Here are some examples of studies commonly conducted via private online panels:

Remember, the basic assumption is that most private panels are largely made up of customers. Private panels generally cannot be used for research studies that require a representative sample.

Should You or Shouldn’t You?

One of the first questions you must ask is, “Will my company really use the private panel once it is set up?” We have set up a number of private panels for clients with big research plans that never materialized. Each one planned to use their private panel frequently, but the burdens and mechanics of proposing studies to internal audiences, getting approvals, preparing purchase orders, writing questionnaires, pulling samples, tabulating the results, and trying to write reports bogged the whole process down. Therefore, few studies were actually conducted, and these private panels were never economically viable.

The economic feasibility of private panels all depends on the financial details. Every product category is different, and you’ll just have to sit down and do the math for your particular situation. As a rule, if you don’t conduct at least one research project per month, a private panel will rarely make economic sense. That is, the expense of setting up and maintaining the private panel will be greater than the savings on the research studies.

Another consideration is the subject matter of your research. If you want to set up a private panel of golfers or pilots or sailboat captains or motorcycle owners, a private panel will work like a charm because the respective panel members are very interested in the categories. It would be easy to recruit panel members, and participation rates in surveys would be high.

Conversely, if you want to set up a panel of insurance owners or water-utility customers or banking customers (comparatively lowinterest categories), it will be much more difficult to recruit panel members, and response rates to surveys will be low. High-interest product categories are the best candidates for private panels.

Lastly, the lower the incidence of your product category, the greater the potential savings from a private panel is likely to be. For example, if your product category happened to be experimental aircraft, the percent of the population (i.e., incidence) of those who build experimental aircraft is far less than 1% of the U.S. adult population. If you had to screen large probability samples of U.S. adults to find experimental-aircraft builders, every study would cost a small fortune. But, if you owned a private panel of these aircraft enthusiasts, studies could be conducted at modest costs.

Advantages

If a private online panel is set up and managed properly, its advantages are:

• Better Decisions. More marketing questions can be addressed and answered, based on objective consumer feedback.
• Economy. Surveys via private online panels typically save 25–35% versus the price of traditional online surveys. The savings are even greater for low-incidence categories.
• Speed. Most surveys are turned around in 14 days (from questionnaire design through final written report). Complicated projects take longer, of course. Having the private panel in place (and all the paperwork and approvals taken care of) means that a company can launch a survey almost instantly in the event of an emergency or urgent need.
• Accuracy. Private online panels can provide reasonable balanced samples (demographically and geographically).
• Response Rates. Private panels tend to enjoy much higher response rates, compared to commercial online access panels. This could be a greater advantage in the future if response rates for commercial panels continue to fall.
• Longevity. Private panels can last for years if new panelists are added on a regular basis and older panelists are removed from the panel.

Best Practices

Companies tend to reap the greatest benefits from a private online panel when it is properly managed and its use is encouraged. Best practices are:

• Assign one person to be responsible for scheduling, expediting, and coordinating with internal groups or brand teams so as to keep research projects moving ahead.
• Consider quarterly omnibus surveys where each group or brand can submit a question or two.
• Charge the private panel’s cost to a central marketing budget so that it is “free” to individual brands or groups. If the studies are “free,” brands are eager to do as many studies as possible (and thereby make better decisions).
• Simplify the approval process so that a group or brand team can launch a private panel survey with minimum paperwork (no purchase orders, no elaborate approval processes).
• Standardize often-repeated types of studies so that the organization learns over time what the answers mean.
• Create a database of normative data for the standardized studies so that results have greater value and meaning.

Internal Staff Versus Outsourcing

If you decide to set up a private panel, should you do it yourself with internal staff, or should you outsource it to a research firm with “private panel” experience? The larger your company and the greater the number of projects, the more likely it is that you should staff up and do it yourself. If you plan to do more than 100 or 150 research projects a year, then building a professional internal staff might make sense. Remember, you will need to hire some individuals with researchsupplier “operations” experience—folks who really understand the nuts and bolts of panel management, sampling, questionnaire design, coding, tabulation, quality assurance, data security, and email deliverability. Also, your company will need an array of software to support the operation of a private panel. If your company is likely to use a private panel for fewer than 100 projects a year, you should probably outsource the private panel to a research company with expertise in private panel development and management. The research firm will manage your panel, answer emails and phone calls, help keep projects on schedule, provide all the software and programming, design questionnaires, conduct the surveys, and write reports for your company.

Purchasing a turnkey private panel service frees you up to be the research consultant within your organization, rather than the research mechanic. You can operate with minimal staff, since most of the work is performed by your private-panel partner.

Recruiting Tips

Recruiting a private panel can be a challenge. The best way is to collect customer email addresses religiously—at every point of contact with customers. Then you will have a massive database of customer email addresses from which to recruit your private panel. If you do not have customer email addresses, then panel recruiting tends to be expensive. You can insert panel invitations in packages. You can mail letters to targeted groups inviting them to join. You can purchase opt-in email lists. You can post a link on your website. You can send out publicity releases announcing your panel. The best methods of recruitment vary by product category, and you will have to experiment to identify the most cost-effective methods.

Threats and Risks

Once you start recruiting your private panel, you will need to be careful that a few panel members don’t join your panel multiple times and/or complete the same survey more than once. A best practice is to use “digital fingerprinting” to identify duplicate respondents. A major risk is the potential liability resulting from hackers gaining access to panelists’ personally identifiable information. A best practice is to encrypt all panelists’ data at a high level of security. If you send large numbers of emails to panelists via a corporate email system, you might find some Internet service providers (ISPs) shutting down delivery of your corporate email. A best practice is to set up a completely separate system to handle private panel emails. Even so, there is always a risk that some ISPs will block delivery of emails to panelists, so it is important to track email delivery and response rates by ISP.

Another risk is panel attrition. If the panel is not nurtured and used, panelists will quickly lose interest and drop out. Best practices are to send birthday cards and other communications to panelists, answer all emails and phone calls promptly, and use each panelist 10 or more times per year. Respondent fatigue is also a risk if questionnaires are long and tedious or poorly designed. A best practice is to keep questionnaires short and concise. Longer questionnaires are possible for some categories, and incentives can make longer questionnaires acceptable to participants.

One final risk in some countries revolves around accessibility. If your corporate communications promise equal access to all, regardless of their disabilities or handicaps, then your surveys must be accessible (a very high standard to meet). Legal risks can be significant in these instances.

Caveats

A private online panel can be a useful tool in your research toolbox, but it is just one tool. You will still need professional research agencies to help your company with studies that require probability samples (market segmentation, benchmarking, volumetric forecasting, etc.). You will need research suppliers to help with centrallocation studies, in-person interviewing, and qualitative research. You will probably still need professional research firms to help with very complex studies (choice modeling, segmentation, optimization, R Language statistics, etc.). You will also need professional research companies to help with major strategic projects where objectivity and outside perspective are crucial. However, for the day-to-day tactical studies among your customers, private online panels may be an option to consider.