Qualitative White Paper

Boxers or Briefs? Pepsi or Coke? In-Person or Online?

A New Debate

Boxers or briefs? Cable or satellite? Charcoal or gas? Mac or PC? Mayonnaise or Miracle Whip? Paper or plastic? Pepsi or Coke? Mail or telephone? Telephone or Internet? In-person or online?

Well, perhaps the last three contests aren’t really debates at all—at least not to those outside of the marketing research industry. Few people outside of researchers have noticed—or cared—about evolving preferences in data collection platforms, from mail to telephone to the Internet. But in marketing research circles, the online platform came of age in the late 1990s and has since climbed to almost 40 percent of all survey research now conducted in the U.S., a $1.4 billion a year industry (Footnote 1).

Most of the online research conducted has remained quantitative, while attempts to convert the in-person experience of individual in-depth interviews and focus groups into real-time electronic correspondence and online chat rooms have often proved to be poor substitutes. In fact, many qualitative researchers scorned the new online platform (Footnote 2), since lost in the equation was the ability to observe facial expressions and body language, listen to speech patterns and intonations, instantly probe responses for clarity, build on group dynamics (ability to react to others’ comments and expand on ideas), and so forth.

Traditionalists had a point. For example, online focus groups in real time usually provided only superficial responses at best, given that average consumers rarely had enough time to type in their answers when reacting to the moderator’s questions and probes (and other participant's responses) within a 60- to 90-minute time frame.

Over time, however, improvements in software and more practical adaptations in technique have made online qualitative research a viable option (Footnote 3). The main breakthrough came from allowing extra time to answer questions. But along with more time, the online platform also offered several advantages over in-person qualitative research.

Participation Time

Moderators overcame the earlier restraints of real-time interviewing online by simply expanding the data-collection window. The paradigm shifted from an hour or two (like in-person focus groups) to days and weeks online. This extra time not only allowed participants time to think through and fully respond to questions at their convenience, but also gave them the ability to (if required) take and submit pictures, complete “homework” assignments, talk to others about a subject, draw and submit detailed diagrams of ideas, and so on.

Geographic Reach

Online qualitative researchers typically access large, voluntary, and representative respondent panels that allow respondents from distant geographies to participate simultaneously. Because the pool is bigger, participants that are screened and recruited from large online panels can often be more representative than those recruited from the databases maintained by local focus group facilities. These facilities are usually located in larger metropolitan areas and cannot access potential respondents from less populated areas of the country or outside of a 25-mile radius of the facility.

Unlike in-person qualitative, a respondent can participate seamlessly in both an individual interview and a group discussion during a single project, if desired. For instance, some studies require that sensitive topics be probed individually and deeply, while other topics receive the benefit of having a group of participants respond and build on each other’s comments (particularly beneficial when stimulating new product ideas).

As mentioned, online projects give respondents ample time to consider topics and answer. While in-person questioning often catches respondents cold, online communication provides a private, relaxed environment in which participants usually feel comfortable enough to share their innermost feelings and thoughts.

Integration With Quantitative Research

This technique is often used during or directly following an online quantitative survey. Researchers identify survey responses that they want to probe further, qualitatively. Responders are then identified and their quantitative responses are explored further during a series of open-ended questions and interactive probes.

For projects requiring a comparison of respondents across multiple markets, moderators sometimes combine in-person sessions with online qualitative. Typically a pair of focus groups (or short series of one-on-one interviews) in a single market serves as a springboard to subsequent online qualitative. This approach offers the best of both worlds, particularly for clients who want to observe respondents in person, but don’t want to puddle jump from city to city. Clients can then access the online discussion from the convenience of their computers.

Consumption Scenarios

Unlike in-person qualitative that relies on consumers’ recent experiences or reactions to artificially created shopping environments at the session, online qualitative allows respondents to report on their experiences in real-world scenarios, in actual settings, as they’re experiencing them, via online diaries and message board feedback and photographs. For instance, respondents can visit a store and buy products and share their reactions about their shopping experiences. Or they can report on their interaction with service representatives visiting their homes—while those agents are still on the service call. Or participants might visit several stores to compare the merchandising of a certain product.

Ethnographic Feedback

Ethnography has become a buzzword in qualitative research, although conducting traditional ethnography in a marketing context becomes illusory at best. Some companies have hired social anthropologists to join their groups, have sent out staff with video cameras to record their own versions of reality TV, or have contracted with boutique firms that conduct “shop-alongs,” video observation, and other forms of the discipline. In most cases, the amount of time, resources, and cost to gain the complete trust and confidence of the targeted individuals and groups, carry out bona fide participant observations, talk to “informants” (i.e., well informed insiders of a targeted group), conduct a mix of individual and group interviews, and receive critical feedback from the target audience to confirm the researcher’s accuracy of her findings, is simply prohibitive and impractical.

The online environment, however, provides a viable platform that in-person methods largely cannot do; namely, a more timely and economical form of consumer ethnography in which respondents can take and download pictures of the subject matter (e.g., show how a specified product is merchandised at stores, how a respondent stores certain items at his home, etc.) and share firsthand interpretation of those images (without being self-conscious or trying to please a shadowing videographer), providing accurate context and meaning.

Projective Techniques

While in-person sessions allow for spontaneous reactions, group-think, and discussion, most average consumers find it difficult to communicate their ideas quickly and articulately enough in front of their peers and the moderator within a few minutes of time. Hence, even standard projective techniques like role-playing often prove ineffective when trying to capture meaningful ideas within a matter of a few minutes.

In online qualitative, a moderator may pose a projective question (sometimes in conjunction with stimuli like pictures, video, audio, etc.) and allow respondents several hours or longer to reflect, brainstorm with others, and answer fully. In fact, respondents may even be encouraged to submit their own diagrams and pictures with explanations. The extra time and freedom often yields more insightful, richer dialogue than can be harvested from shorter, in-person sessions.

Remote Monitoring

Despite the allure of traveling to exotic climes, staying in lavish hotels, and enjoying fine dining (in reality, such travel usually includes only ordinary destinations and budget stays and meals), many clients, over time, simply don’t relish traveling to distant in-person groups and interviews. This aversion to travel has helped, in part, spur the growth of video-streaming and video-conferencing suppliers, though the added cost, setup, and sometimes poor quality have resulted in maintaining the status quo or sending subordinates to observe instead.

The Internet, however, has created the ability to access information and dialogue that’s available wherever there’s a Web connection. In this case, the Internet allows both respondents and clients across geographic boundaries to connect simultaneously, conveniently.

Further, online qualitative allows clients greater access to the live discussion. Rather than pass occasional notes to the moderator from the observation room, clients can submit probes, additional questions, and even change the discussion guide amid the interviewing—all from the convenience of their computers.

Verbatim Transcripts

Transcripts provide a permanent, accessible record of the qualitative sessions that both in-person observers and nonobservers can access. Verbatim quotes offer powerful, first-person insights that can spark new product development and provide testimonials for sales literature.

While most like having a permanent record of the dialogue that took place (even if some don’t have time to comb through all of it), getting written transcripts days and weeks after in-person sessions is often not worth the effort. With online qualitative, however, observers have instant access to the typed dialogue as soon as data collection starts. Moreover, because of the extra time available to participants to respond and answer questions, final transcripts from online studies are richer and more voluminous than in-person verbatims, typically more than a hundred printed pages in length.

Despite its merits, online qualitative has disadvantages, too. For one thing, it’s difficult to execute well. Conducting online qualitative isn’t simply a matter of inviting respondents, posting a few questions on a message board, producing transcripts, and reporting the results. It’s a detailed process that’s labor intensive (see diagram) and can be more expensive than in-person sessions.

For another, the advantages of being present to observe facial expressions and body language, listen to speech patterns and intonations, and instantly probe responses for clarity, simply cannot be duplicated online.

While these advantages persuade many moderators and clients to stay with traditional qualitative, for others who’ve experienced online qualitative, the rewards can exceed that of in-person feedback, depending on the application.

So, when do you order in-person qualitative and when do you order online? Simply put, conduct traditional qualitative when you want to…

• Obtain top-line results quickly (exception would be real-time online groups or interviews).
• Observe respondents firsthand in a physical setting.
• Cultivate and build on personal rapport and group interaction among participants.

Conduct online qualitative when you want to…

• Conveniently access consumers across geographies.
• Integrate several research approaches and techniques within a single study.
• Capture in-depth reaction to actual consumption tasks, behaviors, and settings as they occur.

Conclusion

While each platform has its place, online qualitative has expanded the opportunities for researchers to tap into consumer motivations. So the next time clients have to choose between in-person qualitative and online, their options now go far beyond the confines of the observation room.

Footnote:

1. Inside Research, February 2007, Issue 224, page 10.
2. Chris Mann and Fiona Stewart, Internet Communication and Qualitative Research: A Handbook for Researching Online, London: Sage Publications, Inc., 2000, page 19.
3. Susan Crichton and Shelley Kinash, “Virtual Ethnography: Interactive Interviewing Online as Method,” Canadian Journal of Learning and Technology, Volume 29 (2), Spring 2003.