Qualitative Research White Paper

Qualitative Analytics

Much has been written about how to conduct qualitative research (that is, the techniques of moderating and interviewing), but comparatively little has been published about the far more important task of analysis and reporting.

The purpose of this primer is to share some basic ideas on how to achieve the greatest learning and the most profound insights from qualitative research. The term “qualitative research” generally refers to relatively unstructured, nondirective discussions or interviews (such as focus groups, depth interviews, and ethnography or observation) to explore a topic. Typically, sample sizes are small, fewer than 50 respondents, and sampling methods are rarely scientific.

In recent years, data-collection methods have expanded from in-person focus group discussions, depth interviews, and observation to include online chats, bulletin-board forums, in-the-moment research (such as virtual ethnographies, mobile discussions and assignments) online communities, and Web monitoring. Some researchers would also include neuromarketing research (physiological measures of human reactions) as a part of the qualitative domain. As more is learned in future years, perhaps neuromarketing research can evolve into a quantitative science. The terms “moderator” or “moderator-analyst” or “analyst” will be used in this paper as descriptors for the person who conducts a qualitative project, regardless of project type.

While this article will primarily discuss in-person qualitative, most of the analytical concepts apply equally to online methods. Decisions at every step in planning, executing, and interpreting qualitative research influence the validity, reliability, and relevance of the ultimate analysis.

A Time And A Place

The first step to good analytics is to use qualitative research only when and where it is appropriate. Qualitative research techniques are completely interactive, and that allows us to attack problems with thousands of variables, whereas most quantitative research is limited to fewer than 200 variables. Qualitative research is recommended…

• When you need to understand human motives. Qualitative can answer the “why” of human behavior (for both consumers and business-to-business).
• When you are studying intricate systems or processes. For example, what are all of the steps and actions, sources of information and influence, involved in buying a new car?
• When you are planning large-scale quantitative research and need to understand the consumer’s language, perceptions, issues, and range of answer choices to help design the questionnaire.
• When marketing issues are especially complicated and confusing. The interactive nature of qualitative allows the moderator-analyst to navigate through very complex issues.
• When sample sizes are very limited. If you cannot talk to large numbers of people, then get as much information from each individual as possible.

Not All Respondents Are Equal

The analytic output of qualitative research is influenced by how respondents are chosen. Respondents are not equal in analytical value. Some are articulate, some are not. Some are knowledgeable, some are not. Some are aware of their own motives, while others are not. Some recruiting guidelines:

• Try to make the sample as representative as possible of category users, including heavy users. Heavy users tend to know much more than light users or nonusers.
• Don’t include so many screening hurdles that you end up with a very low-incidence and nonrepresentative sample. Who cares what .005% of the target audience thinks?
• Don’t do all of your qualitative research on the “focus group circuit” in Los Angeles, San Francisco, Chicago, New York, Atlanta, and Dallas. Choose markets based on target-market considerations, not flight schedules.
• Consider screening the sample in some way to ensure that respondents can communicate, but beware of introducing bias. I once discovered that cowboys in the panhandle of Texas who dip snuff do so to avoid talking, so screening for good communicators would have destroyed the greatest single learning from the groups. In general, you will learn more from better communicators than poor communicators.

Free The Moderator

Planning for the actual conduct of the qualitative research is a crucial step. Much of the analytical potential is determined at this stage. The moderator and the client must invest substantial time in sharing information and ideas to ensure the research objectives are correct and precise. This up front planning should involve all of the key stakeholders in the client company. It’s essential that the moderator understands the situation, the background, the issues, the politics, and the risks related to the project.

It’s also essential that the client gives the moderator the freedom to design the interview guide to achieve the objectives. Some clients like to dictate exactly what questions will be asked in what order, subverting group dynamics and discussion into a moderator-led “question and answer” interrogation, which greatly diminishes the reliability and analytic value of the groups. The dynamics of a free-flowing discussion reveal patterns, issues, and motivations that mere questions would never uncover. Let the moderator design the guide; give the moderator the freedom to maneuver, to deviate from the guide, depending on events in the session.

It’s critically important that the client and the moderator build trust and confidence in each other, so that the qualitative research can be conducted in an emotionally relaxed environment. A stressed or distrustful client creates a stressed moderator, who creates stressed respondents, and that inhibits the group discussion.

It’s generally best to limit the number of observers to no more than five or six. The greater the number of observers, the less anyone in the backroom will see or hear. With a large number of observers, the backroom clown or comedian will have an audience to entertain. The greater the number of observers, the greater will be the stresses on moderator and respondents.

Eyeball To Eyeball

The moderator should also be the analyst. Sitting eyeball to eyeball with respondents gives the moderator information and clues that are not visible from behind a one-way mirror. A good moderator will begin to make inferences as the qualitative research unfolds, and will test those inferences in subtle ways (by taking the opposite point of view, for example, to test how strongly people feel on an issue). It’s not always possible for the moderator to be the analyst, but at a minimum, the moderator should have inputs to the analytic process.

The Wallflower Moderator

The philosophy and style of the moderator are key determinants of analytical outcomes. The ideal moderator is soft, low-key, nonthreatening, warm, accepting, and quiet. The goal is to let respondents tell their stories with minimal interference and bias from the moderator. Most questions and most probes risk “leading the witness.” So, fewer words from the moderator mean less bias. A great moderator can guide a group discussion with very few words.

The goal of the moderator is to make respondents the center of attention. The moderator should be a wallflower. The moderator’s job is to make respondents feel totally accepted, relaxed, respected, confident, and competent. Moderators must downplay their own importance by the way they dress, how they act, and what they say. Even if an expert on the topic, good moderators will typically act as though they know nothing.

Moderators must be open to and accepting of all answers and information, and must not take anything for granted, or jump to conclusions, or carelessly assume that something is true before vindicated by sufficient evidence. Objectivity is absolutely essential. If a moderator is highly opinionated on a subject, that moderator should not be permitted to conduct qualitative research on that subject. If a client is highly opinionated on a topic, that client should not be permitted to influence the qualitative research plan or the interpretation of results.

Searching For Black Holes

As the qualitative research progresses, some of the most important dimensions involved in analytics are:

Unconscious Motives

While we all like to think of ourselves as logical and rational, the truth is we are heavily influenced by deeper instincts, genetics, and emotional forces that we are not fully aware of and don’t fully understand. These unconscious factors include our fears, sublimated sexual drives, anxieties, status needs, dependencies on others, and dominance aspirations. The moderator-analyst searches for these underlying motives like an astronomer looks for black holes—inferring their presence by studying what is happening around the edges and by measuring their influence on nearby celestial bodies.

Symbolism

As moderators study the transcripts or videos, they are sensitive to the functional or utilitarian benefits of a product, service, or brand, but also sensitive to its symbolism. For example, a dress provides functional benefits (keeps one warm, prevents sunburn, hides nakedness), but it can also express symbolic values, such as one’s social or economic status, one’s tribal affiliation, or one’s search for a new mate. All products and services provide functional and symbolic benefits, and the analyst must fully understand both. Much of modern marketing is based on understanding and exploiting products’ symbolic values.

Psychological Projection

Often we see in others what we cannot see in ourselves, or refuse to admit about ourselves. The person who accuses others of gossip mongering is most likely the gossip. Good moderators incorporate projective techniques into the interviewing process, overtly and covertly, to help reveal or confirm respondents’ deeper motives. For example, the moderator might encourage respondents to talk about their friends or neighbors, or about imaginary people or situations. The moderator might use pictures, collages, or inkblots as stimuli, or ask respondents to engage in role-playing, draw pictures, or make up stories.

Language

The analyst pays close attention to each respondent’s exact words and phrases, in much the same way as a lawyer or a detective. The details of language reveal a lot about a respondent’s motives, aspirations, emotions, behavior, biases, and perceptions. For example, if a respondent says, “I have to go to work at 8 a.m. in the morning,” that is a far different response compared to the person who says, “I go to work at 8 a.m. in the morning.” The “have to” is the difference, and the analyst must strive to understand. Does the “have to” mean that person doesn’t like to go to work or doesn’t like his or her job? Or, does it mean that respondent feels guilty about leaving the children at home?

Metaphors

We use many different metaphors as we talk. For example, when talking about “love,” we often use the metaphor of “falling.” We don’t say, “I got myself in love.” No, we say, “I fell in love.” What does the metaphor of falling tell us about love? It suggests that love is sudden, unplanned, accidental, uncontrollable, and possibly dangerous. Nicely, since love is an accident, perhaps we don’t have to accept responsibility for our behavior either.

Omissions

The analyst is not only analyzing what was said, but is trying to identify and understand what was not said. These omissions can be clues that lead to new hypotheses and new understanding.

Slips of the Tongue

Do respondents exhibit a pattern of “slips of the tongue,” (that is, using an unintended word or term)? Often these “slips” reveal deeper feelings, issues, or prejudices. For example, if a respondent mentions “Honda” by accident when intending to say “Toyota,” it reveals something about how these two brands are perceived. The analyst must be very attentive to these “slips.”

Emotions

When respondents suddenly burst out in laughter, the analyst circles back to review what triggered the outburst, and looks for the underlying causes. Often laughter is a way of dealing with stress or taboos. What were the stresses or taboos that led to the laughter, and how do these relate to the product or service under study? Tears also tell a story. Many years ago, while conducting a series of focus groups on jewelry, I was startled by female respondents starting to cry soon after each discussion would begin. As the groups unfolded, it became obvious that jewelry is more than “just glitter.” Jewelry is a symbol of, and proof of, a man’s love for his woman.

Intuition

What does the moderator “sense” while interviewing respondents? How do these respondents make the moderator feel? Are they being open and honest, or are they hiding the truth? Are the respondents aware of their motivations, or are they deluding themselves? Are these nice people, or not so nice? Are the respondents selfish, or not so selfish? Are they stylish, or not so stylish? Very often these kinds of learning can only come from actually conducting the focus groups or depth interviews. That’s why it’s so important that the moderator also be the analyst.

Observation

Fish are probably not aware of the ocean. People are like the fish, but the ocean we all swim in is our culture. Often we are unaware of exactly what we are doing and why—because we have never known anything else. A little observation can go a long way.

A study of residents at a large real-estate development in a Central American country illustrates the point. Telephone interviews from the U.S. indicated that the country was very safe and that no one felt security was an issue. When the moderator arrived to personally observe the apartment complex and interview residents, he was shocked to discover that the complex was surrounded by rows of military-type razor wire with guards posted at entrances. Respondents in the country had adjusted to living in a highly insecure environment, and they considered the situation as normal.

Observing people in their homes, or while they are shopping in a store, can raise new questions and lead to new understanding. Observation does not have to be in person. With digital cameras ubiquitous in many countries, respondents can be asked to submit photos of their refrigerators, their bathrooms, their garages, their neighborhoods, or their friends. These photos can lead to new questions and new insights.

Clues And Artifacts

There are no shortcuts, no easy ways to analyze qualitative data. True, software that purports to analyze qualitative data is available, but its value tends to be limited. No software can take the place of an observant, analytical human mind. Analysis involves listening to tapes, watching videos, and reading transcripts of respondents’ verbatim statements. Respondents’ answers are often confusing, inconsistent, entangled, and complicated. The analyst spends many days poring over videos or transcripts again and again, striving to understand what it all means. Instant analysis is no analysis at all.

As moderators read transcripts and watch videos, they are searching for clues, like a detective at a murder scene. Or, like an archeologist excavating a site, sifting for artifacts. These “clues” or “artifacts” are called hypotheses; that is, “facts” that appear to be relevant and true. These “facts” are organized into sections and logical order. Once these hypotheses (i.e., all the clues and artifacts) are logically grouped, the analyst reviews them multiple times, looking for relationships and linkages among the hypotheses and thinking deeply about what the hypotheses mean. Through a deliberate process of inductive reasoning, the analyst arrives at a set of conclusions based on the hypotheses. The last step is developing the marketing recommendations based on the hypotheses and conclusions, as well as on the industry, category, and brand knowledge of the moderator-analyst.

Dog And Pony Show

In the hands of a good moderator-analyst, qualitative research is one of the most powerful analytical tools available to researchers and marketers, but the full value of the analysis is often lost—even in the best of written reports. Bring the moderator in to do a “dog and pony show” for all the key stakeholders at the same time in the same room, so that questions can be answered, ambiguities can be resolved, and marketing implications can be discussed. The goal is for all stakeholders to leave the room with the same understanding of the results. It’s a way to help stakeholders achieve consensus and agree on next steps, which typically involve tests or experiments to confirm the hypotheses.

The ultimate value of qualitative research is determined by the quality of the analysis. Get more from your qualitative research by working with moderators who understand the power of qualitative analytics.