Consumer Insights Blog

Insights, Not Oversights!

(A Market Research Checklist)

As market researchers, our goals and responsibilities are to deliver thoughtful, accurate, data-driven insights to our partners. To do this successfully, we should always ask ourselves these questions:

• Is this methodology right for the audience, topic, and objectives? Sometimes the objectives drive the methodology, but sensitivity to the topic and the needs of the respondents should also be kept in mind. Many times, a mixed methodology (such as qualitative with a homework assignment like a diary or a video log) or a multiphase approach (with qual before or after quant, or a stakeholder workshop prior to the research) is what will work best.
• Am I being too narrow or making assumptions about my target population? Do not limit the scope of the research based on assumptions, old data, or old processes. Surveying your current (perceived) target or biggest buying group doesn’t help to expand your market. Including nonusers in your sample can provide insights into the product’s limitations that you might have otherwise overlooked. (Read more about rediscovering your customers.)
• Am I screening and including a representative set of respondents? Ensuring that you are surveying the correct group is important. If the target population is unknown, use the initial screening completes to identify the target population at its population-appropriate levels, as well as to set and adjust quotas for your final data set. If you or your partner are not sure if more males than females use their product, loosely set quotas to allow for males and females to fall naturally when sending out nationally representative sample. This allows you to identify what the product category demographics look like out of a balanced, nationally representative sample. (Read more about survey sampling plan designs.)
• Is the questionnaire composed of clear, well-framed questions? Overall, questions should be simple to read and understand. They should be concise enough to avoid confusion but with enough context to disallow multiple interpretations. Some questions may require a timeframe or other level-setting qualifier. Industry terms and acronyms should generally be avoided or else thoroughly defined. And be sure to ask a variety of question types – key metrics, diagnostics, and open-ended feedback (an open-end is often most helpful and important after a key metric, such as asking why the respondent would or would not be likely to buy).
• Is the research instrument as short and engaging as possible? Research on research shows us that keeping surveys short and well-flowed will not only lead to better, cleaner data, but it will also help to maintain a positive relationship with respondents. While it seems harmless to add an extra question or two at the end of a 15- or 20-minute survey, stretching the survey length beyond 25 minutes fatigues respondents, possibly causing them to provide invalid or less thoughtful responses. (For instance, open-ended questions towards the beginning of the survey tend to garner longer and more interesting responses.) Asking the most important questions early is helpful, as is asking questions like grids that may be harder for a fatigued respondent to answer at the 20-minute mark. This is why easy questions like demographics are often last in a survey. Lastly, it is helpful to use a variety of question types instead of page after page of the same thing, especially when those are grids! (Read more about how you can maximize survey results with no grid questions.)
• Is the data I am collecting clean and reliable? Every data set should be reviewed for outliers, bots, fraudulent respondents, speeders, and cheaters before the end of data collection (so those found can be replaced with valid respondents). Reviewing survey data and removing bad respondents (including those providing poor or nonresponsive open-ended answers) ensures that the insights are reliable and valid. (Read more about sampling methodology and the sampling industry landscape.)

With years of experience, it can be easy to become engrained in our processes and begin to overlook the many details that ensure a successful market research project. This list just serves to remind us all to ask the important questions to ensure that every research initiative is efficient and effective in yielding reliable, high-quality insights to drive important business decisions.

Stephanie Trevino

Research Director

Stephanie has nearly 15 years of experience in the marketing research field. She is skilled in both qualitative and quantitative research for both consumer and B2B segments. Her experience includes panel and respondent management, project management, data quality assurance, questionnaire design, reporting, and data visualization. Stephanie holds a Bachelor of Arts from Texas State University and a Master of Business Administration from the University of Texas at Austin.

• 
•