|
|
Marketing research is grounded in the scientific method: answering questions by generating a priori hypotheses, collecting data to test hypotheses, and analyzing data to draw conclusions. Adhering to the rules of the scientific method is important to ensure that results are valid and unbiased.
|
|
Sometimes marketing researchers are tempted to use undesirable methods, like conducting many single significance tests, performing statistical tests without hypotheses, and rerunning statistical tests until desired results are discovered. Unfortunately, engaging in these methods has unintended, detrimental consequences: namely, an increase in Type I Error.
What is Type I Error? Type I Error is equivalent to a false positive. It occurs when we believe we have found a significant difference when there isn’t one. Typically, researchers set Alpha (α) to .05, limiting the chance of making a Type I Error to 5%.
Often, researchers do not realize that Type I Error is influenced by the number of statistical tests conducted. Specifically, the probability of making a Type I Error increases with every additional statistical test (correlation, regression, t-tests, etc.) that is performed. This is known as the Familywise Error Rate. For example, if we conduct one significance test, we have a 5% chance of finding a significant difference when there isn’t one—because we have set α = .05. If we conduct three significance tests with an alpha equal to .05, the probability that we will make at least one Type I Error increases to 14.3%1. If we examine ten significance tests, the probability that we will find a significant difference when there isn’t one rises to 40%.
|
Contact Us
|
604 Avenue H East
Arlington, Texas 76011
United States
817-640-6166
|
Follow Decision Analyst
|
|
|
|
|
If you were forwarded this newsletter and would like to have your own subscription subscribe here.
Otherwise, you received this email because you signed up for Decision Analyst newsletters at DecisionAnalyst.com. If you do not want to receive any future Decision Analyst marketing emails, please unsubscribe or write to us at Decision Analyst, Privacy Center, 604 Avenue H East, Arlington, TX 76011.
Copyright © 2020 Decision Analyst, Inc. All rights reserved.
|
|
