Advertising Effectiveness

by: Jerry W. Thomas

The advertising industry, as a whole, has the poorest quality-assurance systems and turns out the most inconsistent product (their ads and commercials) of any industry in the world. This might seem like an overly harsh assessment, but it is based on testing thousands of ads over several decades.

In our experience, only about half of all commercials actually work; that is, have any positive effects on consumers’ purchasing behavior or brand choice. Moreover, a small share of ads actually appear to have negative effects on sales. How could these assertions possibly be true? Don’t advertising agencies want to produce great ads? Don’t clients want great advertising? Yes, yes, they do, but they face formidable barriers.

Unlike most of the business world, which is governed by numerous feedback loops, the advertising industry receives little objective, reliable feedback on its advertising. First, few ads and commercials are ever tested among consumers (less than one percent, according to some estimates). So, no one—not agency or client—knows if the advertising is any good. If no one knows when a commercial is good or bad (and why), how can the next commercial be any better? Second, once the advertising goes on air, sales response (a potential feedback loop) is a notoriously poor indicator of advertising effectiveness because there is always so much “noise” in sales data such as competitive activity, out-of-stocks, weather, economic trends, promotional influences, pricing variation, etc. Third, some of the feedback on advertising is confusing and misleading: agency and client preferences and biases, the opinions of the client’s wife, feedback from dealers and franchisees, complaints from the lunatic fringe, and so on.

Barriers to Great Advertising

Advertising testing could provide a reliable feedback loop and lead to much better advertising, but many obstacles stand in the way. The first great barrier to better advertising is self-delusion. Most of us believe, in our heart of hearts, that we know what good advertising is and that there is no need for any kind of independent, objective evaluation. Agencies and clients alike often think that they know how to create and judge good advertising. Besides, once agencies and clients start to fall in love with the new creative, they quickly lose interest in any objective evaluation. No need for advertising testing. Case closed.

Strangely, after 40 years of testing advertising, we cannot tell you if a commercial is any good or not, just by viewing it. Sure, we have opinions, but they are almost always wrong. In our experience, advertising agencies and their clients are just as inept at judging advertising as we are. It seems that none of us is smart enough to see advertising through the eyes of the target audience, based purely on our own judgment.

A second barrier to better advertising is the belief that sales performance will reveal if the advertising is working. Unless the sales response to the advertising is immediate and overwhelming, it is almost impossible to use sales data to judge the effectiveness of the advertising. So many variables are beyond our control, as noted previously, that it’s impossible to isolate the effects of media advertising alone. Moreover, some advertising works in a few weeks, while other advertising might take many months to show positive effects, and this delayed response can confound our efforts to read the sales data. Also, advertising often has short-term effects that sales data might reflect, and longterm effects that most of us might easily overlook in subsequent sales data. Because of these limitations, sales data tend to be confusing and unreliable as indicators of advertising effectiveness.

Sophisticated marketing mix modeling is one way to measure these advertising effects on sales, but it often takes millions of dollars and years of effort, and it requires the building of pristine databases of sales information along with all of the marketing-input variables. Few companies have the budget, the patience, the accurate databases, and the technical knowledge necessary to succeed at marketing mix modeling. Even so, marketing mix modeling does not help us evaluate the contribution of a single commercial, but rather the cumulative effects of many different commercials over a long period of time. Also, marketing mix modeling does not tell us why the advertising worked, or why it failed to work. Was it the message, the media weight, or the media mix that made the advertising effective? Generally, marketing mix modeling cannot answer these types of questions. So, again, sales data is of limited value when you make critical decisions about your advertising.

A third barrier to better advertising is a pervasive tendency of many (but not all) advertising agencies to delay, undermine, and thwart efforts to objectively test their creative “babies.” Who wants a report card on the quality of their work? It’s very threatening. The results can upset the creative folks. The results can upset clients. The agency can lose control. Agencies can be quite creative in coming up with reasons to avoid copy testing. Some of our favorites:

There’s no time. We have to be on air in five days, so we’ll just have to skip the testing.
These ads are built on emotion and feelings, and you can’t measure such delicate, artful subtleties.
We already tested the ads with a focus group during the development process.
These are image ads, and you can’t test imagery with standard advertising testing techniques.
We have so much equity in this campaign that it doesn’t matter what the testing results are. We can’t afford to change.
We’re in favor of testing, but let’s remove those questions about purchase intent and persuasion from the questionnaire.
We are in a new age, with new media and new messages, and none of the old copy-testing measures apply any more.

The fourth barrier to more effective advertising is the big creative ego. The belief that only the “creatives” in the agency can create advertising—and the conviction that creativity is their exclusive domain—constitute a major barrier. Great advertising tends to evolve over time, with lots of hard work, fine-tuning, and tinkering, based on objective feedback from target consumers. Big creative egos tend to resist such evolutionary improvements. We have seen great campaigns abandoned because agencies would not accept minor tweaks to the advertising. To be fair, big egos are not limited to advertising agencies. Big client egos can also be a barrier to good advertising. Research firm egos are yet another problem. Big egos create barriers because emotion is driving advertising decision-making instead of logic, reason, and consumer feedback. Big egos lead to bad advertising.

A fifth barrier to better advertising is the widespread belief that one’s major competitors know what they are doing. Just copy the advertising approaches of the competition and success will surely follow. We recently had a client who was about to copy the advertising strategy of a major competitor, but we were able to persuade the client to test all major competitive commercials as a precaution before blindly copying the competitor’s advertising approach. This competitor was the industry leader in market share and profitability. Our testing quickly revealed that this industry leader was the industry leader in spite of its bad advertising. The testing also revealed that another competitor, in contrast, had great advertising. Needless to say, the client’s desire to copy the industry leader quickly vanished.

The sixth barrier to better advertising is lack of strategy, or having a poor strategy. The client is most often at fault here. The client has not done his homework, has not thought deeply about his brand and its future, and has not developed and tested strategy alternatives. The client tells the agency to go forth and create great advertising, without providing any strategy guidelines. The agency is left to guess and speculate about strategy. Great advertising is rarely created in a strategy vacuum. If the client cannot define a sound strategy, the agency cannot create great advertising. Again, the responsibility for strategy falls squarely on the client.

A seventh barrier to better advertising is client ineptness. Some clients’ processes, policies, and people tend to discourage the creation of great advertising. Arrogance, ambiguity, impatience, ignorance, risk aversion, and inconsistency tend to be the hallmarks of these “agency killer” clients. Bad clients rarely stimulate or tolerate great advertising.

The eighth and last barrier to better advertising is poor copy testing by research companies. Many advertising testing systems are limited to a few markets (and therefore cannot provide representative samples). Some systems are so expensive that the cost of testing exceeds the value of the results. Research companies have been guilty of relying on one or two simplistic measures of advertising effectiveness, while completely ignoring many other very important variables. For instance, for several years research companies argued publicly over which was more important—persuasion measures or recall measures? The truth is that both are important, but of greater importance is the fact that neither of these measures alone, or in combination, measures advertising effectiveness. To judge the effectiveness of an ad, many different variables must be measured and considered simultaneously.

Creating Better Advertising

Given all of these barriers to better advertising, how can a client, agency, and research company work together to create more effective advertising?

The client must craft a sound strategy for its brand, based on facts, not wishful thinking and self-delusion. The client must carefully define the role of advertising in the marketing plan and set precise communication objectives for the advertising. What exactly does the client want the advertising to convey, to accomplish? Agencies are too often asked to create advertising in an informational vacuum. Agencies are not miracle workers. Once strategy and positioning alternatives are identified and tested, the strategy should be locked down—and rarely changed thereafter.
As creative executions are developed against the strategy, each execution should be pretested among members of the target audience. (“Pretesting” refers to testing advertising before it is aired and/ or before final production. When the term “testing” is used in this article, it is a shorthand term for “pretesting.”) The greater the number of executions tested, the more likely it is that great advertising will emerge. Testing the creative provides a reliable feedback loop that helps agency and client alike become smarter over time. Once a conceptual family of commercials is identified as the optimal campaign of the future, then the campaign should be locked down. Long-term continuity of an advertising message is essential to maximizing effectiveness.
Use the same testing system consistently. There is no perfect advertising testing system. Some are better than others, but any system will help improve your advertising. The secret is to use one system over and over, so that everyone (client, agency, and researchers) learns how to interpret the testing results for the category and the specific brand.
If budget permits, test the advertising at an early stage in the creative process (i.e., the “storyboard” or “animatic” stage) and also test at the finished commercial stage. Earlystage testing allows rough commercials to be tweaked and fine-tuned before you spend the big dollars on final production. Early-stage testing tends to be highly predictive of finished commercial scores, but not always. Testing the finished commercials gives you extra assurance that your advertising is “on strategy” and working.
Build your own “action standards” over time. As you test every execution, you will begin to learn what works and what doesn’t work. Think of the testing company’s norms as very crude, rough indicators to help you get started with a testing program. But, as quickly as possible, develop your own norms for your category and your brand (yes, all of the advertising effectiveness measures vary by product category and brand). Long-term, what you are searching for are not norms, but action standards (that is, the knowledge that certain advertising testing scores will translate into actual sales increases).
Use a mathematical model to derive an overall score for each execution. It doesn’t matter that an ad has great persuasion if it does not register the brand name. It doesn’t matter that an ad registers the brand name if no one will notice the commercial itself. It doesn’t matter that an ad increases short-term purchase interest if it will damage the brand’s quality reputation over time. So all of the key variables must be put together intelligently to come up with a composite or overall measure of advertising effectiveness.
Use the testing results as a guide or as an indicator, but do not become a slave to the mathematical model. Read all of the open-ended questions carefully. Make sure you really understand the underlying reasons. Base your decisions on this comprehensive assessment of the results, and leave yourself some wiggle room. No model or system can anticipate every marketing situation or give a 100% perfect solution every time. Informed human judgment remains important.
Clients and agencies need to accept that “continuous improvement” of the advertising is an important goal. This means that every execution is tested and tweaked based on scientific evidence from the target audience. We are not talking about changing the strategy or changing the campaign, but making sure that every execution is “on strategy” and working as hard as possible.
The ultimate goal of testing is an advertising success formula that works. That is, the goal of advertising creative development, and the goal of advertising testing, is to identify the elements/ ideas essential to advertising effectiveness, and then to make sure that those elements/ ideas are consistently communicated by all advertising executions.

The Power of Advertising

We believe in the power of advertising, based on thousands of studies in our archives. Advertising has the power to persuade, the power to influence the mind, and the power to shape destiny. It has the power to change markets and improve profit margins. Advertising has short-term power (conveying new information, building awareness, enhancing credibility, etc.) and long-term power (conveying brand image, attaching emotional values to the brand, building positive reputation, etc.). The great power of advertising is seldom achieved in practice, but we can’t give up. The potential and the promise are too great. The companies that master the creative guidance and the testing systems to consistently develop and deploy great advertising will own the future and the fortunes that go with it. Great advertising is a cloak of invincibility.

Author

Jerry W. Thomas

Chief Executive Officer

Jerry founded Decision Analyst in September 1978. The firm has grown over the years and is now one of the largest privately held, employee-owned research agencies in North America. The firm prides itself on mastery of advanced analytics, predictive modeling, and choice modeling to optimize marketing decisions, and is deeply involved in the development of leading-edge analytic software. Jerry plays a key role in the development of Decision Analyst’s proprietary research services and related mathematical models.

Jerry graduated from the University of Texas at Arlington, earned his MBA at the University of Texas at Austin, and studied graduate economics at SMU.

Advertising Research Whitepaper