Market Research White Paper

Competitive Best Practices:

Win-Loss Evaluation Research

Successful Win-Loss research programs are built around a well-tailored research tool that collects crucial information from decision makers and influencers who are involved in the sales decision process.

The overall goal of the research is to determine what factors are used as decision criteria in selecting a company for a project. Win-Loss research not only provides a valuable source of competitive intelligence, it also sheds light on the internal working of a company’s sales processes and cycles. Win-Loss research can:

• Improve individual and company-wide competitive win ratios.
• Establish clear benchmarks for understanding performance.
• Increase the number of successful sales per employee.
• Build a successful sales organization that is always improving.
• Discover the reasons behind lost opportunities.
• Increase your competitive advantage.
• Enhance your understanding of competitors.

Unlike pricing research, Win-Loss research goes further, delving into the decision process of clients, client service issues, and the effective handling of proposals. The end result is a tool that can provide actionable information on competitors and a specific market.

Methodology

Many clients ask which methodology is more appropriate: qualitative (in the form of in-depth interviews) or quantitative? A qualitative approach is best for the client who is just beginning the Win-Loss research process. The qualitative methodology allows for more investigation into what factors are actually at play in the sales selection process.

The quantitative approach is then employed once the dynamics of the market and sales process are understood and can be classified into standard metrics used over time.

In terms of quantitative data collection, much of this research is conducted via the telephone. In most cases this means sales representatives calling the client or prospect and enquiring as to why the company did (or did not) win the business. Many companies see this as a cost-effective way to learn more about the sales process and sales cycle. While telephone interviewing is a valid data collection methodology, having sales representatives making the calls is fraught with peril.

The second data collection methodology is face-to-face interviewing. For many companies with Win-Loss research programs in place, this is the preferred method because of the ability to interact with the client, read their body language, and probe for more in-depth responses to questions. Face-to-face interviewing is often used in sales situations where the product or service is expensive and/or complicated and technical.

Finally, online interviewing is also a well-recognized data collection methodology. It provides a timely and cost-effective means of collecting a large number of interviews. And, while a company may be limited in its ability to fully probe on open-ended questions, online interviewing works very well when the audience is technical.

Critical Factors

Regardless of the data collection methodology chosen, creating the right survey instrument is critical. After all, if the right questions are not asked, in the right way, the data may be skewed or simply not informative or actionable.

The interview should last no more than 30 minutes in person, and no more than 15 minutes on the telephone or online. It is unreasonable to assume today’s professionals can devote more than 30 minutes to a survey. Some key areas to focus on:

• The buying process—evaluation and selection criteria
• Decision drivers
• Multiple influencers and decision chain
• Product/Service features and functionality
• External influences on the decision process
• Branding/Marketing issues
• Commercial issues—pricing, contracts, etc.

Many companies that conduct Win-Loss research fall into the trap of using their employees to perform the research. Clients and prospects, however, are unlikely to be candid with the sales representative they’ve been dealing with for some time and who have failed to meet their expectations. It is also unlikely that the sales representatives will be candid in passing along information that may highlight their weaknesses or be viewed in an unfavorable light.

Many clients have found it virtually impossible to get unbiased, factual information without the help of an outside agency. Many clients have reported that in-house researchers have found buyers unwilling to reveal their real decision criteria for fear of offending the sales representative, or having the representative re-engage them in the sales process. In short, data collected in this manner may not be accurate, thus compromising the decision-making process on critical issues.

Conducting this type of research and analysis in-house can lead to a series of problems. A comparison of in-house versus third-party research reveals the following differences:

Category	Third Party	In-House
Performing Win-Loss Research	Experts in data collection and analysis of Win-Loss data	Tend to think of the research as just conducting a series of interviews
Resources	Trained, experienced, and dedicated staff working on the project	Will do the research projects when time and resources permit
Methodology	Various methodologies available—specific to the target audience and market	Often limited to telephone interviewing only
Data Analysis	Experienced analysts use statistical software and techniques to analyze data	Data is often analyzed in Excel and only in
Report Creation	Ability to publish first class reports designed to meet the clients’ specific needs	Reports are often topline only, with little depth or reporting in subgroups
Presentation of Results	Senior researchers analyze data and prepare an executive summary	Data is often presented to executives in raw form
Success of Research	Very high—most customers repeat and expand their projects based on having received actionable and relevant information	Often moderate or low—the project tends to cost more in time and resources than expected and is deemed not worth the cost

Reporting

Win-Loss research reports are usually composed of two types: Individual Performance Reports and Macro Trend Reports.

The Individual Performance Report (IPR) is designed to provide in-depth information on a particular sales opportunity. This report provides insight into how well the sales organization performed on a specific proposal opportunity; in particular, how well they understood the needs of the client and how well the product or service was received by the client/prospect.

This report also provides information on how well the competition performed in the sales process, how it positioned itself in the sales opportunity, and what the prospect saw as its strengths or weaknesses. Ultimately the IPR provides insight into why each prospect is selected in a specific sales opportunity.

The Macro Trend Report (MTR) is designed to present data on a quarterly basis and provides an overview of all the sales opportunities in a specific quarter. Its primary focus is on the business trends appearing in your market, which include:

• The perceived positioning and pricing of the competition
• Emerging competitors in the market
• Strengths and weaknesses of your product or service offerings

Advanced Analytics

Sample sizes in Win-Loss research often vary widely according to the price of the product or service (the higher the costs, the fewer “buyers” there normally are). Where there is enough sample, additional analysis can be conducted.

In addition to standard analytic procedures, Win-Loss research normally employs several multivariate analytical techniques to better understand the motivation and decision-making process of your clients.

Derived importance analysis is a statistical technique designed to understand the underlying factors that drive the decision process. For example, in most stated importance questions price normally rises to the surface as one of if not the most important factor in the selection criteria. Derived importance analysis digs deeper to understand what factors actually drive the decision-making process.

Factor analysis is another technique used to better understand how clients view the marketplace in regard to brand attributes and perceptions. This technique groups brands with their associated attributes and displays them in a graphic format known as a perceptual map.

As the above example indicates, the map identifies the distinguishing characteristics of a brand and those of its competitors, and provides information on which factors are seen as “cost of entry” by the client. This analysis allows for better tailoring of proposals so that they highlight a client’s strengths, while also exploiting the weaknesses of competitors.

In addition to these techniques, there is an entire suite of advanced analytical tools specifically designed to give in-depth information on the performance in the sales cycle and the performance of competitors. These additional techniques include: regression analysis, discriminant modeling, cluster analysis, and conjoint modeling.

Conclusions and Recommendations

A Win-Loss analysis will reveal the risks and opportunities associated with sales strategies, competitors’ strategies, and target markets. A marketer can learn about the marketplace in a way that transcends simply knowing about one’s customers.

A Win-Loss analysis will also reveal the varied buying habits, challenges, and adoption rates of customers competitors’ customers, and mutual prospects. Just remember some guidelines to follow:

By obtaining reliable and unbiased feedback from recent sales opportunities, sales representatives can refine their techniques, learn how to effectively target a client’s needs, and make appropriate presentations that put their companies and products in the best possible light. A Win-Loss research program provides empirical, yet pragmatic, information to drive business, products, and marketing decisions toward an improved competitive advantage.