Data Collection White Papers

Car Clinics (The Head-to-Head Contest)

While this white paper will focus on clinics to evaluate new cars and new trucks, the same concepts and methods can be applied to a wide range of durable goods (bulldozers, construction cranes, lawn mowers, chain saws, vacuum cleaners, refrigerators, washing machines, and hundreds of other long-lasting products).

“Car Clinic” Definition

What is a “car clinic”? You might think of it as a beauty contest among cars, with target-market consumers serving as the judges. It is a head-to-head comparison of competing brands and/or models. These competitive vehicles (usually ranging from 4 to 8 cars) are assembled in a showroom of some type, typically a facility with one-way viewing rooms, good security, and audio and video recording equipment. Target-market consumers (i.e., potential buyers) are recruited to view the cars or trucks, indicate preferences, and provide ratings and opinions.

Purposes

Car clinics are conducted at different times for different purposes. The design and makeup of these clinics vary. The most common purpose is evaluating and learning how to improve new models. A second purpose of car clinics is product line planning, or product line optimization, where all the different models of one brand are displayed and evaluated. A third purpose is pricing optimization, where consumers view the cars within a class of vehicles, and then participate in choice modeling experiments. A fourth purpose is to set the stage for volumetric forecasting of sales.

Choosing the Cars

The automotive market is made up of many different segments, as defined by vehicle type, size, price, and physical characteristics, such as compacts, SUVs, midsized sedans, luxury vehicles, and so on. Typically, the market segment to which the test vehicle belongs would define the universe from which competitive models would be chosen. The most directly competitive cars in the segment (or vehicle class) are generally recommended as the benchmark or comparison vehicles. Not all competitive vehicles in the class need be included in the clinic, but as a rule at least 3 or 4 major competitors are needed for actual and prima facie validity. The competitive vehicles should, to the degree possible, be similar to the test vehicle in size, color, trim package, and price.

Choosing the Facility

If budgets and time permit, multiple facilities located in different cities are the ideal, but rarely do budgets permit more than one clinic in one market. The market should have a Category Development Index (CDI) and Brand Development Index (BDI), if possible, between 75 and 125 (not too far from the national averages). It’s generally best to avoid extremely low or extremely high CDIs and BDIs—to avoid unrealistic or atypical findings. It is also a good idea avoid the hometown crowd (and their biases) and to find a facility far away from corporate headquarters.

The ideal “showroom” offers bright lighting so that cars can be properly illuminated. The ideal facility contains two or more focus group rooms with one-way mirrors, as well as smaller rooms for one-on-one interviews. It also has rooms away from the cars on display where multiple computers can be set up to conduct surveys and choice modeling experiments, before or after exposure to the cars. The facility should provide portable microphones to each respondent, if observers are planning to listen to any in-person interviews. In large display rooms with high ceilings, respondents’ voices are difficult to hear and understand.

Ensuring Security

The research facility must be highly secure, so that vehicles can be unloaded from trucks (and reloaded) without exposure to prying cameras or spies. The ideal showroom has no exterior windows (or few exterior windows—with lightproof coverings). Once the vehicles arrive at the facility, a 24-hour security team remains with the vehicles in the facility—to make sure that only authorized individuals have access to the facility and the vehicles. This same security team would use metal detectors to check all respondents for cameras, phones, and recording devices. The security team stays in place and is vigilant until the last car is loaded on the transport truck and the last PCs and tablet computers are cleaned of all data and programs and packed away.

Displaying the Cars

All of the vehicles included in the clinic must be polished and clean, inside and out, and must be touched up periodically during the clinic, to wipe off dust, fingerprints, and smears. Lighting should be bright and equally distributed, so that all cars are properly illuminated. The order in which participants view the cars is randomized or rotated in a systematic way to minimize bias.

Choosing the Respondents

Who should participate in the clinic? The answers to this question, of course, depend on the purpose of the clinic and the philosophies of the OEM. If the purpose is to evaluate a new model, it makes sense to limit participation to those planning to buy a new car in the next year or two. Some type of household income minimum is often a screening criterion (no point in interviewing people who cannot afford to buy the new car). People who would never consider buying the type or class of vehicle under consideration, or the test brand, are often excluded—but we normally advise great caution in excluding people. These exclusions might bias the sampling. Often psychographic profiles are used as screening criteria, but this can risk the introduction of unknown biases into the sampling. Ideally, the sample of prospective buyers would be as representative as possible.

Recruiting the Participants

Lists of car owners by type and model can be purchased, and this can save time and money, especially for small market segments (such as owners of “sport specialty cars” such as the Mustang and Camaro, for example). Some online consumer panels maintain information on car ownership and can also be a sampling source. Online panels can be screened for intent-to-buy a certain type or category of automobile. Phone interviews are often used as another screening and recruiting method. The more difficult it is to recruit respondents, the higher incentive fees should be; higher fees are cheaper than endless recruiting and also improve the “show rate.”

Conducting the Interviews

Many car clinics are self-administered. Participants are given a tablet computer or iPad (or a clipboard and paper questionnaire) and asked to evaluate the automobiles in a specified order. Depending on the objectives of the research and budgets, interviewers may be used to conduct the surveys, especially if probing of open-ended questions is deemed important. By probing and clarifying open-ended questions, interviewers can enrich the learning from car clinics. Often this open-ended information is collected by conducting focus groups or depth interviews among a subset of the respondents after they have participated in the clinic.

Forecasting Sales

Once participants evaluate the cars in a clinic, and fully understand the major differences among the vehicles, they are better prepared to answer questions about likelihood of future purchases. Choice modeling is the recommended method of determining likely market share and year-one sales volumes. In choice modeling experiments, respondents view a number of scenarios online (sets of automobiles with different features, trim packages, prices, colors, etc.) and choose the vehicle they would purchase, given each scenario (or each set of variables). Participants typically see 6 to 10 different scenarios. Advertising and messaging can also be incorporated into the choice modeling experiments. In addition to forecasting sales volume, choice modeling exercises can reveal the slopes of demand curves and accurately measure price elasticity.

Chances of Success

Car clinics are not perfect. Sampling limitations (e.g., the use of only one market for the clinic) typically mean that the results must be viewed as approximations, not absolute truth. Clinics tend to be expensive, and they can take time to plan and execute. Despite these negatives, nothing can take the place of actually seeing and experiencing the physical vehicle, sitting in the seat and touching the controls, and imagining what it would be like to drive—and be seen driving it. Car clinics yield valuable information that not only improves future vehicle designs, but also improves targeting, messaging, and pricing. The ultimate benefit: car clinics dramatically improve the chances of success.