Choice-Based Conjoint analysis is widely used for multiple applications including feature, price, and assortment optimization. Its properties make it attractive in complex categories, such as tech and durable goods. But sometimes, especially in these categories, selecting the best design for a choice exercise might present a challenge for a researcher. The internal study performed by Big Village focused on the alternative-specific design built from structural relationships between complex attributes. It compared an alternative-specific design with other common CBC designs often used in complex categories, including a traditional CBC, a shelf test, and an adaptive CBC. It considered the impact of a design choice on models, simulations, conclusions, and recommendations. Based on this study, below we summarize the similarities and differences between CBC approaches providing practical hints for researchers using the methodology.
Choice Based Conjoint (CBC) analysis is one of the most popular and trusted tools for product development and price optimization in modern survey-based research. In particular, CBC is widely used in complex categories like tech and durable goods. Often, it is applied to test innovations in these categories. What makes conjoint so attractive for these kinds of studies?
CBC recreates real-world purchasing scenarios and behaviors. In a choice exercise, respondents evaluate products or offerings in a competitive context, comparing different features and prices and making reasonable trade-offs. The data collected in a discrete choice exercise can be used in a Hierarchical Bayesian (HB) estimation to model preferences individually for each respondent and accurately describe market heterogeneity. Using CBC, researchers can simulate consumer behavior in hypothetical scenarios, testing products, offerings, and price points that don’t exist on the market yet. CBC analysis can account for interactions between different product attributes. In simulations, various scenarios on the market can be considered to optimize product features and estimate price sensitivity. The analysis can be used to identify thresholds and optimal prices. The relative importance of attributes can also be estimated in a CBC.
Another advantage of a CBC is its flexibility. Modern conjoint offers multiple design options to describe a large variety of different products and categories and to respond to various research objectives and business needs. Not only does it allow for the description of numerous attributes and levels, but it also accommodates complex layouts with hierarchical attributes. In this case, the level of a primary attribute determines the presence and nature of other conditional attributes.
After learning all the details of a business problem from a client and deciding that a CBC would be a good fit for the study, a researcher has to choose the best design that meets the business objectives and accurately describes the situation on the market. Sometimes, the choice of a design is obvious, but if the research is addressing a complex category and product, it might not be so easy to select the best CBC approach.
A traditional design is the most familiar to researchers. It is usually relatively straightforward to design and implement and still allows incorporating some relationships between attribute levels with prohibitions and conditions. A shelf test has a trivial structure with only two attributes: product and price. It ensures in-depth testing of a limited number of products and an impact on price, but it provides no information beyond fixed configurations of features in a product. Adaptive CBC (ACBC) only presents relevant alternatives in a choice exercise. It allows dealing with a large number of attribute levels, only focusing on what is important for each respondent. This kind of design generally does not need any prohibitions or conditions. The exercise is easier and more engaging for respondents, and the data quality is expected to be higher. Similar to an ACBC, a partial profile CBC is mostly used to deal with a large number of attributes and levels. Respondents are shown alternatives with a subset of all attribute levels in tasks to evaluate.
Sometimes, especially in complex tech or durables categories, products may have a unique set of attributes. As an example, let us consider headphones or earbuds. These products have common attributes – brand, shape, color, price, etc. Levels of these attributes would be presented in every alternative in a CBC. For example, earbuds can be wired or wireless. Also, there will be attributes/features that are presented only for wireless or wired earbuds. For wireless earbuds, these could be charging time and battery life, and for wired earbuds, it could be in-line volume control, plug type, etc. To model a category like headphones and earbuds using a CBC, many researchers are utilizing alternative-specific designs. With this kind of design, every primary attribute (such as wireless or wired earbuds in the example above) will only be paired with a relevant subset of conditional attributes (such as charging time and battery life for wireless earbuds). An alternative-specific design only considers product alternatives containing features that make sense for respondents in this category. Therefore, in an alternative-specific CBC, respondents can make meaningful tradeoffs and choose between products with feasible combinations of attributes. Other examples of appropriate categories to apply an alternative-specific CBC could be computers (desktops, laptops, tablets), and electric floor-cleaning devices (vacuum cleaners, wet cleaners, wet-dry combos).
Overall, the alternative-specific approach makes CBC designs more flexible and “compact” in studies involving complex products or offerings. It provides a more realistic description of alternatives, allowing respondents to choose from the most relevant options, which improves data quality. Classifying and presenting attributes as common, primary, and conditional better informs the estimation and ensures higher accuracy of modeling in CBC studies.
As with any other type of advanced CBC design, alternative-specific conjoint has its limitations. In general, an alternative-specific design can be more detailed than a standard CBC, but it still assumes a certain level of generalization in product descriptions. Introducing additional prohibitions or conditions is not recommended in an alternative-specific conjoint. Alternative-specific CBC is not suitable for estimating interactions since conditional attributes are directly associated with primary attributes in these types of studies. If almost every product in a category has its own set of attributes, a shelf test might be more appropriate for the study than an alternative-specific design. If products have a very large number of attributes, a partial profile approach or an adaptive CBC could be a better fit. Generally, we will characterize the context that leads to appropriate designs for CBC.
Big Village executed an internal research project to better understand the impact of a CBC design on the model, results, observations, and recommendations. It was presented at the Analytics & Insights Summit (hosted by Sawtooth Software) held in May 2023, in Barcelona, Spain. In this research, we were able to characterize the context that leads to appropriate designs for a CBC. Properties of different types of CBC designs dictate the pluses and minuses of each of the methods for analysis in complex categories. A traditional CBC draws attention to particular features, especially to the attribute(s) defining conditions and prohibitions in the design. It might show weaknesses in estimating importance, sensitivity, and a None level. ACBC studies configurations and prices that are the most relevant to each respondent but have restricted capability to capture switching behavior and therefore have limited applications in testing innovations. The shelf test is very accurate in estimating shares of preference and price sensitivities, but it is only limited to testing fixed configurations and cannot be used for feature optimization.
Alternative-specific CBC could be the best-fitting conjoint type for many business problems in complex categories such as tech and durable goods. It naturally recreates the situation where levels of a primary attribute define other attributes in a product or offerings and is suitable for testing and optimizing innovations. If a product or offering is described with a hierarchy of primary and conditional attributes, an alternative-specific CBC would be the best approach for feature optimization. But a researcher has to be careful with the importance score estimation and its interpretation since by design an alternative-specific CBC draws more attention to a primary attribute. Big Village research has shown that price sensitivity can also be affected by an alternative-specific CBC structure if an HB estimation is utilized to build the model.
To summarize, the Big Village research showed that models built based on different variations of CBC if executed properly can be remarkably accurate and stable in the presence of noise. Different models can be applied to solve various business problems, including feature and price optimization, and results can be reported with high confidence. Moreover, the models were sophisticated and detailed enough to underline differences in respondents’ reactions to choice exercises based on different CBC designs.
Written by Faina Shmulyian, Vice President, Data Science.