Work in progress
Validity of the Customized Computerized Conjoint Analysis (CCC) (subject categories: Market Research; Product and Brand Management)
Susanne Hensel-Börner1 and Henrik Sattler2 University of Jena, Germany
Abstract One of the most popular methods for measuring preferences, especially for applications with many attributes, is the Adaptive Conjoint Analysis (ACA) (Johnson 1987). Despite its popularity, however, there are several shortcomings of this method (Green, Krieger and Agarwal 1991). Recently Srinivasan and Park (1997) proposed a new approach, called Customized Conjoint Analysis (CCA) which has several (potential) advantages over ACA. CCA combines self-explicated preference structure measurement with individually designed full-profile conjoint analysis. A disadvantage of CCA are the high costs and time requirements due to the fact that data collection has to be done twice for each individual with a time lag of about two weeks. Based on the idea of Srinivasan and Park (1997), we have developed a new method called Customized Computerized Conjoint Analysis (CCC). Unlike CCA, CCC uses an fully computer-assisted adaptive interview where self-explicated data as well as full-profile conjoint data are collected at the same time. Also CCC uses an improved parameter estimation. In an empirical study with refrigerators we compare ACA, CCC and two self-explicated models with respect to internal and predictive validity. The study is based on a representative sample of 500 respondents. 1
2
Susanne Hensel-Börner, Doctoral Student at the Department of Marketing, University of Jena, CarlZeiß-Str. 3, D-07743 Jena, Germany; Tel. + 49 3641 943114; Fax. + 49 3641 943112; Email:
[email protected] Henrik Sattler, Unilever Cair Professor of Marketing and Management Science, University of Jena, Carl-Zeiß-Str. 3, D-07743 Jena, Germany; Tel. + 49 3641 943110; Fax. + 49 3641 943112; Email:
[email protected]
1 Introduction Conjoint-Analysis as an instrument for measuring preferences of consumers has gained in importance in research and practical applications over the last 20 years. As early as the mid 80s, Wittink and Cattin (1989) estimated the number of commercial applications at 400 per year only in the US. In Europe nearly 1000 conjoint studies were carried out by market research companies between 1986 and 1991 (Wittink, Vriens and Burhenne 1994). In the meantime, more than 1000 Conjoint Analysis are performed each year in practical applications worldwide. While traditional full-profile conjoint analysis with a small number of attributes (up to 6) typically leads to valid results, using a large number of attributes may cause problems of validity due to an information-overload of the respondent (Green and Srinivasan 1990; Wright 1975). In order to address this problem several types of hybrid conjoint analysis have been developed (Green, Caroll and Goldberg 1981; Green, Caroll and Montemayor 1981; Green 1984; Johnson 1987; Green and Krieger 1996; Baier and Säuberlich 1997). Today, Adaptive Conjoint Analysis (Johnson 1987) is the most popular method for hybrid conjoint analysis among researchers as well as managers, especially for applications with many attributes. In a survey Wittink, Vriens and Burhenne (1994) show that ACA is the most frequently used method in Europe, and Green, Krieger and Agarwal (1991) reported a similar development for the U.S. Despite its popularity, ACA entails several problems such as graded paired-comparison (Green, Krieger and Agarwal 1991). Empirical studies that compare ACA with traditional full-profile conjoint analysis and selfexplicated approaches respectively find a slightly poorer or at best the same internal validity of ACA (Green and Srinivasan, 1990; Green, Krieger and Agarwal, 1991; Green, Krieger and Agarwal, 1993; Huber et al. 1993). In view of these results, the question arises if alternative methods lead to more valid results than ACA. Possible alternatives for applications with large numbers of attributes (more than 6) are self-explicated approaches and, in particular, hybrid methods (Green and Srinivasan 1990). Our study aims at testing the validity of a new hybrid method called Customized Computerized Conjoint Analysis (CCC) and comparing it to ACA and to two self-explicated models. We compare CCC with self-explicated models because these methods can be applied to a large number of attributes, and because previous research found a surprisingly high validity of selfexplicated methods (Srinivasan and Park 1997; Green and Srinivasan 1990). 2 Customized Computerized Conjoint Analysis (CCC) Our new method called Customized Computerized Conjoint Analysis (CCC) is an extension of Customized Conjoint Analysis (CCA) proposed by Srinivasan and Park (1997). Therefore, we will first briefly describe CCA in the following section. After this, we will point out the main extensions of CCC (for details concerning CCA see Srinivasan and Park 1997). CCA can be divided into three main stages. First, similar to ACA, it starts with the possibility for respondents to identify completely unacceptable levels for all attributes investigated. For each remaining attribute respondents determine the most-preferred and least-preferred levels. Setting these desirability-ratings to 10 and 0 respectively, the other attribute levels are evaluated using a 11-point rating scale. After rating the single attribute levels the overall attribute
2
importances are evaluated on the same scale using the most important attribute as an anchor. From these data the first stage self-explicated partworths PijkSE can be computed: (1)
SE Pijk = I ij (D ijk / 10)
where: SE Pijk : respondent i`s self-explicated partworth for attribute j`s k-th level, I ij :
respondent i`s importance (0-10) for attribute j, and
D ijk : respondent i`s desirability rating (0-10) for attribute j`s k-th level. Second, based on the results of the first stage, the so-called „core“ attributes are identified for each respondent individually. Core attributes are the most important attributes (not more than 6) which are included in a full-profile conjoint analysis. Full-profile stimuli for the conjoint stage are designed according to a fractional factorial design. Because the core attributes differ from person to person the set of full-profile stimuli (e.g. 8 to 16 stimuli) is customized to each person (Customized Conjoint Analysis). Based on the preferences for the customized profiles normalized conjoint partworths PijkCA can be estimated for each respondent. Third, in a calibration phase the optimal combination of the estimated self-explicated partworths PijkSE and conjoint partworths PijkCA for the core attributes is determined. To do so respondents are again asked to rate or rank full-profile stimuli described by their core attributes. This task is similar to stage 2. However, different combinations of attribute levels are presented. From this data weighted partworths Pijkw can be predicted: (2)
Pijkw = w i PijkCA + (1 - w i )PijkSE
where: Pijkw : respondent i`s weighted partworth for attribute j`s k-th level,
w i : respondent i`s weight, PijkCA : respondent i`s conjoint partworth for attribute j`s k-th level, and Ci :
respondent i`s core attributes.
The preferences are predicted for different weights (0, 0.01, 0.02, ... , 1) to identify the optimal weight wi* that produces the highest correspondence (i.e. cross validity) between actual and predicted preferences. With the optimal w* the predicted preference of respondent i for a new stimulus with level kj for attribute j is given by:
[
]
(3)
U i = ∑ w *i PijkCA + (1 - w *i )PijkSE + ∑ PijkSE
Ui :
respondent i`s predicted preference,
* i
w :
j∈C i
j∈NC i
respondent i`s optimal weight, and
NCi : respondent i`s non-core attributes.
3
CCC contains basically two extensions of CCA. First, CCC uses a fully computer-based interview and all data are collected during one interview session. In contrast, Srinivasan and Park reported a time lag of two weeks between the first and second data collection stage. This time lag in CCA is necessary because the full-profile conjoint-stimuli in stage 2 have to be customized according the results of stage 1. Apart from possible confound due to the time lag, the method is quite time and cost consuming. Combining all three stages in a single interview overcomes these main problems. As a second extension of CCA we modified equation (3) that shows the computation of the preferences toward a new product or stimulus. The estimation procedure proposed by Srinivasan and Park (1997) allows for attribute importance weights of a core attribute to be (substantially) lower than the importance weight of a non-core attribute. To overcome this problem we propose a new approach to calculate weighted partworths: (4)
[
]
SE U i = ∑ w *i PijkCA + (1 - w *i )PijkSE + ∑ (1 - w *i )Pijk j∈C i
j∈NC i
This modification assures that importance weights for core-attributes will be higher than those for the non-core-attributes. Figure 1 summarizes the 3 stages of CCC. Because we intend to compare CCC with ACA, the three main stages of ACA are also summarized in fig. 1.
4
CCC
Stage 1: (Self-Explicated Stage)
Stage 2: (Conjoint Analysis Stage)
Stage 3: (Calibration/ Combination Stage)
ACA
Elimination of completely unacceptable levels (not yet implemeted)
Elimination of completely unacceptable levels
Evaluation of the attribute levels on a 11-point-rating scale
Ranking or rating of preferences for each level of each attribute.
Identification of the most important attribute and evaluation of the importance ratings for the other attributes using the most important attribute as an anchor
Rating of the importance of each attribute in turn on a 4-point-rating scale where each attribute is described by its worst and best level
è SE-CCC
è SE-ACA
Ranking or rating of preferences for 8 to 16 full-profile stimuli described by the customized core attributes for each respondent
Ranking or rating of preferences for 4 to 12 full-profile stimuli described by the same customized core attributes for each respondent but by different combination of attribute levels
è CCC
Evaluation of a set of paired comparisons on a 9-point rating scale. In each paired comparison the respondent indicates which of the 2 profiles is preferred and by how much
Rating 2 to 9 calibration profiles that are each composed of up to eight attributes on a 0-100 likelihood-of-purchase scale
è ACA
Figure 1: Data collection steps for CCC and ACA
3 Research Design The purpose of our empirical study is to compare the validity of CCC with ACA as well as two different self-explicated models. With ACA we chose a widely used approach which has almost become the standard software for conjoint analysis. A comparison to self-explicated models is also quite interesting, because data collection as well as data analysis is much easier for these procedures. In our study, we use the self-explicated models implemented in stage 1 of CCC and ACA (e.g. SE-CCC and SE-ACA according Figure 1).
5
6
Tests of validity will be performed with respect to 3 criteria: a) face validity with respect to estimated attribute importance weights, b) predicitve validity with respect to a holdout sample, and c) predictive validity with respect to real market shares. In co-operation with the German market research company „GFM-Getas“ (Hamburg) data collection has just been completed in September 1998. Data were collected from 250 respondents with CCC as well as from 250 different respondents with ACA in three different German cities (Hamburg, Frankfurt and Munich). In both subsamples a representative quota sampling procedure was used. Choosing a different subsample for each method is a remarkable difference to other empirical studies described in the literature. In most of them one sample was used for analysing all different approaches tested in the study (Huber, Wittink, Fiedler and Miller 1993; Agarwal and Green 1991). With these within-subject designs several problems, including order and learning effects, could bias the results of the studies (Huber, Wittink, Fiedler and Miller 1993; Agarwal and Green 1991). In this study we use 8 attributes (with 2 or 3 levels) of refrigerators, which have turned out to be relevant in a pretest. First, the respondents evaluate all 8 attributes and their levels in the self-explicated stage. For CCC the respondents are asked for their preferences for 9 full-profile cards, described by 4 customized core attributes. The ACA interview also begins with the selfexplicated model and than presents 13 paired-comparisons of concepts first described by 2, and later by 3 attributes. At the end of the interview each respondent is required to answer demographic questions and to give information on past brand purchases. Currently, data analysis is in progress. The results of the comparison with respect to the different validation criteria will be presented at the EMAC conference in May 99.
References Agarwal, M.K. and P.E. Green, 1991. Adaptive Conjoint Analysis versus Self-Explicated Models: Some Empirical Results. International Journal of Research in Marketing 8, 141146. Baier, D. and F. Säuberlich, 1997. Kundennutzenschätzung mittels individueller HybridConjointanalyse. Zeitschrift für betriebswirtschaftliche Forschung und Praxis 49, 951972. Green, P.E.,. 1984. Hybrid Models for Conjoint Analysis: An Expository Review. Journal of Marketing Research 21, 155-169. Green, P.E., J. D. Caroll and S.M. Goldberg, 1981. A General Approach to Product esign via Conjoint Analysis. Journal of Marketing Research 45 (Summer), 17-37. Green, P.E., J.D. Caroll and M. Montemayer, 1981. A Hybrid Utility Estimation Model for Conjoint Analysis. Journal of Marketing Research 45 (Winter), 33-41. Green, P.E. and V. Srinivasan, 1990. Conjoint Analysis in Marketing: New Development with Implications for Research and Practice. Journal of Marketing 54 (October), 3-19. Green, P.E., A.M. Krieger and M. Agarwal, 1991. Adaptive Conjoint Analysis: Some Caveats and Suggestions. Journal of Marketing Research 28 (May), 215-222. Green, P.E., A.M. Krieger and M. Agarwal, 1993. A Cross Validation Test of Four Models Quantifying Multiattributes Preferences. Marketing Letters 4:4, 369-380. Green, P.E. and A.M. Krieger, 1996. Individualized Hybrid Models for Conjoint Analysis; Ma-
7
nagement Science 42 (6), 850-867. Huber, J.C., D.R. Wittink, J.A. Fiedler, and R. Miller, 1993. The Effectiveness of Alternative Preference Elicitation Procedures in Predicting Choice. Journal of Marketing Research 25 (February), 105-114. Johnson, R.M., 1987. Adaptive Conjoint Analysis. Sawtooth Software Conference on Perceptual Mapping, Conjoint Analysis, and Computer Interviewing, ed. Ketchum, ID: Sawtooth Software, Inc., 253-265. Srinivasan, V. and C.S. Park, 1997. Surprising Robustness of the Self-Explicated Approach to Customer Preference Structure Measurment. Journal of Marketing Research 34 (May), 286-291. Wittink, D.R. and P. Cattin, 1989. Commercial Use of Conjoint Analysis: An Update. Journal of Marketing 53, 91-96. Wittink, D.R., M. Vriens and W. Burhenne, 1994. Commercial Use of Conjoint Analysis in Europe: Results and Critical Reflections. International Journal of Research in Marketing 11 (1), 41-52. Wright, P. 1975. Consumer Choice Strategies: Simplifying vs. Optimizing. Journal of Marketing Research 12 (February), 60-67.
8
