The current issue and full text archive of this journal is available at www.emeraldinsight.com/1460-1060.htm
Images of new product success: a case study in search of local validity Derk Jan Kiewiet and Marjolein C. Achterkamp
Images of new product success
87
Faculty of Management and Organization, University of Groningen, Groningen, The Netherlands Abstract Purpose – This paper aims to measure new product success within a Dutch mailing company and to hypothesize that there exists no definition of new product success which is generally applicable, or valid in all circumstances. It seeks to opine that the best that can be achieved is a “local” definition: a definition valid only in a specific local context. In this article, a method is described on how to develop such a local measure. Design/methodology/approach – To prevent this framing bias, a multidimensional scaling approach is used, in which data collection and analysis have relevant new product success features as output instead of input. Findings – The method was applied to a Dutch mailing company, and it was found that in this case only two dimensions of new product success were prevalent. These were ROI and customer acceptance. From this, it was inferred that local valid measures in this particular situation were only a subset of all measures of new product success mentioned in the literature. Originality/value – In summary, the method used could shed light on the difficulties that sometimes do arise when different parties are working together. As a consequence, not only researchers but also practitioners should become aware of the indefiniteness of the concept of new product success. Keywords New products, Management effectiveness, Product development, Direct mail, The Netherlands Paper type Research paper
Introduction The concept “new product success” plays a major role in the ongoing research on product development. Often, new product success is correlated with variables describing the development process (e.g. Davidson, 1976; Montoya-Weiss and Calantone, 1994; Kratzer et al., 2004). Furthermore, new product success is used as a criterion in the assessment of innovation projects by management (e.g. Hollander, 2002). As such, new product success is one of the major goals of innovation teams. It is obvious that a correct measure of new product success, both in terms of reliability and validity, is crucial. However, measuring success is not an easy, nor a straightforward task. For example, Lynn and Akgu¨n (2001) studied the effect of project visioning on new product success. They measured new product success by looking at 14 new product success criteria covering managerial, cost, profit and technical point-of-view (Cooper and Kleinschmidt (1987)). On the other hand, Di Benedetto (1999) studied the effect of product launch activities on new product success. In this study, only four measures of perceived new product success were used: The authors would like to thank Jan Hollander for his help with the data collection.
European Journal of Innovation Management Vol. 11 No. 1, 2008 pp. 87-102 q Emerald Group Publishing Limited 1460-1060 DOI 10.1108/14601060810845231
EJIM 11,1
88
(1) (2) (3) (4)
perceived overall profitability; perceived profitability; sales; and market share relative to competing products on the market.
A major problem concerning the measurement of success is found in the notion that success is a multidimensional concept. Multidimensionality has negative consequences for the construct validity and content validity of concepts (Campbell and Stanley, 1963; Cook and Campbell, 1979, as well as for the ecological validity. The two aforementioned studies show that the choice of product success dimensions, operational definitions, product success objects and respondents can and do differ between researchers. Unclear however, is how new product development practitioners cope with these differences. Can they translate and use the research findings to their own situation? That is, what is the ecological validity of studies about product success? On several annual PDMA International Conferences practitioners expressed their doubts about the applicability of research in general. If the measures and measurements of new product success in research do not adequately match the use of new product success by practitioners, the ecological validity of studies on product success is (severely) hampered. This is not only caused by the multidimensionality of the new product success concept, but also by the context-dependency (De Wit, 1988), and the time-dependency (Pinto and Slevin, 1988) of the concept. Therefore, we hypothesize that there exists no definition of new product success which is generally applicable, or valid in all circumstances. Even stronger, we argue that such a definition cannot exist. The best we can strive for is a “local” definition: a definition valid only in a specific local context. In this article, we describe a method on how to develop such a local measure. The method we use is multidimensional scaling analysis. This approach determines those aspects of success relevant to reviewers of product development project, without a priori restrictions by the researcher on aspects of success that can be used. The method will be illustrated in a case study in a Dutch mail company. Theoretical background: measures of new product success In the last few decades, several articles have shown that there is a large variety in the use of definitions and measures of product success. Griffin and Page (1993) showed that NPD-practitioners and academics researching NPD use 75 distinct measures of new product success. This large number can be explained partly by the observation that product success can be defined at three different levels (firm, program and project), and that within the project level several dimensions of product success can be distinguished: customer based, financial based, and product based. Griffin and Page (1993), but also others (e.g. Graig and Hart, 1993; Hart, 1993, 2003) conclude that the large number of different measures of product success is due to the multidimensionality of the concept of new product success at the project level. Several researchers (e.g. Chryssochoidis and Wong, 2000, Hultink and Robben, 1995, 1999) use various measures based on the research of Griffin and Page in order to capture as much as possible of the different aspects of new product success. Others (e.g. Lynn and Akgu¨n, 2001) use measures based on the work of Cooper and Kleinschmidt (1987, 1995). Cooper and Kleinschmidt used factor analysis to reduce ten
different performance measures (namely success rate, percent of sales, profitability relative to spending, technical success rating, sales impact, profit impact, success in meeting sales objectives, success in meeting profit objectives, profitability relative to competitors, and overall success) into two underlying dimensions: new product profitability and new product impact. Still, other researchers are not focused on the multidimensionality of new product success, but restrict themselves to certain dimensions of new product success, such as selling performance (Hultink and Atuahene-Gima, 2000; Sujan et al., 1994), financial performance, product design quality, and project time-based performance (Smink, 2000), commercial success (Souder and Jenssen, 1999). Furthermore, in some research the focus is on new services success (e.g. De Brentani, 1989; Cooper et al., 1994; Edgett and Snow, 1996; Storey and Easingwood, 1999; Avlonitis et al., 2001; Alam, 2003). Here, it also holds that the authors differ in dimensions used. For example, De Brentani (1989) identified four performance factors for new services success: sales and market share performance, competitive performance, “other booster”, and cost performance, whereas Storey and Easingwood (1999) use enhanced opportunities next to sales performance and profitability. Alam (2003) provides an overview of new product success for financial product innovations. He distinguishes three performance dimensions: financial criteria, customer criteria and opportunities criteria. Obviously, the definitions of new products success used in the literature differ from each other in the dimensions used. However, the situation becomes even more complicated if contingencies are drawn into the equation. Griffin and Page (1996) show that the definition of product success used in a NPD-project is dependent on the project strategy and the organization strategy. Other researchers have shown that the definition of new product success is also conditional upon the time perspective for which the new product evaluation is done (Hultink and Robben, 1995) or, when using the stage-gate model for NPD-projects, upon the gate the project is situated in (Hart et al., 2003). Incorporating the different stakeholders of a project further complicates the meaning of success. A project can be a success for one stakeholder and a disaster for another at the same time (De Wit, 1988). Furthermore, a project may be perceived as a success one day and a failure the next. Pinto and Slevin (1988) suggest that the relative importance of dimensions changes with time. At the beginning of an NPD-project, internal performance measures are important, such as meeting budget, time schedule, and technical performance. In more advanced phases of the project, external factors, such as customer satisfaction and needs, become more important (Pinto and Slevin, 1988; Griffin and Page, 1996). In this overview, project performance and new product success are intertwined. New product success is only part of the success of a new product development project. However, the previous mentioned literature makes it clear that new product success is by itself multidimensional, time-dependent and context-dependent. Therefore, the unconditional use of the concept “new product success” leads to ambiguous situations. However, this is not realized in the organizational practice. One is seldom explicit on the exact meaning of product success. Most of the time subjective images of success instead of a clear definition are used as base for decision-making and action taking. The differences between the images used can lead to organizational problems in
Images of new product success
89
EJIM 11,1
90
communication, assessments and judgments, and in organizational strategy (Litwin and Stringer, 1968; Browne and Golembiewski, 1974; Lant and Phelps, 1999). The previous discussion makes clear that these organizational problems cannot be solved by posing a generally applicable definition of new product success. Because of the multidimensionality, the context-dependency, and the time-dependency of the new product success concept, such a general definition simply cannot exist. However, it might be possible to provide a local definition, valid only in this particular local context. This is a definition corresponding to the actual use of the concept of new product success by the practitioners in the specific context. To assure this validity, the local definition should be based on the prevailing subjective images of the players in this local context, and not on a priori restrictions by the researcher on aspects of success that can be used. Method A case study in search for local images of new product success Because we are looking for local images of new product success, we use a method for data collection and analysis that follows this focus. The starting point of this method is the personal construct theory of Kelly (1970). Two basic notions are in this theory. First, Kelly does not accept that knowledge reflects the real world; there are as many worlds or realities as there are human beings. The individual is the primary generator of meaning. Individuals make representations or constructs through which they view the world. Kelly describes a construct as an individual representation of the universe, which is made and then tested by that individual. This means that, according to Kelly, the only valid form of inquiry is into the subjective understanding of individuals (see also Midgley, 2000). Second, since the universe is essentially a course of events, the testing of a construct is a testing against subsequent events. This means that individuals construct their reality primarily in terms of activities. Although we follow Kelly in his idea that individuals create their own meanings, we do not think that an individual performs this creation process in isolation of others. People interact, and these interactions contribute to the development of individual constructs. So, based on these findings, we assume that persons who work together closely will probably have the same or almost same constructs for work-related entities. One way of representation of (individual) meanings is the use of cognitive modelling. Although there is a diversity of possible mapping approaches, we focus on maps that show underlying dimensions of categories and cognitive taxonomies (Huff, 1990). As diverse as cognitive modelling methods can be, they have in common that they try to externalize individual cognition. In this research on new product success, this means that the subjective images of new product success will be externalized. To develop these images, four questions about the specific context studied have to be answered beforehand. First, there is the “what-question”: success of which new products? In other words, what are in a particular situation the new products for which we want to assess new product success. The second question is the “when-question”: at what moment in the product life cycle are the new products evaluated. The third question is the “attribute-question” and is closely related to the former two questions: what are the attributes or features the new products should possess in order to be considered a success at a certain time? The last question is the “who-question”: who is evaluating new product success?
Given these questions, a logical way of dealing with them is to ask the individuals involved in a particular NPD setting what they consider the relevant features of success for their new products. However, there are some problems connected with this analytical and explicit method. First, there is the problem of rationalisation. This means that individuals seek a reasonable meaning of product success. Yet, in the everyday life of using the notion in the actual practice, different aspects might be relevant. Second, there is the problem of constraining the notion. This would happen when we, the researchers, ask the individuals involved to mark – on a list of new product success features we made – those features they consider relevant or important for new product success. Since we made this list, we then would have influenced the images that are the result of such research. In other words, we would have imposed our frame of reference on the respondents. Third, there is the problem of receiving socially desirable answers. Organizational members might exhibit a certain perspective on new product success, because they assume that this is what they are supposed to do due to organizational policy or ideas of superiors regarding new product success. These types of problems can be considered different examples of framing effects, which are caused by: “respondents” behaviour, or by imposing the world-view of the researcher on respondents, or by the context in which the respondent acts (see, e.g. Tversky and Kahneman, 1981; Levin et al., 1998; Druckman, 2001; McElroy and Seta, 2003). To overcome the dangers of the framing bias, we focus on a holistic or implicit way of making sense of sustainability. The key idea is that the answer to the aforementioned “attribute question” is not given explicitly before data collection takes place. Instead the answer to this question is the result of the data analysis and is revealed by the derived cognitive maps of new product success in a particular NPD situation. We have done this by employing a multidimensional scaling-analysis (e.g. Borg and Groenen, 1997). In such an analysis, three phases can be distinguished. In the first phase, the “what-”, “when-” and “who-question” are resolved by choosing the products for which we want to asses the extent of new product success, the time frame for which this assessment applies, and the subjects that will do this assessment. For these products, proximities are gathered by asking each subject to indicate how similar two entities are with respect to new product success. This is achieved in a direct way, without referring to attributes and thereby circumventing the “attribute-question” as discussed before. In the second phase, these proximities are used to project the products in a multidimensional space. This projection is done in such a way that the distances (i.e. Euclidean distances) between the products in this space are a monotone regression of the proximities. The idea of an MDS-analysis is that in projecting the proximities in a multidimensional space, the dimensions that span the space reveal some of the underlying causes of these proximities. For this study this means that the dimensions of the space should reflect the aspects of new product success that are responsible for the proximities of the first phase. In other words, the interpretation of the dimensions will give the answer of the “attribute-question”. Finding an appropriate interpretation of these dimensions is the third phase of an MDS-analysis. Subjects and products An important feature of our method for revealing images on new product success is that individual respondents order a set of products in terms of success (resulting
Images of new product success
91
EJIM 11,1
92
into proximities necessary for the MDS-analysis). This means that before the actual data collection could be done, we had to decide upon the products that had to be arranged (i.e. the “what- and “when-question” as discussed before) and the respondents that would perform the arranging of these objects (i.e. the “who-question”). To answer the “who-question”, this case study has been carried out within the Marketing and R&D departments of a Dutch mail company specialized in global delivery services. The company’s main business is mail: collecting, sorting, transporting and delivering letters and parcels. It also specializes in data and document services, direct mail, e-commerce and international mail. The company processes about 17 million postal items a day. Of the total volume of mail, 94 percent is business mail and only 6 percent is consumer mail. The company employs 80,000 people worldwide; nearly 40,000 of these deliver the mail in the Netherlands. In 2004, revenues of 3.9 billion euros were achieved. The company was chosen for this research because it launches new products very regularly; we choose the two departments because new product development is a core process for them. Within these two departments, fourteen senior employees who were very experienced in NPD-projects were selected to participate in the research. Of the more than 100 products for the domestic business market, 19 products were selected (the “what-question”). We did this by applying two criteria. First, the product had to be introduced to the market at least one year and at most five years ago (the “when-question”), and second, the product had to be well known to each of the selected fourteen employees. By employing this second criterion, we followed Kelly’s second notion that individuals construct their reality (in this case: new product success) primarily in terms of activities (in this case: the direct or indirect involvement in the NPD-projects of the chosen products) (Kelly, 1970). The selected products include, for instance, a pick up and delivery service for PO Boxes, stamps by mail, and business moving services. As mentioned before, due to framing considerations the “attribute question” is not answered in the method section. However, we will answer the question: Which attributes constitute new product success for these respondents and products? in the result section, because the dimensions of the multidimensional space derived should reflect these attributes. Data collection Data were collected in two rounds. In the first round, proximities between the 19 products were gathered. This was done by using a pairwise comparison procedure: for a pair of products an employee had to evaluate on a six-point scale which product was more successful (1: product X is much more successful than product Y, 2: product X is reasonable more successful than product Y, 3: product X is just a little bit more successful than product Y, 4: product Y is just a little bit more successful than product X, 5: product Y is reasonable more successful than product X, and 6: product Y is much more successful than product X). It was stated explicitly that the employee could use his or hers own definition of or opinion about “successful”, that it was not necessary to elaborate about the judgment on success he or she made, and that there were no right or wrong choices.
To evaluate 19 products in this way, a respondent would have to judge (19 2) ¼ 171 pairs of products. Because this many comparisons would be a too voluminous task, we decided to use an incomplete pairwise comparison design. In such a design, a respondent is asked to make only a subset of all the possible comparisons. A complete proximity matrix is then obtained by combining the comparisons in the subsets. However, an incomplete design can only be an adequate solution for the task size problem when the respondents are interchangeable with respect to the underlying process of giving meaning to success. This is exactly the assumption about shared meanings we made before when discussing the personal construct theory of Kelly. We were able to evaluate this assumption by special arrangements in our incomplete pairwise comparison design. We randomly assigned each pair of products to one of five groups. One group, the test group, consisted of 11 comparison pairs, the other groups of 40 pairs each. The 11 product pairs in the test group were judged on the 6-point scale by all 14 respondents. The interchangeability of the respondents was tested by calculating for each pair of respondents the correlations between the 11 comparisons (see Table I); large correlations indicate that two respondents are very similar. Based on significance and magnitude of these correlations, we decided to exclude persons 8 and 10 for the rest of the analysis. Assumable these employees have a different conceptualization of new product success; for the other employees our assumption was substantiated. By leaving out these two employees, each of the four groups of 40 product pairs was evaluated by 3 respondents. The proximity matrix was now generated by averaging the comparisons for each pair over the respondents that made the comparison for these two products. In the second round of data collection we asked a senior manager of the Marketing Department, who was not one of the 14 respondents in the first round, to order the 19 products on two criteria: customer acceptance (operationalized as the number of complaints received for a product) and ROI. These two criteria were chosen from the list of success/failure measures from the study of Griffin and Page (1993) because data for these criteria were easily available within the organization. The results of these two orderings were later in this study used for the interpretation of the dimensions in the multidimensional space. Analysis The obtained proximity matrix was analyzed by ordinary non-metric multidimensional scaling, using the ALSCAL module from the SPSS-program. In the analysis we used the primary approach for handling of ties. In this approach, ties in the proximity matrix may be broken in the distances of the multidimensional space. We choose for this approach instead of the secondary approach of ties (in which ties in the proximity matrix are preserved in the distances) because of the way the original comparison data were gathered. By letting the respondents choose from only a limited number of alternatives for the success evaluations, it is possible that respondents were forced to place pairs in the same comparison category while there could be differences between these pairs. Therefore, ties in the data are conceivably the result of the method and not of the data itself. In this situation the primary approach is appropriate. By using cluster analysis and a technique called PREFMAP, we will interpret the dimensions in the multidimensional space.
Images of new product success
93
1 0.629 * 0.653 * 0.710 * 0.761 * * 0.215 0.932 * * 0.293 0.773 * * 0.665 * 0.786 * * 0.691 *
3
Notes: * p , 0.05; * * p , 0.01
1 0.883 * * 0.917 * * 0.709 * 0.806 * * 0.703 * 0.855 * * 0.069 0.903 * * 0.343 0.744 * * 0.798 * * 0.879 * * 0.621 *
1 0.781 * * 0.618 * 0.772 * * 0.646 * 0.853 * * 0.344 0.760 * * 0.469 0.784 * * 0.859 * * 0.775 * * 0.631 *
Table I. Correlation matrix for all the 14 employees; correlations are calculated for the scores of 11 comparisons
1 2 3 4 5 6 7 8 9 10 11 12 13 14
2
1 0.739 * * 0.763 * * 0.794 * * 0.169 0.703 * 0.349 0.852 * * 0.500 0.849 * * 0.738 * *
4
1 0.750 * * 0.760 * * 0.035 0.652 * 0.397 0.744 * * 0.709 * 0.819 * * 0.450
5
1 0.600 2 0.040 0.722 * 0.154 0.739 * * 0.378 0.684 * 0.541
6
1 0.371 0.843 * * 0.721 * 0.872 * * 0.863 * * 0.942 * * 0.780 * *
7
1 0.181 0.538 0.489 0.346 0.219 0.496
8
1 0.432 0.749 * * 0.707 * 0.877 * * 0.686 *
9
1 0.541 0.715 * 0.629 * 0.419
10
1 0.652 * 0.820 * * 0.877 * *
11
1 0.822 * * 0.446
12
94
1
1 0.639 *
13
1
14
EJIM 11,1
Results Multidimensional scaling In an MDS-analysis one is looking for a multidimensional space in which the distances between the stimuli correspond, as good as possible, with the proximities in the proximity matrix. This means that the first decision to be made is the number of dimensions of the space. Several strategies exist for extracting this number. Spence and Ogilvie (1973) calculate the stress of a random multidimensional configuration. This “random” stress is compared with the stress of a proposed model. In the present study, the Kruskal’s Stress 1 values were 0.275, 0.173, 0.121, 0.079, and 0.048 for 1, 2, 3, 4 and 5 dimensions respectively. The null hypothesis that these stress values result from a random formation of points could be rejected for all dimensions. Another strategy is proposed by Spence and Graef (1974) and Wagenaar and Padmos (1971). They used the Hefner model for calculating the stress in situations where an evenly scattered configuration in a known number of dimensions is perturbed with random error from an N(0,s2)-distribution. We copied their method for n ¼ 19 objects, and found that the underlying number of dimensions should be two. Although the stress value for the configuration in two dimensions is relatively large, it is acceptable for further examination (Kruskal, 1964) described this value as “fair”). Based on the results of both methods, we chose an MDS-solution in two dimensions (see Figure 1).
Images of new product success
95
Cluster analysis The next step in an MDS-study is to interpret and label the dimensions of the chosen configuration. This interpretation process can be aided in several ways. We used a
Figure 1. Multidimensional space for 16 products
EJIM 11,1
96
combination of cluster analysis and preference mapping. The original proximity matrix was used as input for two different hierarchical cluster analyses: average linkage analysis and Ward linkage analysis (Gordon, 1980). For the average linkage analysis, the largest gradient in the similarity coefficients of the agglomeration schedule was between a clustering with two clusters and a clustering with one cluster. This indicates that two clusters are appropriate for grouping the data. However, cluster 2 had only one member (object 17). Therefore, it was decided to look for the second largest gradient. This was between three and two clusters, resulting in a clustering with three clusters with seven, 11 and one members. For the Ward linkage, the largest gradient in the agglomeration schedule was between two and one clusters, suggesting a clustering with two clusters, with six and 13 members. The cross-classification of these two clusterings shows that the overlap between the clusters is good (Cramer’s V ¼ 0.777, x2 ¼ 10.88, df ¼ 1, p ¼ 0.002). Only three of the 19 objects are not classified consistently over the two clusterings. Preference mapping For the preference mapping we used the orderings of the objects based on the two criteria customer acceptance and ROI[1]. These orderings were projected in the MDS-solution using the PREFMAP-program. Four hierarchical models can be used for this projection (Davison, 1992). The simplest model is the vector model. In this model, for each criterion a vector is placed in the obtained MDS-space, in such a way that the order of the projections of the objects on that vector is an ordinal transformation of the ordering on the corresponding criterion. The second simplest model for the projections is the simple Euclidian model. In this model the criteria are represented by ideal points in the obtained MDS-space. The objective is to position the ideal points in such a way that the Euclidian distances of the objects to an ideal point are the optimal ordinal transformations of the ordering on the corresponding criterion. The third model is the weighted Euclidean model. Here, the criteria are also represented as ideal points, however the axes of the multidimensional space now can be weighted independently of each other. The fourth model is the general Euclidean model. In this model the ideal point representing a criterion is placed in a MDS-space where the axes can be rotated and independently weighted. The results of these mapping are presented in Table II. We see that for both criteria all four models give a significant mapping. If we look at the improvement of one model Change for a model df2 Fc P
Model
R2
df1
Customer acceptance
Vector Simple Weighted General
0.58 0.79 0.87 0.89
2 3 4 5
16 15 14 13
11.00 18.48 22.56 21.89
0.001 0.000 0.000 0.000
1 1 1
15 14 13
14.66 8.19 3.45
0.002 0.013 0.086
Vector Simple Weighted General
0.69 0.81 0.82 0.83
2 3 4 5
16 15 14 13
17.93 21.15 16.34 12.78
0.000 0.000 0.000 0.000
1 1 1
15 14 13
9.20 1.17 0.56
0.008 0.297 0.466
ROI Table II. Results from the PREFMAP modelling
Explained by model df2 F p
Criterion
df1
to the next in the hierarchy of models, we find that for the customer acceptance criterion the weighted Euclidean model is a significant improvement of the simple Euclidean model (F ¼ 8.19, df ¼ 1,14, p ¼ 0.013), whereas the general Euclidean model is not a significant improvement of the weighted Euclidean model (F ¼ 3.45, df ¼ 1,13, p ¼ 0.86). We therefore conclude that the weighted Euclidean model is the best representation of the customer acceptance criterion. The explained variance for this criterion is R 2 ¼ 87 percent. The highest model for the ROI criterion is the simple Euclidean model (F ¼ 9.20, df ¼ 1,15, p ¼ 0.008). The explained variance by using this model is R 2 ¼ 81 percent. Interpretation The previous analysis demonstrates that in this particular case study new product success is a two-dimensional concept. Figure 1 shows the projections of the MDS-solution in these two dimensions. The results of the clustering are shown in the figure too (for the interpretation we will only use the 16 products that were consistently classified over the two cluster techniques), as are the ideal points of the preference mapping. Cluster membership and the position of the ideal points were used for the interpretation of this multidimensional space. When we start with the cluster results, we see that the arrangement of the two clusters corresponds with a vertical division of the multidimensional space: five products lie in the right part of the plane, whereas 11 lie in the left part. After substantive examination of these products, we labeled the right group “core products”, and the left group “additional products”. We can infer from this result that the first (horizontal) axis is capable of explaining the clustering in two groups completely. Because the core products are those products that are responsible for long-term profit, we can interpret the first dimension as a dimension that has a substantially profit component. When we look at the position of the ideal points, we see that they lie very close to the two axes of the multidimensional space: the Customer acceptance ideal point lies near to the vertical axis, the ROI ideal point near to the horizontal axis. This leads to a straightforward interpretation of the two axes. We conclude from these observations that the first dimension has an ROI component, and the second dimension has a Customer acceptance component. The two methods for interpretation of the multidimensional space point to the same meaning for the horizontal dimension. Because the clustering method used data different from the PREFMAP data (in meaning as well as in collection), we could say that our interpretation of the first dimensions was cross-validated. The interpretation of the second dimension was only inferred from the PREFMAP analysis. However, because the ideal point is positioned very closely to this dimension, we believe to have made a valid interpretation for this dimension. Discussion In this article we presented a way to measure new product success, without framing the respondents’ answers into aspects of success determined and restricted by researchers. Instead, the obtained definition was based on the prevailing subjective images of the players in their local context. The results of the analyses support the multidimensionality of the concept new product success. However, we observe only
Images of new product success
97
EJIM 11,1
98
two different dimensions of new product success. In the literature, a much larger amount of potential dimensions is distinguished. Ergo not all of the dimensions mentioned in the literature could be recognized in the empirical findings of this research. This corresponds with our expectation that a local definition, valid for this specific context, differs from the general theoretical definitions. We reasoned that there can not exist one definition for new product success which is valid for all possible contexts. Instead, local validity should be the aim, and the highest claim of researchers of new product success. Of course, this also implicates that the dimensions, or even the number of dimensions, found in this particular research (ROI and customer acceptance) could never be the final answer to the question which dimensions to use in measuring new product success. It is to be expected that the dimensions used in the organizational practice differ over organizations, departments of organizations, projects in organizations, and over time. Therefore, we argue that in measuring new product success, researchers should be aware of the time dependency and context dependency of the concept. They should use a measurement that suits the practice of the situation under study. Multidimensional scaling techniques provide a method to analyse, in a preliminary research, the (number of) dimensions actually perceived by parties in this situation. The results of the multidimensional scaling analysis can then be exploited in developing the measurement instrument to be used in the main research in this particular setting. In this research we have studied a homogeneous group of respondents with respect of their image of new product success. However, such homogeneity will not always be the case. Different images can emerge when exploring local validity, but they could also be the subject of a multidimensional scaling study. For example, a research question can be whether different stakeholders of innovation projects differ in their view on new product success. More concrete, do developers use another “definition” of new product success than managers or customers do? Within the family of multidimensional scaling techniques there is a method developed especially for answering questions about differences between (groups of) respondents. This technique, INDSCAL, can be used to explore if respondents differ in their weighting of the dimensions of the multidimensional space (Davison, 1992). In INDSCAL the overall space is based on the same dimensions for all the respondent groups, however the importance of these dimensions may vary. By comparing the weights of the dimensions, one can assess if respondent groups are different, and to what extent. When different images are found, the logical follow-up question is what these differences mean for the organization? We claim that this question cannot be answered in one way but that it depends on the situation how to deal with these differences. For example, multifunctional teams in product development projects are one situation in which dissimilar meanings of the success concept could arise. Because members of such teams have diverse backgrounds and different objectives, the possibility of different images of new product success is real. In this case, diversity in success definitions does not have to be detrimental for the functioning of the team. Diversity can also enhance the creativity of the team (Kratzer et al., 2006). However, variety in implicit images may be harmful for the development process when it leads to communication problems. The non-framing multidimensional scaling approach can help a multifunctional team to become aware of the range of success definitions in the
team. This insight can be useful in avoiding communication problems as well as in improving the creativity of the team. Another situation for which our approach is valuable can be found in the communication between multifunctional teams and the management of the organization. Different ideas about what new products success is can hinder the information exchange between the team and the management and as a result bias the decision making process of the management. Innovation discussions within the organization can be facilitated by revealing that management and members of the product development team have different viewpoints with respect to success. Management can even consider to try to change the images of the team members in such a way that these images are more aligned with the innovation strategy of the organization. A third situation we want to discuss in this respect is co-development. Because of differences in cultures, strategies and goals it is very likely that parties within this co-development context have varying opinions regarding new product success. Although aligning these opinions is probably too difficult to achieve, awareness is already a major gain. Therefore, measuring new product success without framing respondents’ answers can also be very supportive for a successful co-development process. In summary, our method could shed light on the difficulties that sometimes do arise when different parties are working together. As a consequence, not only researchers but also practitioners should become aware of the indefiniteness of the concept of new product success. Note 1. As said in the Method section, these orderings were collected separately from the comparison data which lead to the MDS-solution. References Alam, I. (2003), “Innovation strategy, process and performance in the commercial banking industry”, Journal of Marketing Management, Vol. 19 Nos 9-10, pp. 973-99. Avlonitis, G.J., Papastathopoulou, P.G. and Gounaris, S.P. (2001), “An empirically-based typology of product innovativeness for new financial services: success and failure scenarios”, Journal of Product Innovation Management, Vol. 18 No. 5, pp. 324-42. Borg, I. and Groenen, P. (1997), Modern Multidimensional Scaling: Theory and Applications, Springer-Verlag, New York, NY. Browne, P.J. and Golembiewski, R.T. (1974), “The line-staff concept revisited: an empirical study of organizational images”, Academy of Management Journal, Vol. 17 No. 3, pp. 406-17. Campbell, D.T. and Stanley, J.C. (1963), Experimental and Quasi-experimental Designs for Research, Houghton Mifflin, Boston, MA. Chryssochoidis, G.M. and Wong, V. (2000), “Customization of product technology and international new product success: mediating effects of new product development and rollout timeliness”, Journal of Product Innovation Management, Vol. 17 No. 4, pp. 268-85. Cook, T.D. and Campbell, D.T. (1979), Quasi-Experimentation: Design & Analysis Issues for Field Settings, Rand McNally, Boston, MA. Cooper, R.G. and Kleinschmidt, E.J. (1987), “Success factors in product innovation”, Industrial Marketing Management, Vol. 16 No. 3, pp. 215-23.
Images of new product success
99
EJIM 11,1
Cooper, R.G. and Kleinschmidt, E.J. (1995), “Benchmarking the firms’ critical success factors in new product development”, Journal of Product Innovation Management, Vol. 12 No. 5, pp. 374-91. Cooper, R., Easingwood, C., Edgett, S., Kleinschmidt, E. and Storey, C. (1994), “What distinguishes the top performing new products in financial services?”, Journal of Product Innovation Management, Vol. 11 No. 4, pp. 281-99.
100
Davidson, J.H. (1976), “Why most new consumer brands fail”, Harvard Business Review, Vol. 54 No. 2, pp. 117-22. Davison, M.L. (1992), Multidimensional Scaling, Krieger Publishing Company, Malabar, FL. De Brentani, U. (1989), “Success and failure in new industrial services”, Journal of Product Innovation Management, Vol. 6 No. 4, pp. 239-58. De Wit, A. (1988), “Measurement of success”, Project Management, Vol. 6 No. 3, pp. 164-70. Di Benedetto, C.A. (1999), “Identifying the key success factors in new product launch”, Journal of Product Innovation Management, Vol. 16 No. 6, pp. 530-44. Druckman, J.N. (2001), “On the limits of framing effects: who can frame?”, The Journal of Politics, Vol. 63 No. 4, pp. 1041-66. Edgett, S.J. and Snow, K. (1996), “Benchmarking measures of customer satisfaction, quality and performance for new service products”, Journal of Service Marketing, Vol. 10 No. 6, pp. 6-17. Gordon, A.D. (1980), Classification, Chapman & Hall, London. Graig, A. and Hart, S. (1993), “Dimensions of success in new product development”, in Baker, M.J. (Ed.), Perspectives on Marketing Management, Vol. 3, John M. Whiley & Sons, London, Chapter 10. Griffin, A. and Page, A.L. (1993), “An interim report on measuring product development success and failure”, Journal of Product Innovation Management, Vol. 10 No. 6, pp. 291-308. Griffin, A. and Page, A.L. (1996), “PDMA success measurement report: recommended measures for product development success and failure”, Journal of Product Innovation Management, Vol. 13 No. 6, pp. 478-96. Hart, S. (1993), “Dimensions of success in new product development: an exploratory investigation”, Journal of Marketing Management, Vol. 9 No. 1, pp. 23-41. Hart, S. (2003), “Industrial companies evaluation criteria in new product development gates”, Journal of Product Innovation Management, Vol. 20 No. 1, pp. 22-36. Hollander, J. (2002), “Improving performance in business development. Genesis of a tool for product development teams”, PhD thesis, University of Groningen, Groningen. Huff, A.S. (1990), in Huff, A.S. (Ed.), Mapping Strategic Thought, Wiley, Chichester, pp. 11-49. Hultink, E.J. and Atuahene-Gima, K. (2000), “The effect of sales force adoption on new product selling performance”, Journal of Product Innovation Management, Vol. 17 No. 6, pp. 435-50. Hultink, E.J. and Robben, H.S.J. (1995), “Measuring new product success: the difference that time perspective makes”, Journal of Product Innovation Management, Vol. 12 No. 5, pp. 392-405. Kelly, G.A. (1970), “A brief introduction to personal construct theory”, in Bannister, D. (Ed.), Perspectives in Personal Construct Theory, Academic Press, London, pp. 1-29. Kratzer, J., Leenders, R.T.A.J. and van Engelen, J.M.L. (2004), “Stimulating the potential: creative performance and communication in innovation teams”, Creativity and Innovation Management, Vol. 13 No. 1, pp. 63-71.
Kratzer, J., Leenders, R.T.A.J. and van Engelen, J.M.L. (2006), “Team polarity and creative performance in innovation teams”, Creativity and Innovation Management, Vol. 15 No. 1, pp. 96-104.
Images of new product success
Kruskal, J.B. (1964), “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis”, Psychometrika, Vol. 29 No. 1, pp. 1-27. Lant, T.K. and Phelps, C. (1999), “Strategic groups: a situated learning perspective”, Advances in Strategic Management, Vol. 16, pp. 221-47. Levin, I.P., Schneider, S.L. and Gaeth, G.J. (1998), “All frames are not created equal: a typology and critical analysis of framing effects”, Organizational Behavior and Human Decision Processes, Vol. 76 No. 2, pp. 149-88. Litwin, G.H. and Stringer, R.A. (1968), Motivation and Organizational Climate, Division of Research, Harvard Graduate School of Business Administration, Boston, MA. Lynn, G.S. and Akgu¨n, A.E. (2001), “Project visioning: its components and impact on product success”, Journal of Product Innovation Management, Vol. 18 No. 6, pp. 374-87. McElroy, T. and Seta, J.J. (2003), “Framing effects: an analytic-holistic perspective”, Journal of Experimental Social Psychology, Vol. 39 No. 6, pp. 610-17. Midgley, G. (2000), Systemic Intervention. Philosophy, Methodology, and Practice, Kluwer Academic/Plenum Publishers, New York, NY. Montayo-Weiss, M.M. and Calantone, R. (1994), “Determinants of new product performance: a review and meta-analysis”, Journal of Product Innovation Management, Vol. 11 No. 5, pp. 397-417. Pinto, J.K. and Slevin, D.P. (1988), “Project success: definitions and measurement techniques”, Project Management Journal, Vol. 19 No. 3, pp. 67-73. Smink, M. (2000), “Technological innovativeness as a moderator of new product design integration and top management support”, Journal of Product Innovation Management, Vol. 17 No. 3, pp. 208-20. Souder, W.E. and Jenssen, S.A. (1999), “Management practices influencing new product success and failure in the United States and Scandinavia: a cross-cultural comparative study”, Journal of Product Innovation Management, Vol. 16 No. 2, pp. 183-203. Spence, I. and Graef, J. (1974), “The determination of the underlying dimensionality of an empirically obtained matrix of proximities”, Multivariate Behavioral Research, Vol. 9 No. 3, pp. 331-42. Spence, I. and Ogilvie, J.C. (1973), “A table of expected stress values for random rankings in nonmetric multidimensional scaling”, Multivariate Behavioral Research, Vol. 8 No. 4, pp. 511-17. Storey, C. and Easingwood, C.J. (1999), “Types of new product performance: evidence from the consumer financial services sector”, Journal of Business Research, Vol. 46 No. 2, pp. 193-203. Sujan, H., Weitz, B.A. and Kumar, N. (1994), “Learning orientation, working smart, and effective selling”, Journal of Marketing, Vol. 58 No. 3, pp. 39-52. Tversky, A. and Kahneman, D. (1981), “The framing of decisions and the psychology of choice”, Science, Vol. 211 No. 4481, pp. 453-8. Wagenaar, W.A. and Padmos, P. (1971), “Quantitative interpretation of stress in Kruskal’s multidimensional scaling technique”, British Journal of Mathematical and Statistical Psychology, Vol. 24, pp. 101-10.
101
EJIM 11,1
Further reading Hultink, E.J. and Robben, H.S.J. (1999), “Launch strategy and new product performance: an empirical examination in The Netherlands”, Journal of Product Innovation Management, Vol. 16 No. 6, pp. 545-56. Kelly, G.A. (1991), The Psychology of Personal Constructs. Volume One – A Theory of Personality, Routledge, London (originally published in 1955).
102 About the authors Derk Jan Kiewiet is an Associate Professor at the Faculty of Management and Organization, University of Groningen, the Netherlands. He studied mathematics (BSc) and cognitive psychology (BSc and MSc), and has a PhD in management and organization studies (his PhD thesis is about a mathematical foundation of a decision support system for information planning), His current research interests are measurement issues of complex business concepts as sustainability and success, organizational learning, teamwork in product development and methodology. Marjolein Achterkamp is an Assistant Professor at the Faculty of Management and Organization, University of Groningen, the Netherlands. In 1994, she received her MSc in Applied Mathematics, and in 1999, she received her PhD in Sociology on the subject of influence strategies in collective decision making. Presently, her research focuses on sustainable innovation (in particular stakeholder involvement), organizational learning, and research methodology. Marjolein Achterkamp is the corresponding author and can be contacted at:
[email protected]
To purchase reprints of this article please e-mail:
[email protected] Or visit our web site for further details: www.emeraldinsight.com/reprints
