Parallel product alternatives and verification ...

This copy is author’s original pre-print version and may slightly differ from the official published version. Full citation: Harkonen, J., Mottonen, M., Belt, P. and Haapasalo, H. (2009) ‘Parallel product alternatives and verification & validation activities’, International Journal of Management and Enterprise Development, Vol. 7, No.1 pp. 86-97. DOI: 10.1504/IJMED.2009.025267

Parallel product alternatives and verification & validation activities Janne Harkonen*, Matti Mottonen, Pekka Belt and Harri Haapasalo Department of Industrial Engineering and Management, University of Oulu, Finland, P.O. Box 4610, FI-90014 University of Oulu, Finland E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] *Corresponding author Abstract: Business environment in the information and communications technology (ICT) sector is turbulent. Requirements for products change even during new product development (NPD). Profound information during the early part of NPD is critical for success, and verification & validation activities can aid in providing this required information. Also, developing parallel technologies and product alternatives during the early NPD supports gaining this information. Although developing multiple options may seem a waste of resources, this may prove beneficial, even for small and medium sized companies, in a changing environment. This study analyses how ICT companies deal with challenges caused by technological uncertainties.

Keywords: parallel development; product development; verification; validation; requirements; enterprise development.

1 Introduction Top management needs information on research and development (R&D) opportunities, and risks associated with technological change, in order to make business decisions. There is evidence of technological uncertainty influencing the product development approach, and thus being a vital challenge for companies (e.g. Magnusson and Berggren 2001). Although technological uncertainty is considered to be inherent characteristics in any product development process, this characteristic is particularly amplified when it involves cutting-edge technology (e.g. Song et al., 2007). Uncertainty can be understood as management’s challenge relating to decision-making, when all the relevant information is not available (e.g. Song and Montoya-Weiss, 2001). Parallel developments, and set-based approaches, are methods for tackling uncertainty by exploring several alternatives simultaneously, in order to gain more information. Developing parallel options influences the final decision-making by postponing it. Technological development, especially in the information and communications technology (ICT) sector, has resulted in complicated products, offering virtually unlimited feature possibilities for customers. Increased product complexity has caused difficulties in designing and manufacturing electronics gadgets defect-free (e.g. Woodward and Hennell, 2005; Black, 2004), and verification and validation (V&V) activities have thus become increasingly important for the development. There are estimates of V&V taking 30 to 80 percent of total resources in high-tech product development (e.g. Engel & Last, 2007; Gilb, 2005; Runeson et al., 2003; Murray, 2007). V&V is an effective means for collecting information and thus reducing uncertainty. However, V&V activities are typically conducted in an environment with great uncertainties. The above mentioned is the motivation for considering parallel development during new product development (NPD) process for the added ease of dealing with uncertainties and organising V&V. Also, the timing of technological decision-making during new product development (NPD) is covered in this study. The research objective can be condensed into the following research questions: RQ1 what type of possibilities are there for reducing NPD uncertainty in the ICT sector, and improving V&V practices? RQ2 how are this type of companies reducing uncertainty in practice?

2 Theory 2.1 Developing parallel alternatives during NPD process Conventional design practices often come together with a quick solution, and modify it later until meeting objectives. This type of approach is also called point-based product development, where a successful solution depends on the initial choices (e.g. Joglekar and Rosenthal, 2003).

Parallel product alternatives and verification & validation activities There are different types of mental frames for gradually selecting from a large set of possibilities. These include funnel approach, and set-based approach (e.g. Wheelwright & Clark, 1992a; Cooper, 2008; Liker et al., 1996; Yassine et al., 2008). The intention of these approaches is to cope with uncertainty by postponing final decision making. For example, Ford and Sobek (2005) argue that longer time spent in early NPD allow companies to have more options and better solutions later. Recent literature has raised set-based product conceptualisation as an important issue for flexible management of product development, allowing handling multiple options, and gaining timely feedback (e.g. Zhang et al., 2008; Ford and Sobek, 2005). Companies are able to anticipate changes with set-based approaches, and facilitate operational reactions to changes, should they be caused by technical, or engineering problems, or customer preferences. This approach allows offering a number of products effectively, with reasonable costs. This virtually inefficient approach has made, for example, Toyota’s development time fastest in its industry (e.g. Morgan and Liker, 2006). Toyota's set-based concurrent engineering (SBCE) approach of intentionally delaying alternative selection is explained as a real options approach to product development management (Ford and Sobek, 2005; Morgan and Liker, 2006). SBCE utilised by Toyota includes different actors working on rough-cut designs within defined frames, that they have means to work within. This type of approach aids to avoid over-the-wall designs, where different departments do not sufficiently support the realities of other stages facing the design later in the process (Hines et al., 2006).

2.2 Platforms The use of platforms is a mechanism to cope with uncertainty in product development (Zhang & Doll, 2001). Product platforms have been seen as a significant R&D tool (Gershenson et al., 2003; Robertson & Ulrich, 1998; Salonen et al., 2008; Meyer & Utterback, 1993), thus are more often considered as a planning construct rather than a single product construct (e.g. Yang and Jiang, 2006). Platforms are understood as the basic structure of products with a common set of components, modules, and parts from which a stream of derivate products can be effectively created, and launched (e.g. Meyer & Lehnerd, 1997; Yakob and Tell, 2007). The core technology of the products remains the same (e.g. Koufteros et al., 2005). This type of an approach decreases the time used for new product planning, as there is no need to engineer the entire product from scratch, while useful elements already exist (e.g. Robertson & Ulrich, 1998.). Product changes can be made rapidly due to technical and marketing uncertainties being lower (e.g. Koufteros et al., 2005). When platforms are utilised, the product can be tailored for the customers needs, fairly late during the development process (e.g. Ratamaki, 2004). Nevertheless, platforms can be complicated to design, and can also fail in many ways (e.g. Robertson & Ulrich, 1998), and developing them requires involvement of multiple parties (Wheelwright and Clark, 1992b). Platform products are also seen to foster organisational learning while the development process is repeated with a great frequency. This type of continuous improvement helps teams share knowledge over customers, suppliers, competitors, products, and internal capabilities. (Zhang and Doll, 2001; Lee, 2007).

2.3 Roadmaps Roadmapping is increasingly used by companies to derive business visions, and to describe linkages, and necessary actions (EIRMA, 1997; Bullinger et al., 2004). When people can visualise a better future, they can begin to create it – in other words decrease

uncertainty. In the early 1980s, Motorola utilised a combination of techniques to integrate product and technology planning for formulating a technology strategy, while calling the outcome a technology roadmap. (Betz, 1993; Willyard and McClees, 1987). The purpose of a roadmap is to make sure the right capabilities are in place, at the right time, to achieve this objective (e.g. Probert and Radnor, 2003; Lee et al., 2007). A technology roadmap coordinates and synchronises different functions. It is important that people have the same understanding. The essence of roadmapping, for organisational knowledge-sharing and learning have been emphasised by different authors (e.g. Groenveld, 1997; Phusavat and Kess, 2008).

2.4 Project portfolio approach Typically technological uncertainty is seen to be relevant for decision making when it is considered in conjunction with market uncertainties, or revenue uncertainties (e.g. Sengupta, 2005; Murto, 2007). Also, both the supply and demand side of the market are seen to be characterised by technological uncertainty (Gressgard and Stensaker, 2006). Different models, with different levels of distinction, are used to communicate issues related to technological uncertainty, Matthews (1991) probably being among the most unambiguous. Yet, also Wheelwright and Clark's (1992) product development funnel has been developed into versions that have uncertainty as a more primary concern (Feland et al., 2004). Matthews (1991) categorises R&D projects into three different groups according to the level of uncertainty and allocated resources. The first category is long-term research for developing future technologies. The level of uncertainty is high, while these projects are considered as overhead costs necessary for companies’ future success. Matthews calls the second category strategic options and locates the category between long-term research and short-term R&D. The purpose of this category is to assess and select the right technologies by reducing uncertainty. The third category of R&D projects can be considered as investments, while the development costs, and manufacturing costs, as well as the markets are known for calculating return on investment (ROI). The level of uncertainty at this stage is low. In other words, companies should have separate projects, between long–term research, and operative development, for addressing uncertainty. This can be done by evaluating potential options, resulting in lower level of uncertainty, allowing decisions on large scale development projects. (e.g. Suomala, 2004; Matthews, 1991; Suomala and Jokioinen, 2003; Suikki, 2007).

2.5 Verification & validation and NPD uncertainty Verification and validation (V&V) is an effective way for collecting information, and thus reducing uncertainty. However, these activities are typically conducted in an environment with great uncertainties. There is evidence of both, acknowledging the need for earlier consideration of V&V, and the current situation being non-ideal (e.g. Boehm, 1981; Belt et al., 2008; Host and Johansson, 2000; Harkonen et al., 2008; Mahanti and Antony, 2006). It is a known fact how errors, or misunderstood requirements, are more expensive to fix, the later they are addressed during the development (e.g. Boehm, 1981; Chun, 2006; Firesmith, 2007). Cost reduction has been a traditional motivation for searching ways to shift the emphasis towards earlier phases in the NPD. There are methods that allow learning from technologies during the development, including simulation, ijiwaru testing (Morgan and Liker, 2006), set-based methods, and such. The distribution of relative V&V efforts is illustrated in Figure 1.

Parallel product alternatives and verification & validation activities Figure 1. Relative efforts of V&V activities and desired development directions

Traditional pass/fail approach to V&V does not decrease, nor aid coping with technological uncertainty, as the information provided is of on/off type compliance indication. Typically, the requirements are stabilising during the NPD process (e.g. Ebert, 2007; Beecham et al., 2005) and the role of V&V changes. V&V, when sufficiently organised, can be utilised to provide valuable information about the subject. These ‘setbased’ type V&V methods are a potential opportunity for improving the performance of V&V during the early NPD, and for decreasing uncertainty. Verification should be more of an integral part of the design, for which typical methods are measuring, simulation, and modelling. In the later stages of the NPD chain (product integration), pass/fail verification is more effective, when efficient execution is crucial. Some elements of verification can be conducted outside the design and are therefore mostly pass/fail type. (e.g. Perttula, 2007). Ideal division of V&V efforts between set-based type, and pass/fail V&V is illustrated in Figure 2. Figure 2. Ideal division of V&V efforts between set-based type, and pass/fail V&V

3 Empirical study 3.1 Research Process The empirical study was conducted to obtain data for analysing how ICT companies address technological uncertainty. The empirical study consisted of 48 interviews, comprising a representation from different phases of the product development process. Interviews were conducted informally, allowing the interviewees to explain and clarify the cases and topics as entities. All the individual interview results were analysed

separately for the following aspects of uncertainty reduction: parallel development, platforms, roadmaps, project portfolio approach, and others identified. The interviewed companies provide products for both business-to-customer, and business-to-business markets. Some of the companies can be seen as subcontractors, whose products are inputs into the first category. Therefore, these interviews represent uncertainty and V&V activities in a versatile manner and provide understanding of managing diverse issues. The participants interviewed were carefully selected on the basis of their professional background and expertise. Selected participants hold responsible positions in the interviewed companies. The experience and the current interests ensured the high motivation among the participants and up-to-date knowledge with respect to the discussed topics.

3.2 Results 3.2.1 Parallel development The study indicates the majority of the interviewed companies not developing parallel technological alternatives. In some cases, two parallel options are used, seldom more. According to the companies, the most important reason is resource limitations. Only the largest of the studied companies use the strategy of developing several parallel alternatives, thus making postponing the final technological selection possible. Typically, the companies fix technologies as early as possible. The milestone models utilised by the companies do not promote open options, but rather guide for final decisions. In the case of an unexpected situation, the strategy of developing a single solution leads to a panic mode, causing unnecessary costs and delayed schedule. Some of the studied companies attempt addressing this type of problems by preparing back-up plans for situations when a chosen critical technology proves inadequate. However, companies are not developing parallel alternatives, but rather, back-up plans are seen to be developed with much less resources. This does reduce the overall development time but only when the selected technology gives the desired results.

3.2.2 Platforms All the interviewed companies utilise platform-based product development, for freezing some common elements, and thus decreasing the associated uncertainty. This also enables faster product variants. The share of this type of development is identified to be increasing. The purpose of platforms is to act in a Lego block manner, so that they can be utilised rapidly for reacting to changes in the marketplace. The life-cycle of a platform is typically several years, and it thus determines the basic technologies for a long time. According to the interviews, standardising critical components, and interfaces, is another approach for tackling uncertainty. Platforms ease the pressure on verification and validation activities shifting the focus onto interfaces. However, in practice it is a challenge for the V&V activities, as according to the interviewees, too often the platforms are taken into product development before they are finalised and properly tested.

3.2.3 Roadmaps All the studied companies are striving for technological leadership, and as a consequence technology management is important to them. They systematically follow the

Parallel product alternatives and verification & validation activities development of technology and customer needs. Roadmapping, covering typically next five years, is used for describing these developments. Decisions on feasibility studies, which assess the importance of technologies, are based on the roadmaps. Despite of striving for technological leadership, a part of the studied companies do not typically try to patent their inventions, but rather analyse competitors’ patenting for learning purposes. The interviewed companies have realised that the ability to react to changes is significantly improved when actively listening and analysing the market. Should there be any radical changes in customer needs, or technologies, the studied companies aim to adapt to these changes rapidly. This is attempted also during product development projects. Regardless of all the companies utilising roadmapping, the profoundness, level, and the extent of this varies significantly. Some of the larger, and more advanced companies, invest considerably into drawing up this type of plans, when the smaller ones tend to have a more shallow approach.

3.2.4 Project portfolio approach The studied companies screen potential technologies, and then select the ones to be utilised, based on their company strategy and market & customer needs. Different types of feasibility studies are used for potential technologies, in order to reduce the level of uncertainty, prior to committing significant resources. These decisions are considered as strategic choices for the companies. Milestones are used in most of the interviewed companies during the NPD process, and they are considered as natural points for assessing the maturity, and the level of uncertainty of projects. V&V are seen as a crucial part of these assessments. The final technology selections are typically attempted as early as possible, as most of the interviewed companies do not develop parallel solutions. The motivation for this is to avoid wasting resources on projects that will not be finalised.

3.2.5 Others The interviewees highlighted few alternative ways for reducing uncertainty associated with technological choices. Prototyping is seen to be used to complement designs, as a means for studying the subject under development in a tangible manner, and to learn from the new technology. Prototypes are also used together with pilot customers to check whether a product meets the customer needs. The interviewees emphasise the importance of the earlier part of the product development process, including technological choices and project planning. Some of the studied companies aim to influence the future standards, and thus to minimise the negative effects of technological uncertainty. By doing so, the companies gain significant know-how on future technologies likely to dominate. The interviewees feel that the business environment is getting increasingly turbulent, and as a consequence, companies must be able to react quickly to any changes in the market place. The companies, especially those developing software, are addressing this challenge currently by adopting agile methods. In agile development, features are developed and launched before completing the final products.

3.3 Summary The interviewed companies are aware of the necessity of emphasising the early part of the development cycle. Roadmapping is used in the beginning of development projects to clarify customer needs and technological opportunities. Feasibility studies are used after roadmapping to decrease the uncertainty with technologies, prior to committing significant amount of resources. Feasibility studies are utilised as a basis for deciding in which technologies to invest. The study showed that only the most successful companies utilise, in practice, the principles of parallel development. In addition, companies try to minimise the uncertainty in product development, participating in creation of business sector standards, and using platforms, prototyping and agile development methods. Most of the interviewed companies do not postpone final technological decisions, rather they tend to make the decisions as early as possible. Figure 3 describes the methods the companies are using for coping with uncertainty. Figure 3. The share of companies applying the studied methods for reducing uncertainty

% 100

80

60

40

20

0 Parallel development

Platforms

Roadmaps

Project portfolios

4 Conclusions Companies are constantly living in uncertain, turbulent business environment. It is crucial for business success to minimise R&D risks and develop right products at the right time. The theoretical part of this paper identified the following methods for reducing uncertainty in this type of environment: parallel development, platforms, roadmapping, project portfolio approach, and V&V activities. The empirical part of the study showed that all the studied companies utilise platforms and roadmapping, the majority uses project portfolio approach, but only the

Parallel product alternatives and verification & validation activities largest and most successful apply parallel development in a systematic manner. This study indicated that especially small and medium sized ICT companies tend to make the technological decisions as early as possible. In addition, companies try to minimise uncertainty in product development through participating in business sector standard creation, by using prototyping, and through agile development methods. The tendency of small and medium sized companies to freeze their technological solutions as early as possible leads to a great technological uncertainty, and thus is a challenge for managers. Should these decisions prove wrong, significant costs are created in the form of wasted resources and delayed development. Especially, SMEs should consider possibilities of using parallel development, and thus follow the more advanced companies. Managers in all enterprises should also strive for shifting the emphasis of V&V activities to the earlier phases of the NPD chain. In addition, the share of understanding increasing V&V activities, such as simulation and modelling, should be highlighted. The current emphasis of V&V is at the end of late product development, and in production, not in the early product development where V&V would be the most effective. This study covered a collection of methods, identified potential for SMEs, for reducing uncertainty. Therefore, should a wider scope be selected, the results could vary somewhat, and potentially describe the set of available methods more thoroughly. The empirical part of the research analysed whether the selected methods are in use, yet the profoundness of utilising these methods requires further study. Additional research is required to clarify how to find balance between the benefits of parallel development, and cost control, in product development. In addition, managing customer requirements in turbulent business environment requires further research, as well as, widening the perspective outside ICT.

References Beecham, S., Hall, T. and Rainer, A. (2005) 'Defining a Requirements Process Improvement Model', Journal of Systems and Software, Vol. 76, No. 3, pp. 251-275. Belt, P., Harkonen, J., Mottonen, M., Kess, P. and Haapasalo, H. (2008) ‘Improving the efficiency of verification and validation’, International Journal of Services and Standards, Vol. 4, No. 2, pp. 150 - 166. Betz, F. (1993) ‘Strategic Technology Management’, McGraw-Hill, New York, pp. 476. Black, R. (2004) ‘Critical Testing Process – Plan, Prepare, Perform, Perfect’, Boston: AddisonWesley. Boehm, B. W. (1981) 'Software Engineering Economics', Prentice-Hall, Englewood Cliffs, NJ. Bullinger, H.J., Auernhammer, K. and Gomeringer, A. (2004) 'Managing innovation networks in the knowledge-driven economy', International Journal of Production Research, Vol. 42, No. 17, pp. 3337-3353. Chun, Y.H. (2006) 'Estimating the number of undetected software errors via the correlated capture– recapture model', European Journal of Operational Research, Vol. 175, No. 2, pp. 1180-1192. Cooper, R.G. (2008) 'Perspective: The Stage-Gate® Idea-to-Launch Process—Update, What's New, and NexGen Systems', Journal of Product Innovation Management, Vol. 25, No. 3, pp. 213-232. Ebert, C. (2007) 'The impacts of software product management', Journal of Systems and Software, Vol. 80, No. 6, pp. 850-861. Engel, M. and Last, M. (2007) ´Modeling software testing costs and risks using fuzzy logic paradigm´, The Journal of Systems & Software, Vol. 9, No. 3, pp. 259-280.

EIRMA (1997) ‘Technology Roadmapping - Delivering Business Vision’, Working Group Report, European Industrial Research Management Association, Paris, No.52, pp.61. Feland, J.M., Leifer, L.J. and Cockayne, W.R (2004) 'Comprehensive Design Engineering: Designers Taking Responsibility', International Journal of Engineering Education, Vol. 20, No. 3, pp. 416-423. Firesmith, D.G. (2007) 'Common Requirements Problems, Their Negative Consequences, and Industry Best Practices to Help Solve Them', Journal of Object Technology, vol. 6, no. 1, pp. 17-33. Ford, D. and Sobek, D. (2005) ‘Adapting real options to new product development by modeling the second Toyota paradox’, IEEE Transactions on Engineering Management, Vol. 52, No. 2, pp. 175–187. Gershenson, J.K., Prasad, G.J. and Zhang, Y. (2003) 'Product modularity: definitions and benefits', Journal of Engineering Design, Vol. 14, No. 3, pp. 295–313. Gilb, T. (2005) ‘Competitive Engineering: A Handbook for Systems Engineering, Requirements Engineering, and Software Engineering Using Planguage’, pp.480. Gressgard, L.J. and Stensaker, I. (2006) 'The mobile service industry: strategic challenges and future business models', International Journal of Mobile Communications, Vol. 4, No. 5, pp. 509 - 531. Groenveld, P. (1997) ‘Roadmapping Integrates Business and Technology’, Research – Technology Management, Vol. 40, No. 5, pp. 48-55. Harkonen, J., Belt, P., Mottonen, M., Kess, P. and Haapasalo, H. (2009) ’Maturity of verification and validation in ICT companies’, International Journal of Innovation and Learning, Accepted – forthcoming Hines, P., Francis, M. and Found, P. (2006) 'Towards lean product lifecycle management: A framework for new product development', Journal of Manufacturing Technology Management, Vol. 17, No.7, pp. 866-887. Host, M. and Johansson, C. (2000) 'Evaluation of code review methods through interviews and experimentation', Journal of Systems and Software, Vol. 52, No. 2-3, pp. 113-120. Joglekar, N.R. and Rosenthal, S.R. (2003) 'Coordination of Design Supply Chains for Bundling Physical and Software Products', Journal of Product Innovation Management, Vol. 20 No. 5, pp.374-390. Lee, C-W. (2007) 'The innovation and success of consumer electronics using new product development process', International Journal of Innovation and Learning, Vol. 4, No. 6, pp. 587-611. Lee, S., Kang, S., Park, Y. and Park, Y. (2007) 'Technology roadmapping for R&D planning: The case of the Korean parts and materials industry', Technovation, Vol. 27, No. 8, pp. 433-445. Liker, J.K., Sobek, D.K., Ward, A.C. and Cristiano, J.J. (1996) '‘Involving Suppliers in Product Development in the U.S. and Japan: Evidence for Set-Based Concurrent Engineering', IEEE Transactions on Engineering Management, Vol. 43, No. 2, pp. 165–177. Koufteros, X., Vonderembse, M. and Jayaram, J. (2005) 'Internal and External Integration for Product Development: The Contingency Effects of Uncertainty, Equivocality, and Platform Strategy', Decision Sciences, Vol. 36, No. 1, pp. 97-133. Magnusson, T., Lindstrom, G. and Berggren, C. (2003) ‘Architectural or modular innovation? Managing discontinuous product development in response to challenging environmental performance targets’ International journal of innovation management, Vol. 7, No. 1, pp. 1 26. Magnusson, T. and Berggren, C. (2001) 'Environmental innovation in auto development managing technological uncertainty within strict time limits', International Journal of Vehicle Design, Vol. 26, No. 2-3, pp. 101 - 115. Mahanti, R. and Antony, J. (2006) 'Six Sigma in software industries: some case studies and observations', International Journal of Six Sigma and Competitive Advantage, Vol. 2, No. 3, pp. 263-290.

Parallel product alternatives and verification & validation activities Matthews, W.H. (1991) ‘Kissing technological frogs: Managing technology as a strategic resource’, European Management Journal, Vol. 9, No. 2, pp.145-148. Meyer, M.H., Lehnerd, A.P., (1997) ‘The Power of Product Platforms: Building Value and Cost Leadership’, The Free Press, New York, pp.288. Meyer, M.H. and Utterback J.M. (1993) 'The product family and the dynamics of core capability. Sloan Management Review, Vol, 34, No. 3, pp. 29-47. Morgan, J.M. and Liker, J.K. (2006) 'The Toyota product development system: integrating people, process, and technology', Productivity Press, pp. 377. Murray, R.M. (2007) 'Recent Research in Cooperative Control of Multivehicle Systems', Journal of Dynamic Systems, Measurement, and Control, Vol. 129, No. 5, pp. 571-583. Murto, P. (2007) 'Timing of investment under technological and revenue-related uncertainties', Journal of Economic Dynamics and Control, Vol. 31, No. 5, pp. 1473-1497. Phusavat, K. and Kess, P. (2008) 'Roadmap for knowledge sharing and transfer: sustaining outsourcing relationships', International Journal of Innovation and Learning, Vol. 5, No. 5, pp. 496-511. Probert D. and Radnor M. (2003a) ‘Frontier experiences from industry-academia consortia Corporate roadmappers create value with product and technology roadmaps’. Research – Technology Management, Vol. 46, No. 2, pp. 27-30. Probert, D. and Radnor, M. (2003b) 'Frontier experiences from industry-academia consortia', Engineering Management Review, Vol. 31, No. 3, pp. 28-28. Ratamaki, K. (2004) 'Product Platform Development from the Product Lines’ Perspective: Case of Switching Platform', Acta Universitatis Lappeenrantaensis, publication No. 201. Robertson, D. and Ulrich, K. (1998) ‘Planning for Product Platforms’, Sloan Management Review, vol. 39, No. 4, pp. 19-31. Runeson, P., Andersson, A. and Host, M. (2003) 'Test processes in software product evolution - a qualitative survey on the state of practice', Journal of Software Maintenance and Evolution: Research and Practice, Vol. 15, No. 1, pp. 41-59. Salonen, M., Holtta-Otto, K. and Otto, K. (2008) 'Effecting product reliability and life cycle costs with early design phase product architecture decisions', International Journal of Product Development, Vol. 5, No. 1-2, pp. 109-124. Sengupta, J.K. (2005) 'Nonparametric efficiency analysis under uncertainty using data envelopment analysis', International Journal of Production Economics, Vol. 95, No. 1, pp. 39-49. Song, Y-I., Lee, D-H., Lee, Y-G. and Chung, Y-C. (2007) 'Managing uncertainty and ambiguity in frontier R&D projects: A Korean case study', Journal of Engineering and Technology Management, Vol. 24, No. 3, pp. 231-250. Song, M. and Montoya-Weiss, M.M. (2001) 'The effect of perceived technological uncertainty on Japanese new product development", The Academy of Management Journal, Vol. 44, No. 1, pp. 61-80. Suikki, R. (2007) 'Changing business environment—effects of continuous innovations and disruptive technologies', Acta Universitatis Ouluensis Technica, C 265. Suomala, P. (2004) 'The Life Cycle Dimension of New Product Development Performance Measurement', International Journal of Innovation Management, Vol. 8, No. 2, pp. 193-221. Suomala, P. and Jokioinen, I. (2003) 'The patterns of success in product development: a case study', European Journal of Innovation Management, Vol. 6, No. 4, pp. 213-227. Wheelwright, S.C. and Clark, K.B. (1992a) 'Revolutionizing Product Development', The Free Press, New York. Wheelwright, S.C. and Clark, K.B. (1992b) ‘Creating Project Plans to Focus’, Harvard Business Review, Vol. 70, No.2, pp. 70-82. Willyard C.H. and McClees C.W. (1987) ‘Motorola’s Technology Roadmap Process’, Research – Technology Management, Vol. 30, No. 5, pp.13-19.

Woodward, M.R. and Hennell, M.A. (2005) ‘Strategic benefits of software test management: a case study’, Journal of Engineering and Technology Management, Vol. 22, Nos. 1–2, pp.113–140. Yakob, R. and Tell, F. (2007) 'Managing near decomposability in complex platform development projects', International Journal of Technology Intelligence and Planning, Vol. 3, No. 4, pp. 387-407. Yang, C. and Jiang, S. (2006) 'Strategies for Technology Platforms', Research-Technology Management, Vol. 49, No. 3, pp. 48-57. Yassine, A.A., Sreenivas, R.S. and Zhu, J. (2008) 'Managing the exchange of information in product development', European Journal of Operational Research, Vol. 184, No. 1, pp. 311326. Zhang, Q. and Doll, W.J. (2001) 'The fuzzy front end and success of new product development: a causal model', European Journal of Innovation Management, Vol. 4, No. 2, pp. 95-112. Zhang, Q., Vonderembse, M.A. and Cao, M. (2008) 'Product concept and prototype flexibility in manufacturing: Implications for customer satisfaction', European Journal of Operational Research, Forthcoming