A performance-oriented risk management framework for innovative ...

Technovation ] (]]]]) ]]]–]]]

Contents lists available at ScienceDirect

Technovation journal homepage: www.elsevier.com/locate/technovation

A performance-oriented risk management framework for innovative R&D projects Juite Wang a,n, Willie Lin b, Yu-Hsiang Huang c a

Institute of Technology Management, National Chung Hsing University, 250 KuoKuang Road, Taichung 402, Taiwan, ROC Program of Technology Management, College of Management, Fu Jen Catholic University, 510 Jhong-Jheng Rd, Sinjhuang, Xin Taipei City 24205, Taipei 100, Taiwan, ROC c Department of Business Administration, College of Business, University of Illinois at Urbana-Champaign, 1206 South Sixth Street, Champaign, IL 61820, USA b

a r t i c l e in fo

Keywords: R&D management Risk management Project management Balanced scorecard Quality function deployment

abstract Uncertainty is one of the major inherent difficulties in developing innovative products, due to their highly dynamic markets and technologies. The presence of a large degree of uncertainty leads to high R&D risks, resulting in many R&D failures. Therefore, it is important to manage R&D risks through all R&D stages to improve R&D project success rates. This paper proposes a new risk management framework that aligns project risk management with corporate strategy and a performance measurement system to increase success rates of R&D projects and to accomplish corporate strategic goals. The balanced scorecard is used to identify major performance measures of an R&D organization based on the firm vision and strategy. Quality function deployment is adapted to transform organizational performance measures into project performance measures and a systematic procedure is developed for risk identification, assessment, response planning, and control. The proposed risk management framework enables an R&D project to be focused on achieving the corporate goals and provides a more effective way to identify, assess, analyze, and monitor R&D risks along the project cycle. The proposed methodology is illustrated with a drug development project. & 2010 Elsevier Ltd. All rights reserved.

1. Introduction In the increasingly competitive and globalized marketplace, technological innovation is one of the important key strategies for high technology firms to survive and achieve corporate growth (Teece, 1986; Freeman and Soete, 1997). However, various types of innovation (Dewar and Dutton, 1986; Henderson and Clark, 1990) involve different degrees of uncertainty in technologies and markets that may cause failures of R&D projects (Doctor et al., 2001; Raz et al., 2002; Lee et al., 2010). For example, in the pharmaceutical industry, the success rate of a drug development project from the first study in humans to launch is less than 10% (CMR, 2006). Therefore, it is important to manage risks for innovative R&D projects through all the development stages to improve their success rates (Smith and Merritt, 2002; Keizer et al., 2002; Bush et al., 2005; Pisano 2006). Risk management is a structured approach for the identification, assessment, and prioritization of risks followed by planning of resources to minimize, monitor, and control the probability and impact of undesirable events (Smith and Merritt, 2002). It has

n

Corresponding author. Tel.: +886 4 22840547x605; fax: + 886 4 22859480. E-mail addresses: [email protected] (J. Wang), [email protected] (W. Lin), [email protected] (Y.-H. Huang).

been widely applied in many disciplines, such as management, engineering, insurance, finance, environment, politics, etc. In R&D management, the major purpose of risk management is to increase success rate of an R&D project, which will lead to corporate success. Most literature in the R&D risk management literature is more focused on an individual project level, and so the ways to identify, assess, and prioritize risks are limited within a single project scope (Smith, 1999; Browning et al., 2002; Keizer et al., 2002; Raz et al., 2002; Saari, 2004; Keizer et al., 2005). The main problem is that if the identified risks are improperly identified and prioritized, then time and cost can be wasted in dealing with risk of losses. Therefore, there is a need to link individual project risk management with the corporate strategic management to ensure that managed risks are coped with by the corporate strategy and corporate objectives can be eventually achieved. This research considers risk to be an event having a negative impact on project outcomes (Browning et al., 2002; Raz et al., 2002; Smith and Merritt, 2002; Keizer et al., 2002, 2005; Perminova et al., 2008) and develops a new risk management framework that aligns project risk management with corporate strategy and a performance measurement system to increase success rates of R&D projects and to accomplish the corporate strategic objectives. The proposed framework, which follows the risk management process that have been widely used in industry,

0166-4972/$ - see front matter & 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.technovation.2010.07.003

Please cite this article as: Wang, J., et al., A performance-oriented risk management framework for innovative R&D projects. Technovation (2010), doi:10.1016/j.technovation.2010.07.003

2

J. Wang et al. / Technovation ] (]]]]) ]]]–]]]

integrates the balanced scorecard (BSC) (Kaplan and Norton, 1992) and quality function deployment (QFD) (Hauser and Clausing, 1988) to help project managers organize risk management activities in a top–down manner. The BSC is used to identify major performance measures of an R&D organization based on the firm vision and strategy. Furthermore, QFD is adapted to transform organizational performance measures into project performance measures and a step-by step procedure is developed for risk identification, assessment, response planning, and control. The proposed risk management framework enables an R&D project to be focused on achieving the corporate goals and provides a more effective way to identify, assess, analyze, and monitor R&D risks along the project cycle. To our best knowledge, there has been no research study that provides an integrated risk management framework based on the BSC and QFD to link R&D risk management with corporate strategy and a performance measurement system. This paper is organized as follows. Section 2 reviews the related literature. The proposed risk management framework is developed in Section 3. In Section 4, the proposed methodology is illustrated with a hypothetic drug development project. Section 5 concludes the paper.

2. Literature review 2.1. R&D risk management There are many definitions of risk that vary by different application domains. In economic theory, risk refers to situations where the decision maker can assign probabilities to different possible outcomes (Knight, 1921). Similarly, in decision theory, risk is the fact that the decision is made under the condition of known probability over the states of nature (Luce and Raiffa, 1957). In project management, there is no consistent definition for risk (Ward and Chapman, 2003; Perminova et al., 2008). In the project management body of knowledge (Project Management Institute, 2004), risk is considered as ‘‘an uncertain event or condition that, if it occurs, has a positive (opportunity) or negative (threat) impact on project objectives.’’ However, many practitioners and researchers in project management still consider risk to be more related to adverse effects on project performance (Williams, 1995; Boehm and DeMarco, 1997; Smith and Merritt, 2002; Ward and Chapman, 2003). From this perspective, project risk management seems to be about identifying and managing threats to the project. Furthermore, in the literature of R&D management, uncertainty is defined as unpredictability of the environment, inability to predict the impacts of environmental change, and inability to predict the consequences of a response choice (Milliken, 1987; Doctor et al., 2001; Sicotte and Bourgault, 2008). Risk is often defined as undesired project outcomes, exposure to uncertainty (Smith, 1999; Browning et al., 2002; Raz et al., 2002; Smith and Merritt, 2002; Keizer et al. 2002, 2005). This research follows the definition that is mostly used in the literature of R&D risk management and defines the risk as an event having a negative impact on project outcomes. Managing R&D uncertainty to enhance project success rates has been studies for many years (Doctor et al., 2001; Loch et al., 2006). Risk management is one of the approaches that have been widely applied in practice (Williams, 1995; Smith, 1999; Keizer et al., 2002; Raz et al., 2002; Cooper, 2003; Smith and Merritt, 2002). In the literature of R&D risk management, several studies have found that applying risk management techniques to innovative R&D projects can improve their success rates (Raz et al., 2002; Salomo et al., 2007; O’Connor et al., 2008). Smith

(1999) described principles and guidelines for effective risk management and emphasized the importance of active risk management for accelerating projects and improving their success rates. Raz et al. (2002) performed an empirical study and reported that risk management practice is more applicable for higher-risk projects and appears to be related to project success. Salomo et al. (2007) investigated the effects of business planning and control on the performance of new product development projects and found that project risk planning and goal stability throughout the development process are found to enhance performance significantly. O’Connor et al. (2008) defined three learning oriented risk management practices, including option mentality, use of experimental and learning processes, and use of harvest strategy, and found that using the first two practices has a significant positive effect on the success of radical innovative project. Mu et al. (2009) conducted an empirical study and showed that risk management strategies targeting technological, organizational, and marketing risk factors influence the performance of new product development. Several researchers have developed risk management methodologies to improve success rates of R&D projects. Browning et al. (2002) proposed a risk value methodology that quantifies technical performance risks to identify, assess, monitor, and control the identified risks throughout the project. However, their research is only focused on technical risks. Keizer et al. (2002) presented a case study of the risk diagnosing methodology (RDM) developed by Philips Electronics Co. to identify and evaluate technological, organizational, and business risks in product innovation. Since R&D is people and knowledge intensive, Cooper (2003) suggested using knowledge management systems and collaboration tools that capture practitioner experience for reducing R&D risks. Keizer et al. (2005) proposed a risk reference framework for diagnosing risks in technological breakthrough projects and concluded that the success of breakthrough innovation projects could be improved through formal risk assessment. Gidel et al. (2005) developed a decision making framework for risk management from the cognitive science viewpoint. Ogawa and Piller (2006) suggested integrating customers into the innovation process and proposed a new market research concept called ‘‘collective customer commitment’’ to reduce the risk of unmet customer needs. In addition, several studies have been published on determinants of new products success and failure (Maidique and Zirger, 1984; Cooper et al., 2004). The key success factors identified in these studies can be used for identifying potential risks. Due to the long development lead-time, rising development cost, and high failure rate for drug development projects, effective management of R&D risks is important to the pharmaceutical industry. Most of the pharmaceutical risk management has been focused on managing drug safety issues including detection, assessment, understanding and prevention of long-term and short-term adverse effects of medicines (Bush et al., 2005). Some researches have studied the pharmaceutical risk management at the drug development project level. For example, Saari (2004) applied the project risk management framework to the drug development project. Vanderbyl and Kobelak (2008) identified growth and risk factors for Canadian biotechnology industry and suggested that a risk mitigation plan is required to manage those risk factors for project success. Some studies have developed portfolio/pipeline management approaches to select appropriate projects for increasing success rates of product launch and to capture the business opportunity and keep the constant revenue for the company (Blau et al., 2000, 2004; Rajapakse et al., 2005). There is a lack of research on providing an integrated framework that links operational risk management with corporate strategies and provide a systematic approach for risk identification, assessment, response planning, and control.



3

2.2. Balanced scorecard

2.3. Quality function deployment

This research applies BSC to identify diverse and critical performance measures of an R&D organization based on the firm vision and strategy. The basic concept of the BSC and its applications in R&D performance measurement are presented next. BSC is a management system that enables organizations to clarify their vision and strategy and then translate them into action (Kaplan and Norton, 1992). Since it was introduced by Dr. Kaplan and Dr. Norton in 1992, BSC has been implemented in commercial companies, governments, and nonprofit organizations worldwide (Kaplan and Norton, 2000). In the past, only financial measures were used to determine the performance of an organization. However, these financial measures can only tell the results of past events and they are inadequate to evaluate the performance of a corporation that has to make to create future value through investment in customer–supplier relationship, human resources, and innovation process. With the BSC, managers are able to view organizations from four perspectives: customers, internal business processes, learning and growth, and financial performance, and thereby develop metrics relative to each of these four perspectives. Since its beginnings as a performance measurement system, the BSC has evolved into a strategy management system that not only measures performance but also describes, communicates, and aligns the strategy throughout the organization. There are five basic principles to implement the BSC including (Kaplan and Norton, 2000): (a) translating the strategy to operational terms, (b) aligning the organization to the strategy, (c) making strategy everyone’s everyday job, (d) making strategy a continual process, and (e) mobilizing leadership for change. It can ensure that senior managers can take a balanced view about the performance of an organization and alignment of individual goals with the corporate strategic objectives. Little research has applied the BSC to the measurement of R&D performance (Kerssens-van Drongelen and Bilderbeek, 1999; Bremster and Barsky, 2004). Since it is necessary to have long-term investment for acquiring the required R&D capability for developing innovative products, the BSC structure may provide an adequate framework to determine, organize, and balance R&D performance measures. Kerssens-van Drongelen and Bilderbeek (1999) proposed a BSC framework for an R&D organization and reported that performance measurement systems have a positive impact on R&D performance. The highly effective performance measurement system that is linked with the customer perspective and measured monthly performs better than much less effective systems that focus only on financial and output metrics and are measured on a semiannual or annual basis. Bremster and Barsky (2004) extended the work of Kerssens-van Drongelen and Bilderbeek by integrating the stage-gate approach to R&D management with the BSC to develop a framework such that firms can link resource commitments to development activities and the firm’s strategic objectives. From the above literature survey, it appears that it is important to provide a balanced performance measurement system that is connected with the corporate strategy and has much broader perspectives to enhance R&D performance. In addition, for the innovative R&D project with high uncertainty in market and technology, the achieved performance may often deviate from the target performance during R&D stages (DiMasi, 2001; Raz et al., 2002; Salomo et al., 2007; O’Connor et al., 2008). Therefore, the R&D performance measurement system should be integrated with the risk management that can reduce threats of failing to achieve desired performance levels for achieving successful product launch.

In this research, QFD is used to transform organizational performance measures, established by the BSC, into project performance measures to ensure the achievement of desired organizational goals. This research adapts the QFD planning framework for managing R&D risks, because of its effective planning and organizing ability (Lager, 2005). The basic concept of QFD and its major benefits are presented next. QFD is a tool that originally provides a comprehensive and systematic approach to support the development of products that meet customer requirements by integrating the ‘‘voice of the customer’’ into the product development process (Hauser and Clausing, 1988). Since it was initially developed by Dr. Akao in late 1960s, QFD has been widely used by industries around the world, especially in Japan and the US (Akao, 1990). The entire QFD methodology is a four-phase model which includes product planning, part deployment, process planning, and production planning. The most commonly used stage is in product planning and its purpose is to relate attributes that represent the overall customer concerns to the design requirements that represent the technical performance specifications. The QFD for product planning can be summarized into the following seven steps: (a) obtaining customer attributes and their relative importance, (b) developing design requirements responsive to the customer attributes, (c) relating the design requirements to the customer attributes, (d) completing the customer competitive survey, (e) performing competitive technical benchmarking, (f) determining relationships among the design requirements, and (g) determining the priority of design requirements based on their technical difficulties and estimated costs of meeting target specifications. Ideally, the QFD should be developed by a cross-functional team made up of members from various departments. The major benefits of QFD include increased efficiency, reduced cost, shorter lead time, reduced prelaunch time and after-launch tinkering, and better customer satisfaction (Akao, 1990). Due to its effectiveness in product design and development, there have been many applications and studies of QFD in other areas, such as: planning, decision-making, engineering, management, teamwork, timing, and costing (Chan and Wu, 2002; Lager, 2005).

3. Performance-oriented risk management framework for the R&D project Risk management is a structured approach for managing uncertainty through a sequence of activities: risk identification, risk assessment, risk response planning, and risk monitoring and control (Smith and Merritt, 2002). The purpose of risk identification is to distinguish risks that may affect project outcomes and determine their characteristics. Risk assessment evaluates the probability of an identified risk and its effects on project objectives. Risk response planning determines suitable actions for reacting to specific risks to reduce threats to the project success. The identified risks can then be monitored and controlled throughout the project. This research proposes a performance-oriented risk management framework that integrates the BSC with QFD in a top–down manner for managing the risks that have adverse effects on project outcomes and corporate performance measures. Based on the corporate strategic goals, BSC is used to define performance measures of an R&D organization from four perspectives: finance, customers, internal business processes, and learning and growth. Then the QFD planning framework is applied to translate organizational performance measures into performance measures of an R&D project, which usually can be divided into several


4


R&D project Stage I

R&D organization

Stage II

…

Stage k

…

Stage n

Relationship matrix between R&D organization & R&D project

Customer Perspective Internal Process Perspective Learning and

Mapping process

Growth Perspective

Benchmarking analysis

Financial Perspective

P r i or i t y of r i sk s Fig. 1. Mapping process between an R&D organization and an R&D project.

stages (see Fig. 1). A systematic procedure that follows a typical risk management process is developed for risk identification, assessment, response planning, and control. The proposed risk management procedure is divided into eight steps. Steps 1–4 constitute the phase of risk identification. Risk assessment is divided into steps 5 and 6. Steps 7 and 8 are risk response planning and risk monitoring and control, respectively. Note that the multi-disciplinary team approach is recommended to implement the proposed methodology. The proposed risk management procedure is as follows:

Step 1: Define performance measures for an R&D organization based on the BSC. Step 2: Determine relative importance of organizational performance measures. Step 3: Define project performance measures and corresponding risks for each stage of an R&D project. Step 4: Develop a relationship matrix between the organizational performance measures and the project performance measures. Step 5: Perform risk assessment for each project performance measure. Step 6: Prioritize the risks. Step 7: Identify risk sources and develop action plans to minimize the critical risks. Step 8: Monitor and control the risks identified.

Step 1: Define performance measures for an R&D organization based on the BSC. The performance measures of an R&D organization are initially defined using the BSC to translate the firm’s strategic goals into R&D relevant performance measures. With the defined business strategy, the BSC planning process begins with a statement of strategic indicators at the firm level to establish a corporate-wide BSC. According to the firm’s organizational structure, the corporate-wide BSC is cascaded down to strategic business units and support departments (e.g., R&D department). Based on the concept of BSC, the R&D department then builds its own BSC from four perspectives: financial perspective, customer perspective, internal business process perspective, and learning and growth perspective. Cascading the corporate BSC to R&D department enables R&D efforts to be tightly integrated with the overall business

strategy (Bremser and Barsky, 2004). For example, new product profitability is one of the major strategic indicators for a corporation. At the R&D department level, market value, customer satisfaction, average R&D lead-time and cost, as well as core skill coverage ratio are typical performance measures from the four perspectives. Note that the performance measures selected at a lower level should be aligned with the performance measures defined at the upper level by mutual agreement of employees at the upper and the lower levels. The established departmental performance measures make up the vertical axis of the top portion of the QFD planning matrix (Fig. 1). The BSC framework provides a comprehensive performance measurement for managing and directing the R&D organization. Step 2: Determine weights of organizational performance measures. Since the relative importance of each organizational performance measures may vary from each other, step 2 assesses the weight of each performance measure according to its alignment with the company R&D strategy. A greater impact on the upper tier performance measures will indicate a higher weight for the performance measure at this level. Since achieving consensus agreement between the upper-tier and lower-tier is very important, communication and changes are necessary. In addition, a corporation may collect their competitors’ information to compare the corporation’s R&D organizational performance measures with its competitors to identify opportunity for possible improvement. If enough information can be collected, it will highlight the areas of strengths and weaknesses of an R&D organization and allow the R&D organization to set priorities for aspects of organizational performance that require further improvement. However, the difficulty is how to ensure the correct and enough information is collected for objective evaluation and comparison. Step 3: Define project performance measures for each stage of an R&D project. The missions of R&D organization are usually achieved and realized by a portfolio of projects. This step lists the project performance measures that are ensured to meet the organizational performance measures across the top horizontal row of the matrix. These project performance measures are directly related to the determined organizational performance measures. This can help project performance measures to be aligned with the business strategic goals. In addition, if there are too many project performance measures but the resources



for risk management are limited, then the right ones that are more relative to the important organizational performance measures can be selected. Each project can be divided into several stages, where each stage may be directed by a different functional group or department or even by external organizations. Allocating resources and responsibility to internal or external functional groups and keeping track of the performance of each group are important issues to increase the success rate of product launch. Translating organizational performance measures into project performance measures for each R&D stage ensures that the identified project performance measures are in alignment with the goals of the R&D organization. In addition, it can also improve the coordination among different functional groups to achieve the project goals in a collaborative R&D environment. Step 4: Develop a relationship matrix between the organizational performance measures and the project performance measures. This step develops a relationship matrix between the organizational performance measures and the project performance measures. Since there may be varying degrees of correlation between the organizational performance measures and the project performance measures, the significance of a relationship is considered as: strong, medium, and weak. The benefit of using the relationship matrix is that it can quickly check whether the determined project performance measures cover all organizational performance measures. It also allows us to assess the significance of a project performance from the viewpoint of the R&D organization. Within the matrix, if a cell is empty, then it indicates that some organization performance measures are not addressed or have a weak relationship to the project performance measures and therefore that the project has little probability of meeting those particular organizational performance measures. Step 5: Perform risk assessment for each project performance measure. Since this research considers the risk as an event having a negative impact on project outcomes, we measure the R&D risk in terms of the expected loss from unsatisfied performance measures. The risk measure developed by Browning et al. (2002) is applied to determine R&D risks. The possible value of a performance measure is represented by a probability density function, because it is difficult to know what the actual performance will be realized in the future. For ease of implementation, the triangular probability distribution that is characterized by three values, namely, the most likely value, the worst-case value, and the best-case value, is used. In addition, the preference for various levels of each performance measure may be different. Utility theory (Keeney and Raiffa, 1993) is used to represent the preference of the team to various levels of a performance measure. The utility function U: x-[0, 100] ranks each possible value of a performance measure. If U(x)ZU(y), then the team strictly prefers x to y or is indifferent between them. The probability distribution and utility function of each performance measure should be determined based on group consensus. Other methods in multi-attribute analysis (Bose et al., 1997) can be used to determine group utility functions. For a performance measure with the property of the largerthe-better, its risk measure is defined as follows: R¼

Z

xT

PðxÞ½UðxT ÞUðxÞdx

ð1Þ

1

where x is the outcome of a performance measure; xT is the target value of a performance measure; P(x) is the probability of performance outcome equal to x; and U(x) is the utility value of performance outcome equal to x.

5

The risk measure is the integral of the products of probability P(x) and the loss for each unachieved outcome which is calculated by [UðxT ÞUðxÞ]. Similarly, risk measures for performance measures characterized by the smaller-the-better or the nominal-the-better can be defined accordingly. Step 6: Prioritize the risks. Since effective risk management takes a substantial amount of effort, it is impossible to manage all risks. However, the team usually identifies far more risks than it has time and resources to resolve. In addition, the risk to be managed should be important to the R&D organization. Therefore it is required to prioritize the identified risks based on their impacts both on the project and the R&D organization. In this paper, the significance of a project performance measure is calculated according to the relative importance of the project performance measure to the R&D organization. The weighting factor wj of project performance measure j is calculated as the sum of relative importance of each organizational performance measure multiplied by its strength of relationship to project performance measure j: X wj ¼ qi rij ð2Þ i

where qi is the weight of organizational performance measure i, i¼1,y,m; rij the strength of relationship between organizational performance measure i and project performance measure j; i¼1,y,m, j ¼1,y,n. Then the project risks are ordered according to the weighted project risk wuj Rj , where Rj is the project risk with respect to performance measure j and wuj is the normalized weight P (wuj ¼ wj = nk ¼ 1 wk ). The ordered list of project risks can help project managers identify very large risks that should be handled in advance. Step 7: Identify the risk sources and develop plans to minimize the critical risks. After critical project risks that have serious impacts on the project have been determined in step 6, step 7 identifies risk sources or events that cause the critical risks based on experience from previous projects and the multi-disciplinary teamwork from a variety of perspectives. The action plans are developed to avoid, transfer, mitigate, or absorb the risks as much as possible (Smith and Merritt, 2002; Vanderbyl and Kobelak, 2008). For example, the market requirements for electronic products are difficult to predict accurately, due to changing customer demands and technology dynamics. Therefore, dual design or parallel design can be used to avoid or reduce the development risks (Thomke and Reinertsen, 1998). Step 8: Monitor and control the risks identified. Risk monitoring and control is the ongoing phase throughout an R&D project, wherein the identified risks are monitored and the additional risks may be required to be identified. The proposed methodology is especially suitable for tracking the project performance measures and risks during the project execution. Based on the project performance measures identified, the possible range and target value of each performance measure can be collected and updated at each review stage or after a pre-specified period. Prioritization of risks is revised and exceptions as well as changes can be reported. The project risks can be easily updated and monitored based on the methodology discussed above. The tracking chart and risk reduction profile can be used to check and examine evolving performance metrics and their corresponding risks (Browning et al., 2002). In general, two main types of decision-making can be identified from the proposed risk management process: the generation of alternatives which represents the more


6


creativity-oriented process (e.g., generation of possible performance measures, risks, etc.) and the evaluation of alternatives which represents the more analytical-oriented process (e.g., evaluation of performance measures, risks, etc.). Since R&D is often characterized by a high degree of uncertainty (lack of information) and ambiguity (lack of clarity or consistency of information), it is beneficial to use the multi-disciplinary team approach that gathers groups of professionals from diverse disciplines to improve the decision making in the proposed risk management framework (Schmidt et al., 2001; Keizer et al., 2005). There are at least two advantages of using the team approach: (1) the pooling of knowledge, expertise, and skills to broaden perspectives and solution approaches and (2) the commitment to the team and to its consensus decisions. However, the diversity of perspectives, expertise, and skills may become disadvantages, if the team is not well integrated and managed. This diversity can lead to unproductive conflicts between team members, which further increase uncertainty and ambiguity of R&D. Therefore, it is still required to manage negative group dynamics, such as inter-party conflicts, specious persuasions, and bandwagon effects (Souder, 1977; Forsyth, 2006), to avoid negative outcomes. We will further discuss this issue in Section 5.

4. Hypothetical example in the pharmaceutical industry This section presents a hypothetical example of a Hepatitis C drug development project in the pharmaceutical industry to illustrate the approach developed in this paper. A pharmaceutical product development project generally consists of three stages: discovery and preclinical testing, development, and launch (Blau et al., 2004). The discovery stage is to select the most promising compounds from the huge set of candidate compounds. One or a few compounds that best affects the targets and has no obvious toxicological effects are selected. In the preclinical testing stage, the compound is tested for safety and efficacy in laboratory and animal studies. Preformulation studies covering the physical and chemical characterization of the new drug substance are also conducted. The development stage involves three phases. Phase 1 clinical trials test the developed drugs on healthy human volunteers. This attempts to obtain information about drug absorption and metabolism effects on humans and side effects for different dosages. Phase 2 clinical trials are performed on several hundred patients to determine the effectiveness of the drug in treating disease and the short-term side effects in those patients. Phase 3 clinical trials involve hundreds or thousands of patients to find out benefits and risks of the developing drug and monitor its adverse reactions from comparatively long-term use. If the drug is found to be effective without unacceptable adverse effects, and is approved by governmental health regulatory authority, such as the US Food and Drug Administration (FDA), then the product can be launched. Hepatitis C is a blood-borne infectious disease caused by the hepatitis C virus (HCV), affecting the liver (Vrolijk et al., 2004). It is a common disease with high morbidity especially in Asian countries and many pharmaceutical companies have attempted to develop a new drug to control the hepatitis C virus. The standard treatment for hepatitis C is a combination of Peg-Intron and the antiviral drug ribavirin (Vrolijk et al., 2004). Peg-Intron has been approved by the FDA as a once-weekly monotherapy for the treatment of chronic hepatitis C on Jan 22, 2001. This research took PEG-Intron, which was developed by the Schering-Plough Co., as an example and focused on its preclinical testing phase. Most of the data used in this paper was obtained from the literature (Davis et al., 1999; Lesko et al., 2000; Masci

et al., 2003; Oudheusden, 2003), FDA (2000), and interviews with three senior managers in Taiwan’s biotechnology and pharmaceutical industry. Each of the chosen experts has over 14 years of working experience in the industry and has management experience in the drug development. Since R&D data is usually confidential and some data is difficult to obtain from the industry, we have made required assumptions for those data. Step 1: Define organizational performance measures based on the BSC. We assume that an R&D department is responsible for definition, measurement, and control of R&D performance. In step 1, the performance measures of the R&D department are defined based on the goals of the firm. Fig. 2 lists the corporate-wide strategic indicators and the corresponding performance measures of the R&D department from the four perspectives based on the literature (Kerssens-van Drongelen and Bilderbeek, 1999; Bremser and Barsky, 2004) and expert interviews. This ensures that the organizational performance measures can be defined from much broader perspectives and can be aligned with the corporate strategic goals. For example, ‘‘approval ratio of new drug’’, ‘‘average development cycle time’’, and ‘‘total R&D hours’’ at the departmental level are linked to ‘‘R&D efficiency’’ at the firm level from the internal business process perspective. Step 2: Determine relative importance of organizational performance measures. The relative importance of organizational performance measures is determined by their impacts on the corporate strategic goals. Fig. 3 shows the weight of each organizational performance measure collected from expert interviews. The average rating of expert judgments was used. For example, the approval ratio of a new drug and the drug-related adverse events and efficacy are the two most important factors for the internal process. Step 3: Define project performance measures for each stage of an R&D project. The performance measures defined for the preclinical testing of the drug development project are shown on the top

Perspectives

Strategic indicators at firm level Survive (Basic R&D Market value)

Financial

Succeed (Medium R&D Market value)

Prosper (High R&D Market value)

Customer satisfaction

Customer

Anticipative-ness to internal and external customers' needs Customer loyalty R&D efficiency

Internal

business process

Timely and efficient execution of investment portfolio Quality of output

Learning and growth

Learning organization

Attract and retain the best Long-term focus

Performance measures at the R&D department R&D revenue / R&D expenditure Average development cost per new drugs Current percentage of sales of new drugs Market share of new drug Accuracy of pricing and revenue planning Customer lifetime value Customer satisfaction with new drugs Personalized drug treatment Sales from new drugs National insurance coverage Approval ratio of new drug Average development cycle time Drug-related adverse events and efficacy Quality of new drug Total R&D hours Employee training hours Percentage of collaborative projects with third parties % of evaluation ideas applied in new projects Strategic skills coverage ratio Employee retention rate

Fig. 2. Performance measures of the R&D department using the BSC structure.



determine the drug’s pharmacodynamics, pharmacokinetics, toxicity, etc., through animal testing. This data allows researchers to estimate a safe starting dose of the drug for clinical trials in humans. Most of the performance measures (e.g., mortality rate) are more concerned with the drug’s toxicity, including which organs are targeted by that drug, and if there are any long-term carcinogenic effects or toxic effects on mammalian reproduction. For example, in Fig. 4, the predictability is one of the important performance measures to evaluate the quality of an animal model in the preclinical testing. The quality of an animal model is proportional to its ability to mimic the response would have in humans (Lesko et al., 2000). Appropriate animal models can afford a more focused examination of drug action and provide, for new compounds, better prediction of the human response. Other management-oriented performance differentiators, e.g., speed, timeliness, data quality, and staff performance, were also included for managing the risks in this example. Shorter lead time for preclinical testing could affect the speed at which drugs get to market and capturing the planned completion date of milestones and the actual completion dates could be used to evaluate the efficiency of preclinical testing. In addition, if the study data is clean and free of discrepancies, this can ensure the accuracy of the study results and drug quality, and it directly affects the time needed to get a product to market. Finally, the performance of individual team members involved in the preclinical trial will affect the R&D productivity and the quality of the study results. Step 4: Develop a relationship matrix between departmental performance metrics and project performance metrics. The relationship matrix is used to identify how much each project performance measure affects each organizational performance measure. This example uses a 1–5–9 numerical scale to denote weak, medium, and strong relationships between organizational performance measures and project performance measures. Fig. 4 shows the relationship matrix for the preclinical testing of the PEG-Intron development project. Organizational performance measures were all addressed in this project. The relationship values were obtained from expert interviews and the average rating of expert judgments was used.

horizontal row of the matrix in Fig. 4. The performance measures were collected from the literature (Lesko et al., 2000; Masci et al., 2003; Oudheusden, 2003) and expert interviews. Note that the project performance measures should be defined based on the organizational performance measures determined in step 1. The main goal of preclinical test is to

Perspectives

Financial Perspective

Customer Perspective

Internal Process Perspective

Learning and Growth Perspective

Performance measures at the R&D department

Relative importance

R&D revenue / R&D expenditure

9

Average development cost per new drugs

8

Current percentage of sales of new drugs

7

Market share of new drug

6

Accuracy of pricing and revenue planning

6

Customer lifetime value

8

Hospital/doctors compliance with the new drug

9

Personalized drug treatment

7

Sales from new drugs

4

National insurance coverage

9

Approval ratio of new drug

9

Average development cycle time

5

Drug-related adverse events and efficacy

9

Quality of new drug

4

Total R&D hours for new drugs

4

Employee training hours

4

Percentage of collaborative projects with third parties

6

% of evaluation ideas applied in new projects

8

Strategic skills coverage ratio

9

Employee retention rate

6

7

Fig. 3. Relative importance of performance measures for the R&D department.

Preclinical trial - project performance measures On-time schedule

Lead time

Data quality

% of working hours on R&D

Staff performance

Average development cycle time Drug-related adverse events and efficacy R&D efficiency/quality of works

Predictability

Weights: Most important - 9 Most unimportant 1

Approval ratio of new drug

Menstrual cycle disorder rate (female monkey)

Depart. Performance measure: Internal process

Mortality rate (monkey)

Organization

Weight loss rate

Dose range in animals (female rat)embryotoxicity Dose range in animals(monkey) repeated dose toxicity

Project

6 7

7

7 5

Medium-5

Weak-1)

Fig. 4. The relationship matrix between organizational performance measures and project performance measures from the internal process perspective.


8


Table 1 Risks of performance measures in the preclinical trial phase. Performance measure

Target value

WCV

MLV

BCV

1. Dose range in animals (monkey)—repeated dose toxicity (mg/m2) 2. Dose range in animals (female rat)—embryotoxicity (mg/m2) 3. Weight loss rate (monkey) 4. Mortality rate (monkey)

5500

1414

4239

14126

450

113

400

0.857 0.125

0.429 0.0625

5. Menstrual cycle disorder rate (female monkey)

0.143

0.857

0.571

6. Predictability 7. Staff performance 8. Data quality 9. Lead time 10. On-time schedule

8 8 9 2.5 0

4 7 4 5 0.25

7 8 7 3.5 0.083

Risk

{(2000,0), (5000, 80), (14 800,100)} 800 {(110, 0), (400, 75), (1000, 100)} 0.286 {(0, 100), (0.5, 70), (0.9, 0)} 0.0313 {(0, 100), (0.07, 80), (0.15, 0)} 0.143 {(0.1, 100), (0.5, 70), (0.9, 0)} 8 {(1, 0), (7, 60), (10, 100)} 9 {(1, 0), (7, 70), (10, 100)} 9 {(1, 0), (7, 70), (10, 100)} 2 {(1, 100), (3.5, 60), (5, 0)} 0 {(0, 100), (0.0833, 80), (0.167, 0)}

Weight Weighted risk

Rank

11.60 0.109

1.27

6

11.35 0.072

0.82

8

13.97 0.072 19.02 0.092

1.01 1.75

7 5

8.26 0.092

0.76

9

1.99 0.5 2.31 2.04 5.37

4 10 2 3 1

12.78 4.53 24.33 22.15 48.82

0.155 0.110 0.095 0.092 0.110

1.0

1.0

Utility

Probability

0.4 0.05

Utility function {(x, y)9 x, y AR}

0

1

2 2.5 3 4 Lead-time

5

0.6

0

1

2

3 4 3.5 Lead-time

5

Fig. 5. Probability distribution and utility function of performance measure ‘‘lead-time’’.

Step 5: Perform risk assessment for each project performance measure. This research considers the expected loss from an unsatisfied performance measure as the risk. The target performance value, the triangular distribution with the most likely value, the worst-case value, and the best-case value, and the utility function for each performance measure are defined in Table 1, based on the preclinical experimental data in the FDA review report (FDA, 2000) and the paper by Masci et al. (2003). The performance measure ‘‘lead-time’’ is used as an example to illustrate how to calculate its risk. Assume that its target value is 2.5 years and the best-case value, the most likely value, and the worst-case value are (in years): 2, 3.5, and 5, respectively. Its utility function that depicts the satisfaction level of the lead-time is displayed in Fig. 5. Using Eq. (1), the risk of performance measure ‘‘lead-time’’ was calculated as 48.82. The risks of other performance measures are shown in Table 1. Step 6: Prioritize the risks. According to Eq. (2), the relative importance of each risk with respect to the organizational performance measures was computed. Table 1 shows the weighting factors, weighted risks, and the ranking order of risks. The total risk was about 17.81. The top-five ranking risks were on-time schedule, data quality, lead-time, predictability, and mortality rate, and these five risks are selected for proactive risk management in the next step. Step 7: Identify risk sources and develop action plans to minimize the risks. After the critical risks have been determined, step 7 identifies the sources of these high-priority risks that are required to be proactively managed. The possible sources of risks for the topfive risks in the preclinical testing and the corresponding risk

actions are summarized in Table 2, based on the literature (Davis et al., 1999; Lesko et al., 2000; DiMasi, 2001) and expert interviews. For example, on-time schedule risk may be due to a lack of efficient preclinical testing operations, lack of required skills and resources, heavy or imbalanced workloads, etc. The on-time schedule risk may be improved by streamlining the preclinical operations, allocating additional resources, providing in-depth training, initiating workload balancing, reducing the number of projects in the pipeline, and adapting a new information system for improving information/ knowledge sharing. Note that it is important for the team to have capabilities for identifying risk sources and determining suitable risk actions to mitigate each critical risk without incurring substantial cost. Step 8: Monitor and control the risks identified. Based on the proposed risk management framework, the identified risks can be monitored during the drug development. If the risk value reaches a preset threshold, then the system can notify the R&D team to take required actions to avoid, transfer, or mitigate the risk. Fig. 6 shows the tracking chart for the top-five risks. Although some of the risks were large at the beginning of the preclinical testing, they were under controlled and gradually reduced along the preclinical testing period.

5. Discussion and conclusions The example reported in Section 4 demonstrates that the proposed risk management framework provides a systematic approach to align the project risk management with corporate strategy and a performance measurement system to actively



9

6

Table 2 Risk sources and actions for potential high risks.

On-time schedule Data quality Lead-time Predictability Motality rate

Risk

Risk sources

Actions

On-time schedule

Lack of efficient preclinical development operations Resources allocated are not sufficient Lack of required skills

Actively monitor the tasks on the critical path Training

Allocate more resources on the delayed tasks on the critical path Workloads are too heavy Employ information systems for or imbalanced better information sharing Poor communications Outsource some preclinical tasks among team members Data quality

Lack of required skills

More extensive training of clinical investigators and study personnel Adopt standard operation procedure

Lack of standard quality assurance procedure Lack of data/information Data standardization and sharing mechanism simplification Improvements in quality control and quality assurance methods Purchase clinical information system Contract preclinical tasks to CRO Lead time

Lack of efficient preclinical development process

Predictability Poor predictability of animal models

High mortality rate

Difficulties of new drug itself Incapable R&D competence

SOP problems

Communication gap Unfamiliar with regulations & laws

Streamline the process by reorganizing preclinical development tasks and applying advanced IT or biotechnology Allocate more resources on the critical tasks Outsource some preclinical tasks Parallel trials Adopt advanced evaluation tools (new biomarkers, imaging technologies, etc.) Improve experimental design and research methods Use new animal models with advanced genomic, proteomic, metabomic, and bioinformatic technologies Parallel trials with different animal models Outsource some preclinical tasks Incorporate new scientific information from external sources Conduct parallel trials by exploring several potential technologies, animal models, and candidate compounds Partnership with other firms to acquire new knowledge, patents, and technologies. Training Establish knowledge management system to share knowledge across different people and projects Keep informed of existing or new regulatory standards and policies

identify and manage critical R&D risks throughout the entire project. Although, due to limited space, the example only demonstrates the preclinical testing phase from the internal process perspective, the proposed approach can be extended to other development stages and perspectives as well. It is worth noting that the implementation of the proposed risk management approach needs to consider the following aspects. Firstly, the proposed risk management framework should be based on the multi-disciplinary team approach. The team should consist of individuals from different perspectives to address risks from all possible facets of the project. As we have mentioned in

Risk value

5 4 3 2 1 0 1

2

3

4

5

6

7

8

9

10

11

12

Time Fig. 6. Tracking chart and risk reduction profile for top-5 risks.

Section 3, though the team diversity can bring some advantages, if the team is not properly managed, then the advantages may become disadvantages. These can lead to negative group dynamics and increase information uncertainty and ambiguity in R&D. A number of group decision making techniques (Souder, 1977; Hwang and Lin, 1987), such as multi-voting, nominal group, nominal-interacting group, and consensus decision making techniques, can be used in the proposed risk management process to avoid negative group dynamics and improve efficiency and effectiveness of team decision making (Keizer et al., 2002; Forsyth, 2006). This would help the team gather and share required information, create and identify alternative courses of action, choose among these alternative by integrating the diverse perspectives of members, and enhance group commitment to implement the decisions. Secondly, the entire risk management is an iterative, rather than a one-shot, stepwise process. The proposed risk management approach should start concurrently with the creation of the project schedule, budget, and specifications, proceeding through each step, continuously monitoring and controlling the risks and then going back to the first step to identify new risks as regularly as required. Thirdly, the risk management team should meet periodically to review current project outcomes and results from ongoing risk assessments, and to discuss upcoming changes in product markets and technologies. Since an innovative R&D project may contain great uncertainty in its market and technology, it is important to regularly reassess the project for unforeseen risks and deviation from the original plan. Furthermore, creative thinking techniques (Michalko, 2006), such as brainstorming, can be used to generate a broad range of possible options (e.g., performance measures, risk sources, and action plans), rather than predefined solutions (Sutton and Hargadon, 1996). These will enhance the possibility to develop more resource-effective options for better R&D risk management. Finally, a knowledge management system can be built to capture, store, and disseminate company-specific knowledge for R&D risk management (Cooper, 2003; Keizer et al., 2005). It would help the team to identify issues that occur in the past and have more time to think of less obvious issues. In addition, the QFD planning chart could be adapted to document the entire risk management process for knowledge storage and retrieval. In summary, the proposed performance-oriented risk management framework has the following benefits. Firstly, linking R&D risk management with the firm’s strategy could prevent local measurements from driving inappropriate behaviors. In addition, due to great uncertainties in technology and market for an innovative R&D project, there are unforeseen risks that may be difficult to be identified at early R&D stages. The performanceoriented risk measure is able to help the team update and monitor critical risks efficiently through the R&D process. Finally, the proposed framework that can be integrated with the corporate


10


performance measurement system provides a more objective way to identify and manage risks from the performance perspective and to improve success rates of R&D projects. Since the major cause of the risk is uncertainty which may lead to positive or negative outcomes (Perminova et al., 2008), future research will extend the current R&D risk management framework to manage both opportunities and threats, and study how to gain values from uncertainty based on the real options analysis (Dixit and Pindyck, 1994; Jacob and Kwak, 2003). In addition, further study is required to investigate the effects of group decision making in the proposed risk management framework.

Acknowledgements This research is partially supported by grant no. NSC 96-2221E-005-012 and NSC 97-2221-E-005-066 from the National Science Council of the Republic of China (Taiwan).

References Akao, Y., 1990. Quality Function Deployment: Integrating Customer Requirements into Product Design, Cambridge, MA. Productivity Press. Blau, G., Mehta, B., Bose, S., Pekny, J., Sinclair, G., Keunker, K., Bunch, P., 2000. Risk management in the development of new products in highly regulated industries. Computers & Chemical Engineering 24 (2), 659–664. Blau, G.E., Pekny, J.F., Varma, V.A., Bunch, P.R., 2004. Managing a portfolio of interdependent new product candidates in the pharmaceutical industry. Journal of Product Innovation Management 21 (4), 227–245. Boehm, B.W., DeMarco, T., 1997. Software risk management. IEEE Software 14 (3), 17–19. Bose, U., Davey, A.M., Olson, D.L., 1997. Multi-attribute utility methods in group decision making: past applications and potential for inclusion in GDSS. Omega: International Journal of Management Science 25 (6), 691–706. Bremser, W.G., Barsky, N.P., 2004. Utilizing the balanced scorecard for R&D performance measurement. R&D Management 34 (3), 229–238. Browning, T.R., Deyst, J.J., Eppinger, S.D., Whitney, D.E., 2002. Adding value in product development by creating information and reducing risk. IEEE Transactions on Engineering Management 49 (4), 443–458. Bush, J.K., Dai, W.S., Dieck, G.S., Hostelley, L.S., Hassall, T., 2005. The art and science of risk management—a US research-based industry perspective. Drug Safety 28 (1), 1–18. Chan, L.K., Wu, M.L., 2002. Quality function deployment: a literature review. European Journal of Operational Research 143 (3), 463–497. CMR, 2006. Pharmaceutical R&D Factbook. CMR International Co.. Cooper, L.P., 2003. A research agenda to reduce risk in new product development through knowledge management: a practitioner perspective. Journal of Engineering and Technology Management 20 (1–2), 117–140. Cooper, R.G., Edgett, S.J., Kleinschmidt, E.J., 2004. Benchmarking best NPD practices—I. Research-Technology Management 47 (1), 31–43. Davis, J.R., Nolan, V.P., Woodcock, J., Estabrook, R.W. (Eds.), 1999. Assuring data quality and validity in clinical trials for regulatory decision making, Workshop Report, Roundtable on Research and Development of Drugs, Biologics, and Medical Devices, Division of Health Sciences Policy, Institute of Medicine, National Academy Press, Washington, DC. Dewar, R.D., Dutton, J.E., 1986. The adoption of radical and incremental innovations: an empirical analysis. Management Science 32 (11), 1422–1433. DiMasi, J.A., 2001. Risks in new drug development: approval success rates for investigational drugs. Clinical Pharmacology and Therapeutics 69 (5), 297–307. Dixit, A.K., Pindyck, R.S., 1994. Investment Under Uncertainty. Princeton University Press, Princeton, NJ. Doctor, R.N., Newton, D.P., Pearson, A., 2001. Managing uncertainty in research and development. Technovation 21 (2), 79–90. FDA, 2000. Toxicologist’s Review: Interferon a-2b, BLA 99-1488, Food and Drug Administration, /http://www.fda.gov/downloads/Drugs/DevelopmentAppro valProcess/HowDrugsareDevelopedandApproved/ApprovalApplications/Thera peuticBiologicApplications/ucm086092.pdfS. Forsyth, D.R., 2006. Group Dynamics 4th edition Thomson Wadsworth, Belmont, CA. Freeman, C., Soete, L., 1997. The Economics of Industrial Innovation 3rd edition MIT Press, Cambridge, MA. Gidel, T., Gautier, R., Duchamp, R., 2005. Decision-making framework methodology: an original approach to project risk management in new product design. Journal of Engineering Design 16 (1), 1–23. Hauser, J.R., Clausing, D., 1988. The house of quality. Harvard Business Review 66 (3), 63–73.

Henderson, R.M., Clark, K.B., 1990. Architectural innovation: the reconfiguration of existing product technologies and the failure of established firms. Administrative Science Quarterly 35, 9–30. Hwang, C.-L., Lin, M.-J., 1987. Group Decision Making under Multiple Criteria: Methods and Applications. Springer-Verlag, Berlin. Jacob, W.F., Kwak, Y.H., 2003. In search of innovative techniques to evaluate pharmaceutical R&D projects. Technovation 23 (4), 291–296. Kaplan, R.S., Norton, D.P., 1992. The balanced scorecard: measures that drive performance. Harvard Business Review 70 (1), 71–80. Kaplan, R.S., Norton, D.P., 2000. The Strategy-Focused Organization: How Balanced Scorecard Companies Thrive in the New Business Environment. Harvard Business Press. Keeney, R.L., Raiffa, H., 1993. Decisions with Multiple Objectives: Preferences and Value Tradeoffs. Cambridge University Press, New York. Keizer, J.A., Halman, J.I.M., Song, M., 2002. From experience—applying the risk diagnosing methodology. Journal of Product Innovation Management 19 (3), 213–232. Keizer, J.A., Vos, J.-P., Halman, J.I.M., 2005. Risks in new product development: devising a reference tool. R&D Management 35 (3), 297–309. Kerssens-van Drongelen, I.C., Bilderbeek, J., 1999. R&D performance measurement: more than choosing a set of metrics. R&D Management 29 (1), 35–46. Knight, F.H., 1921. Risk, Uncertainty and Profit. New York, Cosimo (reprint in 2005). Lager, T., 2005. The industrial usability of quality function deployment: a literature review and synthesis on a meta-level. R&D Management 35 (4), 409–426. Lee, J., Veloso, F.M., Hounshell, D.A., Rubin, E.S., 2010. Forcing technological change: a case of automobile emissions control technology development in the US. Technovation 30 (4), 249–264. Lesko, L.J., Rowland, M., Peck, C.C., Blaschke, T.F., 2000. Optimizing the science of drug development: opportunities for better candidate selection and accelerated evaluation in humans. Pharmaceutical Research 17, 1335–1344. Loch, C.H., DeMeyer, A., Pich, M.T., 2006. Managing the Unknown: A New Approach to Managing High Uncertainty and Risk in Projects. John Wiley & Sons, Hoboken, NJ. Luce, R.D., Raiffa, H., 1957. Games and Decisions. John Wiley & Sons. Maidique, M.A., Zirger, B.J., 1984. A study of success and failure in product innovation: the case of the US Electronics industry. IEEE Transactions on Engineering Management 31 (4), 192–204. Masci, P., Bukowski, R.M., Patten, P.A., Osborn, B.L., Borden, E.C., 2003. New and modified interferon alfas: preclinical and clinical data. Current Oncology Reports 5 (2), 108–113. Michalko, M., 2006. Thinkertoys: A Handbook of Creative-Thinking Techniques. Ten Speed Press. Milliken, F.J., 1987. Three types of perceived uncertainty about the environment: state, effect, and response uncertainty. Academy of Management Review 12 (1), 133–143. Mu, J., Peng, G., MacLachlan, D.L., 2009. Effect of risk management strategy on NPD performance. Technovation 29 (3), 170–180. O’Connor, G.C., Ravichandran, T., Daniel Robeson, D., 2008. Risk management through learning: management practices for radical innovation success. Journal of High Technology Management Research 19, 70–82. Ogawa, S., Piller, F.T., 2006. Reducing the risks of new product development. MIT Sloan Management Review 47 (2), 65–71. Oudheusden, P., 2003. Driving Clinical Trial Performance: The Importance of Measuring Clinical Research. White Paper, Cognos Co.. Perminova, O., Gustafsson, M., Wikstrom, K., 2008. Defining uncertainty in projects—a new perspective. International Journal of Project Management 26 (1), 73–79. Pisano, G.P., 2006. Can science be a business? Lessons from biotech. Harvard Business Review 84 (10), 114–125. Project Management Institute, 2004. A Guide to the Project Management Book of Knowledge (PMBOK) 3rd ed. Project Management Institute, Newtown Square, PA. Rajapakse, A., Titchener-Hooker, N.J., Farid, S.S., 2005. Modelling of the biopharmaceutical drug development pathway and portfolio management. Computers & Chemical Engineering 29 (6), 1357–1368. Raz, T., Shenhar, A.J., Dvir, D., 2002. Risk management, project success, and technological uncertainty. R&D Management 32 (2), 101–109. Saari, H.-L., 2004. Risk Management in Drug Development Projects, Report 2004/1, Laboratory of Industrial Management. Helsinki University of Technology. Salomo, S., Weise, J., Gemunden, H.G., 2007. NPD planning activities and innovation performance: the mediating role of process management and the moderating effect of product innovativeness. Journal of Product Innovation Management 24 (4), 285–302. Schmidt, J.B., Montoya-Weiss, M.M., Massey, A.P., 2001. New product development decision-making effectiveness: comparing individuals, face-to-face teams, and virtual teams. Decision Sciences 32 (4), 575–600. Sicotte, H., Bourgault, M., 2008. Dimensions of uncertainty and their moderating effect on new product development project performance. R&D Management 38 (5), 468–479. Smith, P.G., 1999. Managing risk as product development schedules shrink. Research Technology Management 42 (5), 25–32. Smith, P.G., Merritt, G.M., 2002. Proactive Risk Management: Controlling Uncertainty in Product Development. Productivity Press, New York. Souder, W.E., 1977. Effectiveness of nominal and interacting group decision processes for integrating R&D and marketing. Management Science 23 (6), 595–605.



Sutton, R.I., Hargadon, A., 1996. Brainstorming groups in context: effectiveness in a product design firm. Administrative Science Quarterly 41 (4), 685–718. Teece, D.J., 1986. Profiting from technological innovation: implications for integration, collaboration, licensing and public policy. Research Policy 15 (6), 285–305. Thomke, S., Reinertsen, D., 1998. Agile product development: managing development flexibility in uncertain environments. California Management Review 41 (1), 8–30. Vanderbyl, S., Kobelak, S., 2008. Risk management for the biotechnology industry: a Canadian perspective. Journal of Commercial Biotechnology 14 (2), 128–140.

11

Vrolijk, J.M., de Knegt, R.J., Veldt, B.J., Orlent, H., Schalm, S.W., 2004. The treatment of hepatitis C: history, presence and future. The Netherlands Journal of Medicine 62 (3), 76–82. Ward, S., Chapman, C., 2003. Transforming project risk management into project uncertainty management. International Journal of Project Management 21 (2), 97–105. Williams, T.M., 1995. A classified bibliography of recent research relating to project risk management. European Journal of Operational Research 85, 18–38.
