New product development (NPD) generally requires substantial company ... decision making process in the product development of new medical devices. ...... shape the decisions and illustrate with an application in software development. The.
DELIVERABLE 3 Decision-making methods that could be used to assess the value of medical devices P1 D3 V2.0 051025
Sukhvinder S. Johal University of Nottingham Hywel C. Williams University of Nottingham
March 2005
Multidisciplinary Assessment of Technology Centre for Healthcare
EXECUTIVE SUMMARY Purpose New product development (NPD) generally requires substantial company investment. The financial cost to a company can be high if a new product subsequently fails to sell adequately. Managers therefore need robust decisionmaking tools for valuing potential new product investments in order to justify their development strategy and to allow them to screen out new devices that stand little chance of success at an early stage. Two major problems exist in making such complex decisions: •
The need to cast the decision in words, although most analytical tools use numbers.
•
The need to admit a range of factors to a decision and to weight each appropriately
This report is about several methods that address these two challenges, principally in order to make business decisions about products as they proceed along the development pathway. Scope and Methods A consultation exercise with some of our MATCH industrial partners was initially carried out by postal questionnaire on project valuation/capital budgeting techniques currently used by these companies. Following on from the results of this exercise, an extensive literature review was conducted amongst engineering, management and business journal databases, as well as more generally on the internet to identify key tools and techniques that could be used in the valuation of a new medical device, screen out products that are not likely to succeed, and assist decision making in the product development lifecycle. Summary of Key Issues, Findings and Conclusions Most companies are still using the ‘traditional’ valuation techniques, such as payback period and discounted cash flow methods for project valuation. Discounted cash flow techniques address the challenges above by considering everything in monetary terms: numerical assessment is easy, and money can be handled in a uniform way, regardless of the mechanism used to convert each factor into a number. These methods form the standard against which a range of other techniques are assessed. However, a number of significant weaknesses exist with such techniques, indicating the need for better decision making tools that can be applied to the medical device new product development cycle. This literature survey identified a range of alternative decision-making and valuation tools and techniques that could be used to define and estimate a new product project value in the earliest stages of product development in the medical device industry and the report makes recommendations on the strengths and weaknesses as well as specific areas of application. The following tools are discussed: •
The Analytic Hierarchy Process (AHP) breaks decisions down into pairs, which can be assessed qualitatively (e.g.: ‘this is better than that’). Numbers are introduced to weight those assessments and ensure internal consistency.
•
Fuzzy logic provides a framework that seeks explicitly to mirror aspects of qualitative reasoning, while smuggling the maths in by assigning numbers to i
linguistic variables in a non-linear way (e.g. ‘little’ may equate to 0.1 while ‘lot’ might equate to 0.9). •
Real options theory looks to the stock market – especially in handling risk – but breaks away from discounted cash flows, by recognising that companies may exploit their research investment in many ways – not only through the product they set out to develop.
•
Expert systems favour a linguistic approach in capturing and using the sort of rules that experts intuitively apply, with a numerical inference engine to weight and combine rules.
•
Conjoint analysis focuses on product utility in terms of attributes and levels. Again, the natural language façade is supported by numbers in terms of applied scaling factors.
Although the techniques are not especially novel in themselves, their application to the medical device sector is largely novel. Each has advantages in terms of the way in which it reflects company culture, and disadvantages in terms of the investment required to integrate it into a product development cycle. The report does not try to single out approaches, but instead presents a broad range for the reader to consider. Major Recommendations In this report, the author has identified some key techniques that would aid the decision making process in the product development of new medical devices. These techniques will need to be validated by applying them to ‘real’ products in the medical device sector. The real options technique shows promise and is superior to conventional capital budgeting techniques such as pay back or discounted cash flow. A mathematical model will be developed to model the options with a product development environment. The challenges to the real options method are its mathematical complexity and proper estimation of some of the mathematical parameters of the model, both of which will be addressed. AHP and Conjoint analysis are techniques that can be utilized as early screening techniques for new products and identify and weight the key variables managers use when assessing a potential new product. Work is underway to develop a project selection tool and these techniques may be used here.
ii
TABLE OF CONTENTS Executive Summary ............................................................................................................. i Table of Contents................................................................................................................iii Introduction ......................................................................................................................... 1 Section 1: 1
Existing techniques and practice................................................................. 3
Capital Budgeting and Project Valuation Techniques................................................. 4 1.1
Introduction.......................................................................................................... 4
1.2
Capital Investment Appraisal .............................................................................. 4
1.3
Limitations of current techniques ........................................................................ 7
1.4
Previous Studies of UK Capital Budgeting Practices.......................................... 8
1.5
Summary of key points........................................................................................ 9
2 Findings from the MATCH pilot survey: Project Valuation and Capital Budgeting Techniques used in the Medical Device Sector................................................................ 10 2.1
Background ....................................................................................................... 10
2.2
Results .............................................................................................................. 10
2.3
Summary of findings ......................................................................................... 12
2.4
Implications for future research......................................................................... 13
Section 2: 3
4
5
Novel application of other decision-making techniques ............................ 14
Analytic Hierarchy Process ....................................................................................... 15 3.1
Background and Definitions .............................................................................. 15
3.2
Worked Example ............................................................................................... 16
3.3
Application to medical devices .......................................................................... 17
3.4
Possible concerns/limitations ............................................................................ 17
3.5
Summary ........................................................................................................... 18
3.6
Recommendations for MATCH ......................................................................... 18
Fuzzy Logic ............................................................................................................... 19 4.1
Background and Definitions .............................................................................. 19
4.2
Example of fuzzy sets ....................................................................................... 19
4.3
Fuzzy Logic Analysis Process........................................................................... 20
4.4
Examples of application of fuzzy-logic within Industry and Medicine ............... 22
4.5
Potential Application of Fuzzy Logic to New Product Development ................. 23
4.6
Strengths ........................................................................................................... 23
4.7
Limitations ......................................................................................................... 24
4.8
Summary ........................................................................................................... 24
4.9
Recommendations for MATCH ......................................................................... 24
Real Options Based Analysis.................................................................................... 25 5.1
Background ....................................................................................................... 25
5.2
Traditional DCF techniques............................................................................... 25
5.3
History and Terminology ................................................................................... 25 iii
6
5.4
The Black-Scholes Model for Valuing Options.................................................. 27
5.5
Examples of Applications of Real Options in Practice ...................................... 28
5.6
Concerns/Limitations of Real Options analysis................................................. 30
5.7
Summary ........................................................................................................... 31
5.8
Recommendations for MATCH ......................................................................... 31
Expert Systems ......................................................................................................... 32 6.1
6.1.1
The Knowledge Base ................................................................................ 32
6.1.2
Knowledge Acquisition .............................................................................. 34
6.1.3
The Inference Engine................................................................................ 35
6.1.4
The User Interface .................................................................................... 35
6.1.5
Explanation of the Decision Making Process............................................ 36
6.2
7
8
Background ....................................................................................................... 32
Applications of Expert Systems in New Product Development (NPD).............. 36
6.2.1
Investment Decisions ................................................................................ 36
6.2.2
Pharmaceutical Products .......................................................................... 37
6.2.3
Other Industry Applications ....................................................................... 37
6.3
Concerns/Limitations of Expert Systems .......................................................... 39
6.4
Summary ........................................................................................................... 39
6.5
Recommendations for MATCH ......................................................................... 40
Conjoint Analysis....................................................................................................... 41 7.1
What is Conjoint Analysis?................................................................................ 41
7.2
Attributes, Levels and Utilities ........................................................................... 41
7.3
Steps in performing a conjoint analysis ............................................................ 41
7.4
Applications of Conjoint Analysis ...................................................................... 44
7.4.1
Venture Capital Funding ........................................................................... 44
7.4.2
New Product Development ....................................................................... 45
7.4.3
Healthcare Sector...................................................................................... 46
7.5
Concerns/Limitations of Conjoint Analysis........................................................ 47
7.6
Summary ........................................................................................................... 48
7.7
Recommendations for MATCH ......................................................................... 48
Conclusions and recommendations.......................................................................... 49 8.1
Conclusions....................................................................................................... 49
8.2
Recommendations for Further Research.......................................................... 50
References ........................................................................................................................ 52
iv
INTRODUCTION New products are crucial for the growth and prosperity of any company, especially in the rapidly changing medical device market. Companies must innovate to respond to technological advances, meet regulatory requirements, cope with competitive threats and keep pace with market and consumer needs (Ram and Ram, 1989) In a survey of 150 European medical device companies, 63% said that they had produced/offered an innovative product or service in the last 12 months and 82% were planning to launch a new product within the next two years (DJS Research, 2003) However, new technology and new project development projects are associated with risk and uncertainty. Managers are faced with the dilemma of whether or not to invest at any stage in the new product development (NPD) process, with the concept screening stage perhaps being the most critical point for investment (Cooper and Kleinschmidt, 1986). As this uncertainty is unavoidable, the successful identification, analysis and management of uncertainty is a necessary step in enhancing the likelihood of new project success. The NPD process of a medical device is complex and unique in many ways. The decision to develop a device encapsulates many factors, including development and testing costs, production costs, health economic analysis, market share analysis and strategic considerations. Many medical device manufacturers face another, unique hurdle, the nature of the marketplace. In most other markets for goods and service, the consumer is able to take decision on whether they provide value for money. That is, the consumer is the main regulator. For many medical device markets it is firstly the regulating agency of the healthcare system that determines whether a device obtains approval and then government and third-party payers who determine the value of the product or service based on cost-effectiveness and demonstrable patient outcomes. Any new product assessment method should take this complex setting into account. In terms of the assessment of a concept product, attempts have been made to identify the key drivers for new product success. Cooper argues that the number one factor in success is a unique superior product (Cooper and Kleinschmidt, 1993). In a survey of 200 actual new product projects, product superiority - delivering unique benefits to users separated winners from losers more often than any other single factor. In an analysis of US medical device companies that went public (onto the stock market) from June 1995 to July 1997, Faulkner suggests three variables for assessing new medical technologies: clinical value, barriers to entry, and market size. He goes on to conclude that novel proprietary technologies that address important and previously unmet clinical needs are the secret to success (Faulkner, 1998). While it is impossible to predict future conditions accurately, managers and other key decision makers require robust and systematic methods for valuing potential investments so that they can screen new product projects and also justify their NPD strategies. The 1
techniques that they employ should account for risk and uncertainty and provide a far more complete picture of not only technology values but also the drivers of the value of such technology. This value should drive the design and development process. The rationale for this report has been an increasing realisation amongst the MATCH industrial community that existing methods to assess early financial value of new medical devices are frequently inadequate, as elaborated in the MATCH survey in Part 2 of this report. It is also clear that other industry sectors face similar difficulties in dealing with uncertainty in early decision making, yet they have moved on and used other techniques that have proven to be better tools for making financial value predictions for new products. Whether such techniques are of use to the specific problems of the medical device industry remains to be seen, but early indication of applications cited in section 2 of this report seem encouraging. The novelty of this report is not so much related to the decisionmaking techniques described, but their actual and potential application to specific problems encountered for early decision making in the medical device industry.
2
SECTION 1: EXISTING TECHNIQUES AND PRACTICE
3
1 CAPITAL BUDGETING AND PROJECT VALUATION TECHNIQUES 1.1
INTRODUCTION
New product selection and justification are key challenges faced by most companies. Successful new product development (NPD) requires these companies invest a substantial amount of capital and resources in the development of new products even though in practice new product failure rates are substantial and the cost of failure is large (Booz et al., 1982.; Goldenberg et al., 2001).The decision making process can take into account many factors including customer requirements, profitability, resource allocation feasibility, technological innovation and opportunities, competitor analysis and the overall company strategy. However, it is the prediction of the extent of profitability and future cash flows which most often drive an initial business plan and provide senior management with a quantitative prediction for a product’s success even though this is complicated by the inherent subjectivity involved in attempting to predict the future. A number of financial tools and techniques exist to assist in making decisions about financial returns and these will now be discussed. A financial appraisal must give a decision-maker, usually a finance manager, enough information to decide not only whether or not to invest in a project, but also how much to invest and the ability to choose between two or more alternative projects. A systematic evaluation procedure is needed. This procedure, called capital budgeting (CB), and capital investment appraisal (CIA), is one aspect of the process dealing specifically with the financial analysis. CB is often described as the analysis of long term projects which require large initial investments The CB model/process contains the following components:
1.2
•
Concept generation and search for new products
•
Financial analysis (CIA)
•
Alternative products evaluation
•
Project implementation
•
Project monitoring
•
Review or post-audit investment decision
CAPITAL INVESTMENT APPRAISAL
Companies may use a variety of methods for CIA ranging from a simple payback analysis to more sophisticated discounted cash flow techniques. The most frequently used tools, as highlighted by the literature and by the numerous surveys on capital budgeting practices among firms are:
4
1. Payback period (PB) 2. Accounting rate of return (ARR) 3. Discounted cash flow models a
Net present value (NPV)
b
Internal rate of return (IRR)
Payback Period (PB) The payback method of analysis evaluates projects based on how long it takes to recover the amount of money invested into the project. It is widely used because it is easy to apply and hence particularly applicable for ‘quick and dirty’ project evaluations. For example, consider a product whose development requires an outlay of £100 000, which is expected to produce a net profit of £20 000 after one year, £40 000 after the second year and £50 000 after the third year. Then the payback period is 2.8 years (2+£40/£50 years). In other words it would take 2.8 years to recover the initial £100 000. Accounting Rate of Return (ARR) This is sometimes called the return on investment and represents the annual average profit expressed as some percentage of the investment outlay (Mills, 1988a). ARR = Net cash inflow / Investment outlay where net cash inflow is equal to revenue minus expenses. ARR is an average measure and measures the average rate of a return of a product over its entire life of sales. Consider the above example with a development cost of £100 000 and an average annual profit of £40 000 over a 10 year product life. The ARR is 40%. The ARR is usually compared with the required rate of return (or hurdle rate) of a company which depends on the nature and risk of the project. Discounted Cash Flow (DCF) Methods DCF analysis is based on measurable cash flows. It identifies all cash flows (benefits plus costs) related to a project investment, both now and in the future. Those cash flows are adjusted for time and risk, with an appropriate discount rate to establish an indication of the present value of the cash inflow of the project, and then summed up to yield the value of the investment. In other words, DCF methods focus on the time value of money. The discount formula is given by: Present Value = i / (1+r)n
5
where i = investment, r = discount rate of interest and n= number of years. For example the present value of £1, at a discount rate of 5%, in 3 years is £0.86. It is interesting to note that at a recent MATCH Industrial forum1, most companies said that they used a high discount rate (>15%) while most health economists would use a much lower level ( 0 Ö Accept NPV < 0 Ö Reject NPV = 0 Ö Indifferent The higher the NPV the more attractive is the investment in the project. Consider the example above in which a project an outlay of £100 000, which is expected to produce a net profit of £20 000 after one year, £40 000 after the second year and £50 000 after the third year. Assume the hurdle rate or required rate of return is 10% and the product life is 3 years. NPV = -C0 + C1/(1+r)1 + C2/(1+r)2 + C3/(1+r)3 Where
C0 = investment outlay at commencement C1 = cash inflow at end of year 1 C2 = cash inflow at end of year 2 C3 = cash inflow at end of year 3 r = required rate of return
Hence NPV for the above project = -100000 + 20000/(1+0.1) + 40000/(1+0.1)2 + 50000/(1+0.1)3 = -100000 + 18181 + 33057 + 37565 = -£11197
1
MATCH conference, Luton, September 2004. 6
Because the NPV is negative this investment should be rejected. Internal Rate of Return (IRR) The IRR is the interest rate that makes the value of the discounted cash flows equal to zero. That is, The IRR of a project is the rate of return at which the present value of the net cash inflows equals the initial cost. The IRR reflects the return on the original investment. A project would be considered a worthwhile investment if the IRR was greater than the expected rate of return. IRR and NPV are similar in many ways. The critical difference is that the IRR is a ratio whereas NPV is a pound measure of value. Table 1 Advantages and disadvantages of common CIA techniques Method
Advantages
Disadvantages
Payback Period
Simplicity
Accounting Rate of Return
Simplicity Concerned with Profitability
Ignores time value of money (discounted payback overcomes this). Ignores cash flows occurring after the payback period. Concerned with cost recovery rather than profitability. Ignores time value of money Depends on expense calculation method adopted by company
DCF Methods Net Present Value
Concerned with profitability Concerned with time value of money
Internal Rate of Return
Concerned with profitability Concerned with time value of money
The discount rate to apply is subjective. No indication of what rate of return a project is earning The discount rate to apply is subjective. Often gives unrealistic rates of return. IRR method may give different rates of return Maximizing IRR may actually reduce value by rejecting positiveNPV projects.
The techniques described above are some of the most common used by industry for cash flow valuation. There are many other techniques which include variants of the above methods. These include discounted payback, modified IRR, Profitability Index, equivalent annual annuity, NPV breakeven analysis and marginal analysis. Further details of these can be found elsewhere (Brealey and Myers, 1991; Remer and Nieto, 1995)
1.3
LIMITATIONS OF CURRENT TECHNIQUES
As can be seen there is no dearth of tools available for investment appraisal. Theory would suggest that DCF techniques are superior to traditional techniques such as PB and 7
ARR. However DCF techniques have also been cited as having some serious limitations. Table 1 lists some of the advantages and disadvantages of CIA techniques. The inherent weakness of any of these techniques lies in the fact that they assume all future cash flows are static and do not properly account for the flexibility that may be present in a project. There is the assumption of a single line of development for a project which is carried on to completion. It neglects real-world choices to stop investing in a project or change course in response to changing levels of demand. There is also the criticism that the focus of DCF methods is too narrow and does not consider long term goals and strategy of a company. Further, the discounted cash flow (DCF) method has been criticized for its inadequacy to appropriately appraise soft projects, such as research and development (R&D), which lead the management to select such projects on intuition, experience and rule of thumb methods (Akalu, 2003).
1.4
PREVIOUS STUDIES OF UK CAPITAL BUDGETING PRACTICES
There have been a number of surveys regarding capital budgeting and investment appraisal practices of UK firms. Although the results of these surveys share some common conclusions there are also significant differences. It is difficult to make any useful direct comparisons between studies as they invariably ask different questions and have different methods of data collection and analysis. Also the use of mailed questionnaires has been argued to create a significant level of response bias, ‘sophisticated’ technique practitioners being more likely to respond than those using ‘naïve’ techniques (Pike and Sharp, 1989). However, each survey has made its own distinctive contribution. Mills (1988b) conducted a survey of 200 of the UK’s largest firms (according to market capitalisation and turnover) by means of a postal questionnaire. The response rate was 61.5%. The author expressed concern about the difficulties associated with postal questionnaires and with comparing individual studies. Nonetheless, a number of conclusions can be drawn from the study. There was an association between company size and the use of discounted flow techniques. However these techniques had not been adopted at the expense of the more traditional and simple techniques such as payback period and were generally part of a package of techniques used by the companies. IRR was found to be the most widely used DCF method. Only a small proportion of companies employed formal methods to analyse risk, the most popular method being sensitivity analysis of key variables. Pike and Sharp (1989) again surveyed large UK companies and examined how management science applications have changed since 1975. The study was a combination of two identical surveys undertaken at two points in time, 1980/81 (208 questionnaires sent) and 1986 (140 questionnaires sent) and sent to the same 8
companies each time. The difference in number being due to liquidations and amalgamations of companies during the intervening period. The response rate was over 70% in both cases. Again, IRR was found to be the most popular DCF method as opposed to NPV but both were found to have greatly increased in popularity since 1975. The most popular method for assessing risk was sensitivity analysis. DCF methods and sensitivity analysis were found to be standard practice for three quarters of the firms surveyed. Akalu (2003) examined the capital budgeting practices of the top 10 British and Dutch companies. The research focused on how these companies perform investment appraisal, subsequent follow-up and measurement of project success or failure. The findings indicate that most companies follow decentralized project decision-making. Moreover, there is a strong tendency to combine discounted cash flow (DCF) models with the newly emerged value management tools, such as economic value added (EVA), and modified versions of DCF models. However, firms do not apply uniform appraisal techniques throughout the project life cycle. Companies show less emphasis on project risk, and emphasize on financial measures in project success or failure designation.
1.5
SUMMARY OF KEY POINTS •
A range of techniques are currently being used to predict financial returns of a new product
•
The most widely used techniques are PB, ARR, and DCF methods, NPV and IRR.
•
DCF methods are seen as more sophisticated and superior than techniques such as PB and ARR because they take in to account the time value of money.
•
DCF techniques (and the other techniques) have been cited as having some serious limitations in terms of valuing a new project or concept.
•
Despite their limitations, DCF techniques are still widely used in industry. However, this is not at the expense of less sophisticated techniques such as PB. Most firms will use a combination of methods
9
2 FINDINGS
FROM
THE
MATCH
PILOT
SURVEY:
PROJECT VALUATION AND CAPITAL BUDGETING TECHNIQUES USED IN THE MEDICAL DEVICE SECTOR 2.1
BACKGROUND
An initial survey was conducted amongst some of MATCH’s partners to get a better picture of current practices of project valuation/capital budgeting techniques used by companies within the medical device sector. The perspective of this survey was on products that companies wish to bring to market. Some of the initial results are presented below.
2.2
RESULTS
A total of six companies were surveyed by means of a postal questionnaire which was sent to a senior member of the organisation who had sufficient expertise and confidence to complete the survey, see Table 2 for a breakdown of job titles or respondents. While the survey did not focus on size of company, five companies can be considered as ‘large’, turnover exceeds £200 million per annum and one ‘small’ (turnover information not available but headcount less than 15). Table 2 Breakdown of job titles of respondents Company
Job title of respondent
A
R&D Director
B
Technology Valuation Manager
C
Health Economics Manager
D
VP Global Concept Development
E
Medical Technology Manager
F
European Marketing Manager
The first major question of the survey asked whether firms performed a formal economic evaluation on the development proposal in determining the acceptability of the project. Five companies said they did. New product development is composed of a number of steps or stage gates and companies were asked whether formal economic analysis was performed at different stages of product development. Three companies carried this out at the concept stage of development, all carried it out after the basic research stage, four carried it out after the applied research stage, and four carried it out after the ‘development to commercial project’ stage. 10
Financial Techniques used All of the five companies used DCF methods that take into account the time value of money, in the economic evaluation of their products. NPV techniques were prevalent among four companies as was IRR. Three companies used both IRR and NPV. Surprisingly (or perhaps not), the ‘simple’ payback period method was used by all five companies. This can probably be attributed to its ease of use and understanding by key decision makers outside of the financial department. Discounted payback was used by two of the companies. All companies used a combination of techniques for project valuation. Only one company reported the use of real options in project valuation. Limitations of the evaluation techniques Respondents were asked to cite any fundamental limitations (if any) of the techniques they were using. Four companies revealed that they thought that there were weaknesses in the methods and tools they were using in measuring all the dimensions of profitability which are relevant to a product development decision. Table 3 lists the weaknesses that were pointed out and the number of companies that highlighted them. Table 3 Limitations of appraisal techniques. Limitation
Number
Does not account for future uncertainty
2
Does not account for flexibility present in a project
4
Assumes a single line of development for a product
4
Does not take into account long term nature of some projects
2
As the companies highlighted that they mainly only used DCF and payback period as appraisal techniques these limitations can be assumed to apply to them. These results highlight many of the previous criticisms targeted at DCF and payback. There was a lot of emphasis on rapid recovery of outlay using these techniques. Companies were asked to rate on a five point scale to describe the predictive capabilities of these techniques of future cash flow and to account for future flexibility (changes in project). Most of the companies thought that the combination of DCF and payback techniques to estimate future cash flows was satisfactory although one respondent commented that [these techniques] ‘overestimate the revenues and underestimate the costs’. The payback period was also cited as being unsatisfactory by one company. In terms of accounting future flexibility and changes within a project lifecycle four of the companies ranked the DCF methods as being unsatisfactory or very poor.
11
Risk and Uncertainty The practice of handling project risk varies from company to company as it does from project to project. The companies in the sample used a variety of techniques to account for the risk and uncertainty associated with their project investment (Table 4). Table 4 Methods used for risk analysis in evaluating project investments Method used for risk analysis
Number
Risk is not taken into account in the economic analysis
2
Shorten required pay-back period
1
Increase required hurdle rate
1
Make subjective adjustment of estimated cash-flows
2
Sensitivity analysis
2
Comparative analysis at optimistic, pessimistic and most likely cash-flow
4
scenario Probability analysis (simulation based on probability distributions of future
1
cash-flows) Other
-
As can be seen, some form of comparative analysis at optimistic, pessimistic and most likely cash-flow scenario was the most commonly used method for risk analysis. Other Techniques Companies use a variety of other techniques to assess the feasibility of a project. These may not be totally focussed on cash flows but give the company further data to include in the decision making process. They can be very simple, informal techniques, such as an unstructured peer review, or more complex methods such as decision tree analysis. By far the most common techniques were unstructured peer review (4 companies) and scoring checklists (4 companies). Decision tree analysis was used by one company and one of the smaller companies stated that ‘gut feel’ for a project was important.
2.3
SUMMARY OF FINDINGS
The small sample size of the MATCH survey precludes any sophisticated analysis being carried out on the results. However, the survey has begun to give a picture of project valuation techniques, and their limitations, employed by companies in the medical device sector. Some of the key findings are: •
Most firms apply a formal economic evaluation procedure to project appraisal 12
•
Most firms tend to employ multiple evaluation techniques when analyzing proposed projects.
•
A combination of DCF techniques and payback period was widely used. NPV and IRR were used equally.
•
Firms reported that there were some fundamental limitations with these valuation techniques.
•
Companies used a variety of other techniques for project review, the most common ones being unstructured peer review and scoring checklists.
2.4
IMPLICATIONS FOR FUTURE RESEARCH
The results of this study reveal some inherent weaknesses with current DCF and payback techniques. However, despite their limitations, companies still find them of value. These methods are relatively easy to use and understand and are firmly embedded in the mindset of financial departments. Some researchers have proposed the real options technique to overcome some of these limitations but only one company had considered them. This is probably due to the fact that this technique is something that is ‘untested’ in the medical device sector and is seen as complex and requiring extra data. The investigation of this technique could be a useful avenue of research for MATCH. See Chapter 5 for a description of real options. Another fundamental area of research into capital budgeting and project valuation practices is to investigate the type of data that managers use when constructing an investment model and how they use this information to assess likely new profitability. Conjoint analysis (see chapter 7) has been suggested as one possible technique to capture this type of information.
13
SECTION 2: NOVEL APPLICATION OF OTHER DECISION-MAKING TECHNIQUES
14
3 ANALYTIC HIERARCHY PROCESS 3.1
BACKGROUND AND DEFINITIONS
The Analytic Hierarchy Process (AHP) is a general problem-solving method that is useful in making complex decisions (e.g., multi-criteria decisions) based on a balance of quantitative data (data from the various variables) and qualitative data (judgements about the variables from decision makers). It is a technique that was developed by Thomas L Saaty (1980). It has been applied successfully in decision about product/process/project selection (Tang and Insuk, 1993; Davis and Williams, 1994; Karbhari, 1994; Partovi and Hopton, 1994; Barbarosoglu and Pinhas, 1995) It has even been applied to a political problem (Carlsson, 1995). In AHP, a complex decision problem is first broken down into a hierarchical structure of factors or elements. At the top of the hierarchy is the overall goal or prime objective one is seeking to fulfil. The next level below this contains elements that comprise the goal often called the criteria level. Each criterion is broken down into sub-criteria and finally the lowest level contains the decision alternatives or selection choices. In this way the decision problem can be broken down into units of smaller sets of decisions. A hierarchy can be constructed by creative thinking, recollection and using people’s perspectives (Saaty, 2000). The number of levels depends on the complexity of the problem and the quantity of analysis that is required. Once the structure has been established, criteria and alternatives are then compared a pair at a time to obtain relative weights indicating the extent to which elements of one level have influence over those of a succeeding level. The mathematical basis of the weighting procedure has been established by Saaty (Saaty, 1980) and is carried out using mathematical techniques such as Eigenvalue, Mean Transformation, or Row Geometric Mean. This is not discussed here. The pair-wise comparisons generate a matrix of relative rankings for each level of the hierarchy. This comparison is based on a nine-point ordinal scale. The meaning of each scale measurement is explained in Table 5. The pair-wise comparisons are given in terms of a linguistic phrase, for example how much element A is more important than element B. According to this scale, the available values for the pairwise comparisons are members of the set: {9, 8, 7, 6, 5, 4, 3, 2, 1, 1/2, 1/3, 1/4, 1/5, 1/6, 1/7, 1/8, 1/9}
15
Table 5 Scale of preference between two elements Preference
Definition
Explanation
weights/level of importance 1
Equally preferred
3 5 7 9 2,4,6,8 Reciprocals
3.2
Two activities contribute equally to the objective Moderately preferred Experience and judgement slightly favour one activity over another Strongly preferred Experience and judgement strongly or essentially favour one activity over another Very strongly An activity is strongly favoured over another preferred and its dominance demonstrated in practice Extremely preferred The evidence favouring one activity over another is of the highest degree possible of affirmation Intermediates values Used to represent compromise between the preferences listed above Reciprocals for inverse comparison
WORKED EXAMPLE
Suppose one is choosing a computer based on the criteria of hardware expandability. Presume that there are three alternative computer systems. A decision-maker can then set up a decision matrix based on this criterion of hardware expandability and the nine point scale as shown in Table 6. Table 6 Decision Matrix C1:Hardware
Computer
Computer
Computer
expandability
A
B
C
Computer A Computer B Computer C
1 1/6 1/8
6 1 1/4
8 4 1
Hence, for example, comparing system A with B (by reading across the 2nd row in Table 6 from left to right) the decision-maker has determined that A is between strongly preferred and very strongly preferred to B. The next step in AHP is to determine the overall weight or relative importance of each alternative and this can be done by estimating the right principal eigenvector of the matrix in Table 6 (details not shown). For the criteria of hardware expandability one is then left with a weighting (number between 0 and 1) for each system A, B, and C. In order to overcome any inconsistencies in the pairwise comparissons a consistency ratio is also calculated for the decision matrix (Saaty, 1980). One can then repeat this process for different criteria in the hierachy and also do a pairwise comparisson of the different criteria. The data is syntesised until a composite weight is obtained for each of the "alternatives" represented as the final level of the hierarchy. This composite weight, then, is an overall measure of importance for the particular element or alternative.
16
3.3
APPLICATION TO MEDICAL DEVICES
AHP has been used in a limited way in the assessment of medical technologies. Hummel et al (Hummel et al., 2000) used AHP at the early stage of product development of a new blood pump in helping to make a decision between two competitors based on technical, medical and social requirements. After an initial brainstorming session by experts to establish the requirements of a pump a hierarchy of needs was set up and a pair-wise comparison of requirements was undertaken by scoring them from 1 to 9. A software system called Team Expert Choice (2005) then carried out the analysis to calculate weighting factors representing the importance of the requirements and the priorities reflecting the qualities of the alternatives. In addition, inconsistency ratios were derived to reflect the degree to which each redundant comparison did not accord with the remainder of the pair-wise comparisons. As a result, the approach steered technology development and diffusion through discussions about the effectiveness of the blood pump in comparison to its competitors, and thence onto strategy to improve the pump. Sloane et al (2002) describe a case study using the Analytic Hierarchy Process (AHP) to perform a microeconomic Health Technology Assessment (HTA) on neonatal ventilators for a women’s health facility. Ventilators for neonates range in price from $18,000$40,000, and each one has very different features. Also, the ventilators have a very significant life-cycle cost of ownership due to supplies and maintenance requirements, which can dwarf the initial purchase price. A senior clinical engineer (CE) and respiratory therapist (RT) agreed to be help build the AHP model and the AHP was implemented using Expert Choice 2000. Criterion weights were derived by having the CE and RT do pairwise comparisons of the importance of each criterion against every other criterion in the same group. Ultimately a composite score for each alternative identified the best choice of neonatal ventilator.
3.4
POSSIBLE CONCERNS/LIMITATIONS
Despite it’s attractiveness as a decision making tool, analysts have voiced a number of concerns about the AHP. French (1996) provides a succinct critique; Some of the main doubts raised are: •
Inconsistency in AHP can be generated directly as a result of the limits of he 1 - 9 scale. A may be scored 3 in relation to B and B similarly scored 5 relative to C. But the 1 - 9 scale means that a consistent ranking of A relative to C (requiring a score of 15) is impossible.
•
The link between the points on the 1 - 9 scale and the corresponding verbal descriptions does not have a theoretical foundation.
•
Weights are elicited for criteria before measurement scales for criteria have been set.
•
Thus the decision maker is induced to make statements about the relative importance of 17
•
items without knowing what, in fact, is being compared.
•
Introducing new options (or removing old ones) can change the relative ranking of some of the original options even though nothing else has changed. The relative weights can also change dramatically over the criteria because of the removed judgments.
•
In many cases, a large number of pair-wise comparisons may be needed and can lead to responded fatigue. The ability for humans to remain consistent and independent in judgment over a large number of paired comparisons is extremely difficult. Methods for reducing the number of pairwise comparisons have been proposed (Millet and Harker, 1990).
•
AHP forces the choice between different alternatives even though none of the alternatives may be acceptable. There is a danger of ‘picking the best of a bad bunch’.
3.5
SUMMARY
The Analytic Hierarchy Process is a method for formalizing decision making where there are a limited number of choices but each has a number of attributes and it is difficult to formalize some of those attributes. AHP consists of the following steps: 1. a hierarchical formulation of the problem at hand; 2. a procedure for weighting each element at a particular level of the hierarchy, with regard to the "contribution" it makes to elements at the succeeding level of the hierarchy, by means of a procedure of paired comparisons; and 3. Repetition of process 2 until a "composite weight" is obtained for each of the "alternatives" represented as the final level of the hierarchy. This composite weight, then, is an overall measure of importance for the particular element or alternative.
3.6
RECOMMENDATIONS FOR MATCH
Despite the concerns, AHP is a systemayic methodology that can force decision-makers to make consistent judgements when screening new products. NPD project screening in many firms is rather unsophisticated and there is much room for improvement. AHP may be particularly useful in early device development by involving panels of users and experts in determinining criteria and weighting of choices between different real or theoretical competing devices. It may be particularly useful when many factors are involved in a complex decision process, where some factors are more important than others. AHP can provide a broad analysis of a medical technology in its development stage and can help formulate market, technical, medical and social requirements for the performance of a device to support decision making concerning its development.
18
4 FUZZY LOGIC 4.1
BACKGROUND AND DEFINITIONS
Decision-makers tend to make decisions based on a combination of their knowledge, past experience and subjective judgements. Fuzzy set theory and fuzzy logic was introduced by Zadeh (1965) to deal with the vagueness and ambiguity involved in human reasoning. Fuzzy set theory has now become well established as an engineering discipline (Sugeno, 1985; Hess, 1995). Fuzzy sets provide the appropriate framework to evaluate the possibility of events rather than their probability. Probability is related to the frequency of occurrence of events, while the fuzzy sets provide the appropriate framework to evaluate the possibility of events rather than their probability (Garavelli et al., 1999). There is always degree of uncertainty and complexity when making any decision. Zadeh suggested “as complexity rises, precise statements lose meaning and meaningful statements lose precision.” Fuzzy logic simplifies the complexity by making a suitable trade-off between the information that is available to us and the amount of uncertainty we allow (Klir and Folger, 1988). In a sense, fuzzy logic resembles human decision making with its ability to work from approximate data to find precise solutions. An important aspect of fuzzy logic is that it uses linguistic variables, thus performing computation with words. Phrases such as “around 20%”, “approximately between £300 and £450”, “very expensive”, “cheap” can be represented. However, to tackle the ambiguities involved in the process of linguistic estimations, it is better to convert these linguistic terms to fuzzy numbers (Liang and Wang, 1991; Prabhu and Vizayakumar, 1996). This process is called fuzzification. For example, 0.0 represents False (or non-membership of a set) and 1.0 represents True (or membership of a set). Fuzzy reasoning is then guided by "If-Then rules" typically stated in human language. Fuzzy sets are the basis for fuzzy logic. They are mostly used to describe the values of a linguistic variable, for example, tall, expensive. In traditional sets this says that an element of the set S is either a member or a non-member of the subset U. There are no partial members in traditional sets. A fuzzy set is a set whose elements have degrees of membership. That is, a member of a set can be a full member (100% membership status) or a partial member (e.g. less than 100% membership and greater than 0% membership). Essentially, a fuzzy set is a set whose elements have degrees of membership. A membership function is a mathematical function which defines the degree of an element's membership in a fuzzy set.
4.2
EXAMPLE OF FUZZY SETS
Here is an example (Marakas, 2003) describing a set of people of differing heights using fuzzy sets. In general, tall people can be considered as 7 feet or over. If we use this strict 19
interval to define tall people, then a person who is 7 feet is tall is a member of the set, whereas someone who is 1 inch less is defined as small (not a member of the tall set). To remedy this paradox imposed by categorical classification systems the boundary between the strict separation of tall and small can be relaxed. This separation can easily be relaxed by considering the boundary between tall and small as "fuzzy". Figure 1 graphically illustrates a fuzzy set of tall and small people.
Figure 1 Fuzzy set of tall and small people
We define a fuzzy subset TALL, which will answer the question "to what degree is person x tall?” We have to assign a degree of membership in the fuzzy subset TALL. The easiest way to do this is with a membership function based on the person's height. For persons less than 5 feet in height, their degree of membership of the set ‘tall’ is zero. The rest of the member ship function is described by the equations below. tall(x)= {0,if height(x) < 5ft, (height(x)-5ft)/2, if 5ft Strike price (K), then ⇒ net payoff = S – K – c
where c = call premium paid to obtain the option
If the value of the underlying asset (S) < Strike Price (K) ⇒ Buyer does not exercise and the call option is allowed to expire
For a Put option If value of underlying asset (S) < Strike price (K), then ⇒ net payoff = K – S – p
where p = call premium paid to obtain the option
If the value of the underlying asset (S) > Strike Price (K) ⇒ Buyer does not exercise and the put option is allowed to expire
New product development is essentially made up of a sequential nature of investments (opportunities) analogous to options evaluation; there is an opportunity, but not an obligation, to go ahead with that opportunity. The longer an option lasts before it expires and the more volatile is the price of the underlying asset is, the more the option is worth. For example, a real option occurs in a research and development process that enables a company to launch a new product in to the market giving the stakeholders the “right, but not the obligation” to do so. Even if the R & D phase is successful, the market may not yet be favourable for the launch of the new product. In the pharmaceutical sector, drug development process itself contains multiple options, due to the different stages in the process. The initial investment decision to begin preclinical trials is carried out with an expectation of future cash flows (Rogers et al., 2002). However, subsequent investments are made only if preceding R&D is successful. Ultimately, the drug will be brought to market if it is technologically successful and gains regulatory approval. Each stage of investment is analogous to a call option involving a decision to invest further. At any stage the development of a product can be stopped. There are various examples of business situations that can be modelled as real options. Some are listed below (Amram and Kulatilaka, 1999): •
Option to defer: The value of waiting to introduce a product in to the market place, say, until better market information is obtained may exceed the value of immediate entry 26
•
Growth Options: The decision to invest in entry into a new market may open up future growth opportunities.
•
Flexibility Options: An option to reallocate resources or alter operating scale has value. For example, to expand, contract or shut down a product line.
•
Exit (or abandonment) Options: Value the opportunity to walk away from an investment in response to new market data and thereby increase the value of an investment.
•
Learning Options: A staged investment in gathering information significantly enhances the value of the overall investment.
5.4
THE BLACK-SCHOLES MODEL FOR VALUING OPTIONS
To calculate the value of an option one can use the Black-Scholes model, which requires values for five parameters. Stock Price: the economic value of the project being evaluated, i.e., expected present value of the cash flows from the project. It can be calculated by using the DCF approach. Strike Price: the investment necessary to fund the project that would be incurred if the decision is made to commercialize the technology. Clearly, the higher the strike price, the lower the value of the call. For real options this may not be constant over time (unlike financial options) and can vary as economic/market/technology conditions change. Expiration Time: Expected time to build the product. This will depend on technology (a product’s life cycle), competitive advantage (intensity of competition), and contracts (patents, leases, licenses). The longer the time, the more valuable the option. This is because the longer time to expiration provides more time for the value of the underlying asset to move, Dividends: Payments or costs incurred that have a negative impact the value of the real option, for example, cash flows lost to competitors that invest in an opportunity, depriving later entrants of cash flows. Risk-free interest rate: the yield of a risk-free security with the same maturity as the duration of the option. For example in the U.S., the risk-free rate is determined by the yield of short-term government bonds or treasury bills (Rogers et al., 2002). Volatility: the unpredictability of future cash flows related to the asset. More precisely it is the standard deviation of the growth rate of the value of future cash inflows associated with it. In contrast with financial equity options, volatility is sometimes very difficult to estimate for real options because real options are not traded and in many cases will not have historical rates of return associated with them. It may be possible to estimate future market volume with its volatility by using appropriate statistics from similar ‘peer’ products. This technique is called spanning (Perlitz et al., 1999). However when similar peers are not available, a technique called hotelling (Perlitz et al., 1999) can be used to 27
find estimates for the volatility. This uses a DCF valuation to calculate a product’s value probability distribution from which the standard deviation can be calculated. Options become more valuable when uncertainty and risks are higher, since the more variable the possible outcomes, the more valuable the option to abandon or to invest at a future date as uncertainty is resolved. This is perhaps the key difference between options and NPV analysis. Unlike NPV, when buying an option, a company hasn’t bet the entire value of its investment at the beginning. In summary, real options are most valuable when: •
The stock price is high, volatile and not subject to erosion from dividends.
•
The strike price is low
•
Risk-free interest rates are high
•
The time to expiration is long
There are many other models that have been suggested to value real options including decision tree analysis (Morris et al., 1991), binomial analysis (Cox et al., 1979) , and Geske model (Perlitz et al., 1999). Each has different required inputs and is appropriate in different situations.
5.5
EXAMPLES OF APPLICATIONS OF REAL OPTIONS IN PRACTICE
The real options method has been used in industries such as natural resources, information technology, energy, and pharmaceuticals, which have a high degree of uncertainty in product development investments. In the early 1990s, Merck used option analysis to evaluate a proposed business relationship with a small biotechnology company code-named gamma in order to gain access to their new technologies at early stages of development (Nichols, 1994; Bowman and Moskowitz, 2001). Merck wished to licence the technology with a view to commercialisation and market introduction of products. The technologies would need to be transferred from basic to applied stage of research and if the research looked promising, Merck would need to build manufacturing facilities, resolve regulatory issues and
make
the
appropriate
marketing
and
other
start-up
expenditures
for
commercialisation. Merck negotiated with Gamma to license the technology. Under the terms of the proposed agreement, Merck would make a $2 million payment to Gamma over a three year period. In addition, Merck would pay royalties to Gamma if the product was introduced to the market. Merck had the option to terminate the agreement at any time if the research progress was not satisfactory. In short, the project resembled a real option and the Financial Evaluation and Analysis Group at Merck used the Black-Scholes model to determine the project’s option value. The project volatility was measured using a sample of biotechnology stocks and calculating their return volatility (standard deviation of 28
annual returns). A risk-free rate of interest of 4.5% was assumed, representing the U.S. Treasury bond yield rate over the two to four year period considered for the project. Merck calculated the Black-Scholes option value for a base case scenario and for various sensitivity cases. The analysis led to the conclusion that the option value exceeded the option cost in most cases and so Merck ultimately licensed the technology with a view to commercialisation. Faulkner (1996) provides an excellent survey on the use of options thinking in R&D project valuation. He presents a simplified valuation of an R&D project with an option on its commercialization phase and argues the case for R&D project valuation using the real options approach over the traditional DCF approach as borne out by experiences at the Eastman Kodak Company. An options thinking approach emphasises future uncertainty and encourages an adaptive approach that anticipates that changes to product development process will be required. He goes on to compare R&D investment strategies of Japanese and USA companies. While the Japanese approach is not directly derived or based on real options theory, Faulkner suggests that their approach is entirely consistent with options thinking. The product development process in Japanese companies is driven by continual market feedback with product design changes driven by learning from this feedback. The DCF mindset used by many western companies tends to map out a ‘rigid’ product development pathway which is defined by the predicted yield on an investment. Japanese companies have a higher long-term performance because they consider the growth and increase of the market share as being more important that the return on capital. Lint and Pennings (Lint and Pennings, 2001) considered the new product development (NPD) process as a series of real options with reducing uncertainty over time, an approach that originated from applying real options insights into the product development process at Phillips Electronics. NPD is treated as an incremental process, which is an extension of Cooper’s NPD stage-gate process (1990) but uses the option approach to give explicit decision rules in order to derive economic criteria for the go/no go decision before and after the R&D stage, including the decision to launch a new product on to the market. The project value is assumed to follow a geometric Brownian motion as per the Black-Scholes model. (The geometric Brownian motion assumption corresponds to assuming a log-normal distribution for the stock price and in addition the standard deviation of the logarithm of the stock price is proportional to the square root of the time horizon (Hull, 1996)). Also the product launch is regarded as an American perpetual call option, that is, it can be exercised and time after the R&D stage with no limit on the exercise time. A mathematical model is derived to assess the value of options in NPD process as described and a 2x2 matrix of uncertainty versus option value is developed for both the R&D and product launch stages that gives management a tool to decide at which point to abandon a project. This technique has been used for Phillip’s R&D pipeline. 29
Neely and Neufville (2001) develop a hybrid real options approach that combines decision analysis and options analysis into a practical means of accurate valuation of projects. The overall valuation method consists of the usual three elements: set-up, analysis, and examination of the sensitivity of the results to the assumptions made. The analysis is divided into technical and financial parts thereby allowing technical and financial experts to handle these independently. Neely and Neufville go on to describe an application of the hybrid real options valuation method to the Research and Development (R&D) program of a major automotive manufacturer. The R&D project concerned a major effort to optimize the design of engine parts and reduce their failure rate and weight, the benefits of which could be expressed in dollars. Rogers et al (2002) develop a stochastic programming-based model of pharmaceutical research and development (R&D) portfolio management using a real options decision tree approach for making optimal project selection decisions and to value the uncertainty of candidate drugs. A method is developed to model new product (drug) development as a series of continuation/abandonment options to value management flexibility, deciding at each stage in pharmaceutical R&D whether to proceed further or stop development. Jensen and Warren (2001) examine the practicalities of applying real options theory to valuing research in the service sector. Based on a case study of an electronic commerce programme at British Telecom (BT), the authors use a compound options model, the Geske model (Perlitz et al., 1999), based on the e-commerce project lifecycle of research, development and implementation. Kellogg et al (1999) compute the value of a biotechnology firm, Agouron Pharmaceuticals, Inc., as the sum of the values of its current projects. Three methods to value Agouron using a real options approach are discussed: the decision tree method, the influence diagram method and, the binomial-lattice method. McCormack et al (2001) apply real options techniques to evaluate a company’s extensive portfolio of proven undeveloped reserves (PUDs) of oil and gas. Cameron and Bashshur have suggested a real options model for the financial evaluation of telemedicine (Cameron and Bashshur, 2001)
5.6
CONCERNS/LIMITATIONS OF REAL OPTIONS ANALYSIS •
The complexity of the mathematics of real options, such as the use of partial differential questions, means that real options analysis will appear to be ‘black box’ to most managers with a consequent lack of transparency. However, this is being addressed by the increasing availability of real options tools in a spreadsheet format.
•
Questions remain about a reasonable estimation of certain variables of the Black-Scholes Model such as stock price and volatility
30
•
Certain assumptions of option models are difficult to meet in the non-financial world. For example, the assumption of a complete market without any arbitrage2 opportunities.
•
Options discount management realities. Real options don't expire according to a contract as financial options do, and thus managers can't be counted on to pull the plug on a project (i.e. exercise an "abandonment option") when they should (Teach, 2003).
5.7
SUMMARY
The Real options valuation technique is based on the option pricing principles developed by Black and Scholes for the financial world. It aims to capture the value of management flexibility in a world of uncertainty and helps companies quantify value overlooked by traditional discounted cash flow methods. Real options are not just about "getting a number". They provide a far more complete picture of not only technology values but also the drivers of that value.
5.8
RECOMMENDATIONS FOR MATCH
To date most practical application of Options Theory for research evaluation has been in the pharmaceutical and biomedical industries, for example, see Nichols (1994). The drug development process contains multiple options, due to the different stages in this process. This is very similar to the development of a medical device. That is, there is a discovery (or concept) process followed by phased trials, regulatory approval and finally launch. Like the evaluation of a new drug, the value of a medical device at concept stage is difficult to assess because it is subject to considerable uncertainty (technological, regulatory and market demand). A real options framework emphasises this uncertainty, encourages a flexible approach to resolve these uncertainties and anticipates future changes in development process. Various real options models have been applied to new product development and this approach for a medical device would make for an interesting and novel study
2
Arbitrage - The purchase of securities on one market for immediate resale on another market in order to profit from a price discrepancy 31
6 EXPERT SYSTEMS Artificial intelligence (AI) is a discipline that is concerned with research and development of computer hardware and software that is designed to exhibit some form of intelligence and imitate the human mind. As a discipline, AI has grown enormously in recent years as more and more commercially feasible applications become available. Increasingly, industrial companies are using AI technologies and AI applications today span the realm of manufacturing, consumer products, finance, management and medicine. Expert systems are by far the most widely used AI techniques used in complex decision making problems to aid new product development (Rao et al., 1999).
6.1
BACKGROUND
An expert system is a knowledge based system (or computer program) designed for solving problems within a specific domain or discipline by using knowledge and procedures to make decisions that exhibit a degree of expertise comparable to a human expert. For example, an expert system could diagnose a person’s ailment when the symptoms of the ailment are input in to the model. Unlike conventional programs, which use algorithmic processes to achieve a specific result, expert systems use heuristics (rules of thumb) and reasoning to elicit conclusions from stored facts. The development of a conventional program is generally a sequential process, where as developing an expert system is a highly iterative process. Expert system can be thought of as having four components: the knowledge base, inference engine, knowledge acquisition, and user interface. Figure 3 shows the main concepts associated with an expert system. 6.1.1
The Knowledge Base
The Knowledge Base is specific to a particular problem domain in the expert system and is the repository of the facts and rules provided by a human expert. The knowledge the expert uses to solve a problem must be converted into a form that can be used to code into the computer and then be available for decision making by the expert system. There are various formal methods for representing knowledge and the choice of the method of representation depends on the characteristics of the particular problem. One widely used representation is the production rule, or simply rule, which is why expert systems are often referred to as rule-based systems. Production rules permit the relationships that makeup the knowledge base to be broken down into sub-units. Rules consist of two parts, an IF and a THEN part. They can take various forms, for example,
32
Figure 3 The main concepts of an Expert System
IF antecedent….THEN consequent IF condition …..THEN action IF premise…….THEN conclusion Typical rules in a rule-based expert system may take the following forms 1. IF the probability of technical success is less than 20%, THEN discontinue investment – (condition-action). 2. IF the battery is dead, THEN the car will not start – (premise-conclusion). Rules may also incorporate AND or logical OR operators. For example, 3. IF (potential market share for product is greater than 10%) AND (cost of development to prototype is less than £50000) THEN (proceed with development)
33
Typically systems can have from a few hundred to a few thousand rules and rules are usually interrelated with each other. One the most well known examples of a rule-based system is MYCIN, a medical diagnostics expert system to aid doctors with the diagnosis and treatment of blood infections caused by bacteria in the blood and meningitis. A typical rule in Mycin is as follows, IF the strain of the organism is gram negative AND the morphology of the organism is rod AND the aerobicity of the organism is anaerobic THEN there is strongly suggestive evidence (0.8) that the class of the organism is Enterobacteriaceae. Note that a certainty factor (C.F.) is given, reflecting the degree of the original expert's confidence about the knowledge involved. By assigning certainty or confidence factors an expert system has the capacity to deal with uncertainty and to offer solutions to problems without a complete set of data. 6.1.2
Knowledge Acquisition
The quick extraction of knowledge from domain experts to develop the knowledge base is the key to a successful expert system. Very often knowledge acquisition becomes the bottleneck for the whole system (Feigenbaum, 1977). The process of acquiring and representing knowledge is usually called knowledge engineering and the task of endowing expert systems with knowledge is performed by knowledge engineers. The knowledge engineer is usually a computer specialist, who is trained in the extraction of information from experts, and is also familiar with the domain. Knowledge is usually acquired by a series of intensive interviews, both structured and unstructured with an ‘expert’ in the field. The idea here is not just to transfer facts but to understand how the expert reasons with knowledge, that is, to gain an understanding of the expert’s thinking process. Buchanan et al (1983) have described a process model to acquire knowledge for an expert system. The 5 key stages are: Identification: Identify the problem characteristics that the expert system is needed to solve (the problem domain). Identify the experts, users and the resources available for the project. Conceptualisation: Identify the key concepts and the relations between them and how they are related by the experts in the problem domain. Formalization: Structure the knowledge into a formal representation.
34
Implementation: Turn the formalization of knowledge into the chosen knowledge representation formalism, for example, production rules. Testing: The validation of knowledge and reasoning to test if the system is functioning properly. Common sources of error are rules which are either missing, incomplete or incorrect. Competition between related rules can also cause unexpected bugs (Jackson, 1999). Not all expert systems rely on a dedicated knowledge engineer to gather and organise the knowledge. Some expert systems are expert-driven, where an expert enters knowledge directly into the computer. There is software available to help in selfknowledge engineering. 6.1.3
The Inference Engine
The inference engine or inference procedure is the control structure of the expert system. It bestows the expert system with ‘intelligence’ by giving it the capacity to make inferences from premises. The inference engine is the problem processing part of the expert system. When activated by a user (initial state), it searches for rules in the knowledge base that can be matched to the description or facts (supplied by the user) in the working memory and decides which actions to execute and in which order. The inference engine selects a rule from the knowledge base and then the actions of the selected rule are executed. The rule’s conclusion or action is added to the working memory and the inference engine then selects another rule to match this elementary conclusion and executes its actions. This process is continued until the final conclusion is reached and no applicable rules remain (goal state). Two alternative ways that the inference engines can link rules to form a line of reasoning are backward chaining and forward chaining. Backward Chaining: This mode of reasoning is ‘goal driven’. The inference engine tries to prove a hypothesized conclusion (goal state) by chaining backwards and confirming the truth of all of it's premises. Forward Chaining: This mode of reasoning is ‘data driven’. The inference engine examines the current state of data in the working memory and looks for rules which will move that state closer to a final solution. 6.1.4
The User Interface
The user interface is the means of communication between the user and the expert system. The acceptability and successful utilization of an expert system is determined to a great extent on the quality of the user interface. This should be intuitive and easy to use for the intended audience. An expert system user interface will normally take the form of a set of questions or multiple choice menus with explanation facilities and on-line help systems. 35
6.1.5
Explanation of the Decision Making Process
One of the attractive features of expert systems is their ability to explain their reasoning processes, that is, why or how a particular decision or conclusion was reached. This is often termed transparency. Such explanation facilities enable the user to question and challenge the results from the expert system as well as to understand how the results were achieved (Liebowitz, 1995). Various types of explanations have been proposed for expert systems but by far the most common ones are reasoning-trace, justification and strategic explanations (Chandrasekaran et al., 1989). Reasoning-trace: Presents the chain of reasoning developed by the expert system and a complete trace of rules which have been generated or fired (including those that were not used) by a system to solve a problem. The knowledge engineer would have more use for this than the end user to see how the system was behaving and how rules and data were interacting. Justification: These explain why certain questions are put to the user during the consultation or to justify why a rule-strategy was used rather than merely describe it. Strategic: This focuses on the problem-solving approach and involves explaining the strategy used in selecting rules for problem solution.
6.2
APPLICATIONS OF EXPERT SYSTEMS IN NEW PRODUCT DEVELOPMENT (NPD)
Expert systems have been used in many applications. However it has only been in the last decade or so that they have begun to be applied to new product development decisions (Rao et al., 1999). 6.2.1
Investment Decisions
Ram and Ram (1989) describe an expert system, called INNOVATOR, that was developed to assess the success potential of new products in the financial services industry and provide GO/NO GO, GO/REVALUATE type decisions to the user. Their research was focussed on the concept screening stage of product development. INNOVATOR is comprised of the major components of an expert system and knowledge was acquired from planning experts from five financial service organizations by conducting in-depth personal interviews with them. Experts were asked to generate a list of attributes that they perceived were vital for evaluating a product line and comment on their relevance and utility. From these interviews a final list of twenty key attributes covering market features, financial expectations, product features, corporate fit and competition was compiled. Similar lists were compiled for evaluating a single product and brand. All attributes were subsequently given a weighting and value by the experts. A secondary source of knowledge was acquired from general investment surveys and publications such as Forbes and the Wall Street Journal.
36
By using IF THEN production rules together with confidence factors or probabilities, the expert system calculates scores for the attributes and classes of attributes and an ‘Index of Innovation Acceptance’ is obtained to help provide the user with a Go/No Go/ Reevaluate recommendation. 6.2.2
Pharmaceutical Products
Akoka et al (1994) present an expert system within a commercial environment for assessing the chances of success of a new pharmaceutical product. The model presented considers three fundamental aspects of the decision making process in new product development: finance, marketing, and economy. This is far from a complete model of the process but the authors argue that other components such as technical and human resources could be incorporated in to the expert system. The product development process of the new drug is mainly considered for the French market in which the national health insurance agency regulates the price of pharmaceutical products and also decides the rate of reimbursement to patients. Experts were drawn from staff members of the pharmaceutical company and two dedicated knowledge engineers were employed. A rule based expert system (including certainty factors and weightings of variables) with a backward-chaining inference engine was employed. The variables in the premises and conclusions of the rules were given symbolic values, such as strong, strongly increasing, weak, and these symbolic values were further mapped to numerical intervals. For example; labour market: very good (100-90), risky (29-21). By mapping out the dependencies of each variable in the form of decision trees and dependency diagrams it is possible to see how the input variables fit together to give an overall score for project feasibility as well as individual scores for important parameters such as competitors position, market entry barrier, advertising spend, sales etc. The authors conclude their work by comparing the expert system results with the actual results of the product after seven months and note that most of the values of the various factors of the project provided by the expert system differ only slightly from the actual results for the product. However, it is interesting to note that an important discrepancy occurs between actual market share and predicted market share of the product, a discrepancy all too common in business cases for a new product development. The authors argue that the expert system can be used to identify and analyse why this discrepancy exists. This paper has resonance with new product development for medical devices as it shares many of the same issues. 6.2.3
Other Industry Applications
Liberatore and Stylianou (1995) consider the strategic decision to commit a new product to full scale development within Armstrong World Industries, a manufacturer of flooring products. They develop an expert support system, called PRAS (project assessment system), which incorporates management science methods, statistical methods, decision support systems and knowledge based systems to create an integrated project evaluation 37
model. Two systems are developed, one based on a single expert (a product development manager) and the other based on a team of experts (the strategy team). Technology, manufacturing, market assessment, customer satisfaction and finance were seen as the major systems in product development evaluation. Again, the expert system was based on production rules and techniques such as the Analytic hierarchy process (AHP), logic tables and ratings scales were used to weight and score the criteria. A key output of the system was an overall score for the product being evaluated together with a GO, NO GO, UNCERTAIN (more data required) decision. Cohen, Eliashberg, and Ho (1997) present a decision support system, called Product Portfolio Support System (PS2), for evaluating the financial prospects of new line extension concepts in a fast moving consumer packaged goods industry. The system is based on an in depth analysis of 51 new products launched over a three-year period at a major food manufacturer. The system was designed for line extension projects in which historical data of other NPD projects can be used to predict the success of the new product. It was not designed for revolutionary or novel products. The system allows for continuous assessment of a new product concept through its development process and also allows one to estimate the cumulative revenue and cost of the new product over its life cycle. PS2 seeks to go beyond focussing only on the individual new product and is tailored towards evaluating an entire product line and evaluating the incremental financial impact of the new product on the existing product line. While not fully implemented by the food manufacturer it was customized for, the authors argue that PS2 led to a formal product development protocol within the company which combines the insights from the expert system and best practice from within the company. It has also facilitated crossfunctional and inter-project learning. A slightly different approach to expert system development is taken by Balachandra (2000), who firstly attempts to develop a framework model that classifies NPD projects into three different contextual variables: (1) nature of the innovation; (2) nature of the market; and (3) nature of the technology. The overall model is presented as a 3dimensional cube of which the three variables are its axis. Any NPD project may be classified to fall into a location within the cube, depending on the values assigned to the project along the three dimensions (variables). The location then dictates the type of success/failure factors that need to be used in the evaluation of the project (and determines their appropriate weights) as well as the managerial organization and approach suitable for the project. Following on from this a rule based expert system was developed in which the rules were based on characteristics in the three variable areas. The expert system then determines the location of the project within the cube to which the project belongs using a set of rules and suggests the appropriate success factors and the appropriate management approach that needs to be taken. The author concedes that the system is not fully functional at the time of writing but that the system was being tested with the active involvement of participating firms. 38
6.3
CONCERNS/LIMITATIONS OF EXPERT SYSTEMS •
There are a number of problems associated with knowledge acquisition. It is extremely time consuming and experts may be reluctant to commit so much time and effort to writing rules and explaining how they came to their conclusions. Experts tend to make decisions based on personal experience, and although sub-consciously they may have followed a logical and stepwise route in making a decision, this will not always be easy to express in terms of rules and confidence factors. This type of knowledge is extremely difficult to elicit.
•
Knowledge engineers, who are generally computer specialists, need to have a very good grasp of the problem domain to ensure effective communication and a common frame of reference with the expert.
•
Expert systems cannot learn or change over time from experience as can a human expert. Expert systems must be explicitly updated and thus require constant maintenance. One field of AI called machine learning and case-based reasoning may help overcome this problem (Jackson, 1999) (the discussion is beyond our scope here).
•
Human experts are capable of dealing with unusual situations, different forms of knowledge representation and have ‘common sense’. Expert systems still fail to a degree on these points.
•
6.4
Only partial validation so far for medical products
SUMMARY
Expert systems aim to solve real world problems by emulating human behaviour. Knowledge is extracted from human experts and is represented in the form of facts and rules in the knowledge base. The use of certainty factors allows the uncertainty of certain knowledge to be represented. The inference engine of the system decides which facts and rules to execute to solve a problem. A user interface facilitates communication between user and expert system and an explanation system allows the user to trace the line of reasoning in reaching a decision. Despite the high expense and resources needed to build and develop an expert system it can be maintained relatively inexpensively and is inexpensive to operate. In return, an expert system offers a consistent and reproducible decision making process with a knowledge base that is permanent (humans can forget) which can also be used to preserve scarce expertise. In NPD expert systems have been shown to be very flexible and effective in not replacing but in aiding and improving decision making in terms of product evaluation and concept screening. It forces experts and other key decision makers to systematically consider all the factors and attributes needed to make a decision as well as elucidating the interdependencies and interrelationships between the different factors. In common with other decision making tools an expert system provides a structured approach to decision 39
making. However to be truly effective expert systems need constant updating to adapt to changing markets, product lines etc and hence for the continual success of such a system it would need a managerial ‘champion’ and dedicated resource within a firm to maintain and sustain it.
6.5
RECOMMENDATIONS FOR MATCH
One of MATCH’s aims is to develop tools for industry to help them assess value of medical device products, particularly at a concept stage and support/advise operational, tactical and strategic decision making. Given the complexity of the NPD process, expert systems appear a good way of systematically collecting data (quantitative and qualitative) and expertise as well as automating the evaluation of new products and concepts. Providing one can overcome the problems of knowledge acquisition and get the user interface right, developing an expert system that also incorporates other techniques such management science methods (and other techniques cited in this report) as inputs or knowledge processing elements within the system would offer the medical device industry with a useful and valuable tool for project evaluation. It would also allow managers to track the line of reasoning behind a decision.
40
7 CONJOINT ANALYSIS 7.1
WHAT IS CONJOINT ANALYSIS?
Conjoint analysis (CA), also called trade-off analysis, is a widely used marketing research tool to estimate the value people place on the attributes or features of a new product and to determine the overall utility of the product. CA recognizes that consumer decisions are based not on a single factor or criterion, but on several factors or attributes considered jointly, hence the term conjoint. It was first applied to the analysis of consumer choices in the early 1970s and has since been used in hundreds of applications of judgement and complex decision making ranging from product concept testing and health care facility preferences to the evaluation of energy policies on conservation (Green and Srinivasan, 1978). CA has potential for use in almost any scientific field where measuring people’s perceptions or judgements are important (Riquelme and Rickards, 1992).
7.2
ATTRIBUTES, LEVELS AND UTILITIES
CA attempts to replicate the decision processes of experts by breaking down a decision into its component parts. In the field of CA products or services often referred to as profiles, are described consistently in terms of attributes and levels. Attributes represent dimensions in which the product can be defined, such as size, colour, and price. Each attribute is then made up of specific levels which represent the value of the attributes, such as small, blue and £100. A specific product concept is therefore characterized by its attribute levels. As an example, one could describe a new product concept such as a medical device in general terms using the three main attributes such as clinical value, level of regulatory hurdles and product uniqueness. A specific concept would then be described just by ranking the level of importance of each of these attributes on a scale of say 1-10. For example, for clinical value measured on a scale of 1 to 10, 1 represents ‘explicitly does not meet clinical need’ whereas 10 represents ‘meets an unmet clinical need’. In conjoint analysis, utilities quantitatively represent the value that the respondent group places on each attribute (level) and are often called part worth utilities. These part worth utilities are then added to compute a “total utility” for the profile. Part worth utilities are derived from respondent’s rank orders of preference and his or her ratings of attribute importance.
7.3
STEPS IN PERFORMING A CONJOINT ANALYSIS
There are several basic steps in conducting a conjoint analysis. 1. Attribute selection and definition of corresponding attribute levels 2. Determining which conjoint methodology will best fit the research problem
41
3. Determining data collection method and choice of the way in which attribute profiles are presented to the respondent. 4. Analysis and evaluation of data It is vital to have given a lot of prior thought to the selection of attributes. Each attribute must be distinct and represent a single concept that is salient to the target market. The list of attributes must seek to encompass all the lines of thought in the decision environment. However, too many attributes will overstrain the decision makers with choices and too few attributes will severely reduce confidence in the forecasting capability of a CA model because not all the key factors will have been covered. A number of methods exist to identify attributes of interest including literature review such as focus group discussions, and individual interviews with industry professionals and target customers. Typically, a list of attributes is derived from several groups until the yield of additional themes becomes saturated. Once the attributes have been selected the corresponding levels within each attribute must also be chosen. Conjoint analysis combines these attribute and level descriptions into hypothetical profiles and deploys such profiles in interviews by asking people to make a number of choices between different criteria and to rate or rank them. Perhaps the examples that are most familiar to readers are surveys of domestic products such as washing machines and televisions. Having extracted the various attribute levels from a series of interviews with key informants, one can then work out numerically how valuable each of the levels is relative to the others around it, i.e. the part worth utility. A number of data collection procedures that may be used obtain individual part worth utilities. The main ones are •
Trade-off matrices
•
Full profile (ratings-based) card sort
•
Hybrid (ratings-based) conjoint
The trade-off method requires respondents to consider attributes on a two-at-a-time basis and rank the various combinations of attributes from the most to the least preferred within a matrix structure. For example, when considering the possible development of concept medical device, a typical trade-off matrix could be: Level of Regulatory Hurdles Low
Medium
High
7 4 1a
8 5 2
9b 6 3
Market Share Less than 10% Between 10 and 19% Greater than 20%
a least preferred, b most preferred.
42
In this example, a respondent has stated that his most preferred option is a device which has a potential market share greater than 20% which only requires a low level of regulatory approval. This is perhaps obvious, but the most useful information is the respondents stated preferences between the most and least preferred options. The full-profile method, more closely represents real-life decision-making situations because it utilizes the complete set of attributes. The respondent is asked to rank, order, or score a set of cards according to preference. On each of these cards, all attributes of interest are represented and a different combination of attribute levels appears. Thus, each card describes a complete product concept (Riquelme and Rickards, 1992). An example of a full profile card is shown below. New Medical Device Concept 7 Does the product meet an unmet clinical need? Level of Regulatory Hurdles? Any unique features of the product? Is there a patent to protect the product? Potential Market share Educational capability (to overcome market ignorance) Level of competition rivalry? Expected product gross margin?
No Medium Yes Yes Over 25% Medium Medium Over 50%
Rating_________ Rate this concept on a 0 to 100 scale where 0 means “I would not develop this concept” and 100 means “I would definitely develop this concept” However, it is clear to see that this procedure can lend itself to information overload, as the total number of possible profiles can be very high even for a relative low number of attributes and levels. For the example above, 1944 (35 x 23) different profiles can be constructed which is obviously too many cards for respondents to rank. Fortunately, various types of fractional factorial designs exist that use small subsets of possible combinations. Such fractional designs reduce the number of profiles that respondents have to evaluate and hence reduce the decision-making task to a more manageable size (Green and Srinivasan, 1990). The Hybrid (ratings-based) conjoint method is a combination of the full-profile and the trade-off methods. Respondents are first asked to indicate rank order of preference for levels within each attribute and then to rate the importance of the attribute. They are then asked to make paired comparisons in which two alternative profiles (normally partial profiles) are considered at a time and asked to indicate their preference using a rating scale (Johnson, 1987). For example,
43
High Level of Regulatory Hurdles Unique features of the product Over 25% Potential Market share
Low Level of Regulatory Hurdles Unique features of the product 15-25% Potential Market share
Strongly Prefer Product on left 1
Strongly Prefer Product on right 2
3
4
5
6
7
8
9
As before, statistical analysis techniques such as orthogonal arrays exist to reduce the number of choices that need to be evaluated (Addelman and Kempthorn, 1962). The ultimate objective from all of these conjoint approaches is to estimate individual level part-worth utilities so that when these are appropriately added, one can find a total utility for each combination.
7.4
APPLICATIONS OF CONJOINT ANALYSIS
Conjoint analysis has been used in research for many years in studies of judgment and decision-making. The technique has been successfully employed in hundreds of studies to elicit consumer preferences for new products/services, for conducting competitive analyses, segmenting markets, making product design decisions and concept evaluation. The technique has also been gaining wide-spread use in health care decision making (Ryan and Farrar, 2000). 7.4.1
Venture Capital Funding
Shepherd et al (2000) use a conjoint approach to investigate how venture capitalists go about making investment decisions when selecting start-ups for financing. Their research strategy focussed on eight key attributes (derived from strategy literature) regarded as most important in discriminating among competing start-up proposals. See Figure 4 for a list of these attributes used in the study. A sample of 66 venture capitalists representing 47 venture capital firms in Australia were asked to evaluate a series of conjoint profiles that described new ventures based on the eight attributes. Using a fractional factorial design a total of 39 profiles were evaluated which also included an evaluation of the twoway interaction between one of the attributes, timing of entry, and each of the other variables. Analysis of the results revealed that industrially related competence defined as ‘Venturer has considerable experience and knowledge with the industry being entered or a related industry’, was the strategy that emerged significant most often for each of the venture capitalists in their assessment of new venture profitability. The study goes on to examine the significance of the interaction of timing of entry (point at which start-up enters new industry) with other attributes and determines that interaction with lead time (period of monopoly for start up firm) is significant. Conjoint analysis provides an insight into the venture capital decision-making process when selecting start-up in terms of its likely profitability. The authors conclude that such an understanding could help potential entrepreneurs better target their business plan and presentations in order to increase 44
their chances of raising venture capital, although further research is needed before developing an actuarial model such as an expert system that incorporates the strategy variables used in the study for use by venture capitalists themselves.
Figure 4 Attributes used by Shepherd et al (2000) 7.4.2
New Product Development
In discussing the problem of designing “really new” products and services (using conjoint analysis) Vavra et al (1999) describe a real application based on the EZ-Pass electronic toll collection system aimed at speeding up and simplifying vehicle passage on toll highways, bridges, and tunnels. The system, based on electronic tag technology, was subsequently adopted by a task force of New York and New Jersey transportation agencies. A large conjoint exercise (over 3000 respondents) was carried out to gather information about what commuters wanted for the implementation of EZ-Pass and two answer two main questions: •
How should EZ-Pass be configured?
•
What level of resources should be allocated to its implementation?
Analysts used seven conjoint attributes dealing with such issues as number of lanes available, tag acquisition, cost, toll prices, invoicing, and other uses of the tag. A hybrid conjoint analysis model was chosen that incorporated a fractional factorial design resulting in 49 profiles to be evaluated. Seven years on the authors report that rush hour use of E-ZPass has steadily increased at Port Authority facilities, climbing to 60% in mid1999. Average daily use reached 44% in 1998. Nearly 200,000 E-ZPass accounts have been registered with the Port Authority with about 300,000 tags distributed by the agency. Another agency, the MTA Bridges & Tunnels, reports average daily use of 61%. Bowditch et al (2003) describe the use of conjoint analysis by a major pharmaceutical company which already had a significant franchise in a specific prescription market sector, who wanted to examine: 1. What would be the ‘key drivers’ (product features) that would be essential for newly developed products to offer to potential customers. 2.
The likely market developments with existing brands: 45
a
New formulations likely to be introduced
b
Loss of patent of a key competitor.
3. The introduction of entirely new products. 4. As a result of 2 and 3, how the existing brand franchise could: a
Be protected (defensive strategy)
b
Be expanded (offensive strategy).
In 200 face-to-face interviews, physicians in the USA were asked to evaluate a range of some 32 scenarios subdivided into four balanced sets of eight per respondent. Attributes covered drug type, form, regimen, and a variety of different efficacy levels, including speed, etc. The authors conclude that the conjoint approach was able to accurately estimate numerous product attributes, including price, in order to assess present and future opportunities within the target market. However, as with any assessment technique market share forecasts need to be taken with caution and the authors admit that it is very difficult to work out the confidence levels of market share forecast using conjoint modelling 7.4.3
Healthcare Sector
In recent years CA has begun to be applied within the health care sector. Ratcliffe et al (1999) use conjoint analysis to quantitatively examine patients’ preferences for liver transplantation services by assessing the relative importance of six attributes viz: health outcome (i.e. chance of successful liver transplant) versus process characteristics of the service provided (waiting time, continuity of contact with the same medical staff, amount of information received about the transplant, follow-up support received and distance of the transplantation centre from home). Following a pilot study of 40 patients, a sample of patients (n = 213) who have received a liver transplant were surveyed. The results of the study suggest that, even in the extreme case of a life-saving intervention, the majority of respondents were prepared to exchange a reduction in health outcome for an improvement in the process characteristics of the liver transplantation service. There have been many other studies eliciting patients' and the community's preferences in the delivery of health services (Chakraborty et al., 1993; Propper, 1995; Bryan et al., 1998; van der Pol and Cairns, 1998; Ryan, 1999; Ratcliffe et al., 2004). For a general review on using conjoint analysis to elicit preferences for health care, see Ryan et al (2000). Where cost is included as an attribute, conjoint analysis has recently been used estimate willingness to pay for treatments and/or services within health care systems, in which decisions have to be made concerning the allocation of scarce health care resources. San Miguel et al (2000) applied conjoint analysis to consider women’s preferences for two surgical procedures in the treatment of menorrhagia (excessive menstrual bleeding): hysterectomy and conservative surgery. It is then used to estimate 46
marginal rates of substitution (MRS), willingness to pay (WTP) and utilities for different ways of providing a service. (The MRS provides an indication of the extent to which respondents are prepared to trade an improvement in one attribute for a detriment in another attribute (Bryan et al., 1998)). The results suggest conservative surgery is preferred to hysterectomy, as indicated by higher utility scores for the former and a marginal WTP of £7593 to have conservative surgery rather than hysterectomy. Telser et al (2002) apply a conjoint approach to estimate the marginal willingness-to-pay (MWTP) of elderly individuals for a reduction of the risk of fracture of the femur in Switzerland. The device in question is a hypothetical hip protector which lowers the risk of a fracture by different amounts. Other attributes are ease of handling, wearing comfort, and out-of-pocket cost, which are traded against risk reduction. In 500 face-to-face interviews, pensioners stated whether or not they would buy the product. Results suggested that while individuals are interested in risk reduction in the context of fracture of the femur, they are willing to trade this off against other product attributes, specifically against wearing comfort in the case of a hip protector. The overall WTP for the product was negative indicating that it should not be included as a mandatory benefit in health insurance. Within the health care sector, an insurer (or any payer of treatment) may want to trade off preventive against curative benefits when defining its benefit package and WTP estimates can assist in determining whether the user values a product enough to use it.
7.5
CONCERNS/LIMITATIONS OF CONJOINT ANALYSIS •
Products and especially services may be hard to describe with attributes and levels. Respondents must have a common understanding of how the attributes and levels combine to produce a valid description of a product or service
•
For real applications, the number of attributes is high (greater than 10). Hence the number of product profiles to be rated becomes very large and this increases the complexity of the project analysis. It also overburdens the respondent and dealing with a large number of attributes can lead to respondent fatigue. Various hybrid conjoint models have been developed to overcome this problem by simplifying the respondent’s rating task.
•
All products and services are considered to have the exact same level of advertising, marketing and distribution. This is not the case in the ‘real world’ and adjustments need to be to account for marketing factors such as level of advertising and promotion.
•
A respondent’s decision making may be different when judging a new product in isolation to judging a new product as part of a portfolio. This must be a consideration in the development, testing and interpretation of a conjoint based study (Shepherd and Zacharakis, 1999).
47
•
There is a fair degree of complexity involved in designing the conjoint survey and administering the survey, although off-the shelf software is available to do this.
7.6
SUMMARY
Conjoint analysis is a sophisticated technique developed for measuring human perceptions and preferences in order to know which characteristics of products or services are important to consumers. The most crucial step in a conjoint study is the selection of attributes and levels; poorly defined attributes will seriously degrade the quality and validity of the results. Although different conjoint methods are used dependant upon the application, the overall objective of any conjoint model is to estimate individual level part-worth utilities so that when these are appropriately added, one can find a total utility for each combination of attributes. The technique has been successfully employed in hundreds of studies to predict preference for transportation services, financial services, automobiles, consumer durables, healthcare services and many other industrial and consumer products and services. For manufacturing firms and service providers conjoint analysis has the potential to input the “voice of the customer” into their decision-making process as early as possible.
7.7
RECOMMENDATIONS FOR MATCH
The key strength of conjoint analysis lies in its ability to elicit the preferences of users of products and services to help develop a market strategy and to aid in product design. For this reason it could be an extremely effective tool in a user needs survey to establish the preferences of patients and the community (as well as those of clinicians and policymakers) at an early stage of the decision-making process when developing a product. It has already been successfully in healthcare both to estimate willingness to pay values in the evaluation of healthcare technologies and in eliciting patients' and the community's preferences in the delivery of health services. Further methodological work is needed to validate the conjoint approach (Ryan and Farrar, 2000). In terms of an overall value assessment of a concept product, the complexity of the product development process and the market (i.e. number of attributes will be very high) may hamper the conjoint approach in this respect, but it could be used to quickly screen concepts based on a few key attributes. CA could also be used to evaluate certain key aspects of product value such as market share-product attribute sensitivity, optimal product profile, effect of a product line extension or addition etc. Another potential application of conjoint analysis that would make for an extremely insightful study would be its use to identify the attributes that product managers and other key decision makers utilize when evaluating new products and how they use this information to assess likely new profitability.
48
8 CONCLUSIONS AND RECOMMENDATIONS 8.1
CONCLUSIONS
Assessing the value of a medical device and predicting its success during development is difficult and fraught with uncertainty. A manufacturer has to navigate regulated pathways, determine market potential, prove clinical and cost effectiveness, predict return on investment and so on. Further complication comes from the diverse range of customers for medical devices, nationally and globally, with different reimbursement systems and regulatory environments. The idea of “picking a winner” is attractive but elusive in such a fragmented sector. The first step, and probably the hardest, is to assess the ‘true value’ of a device. This report has presented several decision making tools and techniques that may be used for the value assessment of a medical device. Many of these methods have been used in other industrial sectors. •
The Analytic Hierarchy Process is a method for formalizing decision making where there are a limited number of choices but each has a number of attributes and it is difficult to formalize some of those attributes.
•
Fuzzy logic is based on the ideas of fuzzy set theory – a concept that is found in our natural language in discussing issues, such as “costly” or “large”, which are not precise. It provides a simple way to suggest definite conclusions from vague, ambiguous or imprecise information and has been used extensively in new product models in other sectors.
•
The Real Options method uses options pricing concepts from the financial world to capture the value of management flexibility in a world of uncertainty and helps companies quantify value overlooked by traditional discounted cash flow methods. It has been widely used in the pharmaceutical sector.
•
An Expert system is more of a tool than a technique and aims to solve real world problems by emulating human behaviour. In NPD, expert systems have been shown to be very flexible and effective in not replacing but in aiding and improving decision making in terms of product evaluation and concept screening.
•
Conjoint analysis is a sophisticated technique developed for measuring human perceptions and preferences in order to know which characteristics of products or services are important to consumers. It is based on the selection of attributes and levels.
This list is not exhaustive. Other techniques have been studied for concept screening and new product development such as actuarial models, neural networks and technology road mapping. Also, a journal paper in MATCH is in the advanced stages of preparation and seeks to apply the Dempster-Shafer theory of evidence in the context of prioritization of a 49
range of competing medical device prototypes. The paper will indicate how this theory might facilitate intelligent modelling of the selection processes when different prototype products are in competition for funding, including venture capital based sources (Webb, 2005). However, the techniques presented in this report are intended to reflect some of the more common methods that have been used or considered by industry. It would be unwise and incorrect to suggest that one technique could encompass all the factors that may contribute to a product’s success such as the market, technology, company strategy, regulatory hurdles etc. In practice a complete value assessment model would probably require a combination of these techniques under the umbrella of say a software tool such as an expert system. An additional hurdle is that many projects in industry are chosen, not necessarily because they have the highest probability of success, but because they have the most influential managers backing them. Also, managers can become committed to failing NPD projects and consequently are less likely to terminate the projects after the go decision has been made (Schmidt and Calantone, 1998). However each technique can provide an effective input to the decision making process of new project selection. Perhaps the biggest contribution that any of decision making tools can make is that it forces experts and other key decision makers to systematically consider all the factors and attributes needed to make a decision regarding project selection and development as well as elucidating the interdependencies and interrelationships between the different factors.
8.2
RECOMMENDATIONS FOR FURTHER RESEARCH
The aim of this report was to present a broad range of techniques/tools that that could be used to define and estimate a new product project value in the earliest stages of product development in the medical device industry. The next stage of work will apply these techniques to ‘real world’ situations in order to validate the models. Some of the key recommendations from this report are: 1. Use real options based theory to model a new product development process of a medical device as a series of options and hence provide a framework of assessment at various stages of the NPD process. This model will be validated by data provided by companies. 2. Investigate the use of an AHP model as screening tool for new product concepts, i.e., in early device development stage. Develop a ‘simple’ model in the first instance, possibly using software system, Team Expert Choice (2005), and validate using company data. 3. Use of Conjoint Analysis to identify the attributes that product managers and other key decision makers utilize when evaluating new products and how they use this information to assess likely new profitability.
50
4. Investigate the development of a software tool, possibly an expert system, which can systematically collect data (quantitative and qualitative) and expertise concerning the NPD process and also automate the evaluation of new products and concepts. The aim would be to offer the medical device industry with a useful and valuable tool for project evaluation. It would also give managers the ability to track the line of reasoning behind a decision.
51
REFERENCES (2005). Expert Choice Inc. http://www.expertchoice.com/. Addelman, S. and O. Kempthorn (1962). "Orthogonal main-effect plans for asymmetrical factorial experiments." Technometrics 4(1): 21-46. Akalu, M. M. (2003). "The process of investment appraisal: the experience of 10 large British and Dutch companies." International Journal of Project Management 21(5): 355-362. Akoka, J., B. Leune, et al. (1994). "An expert system for feasibility assessment of product development." Expert Systems with Applications 7(2): 291-303. Amram, M. and N. Kulatilaka (1999). "Real options: Managing strategic investment in an uncertain world", Harvard Business School Press, Boston. Balachandra , R. (2000). "An expert system for new product development projects." Industrial Management & Data Systems 100(7): 317-324. Barbarosoglu, G. and D. Pinhas (1995). "Capital rationing in the public sector using the analytic hierarchy process." Engineering Economist 40(4): 315-341. Bates, J. H. and M. P. Young (2003). "Applying Fuzzy Logic to Medical Decision Making in the Intensive Care Unit." American journal of respiratory and critical care medicine. 167(7): 948-952. Ben Ghalia, M. and P. P. Wang (2000). "Intelligent system to support judgmental business forecasting: the case of estimating hotel room demand." Fuzzy Systems, IEEE Transactions on 8(4): 380-397. Black, F. and M. Scholes (1973). "The Pricing of Options and Corporate Liabilities." The Journal of Political Economy 18(3): 637-654. Booz, Allen, et al. (1982.). "New Product Management for the 1980's", Booz, Allen, and Hamilton, Inc., New York. Bowditch, A., G. Gurrieri, et al. (2003). "The use of combined conjoint approaches to improve market share predictions." International Journal of Market Research. 45(3rd Quarter): 389-405. Bowman, E. H. and G. T. Moskowitz (2001). "Real options analysis and strategic decision making." Organization Science. 12(6): 772-777. Brealey, R. A. and S. C. Myers (1991). "Principles of corporate finance", McGraw-Hill, New York. Bryan, S., M. Buxton, et al. (1998). "Magnetic resonance imaging for the investigation of knee injuries: an investigation of preferences." Health Economics 7(7): 595-603. Buchanan, B. G., D. Barstow, et al. (1983). "Constructing an expert system". Building Expert Systems. F. Hayes-Roth, D. Waterman and D. Lenat. Reading, Massachusetts, Addison-Wesley Publishing Company. Buyukozkan, G. G. and O. Feyzioglu (2004). "A fuzzy-logic-based decision-making approach for new product development." International Journal of Production Economics 90(1): 27-45. Cameron, A. C. and R. L. Bashshur (2001). "A real options model for the financial evaluation of telemedicine". American Telemedicine Association Annual Meeting, Ft Lauderdale. Carlsson, C., Walden, Pirkko (1995). "AHP in political group decisions: A study in the art of possibilities." Interfaces 25(4): 14-29. Chakraborty, G., G. J. Gaeth, et al. (1993). "Understanding consumers' preferences for dental service." Journal of Health Care Marketing. 13(3): 48-58.
52
Chan, F. T. S., M. H. Chan, et al. (2000). "Evaluation methodologies for technology selection." Journal of Materials Processing Technology 107(1-3): 330-337. Chandrasekaran, B., M. C. Tanner, et al. (1989). "Explaining Control Strategies in Problem Solving." Intelligent Systems 4(1): 9-15,19-24. Chao, L.-C. and M. J. Skibniewski (1998). "Fuzzy logic for evaluating alternative construction technology." Journal of Construction Engineering & Management 124(4): 297-304. Chen, H.-K., W.-K. Hsu, et al. (1998). "A comparison of vertex method with JHE method." Fuzzy Sets and Systems 95(2): 201-214. Chen, K. and N. Gorla (1998). "Information system project selection using fuzzy logic." Systems, Man and Cybernetics, Part A, IEEE Transactions on 28(6): 849-855. Cohen, M. A., J. Eliashberg, et al. (1997). "An anatomy of a decision-support system for developing and launching line extensions." Journal of Marketing Research 34(1): 117-129. Cooper, R. G. (1990). "Stage-gate systems: A new tool for managing new products." Business Horizons 33(3): 44-54. Cooper, R. G. and E. J. Kleinschmidt (1986). "An investigation into new product process: steps deficiencies and impact." Journal of Product Innovation Management 3(2): 71-85. Cooper, R. G. and E. J. Kleinschmidt (1993). "Screening new products for potential winners." Long Range Planning 26(6): 74-81. Cox, J. C., S. A. Ross, et al. (1979). "Option pricing: A simplified approach." Journal of Financial Economics 7(3): 229-263. Davis, L. and G. Williams (1994). "Evaluating and Selecting Simulation Software Using the Analytic Hierarchy Process." Integrated Manufacturing Systems 5(1): 23-32. Dixit, A. K. and R. S. Pindyck (1994). "Investment Under Uncertainty", Princeton University Press, Princeton, NJ. Medical Device Technology (2003). "European Medical Device Market Research Report 2003". Faulkner, R. C. (1998). "How to Evaluate Medical Device Technologies." Medical Device & Diagnostic Industry Magazine, August Faulkner, T. W. (1996). "Applying 'options thinking' to R&D valuation." Research Technology Management. 39(3): 50-56. Feigenbaum, E. A. (1977). "The art of artificial intelligence: Themes and case studies of knowledge engineering". 5th International Joint Conference on Artificial Intelligence, Cambridge, Massachusetts, USA. French, S. (1986). "Decision theory: an introduction to the mathematics of rationality", Ellis Horwood, Chichester. French, S. (1996). "Multi-attribute decision support in the event of a nuclear accident." Journal of Multi-Criteria Decision Analysis 5(1): 39-57. Garavelli, A. C., M. Gorgoglione, et al. (1999). "Fuzzy logic to improve the robustness of decision support systems under uncertainty." Computers & Industrial Engineering 37(1-2): 477-480. Goldenberg, J., D. R. Lehmann, et al. (2001). "The Idea Itself and the Circumstances of Its Emergence as Predictors of New Product Success." Management Science 47(1): 69-84. Green, P. E. and V. Srinivasan (1978). "Conjoint Analysis in Consumer Research: Issues and Outlook." Journal of Consumer Research 5(2): 103-123.
53
Green, P. E. and V. Srinivasan (1990). "Conjoint Analysis in Marketing: New Developments with Implications for Research and Practice." Journal of Marketing 54(4): 3-19. Hayes, R. H. and D. A. Garvin (1982). "Managing As If Tomorrow Mattered." Harvard Business Review 60(3): 70-79. Hess, J. (1995). "Fuzzy logic and medical device technology." Medical Device Technology 6(8), October: 37-46 Hull, J. (1996). "Options, futures, and other derivatives", Prentice Hall, Upper Saddle River, NJ. Hummel, J., W. van Rossum, et al. (2000). "Medical technology assessment: the use of the analytic hierarchy process as a tool for multidisciplinary evaluation of medical devices." The International journal of artificial organs. 23(11): 782-787. Jackson, P. (1999). "Introduction to expert systems", Addison-Wesley, Harlow. Jensen, K. and P. Warren (2001). "The use of options theory to value research in the service sector." R & D Management 31(2): 173-180. Johnson, R. M. (1987). "Adaptive conjoint analysis". Sawtooth Software Conference on Perceptual Mapping, Conjoint Analysis, and Computer Interviewing, Ketchum, ID. Kao, C. and S.-T. Liu (1999). "Competitiveness of manufacturing firms: an application of fuzzy weighted average." Systems, Man and Cybernetics, Part A, IEEE Transactions on 29(6): 661-667. Karbhari, V. M. (1994). "The Analytic Hierarchy Process: A viable decision tool for composite materials?" International Journal of Technology Management 9(1): 7793. Kellogg, D., J. M. Charnes, et al. (1999). "Valuation of a Biotechnology Firm: An application of real-options methodologies". 3rd Annual International Conference Real Options: Theory meets Practice, Wassenaar/Leiden, The Netherlands. Klir, G. J. and T. A. Folger (1988). "Fuzzy Sets, Uncertainty and Information", Prentice Hall, New Jersey. Lee, D. H. and D. Park (1997). "An efficient algorithm for fuzzy weighted average." Fuzzy Sets and Systems 87(1): 39-45. Liang, G. and M. Wang (1991). "Fuzzy multi-criteria decision-making method for facility site selection." International Journal of Production Research 29(11): 2313-2130. Liberatore, M. J. and A. C. Stylianou (1995). "Expert support systems for new product development decision making: a modeling framework and applications." Management Science 41(8): 1296-1316. Liebowitz, J. (1995). "Expert systems: A short introduction." Engineering Fracture Mechanics 50(5-6): 601-607. Lin, C.-T. and C.-T. Chen (2004). "A fuzzy-logic-based approach for new product Go/NoGo decision at the front end." Systems, Man and Cybernetics, Part A, IEEE Transactions on 34(1): 132-142. Lint, O. and E. Pennings (2001). "An option approach to the new product development process: A case study at Philips Electronics." R & D Management 31(2): 163172. Liou, T.-S. and M.-J. J. Wang (1992). "Fuzzy weighted average: An improved algorithm." Fuzzy Sets and Systems 49(3): 307-315. Machacha, L. L. and P. Bhattacharya (2000). "A fuzzy-logic-based approach to project selection." Engineering Management, IEEE Transactions on 47(1): 65-73. Marakas, G. M. (2003). "Decision support systems in the 21st century", Prentice Hall, Upper Saddle River, N.J. 54
Martin, J. F. (1994). "Fuzzy control in anesthesia." Journal of Clinical Monitoring 10(2): 77-80. Mason, D. G., J. J. Ross, et al. (1997). "Self-learning fuzzy control of atracurium-induced neuromuscular block during surgery." Medical & Biological Engineering and Computing 35(5): 498-503. McCormack, J. and G. Sick (2001). "Valuing PUD reserves: A practical application of real option techniques." Journal of Applied Corporate Finance 13(4): 110-115. Merton, R. C. (1973). "Theory of Rational Option Pricing." The Bell Journal of Economics and Management Science 4(1): 141-183. Millet, I. and P. T. Harker (1990). "Globally effective questioning in the Analytic Hierarchy Process." European Journal of Operational Research 48(1): 88-97. Mills, R. W. (1988a). "Capital budgeting--the state of the art." Long Range Planning 21(4): 76-81. Mills, R. W. (1988b). "Measuring the Use of Capital Budgeting Techniques with the Postal Questionnaire: A UK Perspective." Interfaces 18(5): 81-87. Morris, P. A., E. O. Teisberg, et al. (1991). "When Choosing R&D Projects, Go with Long Shots." Research Technology Management. 34(1): 35-40. Naj, A. K. (1990). "In R&D, the Next Best Thing to a Gut Feeling". Wall Street Journal. New York: A12. Neely III, J. E. and R. de Neufville (2001). "Hybrid real options valuation of risky product development projects." International Journal of Technology, Policy and Management 1(1): 29-46. Nichols, N. A. (1994). "Scientific management at Merck: An interview with CFO Judy Lewent." Harvard Business Review 72(1): 88-98. Partovi, F. Y. and W. E. Hopton (1994). "The analytic hierarchy process as applied to two types of inventory problems." Production and Inventory Management Journal. 35(1): 13-19. Perlitz, M., T. Peske, et al. (1999). "Real options valuation: The new frontier in R&D project evaluation?" R & D Management 29(3): 255-269. Phillis, Y. A. and L. A. Andriantiatsaholiniaina (2001). "Sustainability: an ill-defined concept and its assessment using fuzzy logic." Ecological Economics 37(3): 435456. Pike, R. and J. Sharp (1989). "Trends in the Use of Management Science Techniques in Capital Budgeting." Managerial and Decision Economics 10(2): 135-140. Prabhu, T. and K. Vizayakumar (1996). "Fuzzy hierarchical decision making (FHDM): A methodology for technology choice." International Journal of Computer Applications in Technology 9(5-6): 322-329. Prabhu, T. R. and K. Vizayakumar (2001). "Technology choice using FHDM: a case of iron-making technology." Engineering Management, IEEE Transactions on 48(2): 209-222. Propper, C. (1995). "The disutility of time spent on the United Kingdom's National Health Service waiting lists." The Journal of Human Resources. 30(4): 677-700. Ram, S. and S. Ram (1989). "Expert systems: An emerging technology for selecting new product winners." Journal of Product Innovation Management 6(2): 89-98. Rao, S. S., A. Nahm, et al. (1999). "Artificial intelligence and expert systems applications in new product development--a survey." Journal of Intelligent Manufacturing. 10(3-4): 231-244. Ratcliffe, J. and M. Buxton (1999). "Patients' preferences regarding the process and outcome of life-saving technology: An Application of Conjoint Analysis to Liver 55
Transplantation." International Journal of Technology Assessment in Health Care 15(2): 340-351. Ratcliffe, J., M. Buxton, et al. (2004). "Patients' preferences for characteristics associated with treatments for osteoarthritis." Rheumatology 43(3): 337-345. Remer, D. S. and A. P. Nieto (1995). "A compendium and comparison of 25 project evaluation techniques. Part 1: Net present value and rate of return methods." International Journal of Production Economics 42(1): 79-96. Riquelme, H. and T. Rickards (1992). "Hybrid conjoint analysis: An estimation probe in new venture decisions." Journal of Business Venturing 7(6): 505-518. Rogers, M. J., A. Gupta, et al. (2002). "Real Options Based Analysis of Optimal Pharmaceutical Research and Development Portfolios." Industrial & engineering chemistry research. 41(25): 6607-6620. Ross, S. A., R. W. Westerfield, et al. (1993). "Chapter 21". Corporate finance. Homewood, IL, Irwin: 910. Ryan, M. (1999). "Using conjoint analysis to take account of patient preferences and go beyond health outcomes: an application to in vitro fertilisation." Social Science & Medicine 48(4): 535-546. Ryan, M. and S. Farrar (2000). "Using conjoint analysis to elicit preferences for health care." BMJ 320(7248): 1530-1533. Saaty, T. L. (1980). "The analytic hierarchy process: planning, priority setting, resource allocation", McGraw-Hill,. Saaty, T. L. (2000). "The Fundamentals of Decision Making and Priority Theory with the Analytic Hierarchy Process", RWS Publications, Pittsburgh, PA. San Miguel, F., M. Ryan, et al. (2000). "Applying conjoint analysis in economic evaluations: an application to menorrhagia." Applied Economics, 32(7): 823-833. Schaublin, J., M. Derighetti, et al. (1996). "Fuzzy logic control of mechanical ventilation during anaesthesia." British Journal of Anaesthesia 77(5): 636-641. Schmidt, J. B. and R. J. Calantone (1998). "Are really new product development projects harder to shut down?" Journal of Product Innovation Management 15(2): 111123. Shepherd, D. A., R. Ettenson, et al. (2000). "New venture strategy and profitability: A venture capitalist's assessment." Journal of Business Venturing 15(5-6): 449-467. Shepherd, D. A. and A. Zacharakis (1999). "Conjoint analysis: a new methodological approach for researching the decision policies of venture capitalists." Venture Capital 1(3): 197-217. Sloane, E. B., M. J. Liberatore, et al. (2002). "Clinical engineering technology assessment decision support: a case study using the analytic hierarchy process (AHP)". Engineering in Medicine and Biology, 2002. 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society] EMBS/BMES Conference, 2002. Sugeno, M., Ed. (1985). "Industrial Applications of Fuzzy Control". North-Holland, Amsterdam. Tang, J. C. S. and N. Insuk (1993). "Sector priority and technology choice in the Korean machinery industry." International Journal of Technology Management 8(3-5): 333-341. Teach, E. (2003). "Will Real Options Take Root?" CFO Magazine, July 1st Telser, H. and P. Zweifel (2002). "Measuring willingness-to-pay for risk reduction: an application of conjoint analysis." Health Economics 11(2): 129-139. Trigeorgis, L. (1996). "Real options: Managerial flexibility and strategy in resource allocation", MIT Press, Cambridge, MA. 56
Tsutsui, T. and S. Arita (1994). "Fuzzy-logic control of blood pressure through enflurane anesthesia." Journal of Clinical Monitoring 10(2): 110-117. van der Pol, M. and J. Cairns (1998). "Establishing patient preferences for blood transfusion support: an application of conjoint analysis." Journal of health services research & policy 3(2): 70-76. Vavra, T. G., P. E. Green, et al. (1999). "Evaluating EZPass." Marketing Research 11(2): 4-13. Webb, J. A. C. (2005). S. Johal. Yuan, Y., S. Feldhamer, et al. (2002). "The development and evaluation of a fuzzy logic expert system for renal transplantation assignment: Is this a useful tool?" European Journal of Operational Research 142(1): 152-173. Zadeh, L. A. (1965). "Fuzzy sets." Information and Control 8(3): 338–353.
57
