PRIME Decisions: An Interactive Tool for Value Tree ... - CiteSeerX

PRIME Decisions: An Interactive Tool for Value Tree Analysis Janne Gustafsson, Ahti Salo, and Tommi Gustafsson Helsinki University of Technology, Systems Analysis Laboratory P.O. Box 1100, FIN-02015 HUT, Finland Abstract. Several methods for the processing of incomplete preference information in additive preference models have been proposed. Due to the lack of adequate decision aiding tools, however, only a few case studies have been reported so far. In this paper, we present PRIME Decisions, a decision aiding tool which supports the analysis of incomplete preference information with the PRIME method. PRIME Decisions also offers novel features such as decision rules and guided elicitation tours. The application of PRIME Decisions is illustrated with a case study on the valuation of a high-tech company. Keywords. Decision analysis, interval judgement, incomplete information, linear programming, decision aiding tools.

1. Introduction The development of additive value tree representations has a solid foundation in multiattribute value theory (Keeney and Raiffa 1976). Traditionally, value tree analyses assume that the decision maker (DM) is able to provide complete information about his or her preferences. Yet, since the DM may find it difficult to do so, several methods for the processing of incomplete information have been proposed (see, e.g., Kirkwood and Sarin 1985; Weber 1985; White et al. 1984; Salo and Hämäläinen 1992, 1995; Kim and Han 1999). While these methods differ in their details, they all accommodate incomplete information and help the DM to explore what conclusions can be inferred from such information. The PRIME method supports the elicitation of incomplete information through the use of interval-valued ratio statements (Salo and Hämäläinen 1999; Lindstedt et al. 2000). These statements are translated into linear constraints so that the dominance structures can be determined from a series of linear programming (LP) problems. From the computational viewpoint, PRIME is thus related to other approaches such as HOPIE (Weber 1985), PAIRS (Salo and Hämäläinen 1992), and the recent approaches proposed by Park and Kim (1997) and Kim and Han (1999). In this paper, we describe PRIME Decisions, a decision aiding tool based on the PRIME method. PRIME Decisions also provides novel features which support interactive decision process, including an elicitation tour which guides the DM through the preference elicitation phase. We see PRIME Decisions as an important catalyst for further applied work, as it helps practitioners benefit from the explicit recognition of incomplete information (see, e.g., Hämäläinen and Leikola 1995;

Hämäläinen and Pöyhönen 1996). This tool is downloadable for academic use at http://www.hut.fi/Units/SAL/Downloadables/. The rest of this paper is structured as follows. Section 2 summarizes the PRIME method. Section 3 illustrates the features of PRIME Decisions, and Section 4 presents an application of PRIME to the valuation of a high-tech company. Finally, Section 5 concludes the paper.

2. PRIME – Preference Ratios in Multiattribute Evaluation 2.1 Preference model In PRIME, the DM’s preferences are assumed to have an additive structure so that the overall value of an alternative equals the sum of its attribute-specific scores, i.e., N

V ( x) = ∑ vi ( xi ).

(1)

i =1

Here, N is the number of lowest level attributes (i.e., twig-level attributes) in the value tree, xi is the achievement level of alternative x with regard to the i-th attribute, and vi(xi) is the single-attribute score associated with the achievement xi on the i-th attribute. The most and preferred achievement levels with respect to the i-th attribute are denoted by xi * and xi 0, respectively. Because the representation (1) is unique up to affine positive transformations, the scores of the least preferred achievement levels can be set equal to zero, i.e., vi (xi0) = 0. It is customary to normalize the value function so that 1) the scores of the most preferred single-attribute scores are mapped to one and 2) all scores are multiplied by scaling constants, or attribute weights wi (i = 1,… ,N) which add up to one. When doing this, however, care must be taken to ensure that the attribute weight wi remains proportional to the value difference between xi * and xi 0 (Salo and Hämäläinen 1995, 1997). This can be seen from the equalities N N v ( x ) − vi ( xi0 )  N V ( x) = ∑ vi ( xi ) = ∑ vi ( xi∗ ) − vi ( xi0 )  i ∗i = ∑ wiviN ( xi ), 0  i =1 i =1 vi ( xi ) − vi ( xi )  i =1

[

]

(2)

where wi =vi (xi *)–vi(xi0), i=1,… ,N denote attributes weights and the normalized scores vi N(xi ) are defined by viN ( xi ) =

vi ( xi ) − vi ( xi0 ) vi ( xi ) − vi ( xi0 ) = . vi ( xi∗ ) − vi ( xi0 ) wi

(3)

From the viewpoint of preference elicitation, it is important to note that ratio estimates about value differences in (2) and (3) are sufficient to support conclusions about the DM’s preferences. This observation, coupled with the possibility of providing interval-valued estimates, is the foundation for the preference elicitation in PRIME.

2.2 Preference elicitation In PRIME, preference elicitation consists of two phases, score elicitation and weight elicitation. For each attribute, score information is obtained through 1) identification of the least and most preferred achievement levels xi0 and xi*, 2) ordinal ranking of other achievement levels, and 3) elicitation of possibly interval-valued estimates for ratios of value differences. The ordinal ranking of achievement levels determines which value differences are positive. It also implies a set of linear constraints on the scores: for instance, if the achievement level xi j is preferred to xi k with regard to the i-th attribute, then the inequality vi (xi j ) – vi(xik ) > 0 must hold. After achievement levels have been ranked, further score information can be obtained through interval-valued statements about ratios of value differences. For instance, putting lower and upper bounds [L,U] on the ratio between 1) the value difference from xi0 to the achievement level xi j and 2) the value difference from xi 0 to xi * leads to the inequalities ví ( xí ) − ví ( xí ) ≤U . ∗ 0 ví ( xí ) − ví ( xí ) j

L≤

0

(4)

This interval-valued judgement implies two linear constraints on the scores. It is analogous to direct rating in that the score vi(xi j) is positioned relative to the worst and best achievement levels xi0 to xi*. The PRIME method supports several approaches to the elicitation of attribute weights. The one encouraged by PRIME Decisions is based on the following extension of the SWING method. First, a reference attribute (e.g., the most important one) is selected and one hundred points are assigned to it. The DM is then asked to assign a range of points [L,U] to the other attributes in accordance with the perceived importance of these attributes. For each attribute, this leads to the inequalities L w U ≤ i ≤ ⇔ 100 w ref 100

L vi ( xi∗ ) − vi ( xi0 ) U ≤ ≤ . ∗ 0 100 vref ( xref ) − vref ( x ref ) 100

(5)

2.3 Preference synthesis The results of preference synthesis consist of 1) value intervals for the alternatives, 2) weight intervals for the attributes and 3) dominance structures and decision rules for the comparison of alternatives. These results are obtained as solutions to LP problems where the relevant objective functions are solved subject to the constraints implied by the DM’s judgements. For example, the value interval for alternative x can be computed from the two linear programs

N N   V ( x ) ∈ min ∑ vi ( xi ), max ∑ vi ( xi ) ,  i =1 i =1 

(6)

while the linear programs wi ∈ [min vi(xi*), max vi (xi *)] give bounds for the weight of the i-th attribute. PRIME provides two dominance structures and several decision rules to help the DM in the comparison of alternatives. Specifically, alternative xj is preferred to xk in the sense of absolute dominance if the least possible value of xj is greater than the largest possible value of xk (i.e., the value intervals of the two alternatives do not overlap). It is also possible that the value of xj is greater than that of xk for all feasible scores (even if the value intervals overlap). In this case, the inequalities N max V ( x k ) − V ( x j ) < 0 ⇔ max ∑ vi ( xik ) −  i =1

(

)

N



∑ v ( x )  < 0 j

i =1

i

i

(7)

hold and xj is preferred to xk in the sense of the pairwise dominance criterion. If these inequalities do not hold, the maximum in (7) gives the greatest possible loss of value (PLV) that may be lost by choosing alternative xk instead of xj (i.e., for some feasible parameters values, the value of xj is greater than that of xk). More generally, the greatest possible loss of value that may be forsaken by choosing alternative xk is given by the maximization problem

 N  PLV (k ) = max max ∑ vi ( xij ) − j i=1 j≠ k  

N

∑ v (x i =1

i

k i

  ).  

(8)

PRIME has four decision rules that can be used to obtain a decision recommendation. These rules are helpful in situations where dominance relations do not hold and possibilities of obtaining further preference information are limited. They are 1) maximax (i.e., the alternative with the largest possible value), 2) maximin (i.e., the alternative for which the least possible value is greatest), 3) minimax regret (i.e., the alternative for which the greatest possible loss of value (8) is smallest), and 4) central values (i.e., the alternative for which the midpoint of the value interval (6) is greatest).

3. PRIME Decisions In PRIME Decisions, problem structuring is carried out in the same way as in traditional value tree analysis (see, e.g., Clemen 1996; French 1986). The main differences are in preference elicitation and preference synthesis. These phases are not isolated, but form an iterative process where the steps in preference elicitation are guided by intermediate results from preference synthesis. In this section, we illustrate some features of PRIME Decisions with a problem of choosing one of five computer configurations which differ in terms of their

performance and display, among others. This example is also included in the installation package at http://www.hut.fi/Units/SAL/Downloadables/. Due to space limitations, our coverage of features is not exhaustive. 3.1 Preference elicitation To help the DM make use of different possibilities in preference elicitation, PRIME Decisions has a feature called elicitation tour which guides him or her through a specific sequence of preference elicitation steps. In score elicitation, for example, the DM may choose the direct rating approach where PRIME Decisions asks the DM to position intermediate achievement levels vi (xi ) using intervalvalued ratios (see equation (4)). Alternatively, the DM may compare successive value differences whereby he or she is asked to supply ratio statements for value differences between adjacent achievement levels in the ordinal ranking. Weight elicitation is carried out with the interval-SWING method. The most important attribute (or possibly some other reference attribute) is assigned 100 points. Then, the DM is asked to assign intervals to the other attributes in keeping with their perceived importance. In Figure 1, for example, the most important attribute is price, followed by the computer’s monitor, the weight of which is between 25% and 40% of the weight that is associated with price.

Figure 1: Elicitation of attribute weights

Figure 2: Value intervals

3.2 Preference synthesis Figure 2 shows the alternatives’ value intervals which have been computed from the equation (6). Since the fourth and fifth intervals do not overlap, Octek 350 MHz with a 17” display dominates the same computer with a larger 19” display. On the other hand, the value intervals of Morse 350 MHz and Octek 350 MHz+17” overlap, i.e., the absolute dominance criterion does not make it possible to determine which one of these two alternatives is better. Figure 3 shows the results of the pairwise dominance criterion (cf. equation (7)). Here, a green dot (which is indicated by an arrow) highlights that the value associated with Octek 350 MHz+17” is higher than that of Morse 350 MHz for all

Figure 3: Dominance structures. Arrows indicate green dots.

feasible parameters values; thus, it is safe to conclude that Octek is indeed the better one of the two. This figure also indicates that there are two nondominated alternatives, Octek 350 MHz+17” and Octek 333 MHz, as the columns for these two alternatives are the only ones without arrows. Further support for Octek 350 MHz+17” is provided by the four decision rules listed in section 2.3. Specifically, straightforward computations show that they all suggest that this alternative is indeed the preferred one. The possible loss of value for Octek 350 MHz+17” is 0.07; this is equal to the difference by which the value of Octek 333 MHz would exceed that of Octek 350 MHz+17”, if the analysis were to continue through statements which are most favourable to Octek 333 MHz and least favourable to Octek 350 MHz+17”. 3.3 Computational performance PRIME Decisions does not put any a priori restrictions on the number of attributes, the number of alternatives, or the number of levels in the value tree. However, our experiments suggest that the computation time is roughly proportional to third power of the number of LP problems (Gustafsson 1999a; Gustafsson 1999b). The number of these problems, in turn, depends on the number of attributes (N) and alternatives (K): at most K×(K–1) linear programs are needed to determine the dominance relations while the computation of value and weight intervals involves 2×(K+N) linear programs. Because the computation time may increase rapidly with the number of alternatives, PRIME Decisions is best suited for problems where are relatively few nondominated alternatives. For example, solving a PRIME model with 19 attributes and 5 alternatives led to 491 linear programs and required three minutes of computation time on a Pentium II 350 MHz with 256 MB of RAM. When only the dominance structures, and value intervals were computed, the number of linear programs reduced to 73 and the results were obtained in twenty seconds.

4. Valuation of a high-technology company This section presents a case study in which PRIME Decisions was used to support the valuation of a new technology venture. This application is of interest because it demonstrates that PRIME Decisions can be employed to support judgemental forecasting problems which often share similarities with problems in multicriteria decision analysis (see, e.g., Salo and Bunn, 1995). 4.1 Background In the spring 2000, we employed PRIME Decisions to estimate the market capitalization value of a new technology-based company Sonera SmartTrust

(Gustafsson et al. 2000). This study was carried out in close collaboration with Merita Securities, a Scandinavian investment bank, where it attracted considerable interest in interval-based methods. SmartTrust is a provider of novel security technologies for mobile transactions. It is fully owned by Sonera which is the largest telecommunications operator in Finland with more than 10000 employees and a turnover of about 1.85 billion EUR. The products of SmartTrust are based on public key infrastructure (PKI) which relies on asymmetric encryption techniques to ensure the security of mobile transactions. It is expected that SmartTrust will gain significant sales around 2003. However, the size of these sales and associated profits involve uncertainties which pertain to the future growth of markets for wireless services and the competitive offerings of other enterprises, among others. In order to acknowledge these uncertainties, PRIME Decisions was used for estimating the lower and upper bounds for the size of the PKI market in 2007. This, in turn, provided the foundation for the valuation of SmartTrust. The case study was carried out by a team of four members which consisted of two experts from industry and two experts from the Systems Analysis Laboratory at the Helsinki University of Technology (HUT). The experts from HUT provided the team with expertise in decision analysis, while the industrial experts (an equity analyst at Merita Equities and a mobile business consultant at Omnitele Ltd.) supplied domain knowledge. 4.2 Estimation of market share through value trees A hierarchical problem representation was established to support the analysis. This representation was largely developed by the mobile business consultant (see Figure 4). It was based on the realisation that by interpreting 1) the attribute weights as the relative size of the different market segments and 2) the scores as the percentage of services that require PKI technologies in the respective market segments, it would be possible to produce an aggregated estimate for the overall demand for PKI technologies. However, the experts found considerable difficulty in providing point estimates, because the forecast extended into the year 2007 and involved several uncertainties. The experts therefore felt that interval statements would more adequately capture their perceptions about the emerging PKI technologies. This was also the reason for why PRIME was chosen for this valuation problem. In the first phase, estimates for the size of each market segment (relative to the total market for wireless services) were developed. These estimates were produced by asking the experts 1) to consider the revenues from the different market segments in 2007, and 2) to assign weights to these segments accordingly. That is, at each level of the hierarchical problem representation, the experts first chose a reference segment to which the other market segments would be compared. The reference segment was assigned 100 points, whereafter the experts provided lower and upper bounds to the other segments so that the ratios between the points

would reflect the relative size of the markets. The results of this weighting process are shown in the second and third column of Table 1. The elicitation of scores corresponded to the estimation of PKI market share within each of the nine market segments. The scores were specified as intervals which ranged from 0% to 100%. For example, the experts judged that PKI technologies would be required in about 70% – 90% of applications for wireless services in machine-to-machine communication. The scores are shown in the fourth and fifth columns of Table 1. The experts pointed out that security is more important in some segments than others so that the profit margins might vary between different segments. They also noted that difficulties in considering dynamic interdependencies between the segments would imply quite wide intervals within each segment. After examining the average margins of credit card companies, however, they were confident in assuming an average PKI margin of 2%. This margin was consequently applied to the estimate about the total size of PKI markets which was derived by multiplying the results of PRIME Decisions with the PKI margin and the total size of markets for wireless services, as estimated by leading consultancies. 4.3 Results Finally, the judgemental inputs were synthesized with PRIME Decisions. The total market share of PKI technology in wireless services was estimated to belong to the range [3.5%, 13.4%]. This result was exploited by forming three scenarios: pessimistic (3.5%), neutral (8.5%), and optimistic (13.4%). By applying the net present value (NPV) criterion with a discount rate of 12%, the value of SmartTrust was estimated to be approximately 700 million EUR in the neutral scenario, which was much less than the estimates of Merita (about 17 billion EUR) and Merrill Lynch (about 6 billion EUR). Even in the optimistic scenario, the value of SmartTrust was found to be much below these estimates. It is of interest to note that both banks have subsequently reduced their estimates considerably.

Wireless Services

Infotainment

Information

Entertainment

M-Office

M-Banking

M-Commerce

M-Shopping

Voice / Video

M-Ticketing

UMS

VMS

Data

E-Mail

Figure 4: Classification of wireless services

Machineto-Machine

Table 1: Weights and scores of the PRIME model

Voice/Video Data Infotainment - Information - Entertainment M-Commerce - M-Banking - M-Shopping - M-Ticketing UMS - VMS - E-Mail M-Office / VPN Machine-to-Machine

Relative Service Market Size (SWING Weight) Lower Bound Upper Bound 100 100 10 30 2 5 60 140 100 100 2 5 40 120 140 180 100 100 4 20 5 20 100 100 2 8 1 3

Proportion requiring PKI % (Score) Lower Bound Upper Bound 0 0 3 15

100 70

0 0

5 0

100 95 95

100 100 100

0 0

0 4 100 90

5. Conclusions Although methods for the analysis of incomplete information in value trees have attracted plenty of theoretical interest, the application of these methods has been partly delayed by the lack of appropriate software. In this paper, we have presented a decision aiding tool called PRIME Decisions which supports the application of PRIME and related approaches to the processing of incomplete information. We have also described a case study which was recently carried out in close collaboration with a major investment bank in Finland. The experiences from this case study suggest that PRIME Decisions has potential not only in problems of traditional multicriteria analysis (i.e., comparison of well-defined alternatives), but also in forecasting applications where incomplete information is used to capture the uncertainties that pertain to emerging technologies. Early experiences with PRIME Decisions suggest several avenues for future work. At the level of tool development, PRIME Decisions can be enhanced by providing further support for maintaining the consistency of the DM’s preference statements and for the specification of continuous value functions. At the level of empirical work, there is a need for further case studies and comparative research on the behavioural issues raised by the explicit management of uncertainties.

6. References Clemen, R. T. (1986): Making Hard Decisions – An Introduction to Decision Analysis, 2nd Edition, Duxbury Press, Pacific Grove.

French, S. (1986): Decision Theory: An Introduction to the Mathematics of Rationality, Ellis Horwood Limited, Chichester. Gustafsson, J. (1999a): PRIME – An Introduction and Assessment, Systems Analysis Laboratory, Helsinki University of Technology, unpublished manuscript. Gustafsson, T. (1999b): A Comparison of PRIME Decisions with Other Tools for Decision Analysis, Systems Analysis Laboratory, Helsinki University of Technology, unpublished manuscript. Gustafsson, J., Lassila, J., Paloranta, M. and Uskola, J. (2000): PKI Security in Mobile Business – Case: Sonera SmartTrust, Systems Analysis Laboratory, Helsinki University of Technology, unpublished manuscript. Hämäläinen, R.P. and Leikola, O. (1995): Spontaneous Decision Conferencing with TopLevel Politicians - Discovering the Hidden Demand for Support in Political Decision Making, OR Insight 9, 24-28. Hämäläinen, R.P. and Pöyhönen, M. (1996): On-Line Group Decision Support by Preference Programming in Traffic Planning, Group Decision and Negotiation 5, 485500. Keeney, R. L. and Raiffa, H. (1976): Decisions with Multiple Objectives: Preferences and Value Tradeoffs, John Wiley & Sons, New York. Kim, S.H. and Han, C.H. (1999): An Interactive Procedure for Multi-attribute Group Decision Making with Incomplete Information, Computers & Operations Research 26, 755-772. Kirkwood, C.W. and Sarin, R.K. (1985): Ranking with Partial Information: A Method and an Application, Operations Research 33, 38-48. Lindstedt, M.R.K., Hämäläinen, R.P., Mustajoki, J. (2000): Using Intervals for Global Sensitivity Analyses in Multiattribute Value Trees, Proceedings of the 15th International Conference on Multiple Criteria Decision Making Park, K.S. and Kim, S.H. (1997): Tools for Interactive Multiattribute Decisionmaking with Incompletely Identified Information, European Journal of Operational Research 98, 111123. Salo, A.A. and Hämäläinen, R.P. (1992): Preference Assessment by Imprecise Ratio Statements, Operations Research 40, 1053-1061. Salo, A.A. and Bunn, D.W. (1995): Decomposition in the Assessment of Judgemental Probability Forecasts, Technological Forecasting and Social Change 49, 13-25. Salo, A.A. and Hämäläinen, R.P. (1995): Preference Programming through Approximate Ratio Comparisons, European Journal of Operational Research 82, 458-475 Salo, A.A. and Hämäläinen, R.P. (1997): On the Measurement of Preferences in the Analytic Hierarchy Process, Journal of Multi-Criteria Decision Analysis 6(6), 309-319 Salo, A.A., Hämäläinen, R.P. (1999): PRIME – Preference Ratios In Multiattribute Evaluation, Systems Analysis Laboratory, Helsinki University of Technology. Weber, M. (1985): A Method of Multiattribute Decision Making with Incomplete Information, Management Science 34, 431-445. White III, C.C., Sage, A.P. and Dozono, S. (1984): A Model of Multiattribute Decision Making and Trade-Off Determination Under Uncertainty, IEEE Transactions on Systems, Man, and Cybernetics 14(2), 223-229.