Experiments in multiple criteria selection problems ...

Int. J. Operational Research, Vol. 7, No. 4, 2010

Experiments in multiple criteria selection problems with multiple decision makers Maria Angelica Velazquez*, David Claudio and A. Ravi Ravindran Industrial and Manufacturing Engineering Department, The Pennsylvania State University, 310 Leonhard Building, University Park, PA 16802, USA E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] *Corresponding author Abstract: Multiple criteria selection problems deal with ranking alternatives under conflicting criteria. In the resolution of such problems, scaling methods are used to standardise the data and weighting methods are used to assign a preference to each criterion based on the decision maker’s assessment. Several methods have been proposed for weighting and scaling but the most favourable combinations of methods are uncertain. This study presents the results of experiments in which data from human decision makers is used to determine the best combination of weighting–scaling methods for single and multiple decision makers using the weighted sum decision making model. Keywords: AHP; analytic hierarchy process; ideal values; linear normalisation; multiple-criteria selection problems; multi-person decision making; operations research; ranking; rating; scaling methods; vector scaling; weighting methods. Reference to this paper should be made as follows: Velazquez, M.A., Claudio, D. and Ravindran, A.R. (2010) ‘Experiments in multiple criteria selection problems with multiple decision makers’, Int. J. Operational Research, Vol. 7, No. 4, pp.413–428. Biographical notes: Maria Angelica Velazquez is a PhD candidate in the Industrial and Manufacturing Engineering Department at the Pennsylvania State University. She obtained her Master of Science from Rensselaer Polytechnic Institute in Troy, New York. She earned a Bachelors degree in Industrial Engineering at the University of Puerto Rico, Mayaguez Campus. She is a Professional Engineer and has industrial experiences in various areas within the Industrial Engineering field, ranging from quality assurance to process and human factors engineering. Her research interests are related to human–computer interaction and cognitive ergonomics, specifically in the integration of usability assessments in engineering design. She is a Member of the Institute of Industrial Engineers. David Claudio is a PhD candidate in the Department of Industrial Engineering at the Pennsylvania State University. He has an MS in Industrial and Management Engineering from Rensselaer Polytechnic Institute located in Troy, NY. He is a Professional Engineer and is also certified in Production and Inventory Management (CPIM) from the Association for Operations Management (also known as APICS). He is also a Student Member of the Copyright © 2010 Inderscience Enterprises Ltd.

413

414

M.A. Velazquez, D. Claudio and A.R. Ravindran Institute of Industrial Engineers. His interests include human factors, service systems, decision making and engineering design. Dr Ravindran is a Professor and the past Department Head of Industrial and Manufacturing Engineering at Pennsylvania State University. Formerly, he was a Faculty Member in the School of Industrial Engineering at Purdue University for 13 years and at the University of Oklahoma for 15 years. He holds a BS in Electrical Engineering with honours from India. His graduate degrees are from the University of California, Berkeley where he received an MS and a PhD in Industrial Engineering and Operations Research. His areas of specialisation are operations research with research interests in multiple criteria decision making, financial engineering, health planning and supply chain optimisation. He has published two major text books (Operations Research: Principles and Practice and Engineering Optimisation: Methods and Applications) and over 100 journal articles in operations research. He has edited a new Handbook on Operations Research and Management Science that was published in 2008.

1

Introduction

The resolution of multiple criteria selection problems (MCSP) involves the consideration of multiple and conflicting criteria and the ranking of the alternatives available based on a set of goals. The number of areas in which MCSP are present is increasing every year, and they vary from asset acquisition to energy planning, global warming, fishery planning and other decisions involving resource management (Osman, et al., 2005; Olson, 2008). Multiple criteria analysis serves as a method for decision support to allow the evaluation of tradeoffs among alternatives. In most cases, the resolution of MCSP entails the determination of the importance of each criterion by allocating weights to the criteria based on the decision maker preferences. In addition, the diverse nature of the criteria may generate a multi-dimensional MCSP with the criteria expressed in different units, requiring the use of a numerical scaling system to allow comparisons among criteria with different dimensions. The weighted sum model (also known as the scoring model) is the earliest and probably the most widely used method for solving the MCSP (Triantaphyllou, 2000; Masud and Ravindran, 2008). Consider an MCSP with m alternatives and P criteria. Let the matrix F { fij } ( mxP )

represent the criteria values to be maximised where fij is the value of criterion j for alternative i. If wj is the normalised weight of criterion j assessed subjectively such that wj t 0 and ¦ w j 1 , then the weighted sum model computes a score Si for each

alternative i, as follows: P

Si

¦w

j f ij ;

i 1,! , m.

(1)

j 1

The alternatives are then ranked based on their scores. This model is based on the additive utility assumption (Triantaphyllou, 2000). For the single-dimensional cases the data can be directly used in the model. For multidimensional cases of MCSP, the model requires the use of scaling methods to preserve

Experiments in MCSP with multiple decision makers

415

the assumption of additive utility underlying the method. Consequently, the effectiveness of the weighted sum model depends on the determination of subjective weights (wj) and the scaling of criteria values (fij). Given that the weighted sum model is the most common method for the resolution of MCSP, the use of weighting along with scaling methods is often required to find efficient solutions to such problems. Various methodologies have been proposed for scaling and weighting procedures. Both types of procedures have been analysed independently and their performance and efficiency have been studied (Nutt, 1980; Powdrell, 2003). However, uncertainty exits with respect to which weighting–scaling combination of methods yield the most accurate results as they are concurrently used in the weighted sum model, with respect to the true choice made by the individuals. This research provides insights for the selection of the appropriate combination of scaling and weighting methods to maximise the accuracy of the results with respect to the real preference of the decision maker. It provides empirical evidence of the interaction between weighting and scaling methods and its effect on the solutions provided. The scaling methods used in this study are ideal values, linear normalisation and vector scaling (L1 Norm, L2 Norm, L3 Norm and L Norm). These scaling methods were paired with the weighting methods that include rating, ranking, analytic hierarchy process (AHP), L1 Metric, L2 Metric, L3 Metric and L Metric when applicable. This set of scaling and weighting methods is the most commonly used in MCSP (Masud and Ravindran, 2008). The results obtained from the weighted sum model using Equation (1) were compared with the actual choices of the decision makers by implementing the accuracy index and performance index metrics. The best weighting–scaling combination of methods was defined as the one that yields more consistent answers that were similar to the decision makers’ preferences as well as the method combination that consistently appear for group decision making. Although multiple criteria decision making (MCDM) methods are applied to find solutions for a broad range of problems, the effect of the choice of weighting and scaling methods is usually ignored or disregarded, as most of the times the efforts are concentrated in the formulation and solution of the problems. In general, the selection of methods is based on computational efficiency and previous familiarity with the methods. The interaction effects between weighting and scaling methods is not analysed in the selection process. The results of this study should support the selection of weighting–scaling combination methods in order to maximise the accuracy while guaranteeing the reliability and robustness of the results for either single or group decision making. This, in turn, would provide information about the performance of the weighted sum model when diverse scaling methods are used in combination with different weighting methods. This research suggests that different combinations of weighting and scaling methods may yield different solutions to the same problem, which vary considerably with respect to the preference of the decision maker. Therefore, it is important to understand the effect of these combinations of methods. This article is organised as follows. Section 2 contains a brief literature review on the importance of assigning weights and scaling the data in MCSP. Section 3 explains the method used to conduct the experiments. Section 4 discusses the findings from the experiments. Section 5 gives a summary and extensions of the research.

416

2

M.A. Velazquez, D. Claudio and A.R. Ravindran

Literature review

Decision makers often deal with problems that involve multiple, usually conflicting, criteria (Yoon and Hwang, 1995; Figueira, Greco and Ehrgo, 2005; Masud and Ravindran, 2008). MCDM can be broadly classified as ‘selection problems’ or ‘mathematical programming problems’. The focus of multiple criteria selection problems (MCSP) is a selection of the best or preferred alternative(s) from a finite set of predetermined alternatives. Each alternative corresponds to a sure outcome, assumed to be known (Zeleny, 1984). Another way of classifying MCDM is according to the number of decision makers involved in the process. Hence, there are single MCDM methods and group MCDM methods (Triantaphyllou, 2000). Often not all criteria are considered to be equally important. For this reason, subjective weights are assigned to each criterion. These weights state the importance of one criterion with respect to the others. Assigning weights serve the purpose of accounting for changes in the range of variation for each evaluation measure and to differentiate degrees of importance being attached to these ranges of variation (Kirkwood, 1997). One method of obtaining the weights is the rating method in which the decision maker is asked to rate from 1 to 10 each criterion where 1 is least important and 10 is most important. A better method of obtaining weights is the ranking method, in which pairwise comparisons of criteria preferences are obtained using Borda Count. A third method of acquiring the weights is by using the AHP. AHP is used to derive priorities based on sets of pairwise comparisons (Saaty, 1980). It assigns a magnitude of how much one criterion is preferred over another. Basically, it uses a standard scale of numbers from 1 meaning no preference up to 9, meaning that one criterion is extremely preferred over another. The judgments are used in deriving ratio scale priorities for the decision criteria and alternatives (Dyer and Forman, 1992). AHP is one of the most popular and powerful techniques for decision making in use today. Vaidya and Kumar (2006) completed a review of 150 AHP applications which can be categorised as: selection, evaluation, benefit–cost analysis, allocation, planning and development, priority and ranking, decision making, forecasting, medicine and related fields and quality function deployment (QFD). Group decision making using AHP has been studied by several authors (Beynon, 2005; Retchless, Golden and Wasil, 2007). There are many ways of comparing the different alternatives. The ‘weighted sum model’ is the most widely used MCSP method to compare alternatives. This model is based on the assumption that the total value of each alternative is equal to the sum of the products (Zionts, 1980; Triantaphyllou, 2000). Triantaphyllou (2000) argues that there is one difficulty with this method, when it is applied to multi-dimensional MCDM problems. He states that “in combining different dimensions, and consequently different units, the additive utility assumption is violated and the result is equivalent to ‘adding apples and oranges’” (Triantaphyllou, 2000). Yoon and Hwang (1995) propose a way of solving this problem by using a common numerical scaling system, like normalisation, to permit addition among criteria values. After normalising the original data, the total score for each alternative can be computed by multiplying the comparable rating for each criteria by the importance weight assigned to the criteria and then summing these products over all the criteria (Yoon and Hwang, 1995). There are several methods of normalising or scaling the data which includes the use of ideal values, linear normalisation and vector scaling (Masud and Ravindran, 2008).


417

Ideal values represent the best criteria values possible among the alternatives. Linear normalisation uses the maximum and minimum criteria values to scale the data. Vector scaling uses Lp Norm to scale the data. Lp Norms measure the lengths of vectors and Lp metrics measure the distance between points (Steuer, 1986). Usually, after normalising the data with the vector scaling method, one can use the Lp metric in order to obtain the best alternative. The Lp metric between two vectors x, y, where x, y Rn is given by: Lp

ª n º « | x j y j |p » «¬ j 1 »¼

¦

1/ p

.

(2)

If one of these points is the ideal solution, then the distance represents how close each alternative is to the ideal solution. For this reason, the smaller the distance, the closer an alternative is to the ideal solution. Even though there are so many possible ways of obtaining the weights and scaling the data, there is a lack of published studies on comparing discrete MCDM algorithms (Powdrell, 2003). Khairullah and Zionts (1979) published studies that compared different MCDM algorithms. They compared LINMAP, Zionts’ algorithm and three other modifications of LINMAP. In addition, Powdrell (2003) compared seven ranking algorithms which included: simple rating method, simple ranking method, pairwise comparison of criteria, pairwise comparison of alternatives, LINMAP, AHP and Zionts interactive model. In his study, Powdrell (2003) concluded that the pairwise comparison is the optimal algorithm for weights and that “applying the L1 Norm to several of the ranking algorithms increases the accuracy of the algorithm and help eliminate ties in the rankings”. The various weighting and scaling methods have been studied independently. Most of these studies focus on the efficiency of the methods, the tradeoffs between the quality of the results and computational complexity, and the proposal of modifications for such methods. For instance, Pierre (1987) proposed a method for establishing a scaling array for non-linear functions by generalising an existing approach for scaling and by adding weighting factors and target values. In a separate study, Nutt (1980) studied how different weighting methods affected the magnitude of the weights assigned by decision groups. In this study, direct and indirect criteria weighting methods were compared in terms of the magnitudes of the weights, the variance and the similarity of the decisions and the computational complexity. In this case, the linear rating scale was found to be the most efficient and accurate. Some other studies have focused on the measurement of the propagation of errors in the alternatives due to individual attribute weighting and scaling errors or data imprecision (Yoon, 1989). To our knowledge, the interaction of diverse types of weighting and scaling methods and the effect of such interactions in the weighted sum model have not been previously analysed. Some of the MCSP methods have been analysed and compared as well. A comparative study (Zanakis et al., 1998) assessed the performance of methods, such as ELECTRE, TOPSIS, multiplicative exponential weighting among others by using a simulation for the problem parameters. The results indicated that different results may be obtained by using different MCSP methods to solve the same problem. Although the difference among MCSP methods is out of this research scope, it is appropriate to acknowledge the general awareness about the effects of using different methods. Nevertheless, these types of comparative studies for MCSP methods are not very common, as the choice of method occurs automatically in most of the cases.

418


Ozernoy (1992) proposed a systematic approach for the selection of the best MCSP method as a function of the problem attributes. He justified such an approach by pointing out that in most cases the method selection process is not scientifically supported or tested for accuracy, and the possible methods are not compared. He concluded that “the analyst usually selects a method developed by him/her, a method the analyst has the most faith in or a method the analyst is familiar with and has used before” (Ozernoy, 1992). We believe that the same phenomena is observed when choosing the weighting and scaling method. When solving MCSP, the focus is usually given to the problem formulation, resolution and interpretation of results. In many instances, the process of choosing an appropriate weighting and scaling methods occurs automatically and without further analysis on the interaction of such methods and its effect on the results. Although there is some research on the accuracy and efficiency of weighting and scaling methods, there is no information about the combinations of scaling–weighting method that will maximise accuracy with respect to the real preferences of the decision and the effects of the interaction of different weighting and scaling methods in the results. Therefore, there is a need for information on the performance of weighting–scaling combinations of methods. The research presented in this article explores the interaction of weighting and scaling methods when they are used in the weighted sum model. The information presented would provide guidance on the selection of such methods and awareness of the possible effects of combining diverse weighting and scaling methods in the accuracy of the results.

3

Methodology

A total of six subjects, three males and three females, all within the age range of 24–27 years were selected for the study. The subjects were provided with six multi-criteria problems: buying a house, buying a SUV, selecting a job, selecting a restaurant, buying air tickets and buying a laptop computer. Each problem had five alternatives and each alternative had six criteria. Appendix A contains the MCSP presented to the subjects. For instance, when buying an SUV, the subject’s criteria were: cost, miles per gallon, horsepower, frontal impact resistance, rollover resistance rating and manufacturer. The alternatives for this example were: Nissan Pathfinder, Jeep Grand Cherokee, Ford Explorer, Toyota 4-Runner and Hyundai Tucson. The subjects were asked to provide a rating for each criterion based on the individual relevance of the criteria. For example, a typical question was: from 1 to 10 where 1 is not important and 10 is most important, how important is the cost when buying a SUV? The weights for each criterion for each problem were calculated by dividing the rating assigned to criterion over the sum of the ratings for all criteria of that problem, as presented by Equation (3). wj

Vj P

.

(3)

¦V j j 1

where Vj is the rating assigned to criterion j. After the rating exercise, each subject was asked to rank the criteria according to the pairwise comparison method for each problem.


419

An example of a pairwise comparison question is: is cost more important than miles per gallon? With this method a matrix, denoted by A, was calculated as A = [aij] where aij = 1 if i is preferred to j, 0 if j is preferred to i and 1 if indifferent. Subsequently, the weight for each criterion was calculated as the row sum divided by the sum of all the row sums. Finally, the subjects performed pairwise comparisons by providing a numerical value for the degree in which one criterion was more important than the other. This last comparison was used as input for the AHP estimations. According to AHP, the following scale was used: 1 = equal importance, 3 = somewhat important, 5 = very important, 7 = strongly important, 9 = extremely important. A consistency check was done on every problem for each subject. The consistency ratio was compared with 0.1, meaning that if the consistency ratio was higher than 0.1, the matrix was catalogued as inconsistent. In addition to the questions asked for the rating, and pairwise comparisons for ranking and AHP, in some cases, it was needed to ask the subjects about preferences between qualitative values of some alternatives in order to convert them into quantitative values. For example, in the example of buying an SUV, the manufacturer is one of the criteria. Since, different people have different preferences; each subject specified which manufacturer among Nissan, Jeep, Ford, Toyota and Hyundai they liked better. This allows the customisation of each solution for each subject. After obtaining the feedback for the weights, the subjects were asked to provide their preferred alternative for each problem. The responses of the subjects were used as the input data for each weighting method to estimate the weights of each criterion for each problem. The selected alternatives for each problem were obtained by applying the weighted sum model to the combinations of scaling and weighting methods shown in Table 1. The weights obtained from the ranking and rating methods were combined with ideal values, linear normalisation, and vector scaling by using the L1 Norm, L2 Norm, L3 Norm and L Norm. Given that the methods of ideal values and linear normalisation were the simplest and most widely used scaling methods, it was decided to use both methods in combination for the weights obtained with AHP. Each Lp Metric was combined with an Lp Norm. More specifically, it was decided to match L1 Metric, L2 Metric, L3 Metric and L Metric with L1 Norm, L2 Norm, L3 Norm and L Norm, respectively. The Lp Metric approach computes the distance of each alternative’s criteria values from the ideal values and ranks the alternatives based on that distance. Table 1

Weighting–scaling combinations used for this study Scaling

Weight

Ideal values

Linear normalisation

Vector scaling – L1 norm



Vector scaling – L norm

Rating

X

X

X

X

X

X

Ranking

X

X

X

X

X

X

AHP

X

X

–

–

–

–

L1 Metric

–

–

X

–

–

–

L2 Metric

–

–

–

X

–

–

L3 Metric

–

–

–

–

X

–

L Metric

–

–

–

–

–

X

420


For multiple decision makers, the weights were computed by combining the individual responses in the rating method. Pairwise comparisons for each individual were used to rank the criteria in order and then use the Borda Count ranking method, counting how many times a criteria was number one, number two, etc. Thus, two set of weights for each problem were obtained. Furthermore, a matrix for AHP for multiple decision makers was calculated by using the geometric mean approach to combine the responses of the individual decision makers. As before, the same scaling procedures were used as shown in Table 1.

4

Analysis and results

The analysis was divided into two parts. The first part focused on the results for individual decision makers and the second part focused on the effect of multiple decision makers.

4.1 Analysis for single decision maker For the first part of the analysis, each subject was treated independently. In order to determine the best combination of methods, the accuracy index was estimated for each combination. To estimate the accuracy index, a score equal to 1 was given to those combinations of methods for which the resulting best alternative equals the preferred alternative of the decision maker. Otherwise, the combination of methods received a score of zero. The scores obtained for all the decision makers and all the problems were aggregated to estimate the accuracy index for each combination of weighting–scaling methods. The accuracy index provides the percentage of times a weighting–scaling combination gave a result that was in concordance with the subject’s opinion. The deviation is a measure of how distant is the preferred alternative chosen by the decision maker from the first place. For example, if a subject chose Nissan as his or her favourite SUV according to all the criteria presented, and a combination method placed Nissan as the second option, then the method received a 1 since it deviated from the first place by one position. The average deviations were taken as the deviation estimates for each combination of methods. Table 2 summarises the accuracy ratio and the deviation factor for each combination of methods under study. Based on both the accuracy ratio and the deviation factor, the best three combinations of weight–scale methods are: ideal values with ranking, vector scaling L3N with ranking and vector scaling LN with ranking. The least favourable combinations of weight–scale methods are: vector scaling L2N with L2 Metric, vector scaling LN with L Metric and vector scaling L3N with L3 Metric. If the objective is to maximise the accuracy of identifying the best alternative, it is better to use any of the top three combinations of methods and avoid the bottom three combinations of methods. The next set of performance measures leaves out the decision makers and compares each combination method with the outcome of the majority of the combinations of methods. It was decided to do this under the assumption that given the relatively high numbers obtained from the accuracies, the alternative that resulted as best or worst in the majority of the methods should be the best or worst alternative for the subject. Another way to look at, it is by imagining that a decision maker does not trust any particular method but whenever he or she needs to make a decision he/she uses all the combination


421

methods. It is reasonable to think that the decision maker will choose the alternative that the majority of the methods recommend. The performance index for first choice evaluates the concordance of the alternative suggested as first by the corresponding combination of methods with the alternative that was selected as first by the majority of the combination of methods. The performance index represents the number of times a method provided as the preferred alternative, the same that the majority of the combinations of methods provided. For the estimation of the performance index, those methods that fall into the majority receive a ‘1’ while those who do not receive a zero. Alternatively, some methods might place the number one option according to the majority in their second position and still make the method a good one. Following this argument, the first and second choice metric was developed. This metric is similar to the previously discussed metric, but now instead of looking at who has the alternative that the majority dictated as number one in their first position, it asks if that alternative is in either the first or second position. Once more, the same calculations are done in order to obtain a number representing how many times the combination either falls on the majority or placed the top alternative as their second choice. Finally, the performance index for the last choice does the same assessment but in this case, the evaluation is based on the less desired alternative. This metric captures the ability of the combination of methods to suggest as the worst alternative the same alternative that was suggested as the last alternative by the majority of the methods. Table 3 summarises the performance index obtained for each combination of methods when evaluating the first, first and second, and last alternatives suggested. Table 2

Accuracy ratio and deviation factor for each weight–scale method combination Accuracy

Deviation

Scaling method

Weighting method

Ideal values

Rating

23

64

17

47

Ranking

27

75

10

28

AHP

24

67

13

36

Rating

22

61

19

53

Linear normalisation Vector scaling – L1N Vector scaling – L2N Vector scaling – L3N Vector scaling – LN

Accuracy

Percentage (%)

Deviation

Percentage (%)

Ranking

22

61

20

56

AHP

24

67

21

58

Rating

24

67

17

47

Ranking

22

61

12

33

L1 metric

24

67

18

50

Rating

25

69

14

39

Ranking

26

72

21

58

L2 metric

21

58

25

69

Rating

25

69

15

42

Ranking

27

75

12

33

L3 metric

17

47

31

86

Rating

25

69

17

47

Ranking

27

75

14

39

L metric

18

50

35

97

422 Table 3

M.A. Velazquez, D. Claudio and A.R. Ravindran Performance index for each combination evaluating the first, first and second, and last alternatives: single decision maker First choice

Scaling method

Weighting method

Ideal values

Rating

Number 26

First and second choice

Last choice

Performance Performance Performance index Number index Number index 72

29

81

30

83

Ranking

28

78

30

83

29

81

AHP

24

67

29

81

20

56

Rating Linear normalisati Ranking on AHP

31

86

33

92

31

86

28

78

33

92

27

75

27

75

31

86

22

61

Vector scaling – L1N

Rating

29

81

30

83

29

81

Ranking

28

78

32

89

26

72

L1 metric

28

78

29

81

29

81


Rating

31

86

31

86

34

94

Ranking

28

78

32

89

29

81

L2 metric

21

58

25

69

24

67

Rating

31

86

31

86

34

94

Ranking

30

83

32

89

30

83

L3 metric

17

47

24

67

23

64

Vector scaling – L3N Vector scaling – LN

Rating

31

86

31

86

31

86

Ranking

32

89

33

92

32

89

L metric

17

47

20

56

20

56

Based on the performance index for first choice, the top five combinations of weight– scale methods are: vector scaling LN with ranking, vector scaling L2N with rating, vector scaling LN with rating, vector scaling L3N with rating and linear normalisation with rating. The least favourable combinations are: vector scaling L2N with L2 Metric, vector scaling L3N with L3 Metric and vector scaling L with L Metric. Similarly, based on the performance index for first and second choices, the best three combinations of weight–scale methods are: vector scaling L with ranking, linear normalisation with rating and linear normalisation with ranking. The last three combinations of weight–scale methods are: vector scaling L2N with L2 Metric, vector scaling L3N with L3 Metric and vector scaling LN with L Metric. If the objective is to maximise the chances that the method used yields the same results of other methods, then it is better to use any of the top three combinations of methods and avoid the bottom three combinations of methods. Finally, based on the performance index for the last choice, the best three combinations of weight–scale methods are: vector scaling LN with ranking, vector scaling L2 Norm with rating, and vector scaling L3 Norm with rating. The bottom three combinations of weight–scale methods are: vector scaling L with L Metric, ideal values and AHP and linear normalisation with AHP. If the objective is to maximise the chances


423

that the method used will keep the worst alternative in the last position, then it is better to use any of the top three combinations of methods, and avoid the entire bottom three combinations of methods mentioned previously.

4.2 Analysis for multiple decision makers For multiple decision makers, the geometric averaging approach was used to calculate the weights. The Borda Count ranking method was used to estimate the rankings. The AHP matrix for multiple decision makers was estimated by combining the individual decision makers responses. As with the single decision maker case, the first, first and second, and last choice performance measures were used to compare each method combination with the majority. Table 4 summarises the performance index obtained for each combination of methods. The performance index for the first alternative resulted in 12 combinations having a performance index of 100%. This means that any of these combinations will be good for a group decision making analysis. The bottom three combinations of weight–scale methods are: vector scaling L3N with L3 Metric, vector scaling LN with L Metric and linear normalisation with AHP. Table 4

Performance index for each combination evaluating the first, first and second, and last alternatives: group decision making First choice

Scaling method

Weighting method Number

Ideal values

Rating

6


Performance index

Number

100

6

Last choice

Performance Performance index Number index 100

6

100

Ranking

6

100

6

100

5

83

AHP

6

100

6

100

5

83

Linear normalisation

Rating

5

83

5

83

6

100

Ranking

4

67

5

83

6

100

AHP

3

50

6

100

5

83


Rating

6

100

6

100

5

83

Ranking

6

100

6

100

5

83

L1 metric

6

100

6

100

5

83


Rating

6

100

6

100

6

100

Ranking

6

100

6

100

5

83

L2 metric

4

67

4

67

3

50


Rating

6

100

6

100

6

100

Ranking

6

100

6

100

5

83

L3 metric

2

33

4

67

3

50

Vector scaling – LN

Rating

6

100

6

100

6

100

Ranking

6

100

6

100

6

100

L metric

2

33

3

50

3

50

424 Table 5

M.A. Velazquez, D. Claudio and A.R. Ravindran Top three and bottom three combinations of methods for the single and multiple decision makers Performance measure for single DM

Top three combinations

Bottom three combinations

Accuracy

First choice


Ideal values + Ranking

Linear Norm + Rating

Vector scaling L3N + Ranking

Performance measure for multiple DMs

Last choice

First choice


Last choice

Linear Norm + Rating

Vector scaling L2N + Rating

–

–

–


Linear Norm. + Ranking


–

–

–

Vector scaling L N + Ranking




12 of the methods tied for 100%



–


–

–

–

–

–

–

Vector scaling L N + Rating

–

–

–

–

–

Vector scaling L2N + L2 Metric



Linear Norm + AHP

Linear Norm + AHP






Ideal Values + AHP




Vector scaling L N + L Metric







The performance index for the first and second alternative had a similar outcome, where a total of 13 methods obtained a performance index of 100%. The bottom three combinations of weight–scale methods are: vector scaling L2N with L2 Metric, vector scaling L3N with L3 Metric and vector scaling LN with L Metric. Similarly, to the previous two cases, the performance index for the last choice resulted in seven combinations of methods that yield a performance index of 100%. The bottom three combinations of weight–scale methods are: vector scaling L2N with L2 Metric, vector scaling L3N with L3 Metric and vector scaling LN with L Metric. Table 5 summarises the results of the top three and bottom three combinations of methods for the single and group decision makers.


425

In summary, the combination that is found among the top three in all performance measures for both single and group decision making is vector scaling LN with ranking. In contrast, vector scaling LN with L Metric can be found in the bottom three in all the performance measures. As can be seen from the results, the weighting method has a big impact in the final decision. In many cases, for both single and group decision making, there were several differences for a scaling method when different weighting methods were used. It was observed that vector scaling L2N resulted among the top choices when it was done in combination with rating. In contrast, when it was done with L2 Metric it was among the bottom three for most performance metrics. The same behaviour was observed for vector scaling LN. It fell in the top choices when combining with ranking but then it was among the worst choices when combined with L Metric. These results suggest that the ‘goodness of a scaling method may depend on the method used to obtain the weights’. In other words, the selecting the best or worst alternative does not depend on in the scaling or weighing method alone, but in their combination.

5

Conclusions

The main contribution of this research is to provide evidence of the goodness of certain combination of methods to rank the alternatives of a typical decision making problem. This research studied the best combination of weight and scaling method for ranking alternatives under conflicting criteria both for single and group decision making, by performing several experiments with different ranking methods on typical real life problems. The best combinations for single decision maker were the same for group or multiple decision makers proving the consistency of the results. Overall, considering both single and group decision making, the best method was vector scaling L Norm with ranking which outperformed all other combinations of methods. The worst method was the combination of vector scaling L Norm with L Metric. The results of this research work can be used as guidelines for the selection of the combination of weighting and scaling methods in real life applications. It can be used to maximise the accuracy of the rankings as well as guarantee the reliability and robustness of the results obtained when selecting the best and the worst alternatives, for either single or group decision making. Future research should focus on building new problems to test if the results obtained are consistent with the ones presented here. Moreover, we only tested 18 possible combinations of weight–scaling methods. Future research could look more in depth into the combinations not studied here. More weight or scaling methods, like LINMAP, can be added to the experiment and see how it performs with respect to the combinations tested here.

Acknowledgements We wish to thank the reviewers for their valuable comments which improved the quality and scope of the article and its significance.

426


References Beynon, M.J. (2005) ‘A method of aggregation in DS/AHP for group decision making with nonequivalent importance of individuals in the group’, Computers and Operations Research, Vol. 32, pp.1881–1896. Dyer, R.F. and Forman, E.H. (1992) ‘Group decision support with the analytic hierarchy process’, Decision Support Systems, Vol. 8, pp.99–124. Figueira, J.S., Greco, S. and Ehrgo U.M. (Eds) (2005) Multiple Criteria Decision Analysis; State of the Art Surveys. New York, NY: Springer. Khairullah, Z. and Zionts, S. (1979) ‘An experiment with some approaches for solving problems with multiple criteria decision making’, in G. Fandel and T. Gal (Eds), Multi Criteria Decision Making, Springer, Verlag. Kirkwood, C.W. (1997) Strategic Decision Making: Multiobjective Decision Analysis with Spreadsheets. Belmont, CA: Wadsworth Publishing Company. Masud, A.S. and Ravindran, A.R. (2008) ‘Multiple criteria decision making’, in A.R. Ravindran (Ed.), Operations Research and Management Science Handbook, Boca Raton, FL: CRC Press. Nutt, P.C. (1980) ‘Comparing methods for weighting decision criteria’, The Int. J. Management Science, Vol. 8, pp.163–172. Olson, D.L (2008) ‘Multi-criteria decision Support’, in F. Bursdtein and C.W. Holsapple (Eds), Handbook on Decision Support Systems. Berlin, Heidelberg: Springer-Verlag. Osman, M., Kangas, S., Abou-El-Enien, T. and Mohamed, S. (2005) ‘Multiple criteria decision making theory, applications and software: a literature review’, Advances in Modeling and Analysis, Vol. B, pp.1–35. Ozernoy, V.M. (1992) ‘Choosing the ‘best’ multiple criteria decision making method’, INFOR, Vol. 30, pp.159–171. Pierre, D.A (1987) ‘An optimal scaling method’, IEEE Transactions, Vol. SMC-17, pp.2–6. Powdrell, B.J. (2003) ‘Comparison of MCDM algorithms for discrete alternatives’, Unpublished MS Thesis, The Pennsylvania State University. Retchless, T., Golden, B. and Wasil, E. (2007) ‘Ranking US army generals of the 20th century: a group decision making application of AHP’, Interfaces, Vol. 37, pp.163–175. Saaty, T.L. (1980) The Analytical Hierarchical Process. New York, NY: John Wiley and Sons. Steuer, R.E. (1986) Multiple criteria optimization: theory, computation, and application. New York, NY: John Wiley and Sons. Triantaphyllou, E. (2000) Multi-criteria Decision Making Methods: A Comparative Study. Dordrecht, The Netherlands: Kluwer Academic Publishers. Vaidya, O.S. and Kumar S. (2006) ‘Analytic hierarchy process: an overview of applications’, European Journal of Operational Research, Vol. 169, pp.1–29. Yoon, K. (1989) ‘The propagation of errors in multiple-attribute decision analysis: a practical approach’, The Journal of the Operational Research Society, Vol. 40, pp.681–686. Yoon, P. and Hwang, C. (1995) Multiple Attribute Decision Making: An Introduction Thousand Oaks, CA: Sage University Press. Zanakis, S.H., Solomon, A., Wishart, N. and Dublish, S. (1998) ‘Multi-attribute decision making: a simulation comparison of selected methods’, European Journal of Operational Research, Vol. 107, pp.507–529. Zeleny, M. (1984) MCDM: Past Decade and Future Trends: A Source Book of Multiple Criteria Decision Making. Greenwich, CT: JAI Press. Zionts, S. (1980) ‘A multiple criteria method for choosing among discrete alternatives’, European Journal of Operational Research, Vol. 7, pp.143–147.


427

Appendix A Problems with their different criteria and alternatives 1.0 Buying a house* *Source from: www.endi.com

Criterias Town where located

Mayaguez

Ponce

San Juan

Cidra

Rio Grande

$ 300,000.00 $ 156,900.00 $ 715,000.00 $ 150,000.00 $ 145,000.00

Price Lot Size (squared meters) Buying a house

Alternative 1 Alternative 2 Alternative 3 Alternative 4 Alternative 5

Life of the house Number of bedrooms Zone located

613.00

297.00

467.00

302.00

331.56

8

26

5

4

20

5

4

4

3

3

City

City

City

Rural

Rural

Nissan Patfinder

Jeep Grand Cherokee

Ford Explorer

Toyota 4runner

Hyundai Tucson

2.0 Buying a SUV* * based on 2006 all brands

Criterias Cost

Buying a SUV

$ 26,250.00

$ 34,635.00 $ 25,999.00 $ 27,635.00 $ 16,995.00

Miles per gallon

21

21

20

20

24

Horse Power

270

210

292

260

173

Frontal impact resistance

4

3

5

4

5

Rollover resistance rating

3

5

3

3

4

Nissan

Jeep

Ford

Toyota

Hyundai

Criterias

Alternative 1

Alternative 2

Alternative 3

Alternative 4

Alternative 5

Salary

$ 60,000.00 $ 65,000.00 $ 45,000.00

Manufacturer

3.0 Selecting a job* *Source from: www.endi.com

Expected working hours/week

Selecting a job

Type of industry Traveling Requirements/year Size of company

40

46

35

Development Manufacturing Government

$ 30,000.00 $ 80,000.00 30

50

Service

Consulting

12

1

2

0

2

Medium

Large

Large

Small

Medium

Good

Good

Excellent

Average

Average

Quality of Benefits (e.g. health insurance, bonus, etc.)

428


4.0 Selecting a restaurant Criterias Type of food Average cost (per person) Quality of Service ( Good, Regular, Poor) Selecting a restaurant

Food Quality (Good, Regular, Poor) Cleaness (Good, Regular, Poor) Bar/ Ammenities (0-2)

Chilis

Macaroni Grill

Outback

Margarita's

Friday's

Mexican

Italian

Steak

Mexican

American

15

16

18

18

13

Regular

Good

Good

Good

Poor

Good

Good

Good

Regular

Regular

Regular

Good

Regular

Good

Good

2

1

1

2

0

5.0 Buying air tickets to Italy Iberia

Criterias Price

$ 1,300.00

Number of stops Departure time Arrival time

Buying air tickets

Air France Continental

Flight duration Extras (movie, food, seats, etc.)

$

890.00

Swiss

Alitalia

$ 1,100.00

$ 950.00

$ 700.00

0

2

1

1

2

9:00 AM

7:00 AM

5:00 PM

8:00 AM

7:00 PM

5:00 PM

7:00 PM

2:00 AM

7:00 PM 11:00 AM

8 hours

12 hours

9 hours

11 hours 16 hours

Excellent

Excellent

Good

Average

Good

6.0 Buying a Windows based Laptop Computer Dell Inspiron E1505

Criterias Price Brand Processor Speed (GHz) Selecting a laptop

Hard Drive Capacity (GB) Burning Capacity Memory (MB)

$

CompaqPresario V6000T series

699.00 $

LenovoThinkPad R60e

609.99 $

HP-Pavilion Sony- VAIO® Notebook N dv6000z SeriesspacerVG series N-N230E/T

804.00 $

579.99 $

979.99

Dell

Compaq

IBM

HP

Sony

2.16

1.6

1.6

1.8

1.73

100

80

80

60

100

DVD+/-RW DVD/CD-RW DVD-ROM DVD+/-R/RW DVD+/-R/RW 512

512

1000

512

1000

Experiments in multiple criteria selection problems ...

Experiments in multiple criteria selection problems ...

Suggest Documents