Discovering Classes of Decision Models: A Step Towards Discovery of Decision Making Styles Dr. NarasimhaBolloju Department of Information Systems City University of Hong Kong, HONG KONG
[email protected]
Abstract Managers at tactical and operational levels in many organizations frequently encounter similar decision problems. Decisions taken by different managers for a given problem in an organization vary due to differences in their decision making styles and/or subjectivity. Discovering classes or categories of decision makers with similar decision making styles can contribute towards organizational learning through better understanding current decision making patterns and changes in those patterns over long periods of time. In this paper we propose a generic approach for discovering classes of decision models from a large number of decision makers as a step towards discovering decision making styles. This approach for discovering classes of decision makers consists of three phases: decision modeling, unification of individual decision models, and discovering decision model classes. An example of this proposed approach is illustrated using employment preferences of 70 subjects, modeled using analytic hierarchy process, to discover classes of decision makers..
1. Introduction Managers at tactical or operational levels in many organizations frequently encounter similar decision problems. When solutions to such decision problems involve a fair amount of subjectivity, decisions taken by different managers in an organization for a given problem vary due to differences in their decision making styles. Business decision problems such as insurance underwriting, hiring personnel, and loan approval in financial institutions fall under this category. These types of decision problems are fairly recurrent, subjective and usually solved independently by many decision makers.
Many approaches to support decision making in such situations perform some form of aggregation of the judgments of different decision makers rather than considering the differences across decision makers. Discovering classes or categories of decision makers with similar decision making styles can contribute towards the organizational learning through better understanding of current decision making patterns and changes in those patterns over long periods of time. Organizations also require such classification for reasons such as validation of decisions, verification of consistency in decision making, and use in training of new staff. The process of discovery of decision making styles can be performed in many possible ways. For example, if there exists a record of past cases of decisions, it is possible to directly discover the classes of decision models. Alternatively, one may address this problem of discovery by decomposing the task into several phases starting from discovery or modeling individual decision making processes and ending at the classification of decision making styles. In any case, some important issues to be addressed in a specific situation include modeling of the decision problem from the perspective of each decision maker, resolving differences in these models for purpose of generalization and comparison, and classification of the decision models into different categories for the purpose of intra-class generalization or aggregation. In this paper we describe the role of knowledge discovery techniques in addressing some of these issues and propose an approach to discovering classes of decision models. Some of the issues related to discovery of decision models and styles are presented in the next section. In section 3, an overview of the proposed approach to discovering classes of decision models to represent different decision making styles. This approach is illustrated in section 4 using employment preferences elicited from 70 subjects to discover decision model classes. The employment
1060-3425/98 $10.00 (c) 1998 IEEE
preferences of these subjects are modeled using the analytic hierarchy process, AHP (Saaty, 1980). In the last section, directions for further research to exploit the developments in the field of knowledge discovery for this purpose are presented.
2. Issues in Discovering Decision Models and Styles Researchers in the field of decision support systems (DSS) often address either individual decision making or group decision making type of problems at various organizational levels. Many decision problems of the type described in Section 1 are often considered as special cases of individual decision making by aggregating decision making processes of individual decision makers. O’Leary (1993) suggests verifying whether the experts (or decision makers) have similar views as a precondition to aggregating individual judgments. He cautions that if the experts do not have similar views, it is meaningless to aggregate individual judgments. In this paper, we consider subjective type of decision problems where it is not appropriate to aggregate individual models because such an approach neither may represent any decision maker nor can satisfy majority of the decision makers. Elicitation is the process of extracting and representing problem-solving knowledge from decision makers or experts. While structured problems are amenable to quantitative techniques, many unstructured or semistructured decision problems are solved by qualitative means (Bolloju, 1996). Decision makers apply intuition and experience in solving such problems, and they cannot always easily articulate their reasoning processes behind taking such decisions. Many difficulties associated with elicitation or knowledge acquisition are addressed in the field of expert systems. Automated techniques such as machine learning, neural networks, statistical techniques, case-based reasoning and other hybrid techniques can be useful in addressing some of these difficulties (e.g., Chan and Wong, 1991; Billman and Courtney, 1993; Markham and Ragsdale, 1995; Mechitov et al., 1995). The process of elicitation of problem-solving knowledge from a number of decision makers gets even more difficult due to differences individual decision making styles and subjectivity. An immediate problem in this process is the resolution of semantic and structural differences in various aspects of the decision models of different decision makers. The complexity of this task
depends upon the structure of the decision model and the process used in elicitation. For example, subjective decision problems with open-ended factors or attributes are difficult to resolve because different decision makers may have used an attribute to mean different things or used different attributes to mean the same thing. The next major issue is related to the classification and aggregation of decision models representing different types of decision making processes. Any approach to this issue should deal with inconsistencies, conflicts, and decision makers’ subjectivity. Classification problem solving techniques cannot be used directly for this purpose because of lack of structure and sparseness in decision models and possible differences in the problem-solving process. Decision rules used by different decision makers, for example, may refer to different attributes and different intermediate conclusions or decisions. It may be possible to combine with relative ease only the non-contradictory rules in such situations. Many business decision making situations rarely exhibit a structure that is amenable to simple aggregation techniques. We can make use of developments in the field of knowledge discovery to address issues related to the discovery of decision models and classification of decision models. However, we need a well-defined approach for effective application and for directing further research.
3. An Approach to Discovery of Decision Model Classes We propose an approach to discovery of decision model classes which we consider as a step towards the farther goal of discovering decision making styles. Figure 1 illustrates the proposed approach to discovery of classes of decision makers as consisting of three phases: decision modeling, unification of individual decision models, and discovering classes of decision models. In the remaining part of this section we describe the role of knowledge discovery techniques in the first and last phases of this approach. Unification phase of this approach may not be directly supported by the current knowledge discovery techniques. It is possible to employ case based reasoning and developments in schema integration and database interoperability (see Batini et al. 1986; Litwin et al.1990) in the field of database management for this purpose.
1060-3425/98 $10.00 (c) 1998 IEEE
Past Decisions of Decision Makers
Models for Individual Decision Makers
Modeling Individual Decision Making Process
Unified Decision Models
Unifying Different Decision Models
Models Representing Classes of Decision Makers
Classfying Decision Models
Figure 1: An approach to discovering decision maker classes
Role of Knowledge Discovery in Decision modeling: Knowledge discovery techniques are being used for supporting the decision making process in difficult tasks (Widrow et al., 1994). Common application of these techniques is generally limited to discovery of interesting patterns for supporting the problem identification phase of the decision making process. Although the utility of knowledge discovery techniques beyond this form of support to assistance in decision modeling has been identified (Hill & Remus, 1994), there is little research on the effectiveness of such application. Bolloju (1997) presents the results of a quasi-experiment to measure the effectiveness of a neuro-fuzzy classifier in automated elicitation of classification problem-solving knowledge. The results of this experiment favor the use of such techniques for modeling classification decision problems. However, it should be noted that typical situations for discovery of decision models differ significantly from discovery of interesting patterns in databases with respect to the number of cases, inconsistencies, structure and the representation (e.g., mathematical, rule-based). Role of Knowledge Discovery in Classifying Decision Making Styles: Approaches such as statistical techniques, knowledge based techniques, heuristic classification (Clancey, 1985), machine learning and artificial neural networks are commonly used for solving classification problems. Piramuthu et al. (1994) present a brief review of statistical, inductive learning and neural network classifiers with pointers to a large number of comparative studies. Mechitov et al. (1994) discuss various problems associated with decision rule elicitation and the requirement of assistance to deal with the problems of
inconsistencies and incompleteness in the decision rules. Many of these techniques require known classifications for learning or discovery purposes. However, in the case of classifying decision making styles, the purpose is to discover classes rather than using some known classifications for the purpose of classifying the unknown objects. Techniques such as cluster analysis, data envelopment analysis and discriminant analysis can be used for discovering classes based on similarities in the unified decision models. Intra-Class Aggregation: Once the unified decision models are classified, it is required to aggregate various factors within each class. Some useful aggregation techniques for this purpose include conceptual aggregation based on conceptual clustering and case-based learning for real-time (dynamic) decision making (Chaturvedi et al., 1993), a flexible modeling approach based on Bayesian analysis for aggregation of point estimates (Clemen and Winkler, 1993), and aggregation of preference patterns using social choice framework (Dubois and Koning, 1994)
4. Discovery of Decision Model Classes: An Example The generic approach presented in the previous section can be operationalized in different ways. Different tasks in the three phases may be realized through application of different techniques and methods. Tasks that require human participation can be performed by DSS developers or decision makers themselves. Alternatively, one may employ automated modeling techniques to discover models using past decisions (e.g., Billman and Courtney,
1060-3425/98 $10.00 (c) 1998 IEEE
1993). Tasks related to knowledge discovery can be realized either in form human-assisted computer discovery
Intuitive Rankings
AHP Models Representing Individual Preferences
Modeling Individual Preferences Using AHP
or computer-assisted human discovery (Uthurusamy, 1996).
Unified Individual AHP Models
Unifying Different Decision Models Using Content Analysis
AHP Models Representing Classes of Preferences
Classfying Decision Models Using Cluster Analysis
Figure 2: Discovering classes of preference models In this section, we present an operationalization of the proposed approach as shown in Figure 2. In the first phase, we can use the AHP technique for elicitation of preferences from the decision makers in ranking decision alternatives. In the second phase, the AHP models representing individual preferences can be analyzed and a unified set of factors can be identified. As part of the unification, differences among individual models (e.g., factor names and meanings, factor groups) will be resolved. These unified decision models can then be classified into different classes using cluster analysis to define possible classes of decision models. An experiment was designed to investigate the feasibility of the proposed approach with the above described operationalization. A group of 70 undergraduate students were used as decision makers for ranking employment offers. Each subject had identified a
set of five typical employment offers he/she would consider acceptable. Each of those offers were described by the subject giving appropriate values to attributes or factors (e.g., Salary, Type of work, Allowances) considered relevant by the subjects. Then, subjects ranked their own sets of employment offers based on their subjective preferences. This decision making task was then modeled separately by each subject as an AHP model using the Expert Choice software tool. The subjects were suggested to use three levels in their AHP modeling (see Figure 3) to reduce the number of pair-wise comparisons required and to minimize inconsistency problems. Each subject compared the ranking produced by his/her AHP model and verified against his/her intuitive ranking. Figure 3 depicts the AHP model of one of the subjects used in the experiment with the factors, associated factor values in parentheses and the utility function.
1060-3425/98 $10.00 (c) 1998 IEEE
JOBRANK (1.0)
PAYPACK (0.351)
PROSPECT (0.109)
PERSONAL (0.351)
WORK (0.189)
SALARY (0.571)
TRAINING (0.163)
WHOURS (0.5)
TYPE (0.510)
ALLOWANC (0.286)
GROWTH (0.297)
DISTANCE (0.5)
PRESTIGE (0.490)
FRINGE (0.143)
PROMOTN (0.540)
Utility Function:
0.200*SALARY + 0.100*ALLOWANC + 0.050*FRINGE + 0.0178*TRANING + 0.032*GROWTH + 0.059*PROMOTN + 0.176*WHOURS + 0.176*DISTANCE + 0.096*TYPE + 0.093*PRESTIGE
Figure 3: A subject’s model of preferences
Table 1: Factor codes identified after content analysis Factor Code Salary Work Load Allowances Development Work Environment Leave Job Satisfaction Traveling Distance Bonus Prestige Job Security Housing Others
Factor Description Basic Salary Expected workload (hours) and need to work late Allowances such as travel, medical, education, etc. Career development, training, promotion prospects, etc. Working environment, location, etc. Annual leave, sick leave, etc. Job satisfaction, nature of work, work relevance, etc. Distance from home to work place or time spent in travel Amount of bonus offered (a multiple of monthly salary) Prestige, reputation, etc. associated with the job Job security or low risk of losing job Housing benefits, housing loan, etc. Retirement benefits, life insurance, etc.
Content Analysis: A first look at the 70 AHP models revealed that a variety of factors (over 100 different factors) were used in their models. Many of these factors had some major/minor variations (e.g., Distance between home and work place, Travel time between home and office) that were to be resolved. Subjects had used about 7 to 8 factors on an average in their models. These factors were categorized (manually by two researchers for better reliability) and were short-listed into 13 factor codes as listed in Table 1. Although no formal reliability tests were performed, the differences in this categorization
Frequency 70 62 61 58 55 42 37 37 28 28 19 17 12
were identified and resolved manually. Table 1 also shows the frequency distributions of usage of these factors. All intermediate nodes in the hierarchies were discarded for this analysis. Utility functions were produced for each subject using these codes and the corresponding weights to represent unified AHP models. Cluster Analysis: The unified AHP models (i.e., utility functions) were then used as input to the cluster analysis using SPSS for Windows (version 6.0) software. The number of classes (K) was varied from 2 to 10 to generate
1060-3425/98 $10.00 (c) 1998 IEEE
different sets of classifications. Classifications reported by the software tool are presented in Table 2 (only the three most important factors based on factor values are listed in this table). Mean distance of individual utility functions
from the cluster centroid of each class were computed for the selection of the appropriate classification (see Table 3). Based on this distance measure, classification of AHP models with 6 classes is selected.
Table 2: Classes discovered using cluster analysis # of Classes K=2 K=3
K=4
K=5
K=6
K=7
Class Number 1 2 1 2 3 1 2 3 4 1 2 3 4 5 1 2 3 4 5 6 1 2 3 4 5 6 7
Class Size 29 41 9 48 13 8 27 13 22 20 20 12 10 8 4 20 11 18 12 5 15 10 11 8 18 4 4
Three Most Significant Factors (Factor Values) Development(0.3), Salary(0.24), Job Satisfaction(0.11) Salary(0.4), Housing (0.09), Allowances (0.09) Development(0.46), Salary(0.17), Prestige(0.09) Salary(0.40), Development(0.1), Allowances(0.09) Job Satisfaction(0.32), Salary(0.19), Development(0.12) Development(0.47), Salary(0.14), Prestige(0.1) Salary(0.44), Allowances(0.11), Work Load(0.1) Job Satisfaction(0.32), Salary(0.19), Development(0.12) Salary(0.35), Development(0.22), Working Environment(0.07) Salary(0.34), Development(0.23), Work Load(0.07) Salary(0.47), Allowance(0.13), Bonus(0.07) Job Satisfaction(0.32), Salary(0.2), Development(0.13) Salary(0.33), Work Load(0.26), Leave(0.08) Development(0.5), Salary(0.12), Job Satisfaction(0.12) Development(0.5), Salary(0.12), Job Satisfaction(0.12) Salary(0.49), Allowances(0.14),Traveling Distance(0.06) Job Satisfaction(0.32), Salary(0.20), Development(0.14) Salary(0.34), Development(0.25), Prestige(0.07) Salary(0.38), Working Environment(0.17), Bonus(0.12) Work Load(0.28), Salary(0.27), Job Satisfaction(0.12) Salary(0.37), Working Environment(0.14), Development(0.10) Salary(0.51), Work Load(0.09), Leave(0.07) Job Satisfaction(0.32), Salary(0.20), Development(0.14) Development(0.31), Salary(0.30), Prestige(0.14) Salary(0.4), Allowance(0.2), Development(0.1) Development(0.5), Job Satisfaction(0.16), Work Load(0.12) Work Load(0.32), Salary(0.23), Job Satisfaction(0.14)
. . .
5. Discussion and Implications An approach to discovery of classes of decision models representing differences in decision making styles is presented in this paper. The results of the experiment designed to validate the proposed approach confirm the utility of this approach. The experiment used the AHP technique for preference elicitation, content analysis for unification of individual AHP models, and cluster analysis for discovering classes of
unified decision models. The mean distance from the cluster centroid is used as the measure for selecting a classification from a set of 9 different classifications. Following the suggestions regarding the iterative approach to the discovery process, it is necessary to extend this study by iterating with a reduced number of factors from 13 to 8 (since only 8 factors were found to be of important in Table 2). Further work is required in this experimental case in identifying appropriate measures and evaluation of the effectiveness of this approach.
1060-3425/98 $10.00 (c) 1998 IEEE
Table 3: Descriptive statistics for distances from cluster centroids n = 70 No. of Classes (K) 2 3 4 5 6 7 8 9 10
Mean .27 .25 .23 .22 .21 .21 .20 .19 .19
The generic approach presented in this paper offers a great potential for both strategic application of knowledge discovery techniques and other research possibilities in the field of decision support systems. Some of the possible research directions include: • Discovery of decision models using example decision cases: Majority of the knowledge discovery and data mining research considers discovery of interesting patterns or rules from large structured databases. Such techniques should be adapted to the type of application addressed in this paper that requires discovery from few decision examples, and large number attributes. Characteristics such as these introduce a number of difficulties in discovery of decision models (Cheeseman and Stutz, 1996). • Representation of decision models: This aspect can play a significant role in unification of decision models. Representations such as the utility function of AHP models lend themselves easily for unification as compared to rule based or other complex representations. Research in this direction of unification schemes or algorithms for different representations from one or more modeling paradigms is useful. • Representation of decision styles: In the experiment presented we have used the same representation mechanism for representing classes of decision models. There is little explicit information regarding the decision making style in such representations. • Classification of decision models: The structure of AHP models used in this paper is amenable to simple classification technique such as cluster analysis. Classification techniques for other forms (and even AHP models considering the complete hierarchy
StdDev .07 .05 .05 .05 .04 .04 .04 .04 .05
Minimum .18305 .15301 .15111 .13685 .11491 .11912 .10069 .10804 .10804
Maximum .51947 .38729 .37447 .38022 .31368 .29520 .30115 .30593 .29760
rather than just the utility function) would be helpful in operationalizing the proposed approach across different modeling paradigms. Research in the above mentioned directions can raise the utility of knowledge discovery techniques (and other techniques from related disciplines) beyond the direct form of decision support to strategic application such as capturing knowledge for organizational learning and for other strategic business decisions.
References [1] Batini, C., Lenzerini, M., and Navathe, S. B., A Comparative Analysis of Methodologies for Database Schema Integration, ACM Computing Surveys, 1986, 18(4), 323-364. [2] Billman, B. and Courtney, J.F., Automated discovery in managerial problem formulation: Formation of causal hypotheses for cognitive mapping, Decision Sciences, 1993, 24(1) 23-41. [3] Bolloju, N., Qualitative decision modeling using fuzzy logic, Decision Support Systems, 1996, 17, 275-298. [4] Bolloju, N., Effectiveness of a neuro fuzzy classifier in automated modeling of classification problem-solving knowledge: A quasi-experiment, Proceedings of the ICIS, 1997 (to appear). [5] Chan, K.C.C. and Wong, A.K.C., A Statistical Technique for Extracting Classificatory Knowledge from Databases, Knowledge Discovery in Databases, G. Piatetsky-Shapiro, and W.J. Frawley, (eds.), AAAI Press, 1991, 107-123. [6] Chaturvedi, A.R., Hutchinson, G.K. and Nazareth, D.L., Supporting complex real-time decision making through machine learning, Decision Support Systems, 1993, 10, 213-233.
1060-3425/98 $10.00 (c) 1998 IEEE
[7] Davies, M. A. P., A Multicriteria Decision Model Application for Managing Group Decisions, Journal of the Operational Research Society, 1994, 45(1) 47-58. [8] Cheeseman, P. and Stutz, J., Bayesian Classification (AutoClass): Theory and Results in Advances in Knowledge Discovery and Data Mining, Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., and Uthurusamy, R. (eds.), AAAI Press/ MIT Press, 1996, 153-180. [9] Clancey, W.J., Heuristic Classification, Artificial Intelligence, 1985, 27, 289-350. [10] Clemen R.T. and Winkler, R.L., Aggregating Point Estimates: A Flexible Modeling Approach, Management Science, 1993, 39(4), 501-515. [11] Dubois, D. and Koning, J-L., A decision engine based on rational aggregation of heuristic knowledge, Decision Support Systems, 1994, 11, 337-361. [12] Hill, T., & Remus, W., Neural network models for intelligent support of managerial decision making, Decision Support Systems, 1994, 11, 449-459. [13] Litwin, W., Mark, L., and Roussopoulos, N., Interoperability of Multiple Autonomous Databases, ACM Computing Surveys,1990, 22(3), 267-293. [14] Markham, I.S. and Ragsdale, C.T., Combining Neural Networks and Statistical Predictions to Solve the Classification Problem in Discriminant Analysis, Decision Sciences, 26:2, 1995, 229-242. [15] Mechitov, A.I., Moshkovich, H.M. and Olson, D.L., Problems of decision rule elicitation in a classification task, Decision Support Systems, 1994, 12, 115-126. [16] O’Leary, D.E., Determining differences in expert judgement: Implications for knowledge acquisition and validation, Decision Sciences, 1993, 24(2), 395-407.
[17] Piramuthu, S., Shaw, M.J., & Gentry, J.A., A classification approach using multi-layered neural networks, Decision Support Systems, 1994, 11, 509-525. [18] Perez, J. and Barba-Romero, S., Three practical criteria of comparison among ordinal preference aggregating rules, European Journal of Operational Research, 1995, 85, 473487. [19] Ramanathan R. and Ganesh, L.S., Group preference aggregation methods employed in AHP: An evaluation and an intrinsic process for deriving members’ weightages, European Journal of Operational Research, 1994, 79, 249-265. [20] Saaty, T.L., The analytic hierarchy process, Wiley, New York, 1980. [21] Simpson, P.K., Fuzzy min-max neural networks - part 1: classification, IEEE Transactions on Neural Networks, 1992, 3(5), 776-786. [22] Takagi, H., Suzuki, N., Koda, T. and Kojima, Y., Neural networks designed on approximate reasoning architecture and their applications, IEEE Transactions on Neural Networks, 1992, 3(5), 752-760. [23] Uthurusamy, R., From data mining to knowledge discovery: Current challenges and future directions in Advances in Knowledge Discovery and Data Mining, Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P., and Uthurusamy, R. (eds.), AAAI Press/ MIT Press, 1996, 561-569. [24] Widrow, B., Rumelhart, D.E. and Lehr, M.A., Neural networks: applications in industry, business and science, Communications of the ACM, 1994, 37(3), 93-105.
1060-3425/98 $10.00 (c) 1998 IEEE
