A knowledge-based DSS for supporting ill-structured multiple criteria ...

A Knowledge-Based DSS f o r Supporting Ill-Structured Multiple Criteria Decisions

I l y w B. Hong

Doug Vogel

Jay F. Nunamaker, Jr.

Department of Management Information Systems College of Business a n d Public Administration The University of Arizona Tucson, Arizona 85721

methods developed in the OR/MS area such as goal programming have been useful only for decision problems where the analysis involves the use of quantitative data and models. With the recent emergence of DSS concepts, there have been developed computerized systems designed to provide support to semi-structured decisions such as bond trading or capital acquisition analysis [17]. Nevertheless, these systems could hardly lend effective support to highly unstructured choice problems that often form the basis of many top management decisions. As technological advances made it possible to apply artificial intelligence techniques to multicriteria decision problems, some authors suggested the design of knowledge-based DSS for specific applications such as consumer product selection 2341 or for generalized MCDM problems [15,20]. These authors focus, as an area of AI application, on either preference elicitation or datdmodel management. However, they do not yet address such issue as support for qualitative reasoning processes involved in complex decisions. We need a methodology for aiding the DM in solving ill-structured multicriteria decisions. Application of AI techniques must be extended to include facilitation of high-level managemendecision processes that typically require a large degree of human intuition and personal analysis. This paper presents a system architecture for designing a knowledge- based DSS that facilitates structuring and solving intricate MCDM problems for an organization. In the paper, strategies for making choices among altematives are briefly discussed that have been modeled after human choice making processes. Then, we identify two types of multicriteria decisions (well-structured MCDM and ill-structured MCDM), and we discuss them with regards to the nature of expertise, attributes, size of altematives set, task environment, criteria analysis, and real-world examples. A method is developed for computerized support to unstructured multi-criteria decisions within the general framework of human choice making. A system architecture for an MCDM DSS design is presented that shows how expert knowledge and data can be accessed and used by the inference engine to help the DM solve intricate multi-criteria decision problems. Development of a prototype intended to partially represent the architecture will be described. Finally, we describe a real-world case on a bank’s commercial loan approval judgment.

ABSTRACT Focusing on qualitative reasoning processes as a key area of AI (artificial intelligence) application for decision support, this paper proposes an architecture for designing an intelligent DSS that is intended to aid in MCDM (Multiple Criteria DecisionMaking) in ill-structured situations. An MCDM DSS, for its maximum contribution to organizational problem-solving, must be capable of lending effective support to high-level as well as low-level management decisions. A commercial loan approval judgment case is described to illustrate the real-world situation where decisions usually require a high degree of intuition and subjective judgment. Development of a prototype intended to partially represent application of the architecture is described. The paper concludes with suggestions for research extensions.

Introduction

In recent years, there has been a surge of interest in the cognitive processes involved in decision making. One area of decision research that has received a particular focus among researchers is MCDM (multi-criteria decision-making) that is concemed with selecting from the pool of all possible decision altematives one that is mostly likely to solve problems. In a n organizational context, MCDM has been an issue of central concem to managers who make decisions. This is particularly true when an organization is viewed as an entity that constantly strives to solve problems in order to adapt to environmental changes 1251. To make appropriate decisions that will lead to possibly favorable conditions and thus solve organizational problems, managers have to address a wide spectrum of factors that are considered important in aniving at a final decision. Recently, there has been a marriage of MCDM to the AI area. The distinction in roles between the two areas is clear. MCDM provides choice algorithms and AI provides alternative generation and reduction [34]. Mintzberg et al. [21] argue that ”it is at the top levels of organizations where better decision-making methods are most needed.” In fact, past approaches to computerized decision support for organizations have mainly focused on low-level or middle-level management decisions. Quantitative analysis

229

0073-1129/89/oooO/0229$01.00 0 1989 IEEE

The MCDM Literature

Research on MCDM to date has produced literature in two main arenas. One arena has approached MCDM from operations research and management sciences disciplines, resulting in optimization methods for choice making. The other arena is concemed with descriptive research regarding human choice behavior. Our literature review will focus on the latter, examining what knowledge has been compiled on the way people make choices. We will be allowed to gain a perspective on it from the choice models for evaluating decision altematives. Solving problems through MCDM involves choice of one or more altematives by evaluating the altematives according to criteria. The MCDM literature suggests that there are a host of choice strategies, or choice rules, that allow the DM to arrive at a decision. It is widely accepted notion that MCDM consists of two categories: MADM (multiple attribute decision making) and MODM (multiple objective decision making) [13,14,4]. MADM is concemed with choice, while MODM relates to design (for more details on this dichotomy of MCDM, see [13]). Our main concem in this paper is MADM that involves choosing among available options based on multiple attributes, hence we will use the term MCDM to primarily refer to MADM'. These MCDM models include not only the models that describe human choice behavior but also the models that specify methods for making optimal choices. This descriptive/prescriptive classification of choice models represents two major approaches to choice making, which we will describe in this section. The choice strategies are classified into two categories depending on the level of cognitive processing demanded on the DM: compensatory (high processing) strategies and noncompensatory (reduced processing) strategies [ 18,201. The compensatorylnoncompensatory distinction is made on the basis of whether the same information is searched for each altemative, so that advantages of one dimension are traded against disadvantages of another dimension. A choice strategy is compensatory if inter-dimensional compensation of attributes values can be made (i.e., commensurability exists across dimensions), and it otherwise is noncompensatory. Therefore, compensatory strategies are cognitively more demanding but may lead to more optimal decision outcomes than noncompensatory strategies. Compensatory strategies include normative, linear models such as the additive model and the additive-difference model. Noncompensatory strategies include descriptive, non-linear models such as the dominance model, the conjunctive model, the disjunctive model, the lexicographic ordering model, and the elimination-by-aspects model. Below we briefly discuss the general characteristics of each choice model. Additive Model: The additive model, together with the additivedifference model, demands a high degree of cognitive processing. Search is exhaustive, and the goal is to arrive at an opti'MODM methods such as MOLP (multiple objective linear programming) or goal pmgrmming are not to be discussed in this paper. They are fully described in [4].

mal solution through normative linear models. In the additive model, overall "worth" indices are developed for each alternative, and then a final ordinal comparison of these indices is made to pick the best [l,35]. These worth indices represent weights assigned to different dimensions in accordance with their relative importance. The value of an altemative on a dimension is multiplied by the corresponding weight across all dimensions, and the multiplied values are summed up to arrive at an overall value of that altemative. This is repeated over all altematives, and altemative-by-altemative comparisons are made to find the altemative with the highest total [18]. Additive-Difference Model: The additive-difference model is a compensatory choice strategy requiring high cognitive processing like the additive model, but it differs from the additive model in that for each weighted dimension a difference is computed between the values of a pair of alternatives. In this model, two altematives are considered at once, dimension by dimension [ 181; thus the intra-dimensional rather than interdimensional information is utilized. The weighted differences are summed for each altemative. Now, comparisons are made between pairs of options in terms of distance; the better of the first two altematives is compared to the next alternative. This is repeated over the remaining options, until one best altemative is chosen. Dominance Model: The dominance model is the simplest form of noncompensatory choice strategies, and it is usually used for the initial screening of altematives. An alternative is said to be ominated if and only if there is another altemative which is better than that in one or more dimensions and is equal to that in the remaining dimensions [36]. To illustrate the dominance model, consider the following four altematives ( A l , A2, A3, A4) with values associated with three dimensions ( D l , D2, D3).

q--zT D1

D2

D3

A3

IO

3

12

A4

11

15

12

In this example, A I , A2, and A3 are not dominated among themselves, because no single altemative of the three satisfies the dominance condition. However, when these three altematives are compared to the fourth altemative (A4), it is clear that A4 is non-dominated; i.e., A I , A2, and A3 are dominated by A4. In the dominance method, dominated altematives are all eliminated from the set of existing alternatives, and the result of this elimination process is a set of nondominated altematives that can be further evaluated by other non-compensatory choice rules. In this sense, the dominance model is not a stand-alone decision method. But, the power of a decision method can be expanded when the dominance method is used together with other MCDM methods. Conjunctive Model: With the conjunctive model, the DM starts with specifying a set of cutoff values on different dimensions.

tioned, an aspect is a value on an attribute. When an aspect is picked as a baseline or standard, its role is as a criterion or a cutoff value.

Instead of computing each altemative’s overall score, the DM relies on altemative reduction based on cutoff values. The decision rule to use is to select the altemative that will exceed the specified cutoff values on all dimensions, by eliminating all alternatives that do not meet the criteria (cutoffs) on one or more dimensions. This procedure continues until all unsatisfying altematives have been eliminated; then, the options surviving the test are adopted as final choice. It will sometimes be possible for no alternative to be selected as final choice. In such a case, the initially set constraints can be relaxed to allow at least one altemative to be accepted as a decision.

The seven choice models described above represent how an expert DM would structure and solve problems at least for suboptimization of hisher goal. In a crude sense, compensatory models are based on linear algebraic models, whereas noncompensatory strategies are based on non-linear models. While they may not yield a rationally-grounded, optimal decision, they can provide a solution that is sound enough to satisfy the DM’s needs. These choice strategies can be applied to diverse real-world problems. However, successful application of the models requires careful evaluation of the tradeoffs between strengths and weaknesses of each model. This necessitates an analysis of the problem to find a model that captures the characteristics of the given problem. In addition, the applicability of these models seems limited to situations where decision tasks are relatively well-structured. When decisions involve many qualitative attributes, it may be necessary to develop a special version of a choice model.

Disjunctive Model: Like the conjunctive model, the disjunctive model requires the DM to establish cutoff values for the attributes. An alternative is chosen if and only if it exceeds a minimal cutoff on one or more dimensions [20] and if all the aspects of, the other altematives should fall below or be equal to the criterion values [27]. In other words, selection of an altemative is based on a disjunctive combination of values; an altemative satisfies the rule as long as its values exceeds cutoffs on one or more dimensions. Klayman [18] suggests that the disjunctive model bases a choice on quick ’acceptance’ of an altemative, whereas the other noncompensatory models focus on quick ’elimination’ of altematives. Hence, a suboptimal decision is typical of the disjunctive model. As with the conjunctive model, relaxation of constraints is sometimes necessary when the search cannot reach a decision. If all alternatives are disqualified, the DM can reduce the cutoffs of one or more dimensions, and resume the evaluation of altematives.

Two Types of Multi-Criteria Choice Problems Many real-world MCDM problems that organizations encounter typically fall into two broad categories; one where choice is made with a set of straight-forward rules and the other where rules for making choices are so fuzzy that it is often necessary to use intuitive or subjective judgments. This distinction is analogous to the well-structured/ill-structuredproblems dichotomy in the DSS literature (for example, see [ 17,26,21]). Some multi-criteria decisions are well-structured because there exist good guidelines for making choices, whereas others are ill-structured because of uncertainty and intricacy embedded in the choice-making processes. Hereafter, we refer to those two categories as well-structured MCDM and ill-structured MCDM. Well-structured MCDM is very common to our everyday life. For example, choosing an apartment, choosing a brand of product, selecting applicants for admission, etc. are all within this category. For this type of multi-criteria decisions, the DM uses a set of well-defined rules to guide hisher choice. Objectives may sometimes conflict from one another, in w h c h case the DM relies on choice strategies to resolve the conflict problem. On the other hand, ill-structured (or unstructured) MCDM is more intricate than its counterpart in the course of choice making. Unstructured decisions, as Keen & Scott-Morton [ 171 suggests, require human intuition and personal judgment. This is in part related to the observation that the uncertainty of the environment keeps the DM from foreseeing the likely outcome of a decision. In addition, many ill-structured multi-criteria decisions involve a considerable amount of possible risk which is associated with erroneous decisions. This is a major factor that forces the DM toward a careful analysis of the problem and possible solutions. We examine some of the major differences between these two types of multi-criteria choice problems with

Lexicographic Ordering Model: With the lexicographic ordering model, the DM lexicographically orders the attributes in order of decreasing importance. It does not involve the use of dimensional cutoff values, but rather it relies on altemativeby-altemative comparisons. The decision rule is to select an altemative that contains values closest to the best values across all dimensions. Starting with the most important dimension, altematives are ordinally compared to the alternative with the ”best” value for that dimension, and the altematives that have values distant from the best value are eliminated; this process is repeated on next important dimensions until one alternative is left. Elimination-By-Aspects Model: Developed by Tversky as a choice theory, the elimination-by- aspects (EBA) model uses the cutoff values on dimensions specified by the DM to eliminate less attractive altematives in a sequential manner. The model starts by selecting an aspect on a significant dimension, and eliminates all altematives that do not include the aspect [30]. If more than one alternatives pass the test on that dimension, only those passing altematives are tested against the criterion on the next important dimension. If no altemative qualifies for the test on the dimension, either can the constraint be relaxed or can the dimension be ignored to search the alternatives on the next dimension. This process is continued until only one altemative remains. As has been previously men-

23 1

respect to the six aspects shown in Table 1. Table 1. Differences between the two types of multi-criteria decisions

[Attributes

I Size o f alt.

1 I

set

Task environment Criteria analysis

I quantltative-oriented I qualitatlve-oriented I I large I small I

I I

static, predictable relatively straightforward

I I

personal loan approv-

I

Real-world examoles

students admission, selection

I apartment

I

dynamic, unpredictable extremely difficult ccmercial loan approval, merger/acauisition. hir;ng an executive

I I

Nature of Expertise: The nature of expertise is one of the major aspects that distinguish well-structured from ill-structured multi-criteria decisions. Expertise for well-structured MCDM is well-defined procedural knowledge pertinent to the problem domain, and so it is, to a large extent, readily acquirable from an expert. In particular, knowledge acquisition is facilitated through existing formal policies and rules that are easily transformed into a set of criteria or constraints. However, this is not the case for ill-structured MCDM. Expeaise for this type of decisions is characterized by fuzzy rules that may have no direct applicability to the problem at hand. Acquisition of an expert’s knowledge is indeed a problemistic process that can only be attempted at an abstract, superficial level. Attributes: Attributes involved in well-structured multicriteria decision problems are fairly quantitative so as to be more or less easily measured with only a few exceptions. Examples of quantitative attributes in a students admission case might include GPA, GMAT score, and work experience in years, while non-quantitative attributes might be recommendation rating and research capability. On the other hand, ill-structured multicriteria decisions deal with more qualitative-oriented attributes. Such attributes may be highly non-measurable and often very hard to operationalize. For example, a commercial loan approval judgment involves attributes like the good will, credit rating, capacity, and so forth. Size of the Alternatives Set: In general, the size of the alternatives set for well-structured choice problems is large relative to that for ill-structured choice problems. The DM solves such problems by narrowing the initially large altematives set down to a smaller size for which he or she can better work on the evaluation. By contrast, the number of altematives generated by the DM for unstructured choice making is fewer. Consider the commercial loan approval problem. The choice is usually between approving and rejecting the loan for the company that requested it. In an executive hiring problem, the altematives set will consist of a few highly competent and prospective individuals. It makes an intuitive sense that the harder the choice is, the fewer options the DM must review.

Task Environment: Most well-structured decisions are usually routine or repetitive tasks that can be as much programmed [261. This implies that the task environment is static and predictable. The DM is likely to have sufficient experience of a homogeneous group of problems, through which he or she can also classify new problems into some categories. Uncertainty is hardly encountered. However, the task environment for illstructured decisions is very dynamic and unpredictable due to a high degree of uncertainty regarding the expected outcomes of decisions [21]. Criteria Analysis: The method of criteria analysis is also an important factor that differentiates the two types of multi-criteria choice problems. For relatively simple, structured decisions, the choice strategies that we discussed earlier might be applied in a straight-forward manner to analyze criteria. On the other hand, analysis of criteria for unstructured decisions is an extremely difficult process. The difficulty is attributed both to the qualitative nature of the criteria and to a large number of minute details associated with the criteria. Real-world Examples: The majority of choice problems dealt with in the MCDM literature represents real-world examples of well-structured multi-criteria problems. They include the students admission decision, the personal loan approval, the recruition of clerks, and the apamnent selection. Unstructured real-world choice problems could be commonly seen in corporate topmanagement decision making. Examples of this category are commercial loan approval, merger or acquisition of a company, and hiring of a senior executive. These are the kinds of choice problems that may place the DM in an ambiguous and intricate situation.

A 3lethod for Supporting Ill-Structured Decisions The traditional approach to helping the DM solve multicriteria choice problems through a DSS is searching the data base, applying some quantitative models to the data, and presenting the solution to the DM. However, this method is not Likely to work well for such complex choice problems as we previously discussed. To insure effective support for behaviorally-grounded aspects of complex decisions, emphasis must be placed on a structural decomposition of the unstructured decision problem and on an effective manipulation of preference information. Such an emphasis will lead to a decision outcome that can be more convincing to the decision maker. In this section, we show how a decision problem can be hierarchically represented and how the DM’s preference information can be elicited and represented. We conclude with a discussion of evaluation of alternatives. Hierarchical Representation of a Problem To represent a choice problem into a hierarchy, we borrow the notion of structural criteria from the Saaty’s analytical hierarchy process (AHP) method. The AHP is a multicriteria decision method that uses hierarchic or network structures to represent a decision problem and then develops priorities for the

Representation of Knowledge There are three types of knowledge that need to be captured and maintained in the knowledge base. The first one is preference knowledge that encapsulates the objectives, constraints, and policy. We group these as preference knowledge because they are most important aspirations that the DM wants to achieve in organizational decision making. The second categoly is choice knowledge that is intended to guide the choice-making process. The third type of knowledge is metaknowledge that governs the activation of rules of the choice knowledge. The preference knowledge is declarative, while the choice knowledge and metaknowledge are procedural. We below discuss each of these three types of knowledge as to its nature and representation scheme. Preference Knowledge: Three key elements of the preference knowledge include (1) evaluation criteria, (2) criteria weights, and (3) cutoffs. These elements are elicited from the DM through a guided dialogue. The most appropriate scheme for representing the preference knowledge is the frame that can well support the representation of template and hierarchical knowledge. Recall that a decision problem can be decomposed into hierarchical or structural criteria. As shown in Figure 2, a frame contains slots (name, level, weight, and lower-level criteria) associated with the primary decision goal, and each slot has related suborlnate slots, etc. The lowest-level criterion frame, the final degree program completed, has the same format as the above frames except that it specifies a cutoff instead of lower-level criteria. Weights of subcriteria related to a criterion at each level should total to 1; that is, a weight of a criterion is relative to the weights of other related criteria at the level. The frame-based knowledge representation can accommodate any size and possibly any complexity of criteria structure existing in ill-structured decision problems. Thus it can support the structure of the multi-level criteria. In addition, it possesses a certain degree of inheritance feature that is central to the semantic net. That is, each frame is inherited with the properties (e.g., the weight) of the frame above it.

Figure 1: Hierarchical decomposition of the goal in a marketing executive hiring problem altematives based on the decision maker’s judgments throughout the system [28]. Because it is based on rank-ordering of altematives for choice making that is different from our method, we will only discuss its problem representation scheme (for details on the AHP, see [28,29]). In the AHP, the goal of a decision is decomposed into more specific criteria that are further broken down into lower levels as needed. At the lowerest level of the hierarchy are the altematives. The structural criteria concept is useful in analyzing complex decision processes that involve highly intangible criteria. An intangible criterion is broken down to lower, more concrete levels until it is operationalized or it cannot be itemized any further. In this manner, it facilitates the information processing required by ill-structured decisions. To illustrate a realistic example of using structural criteria, consider a marketing executive hiring problem. As can be seen in Figure 1, the decision goal of hiring a highly competent candidate is, for simplicity, decomposed into such criteria as leadership, marketing expertise, and judgmental ability. The leadership criterion can be divided into subcriteria that include experience in years at managerial positions, communication skills, and leadership potential. The marketing expertise may be explained by a combination of factors such as educational background and claimed expertise, and the educational background is further divided into the final degree program completed and the exposure to marketing courses. Finally, the judgmental ability criterion is indicated in two subcriteria such as intuitive skills and analytical skills. Information sources can be located for the lowest-level criteria. The application form may provide information for the experience in years at managerial positions and for the education background (the degree and the relevance of majors to marketing). The interview results tell about the communication skills and the claimed expertise. Recommendation letters or contacts with references may provide a basis for determining the leadership potential, the intuitive skills, and the analytical skills. Notice that we do not include the altematives in the hierarchy as the AHP does.

I

NAME: The b e s t c a n d i d a t e LEVEL: 1 WEIGHT: 0 . 4 LOWER-LEVEL CRITERIA: Leadership M a r k e t i n g ex e r t i s e Judgmental a g i l i t y

* **

I

I

WEIGHT: 0 . 4 LOWER-LEVEL CRITERIA:

WEIGHT: 0.3 LOWER-LEVEL CRITERIA:

LEVEL: 4 ( l o w e s t level WEIGHT: 0 . 7 CUTOFF: The M a s t e r s degree

Figure 2: The frame-based representation of preference knowledge

233

Choice Knowledge: Choice knowledge includes a group of aforementioned choice models that can be retrieved and executed during the decision aiding process. Therefore, choice knowledge simulates human cognitive processes involved in general choice-making. Production systems have proved quite useful in modelling cognitive processes [32]. These production systems use a set of production rules, each of which consists of the antecedent to recognize whether the condition is true and of the consequent to act when the condition is true. Procedural operations required by choice knowledge can be represented using such production rules. Each model would contain a set of if-then rules that can collectively determine a most preferred altemative for the DM. For example, for a conjunctive model:

:F

cbtofi

e x i m ICIIII

1 ; I AND

v a l u e iaiternar.vei11, c r i t ! l ' t t

ThEh fa;i

( a ' : t e r n a i i r e ( 1 ! i AIiD

' cutoff

t

t

t

Cutoffs

Dominance model

Conjunctive model

utilities, and ranks

t

summary report

f

I

Weights for dimensions Additive model

Figure 3: The four-phase process for choosing a best altemative ing at a decision. In her experimental study, Corbin [ 5 ] found that subjects would set cutoffs, but search a bit further than the first acceptable offer as long as that acceptable offer could be guaranteed available. Wright & Barbour [35] examined phased decision strategies under an experiment. Their results revealed that in the first phase, people initially screened the options using multiple cutoffs and in the second phase, they adopted a new choice strategy based on the result of the initial screening. An implication to be drawn from these findings seems that the evaluation of options must start with the initial screening that then precedes the review of the remaining options on dimensions. Choosing a highest-priority option takes a sophisticated evaluation of the alternatives along the problem hierarchy. We followed a top-down method to represent a choice problem. In making a choice, however, we work upward through the hierarchy from the bottom. Choice making is done in four successive phases - initial screening, further reduction, compensatory evaluation, and report generation - as shown in Figure 3.

:Ti