Improving Group Decision Making: A Fuzzy GSS Approach - CiteSeerX

Improving Group Decision Making: A Fuzzy GSS Approach Ron C W Kwok1, Jian Ma2 and Duanning Zhou3 Dept of Information Systems, City University of Hong Kong Tat Chee Avenue, Kowloon, Hong Kong 1 [email protected]

2. [email protected]

3. [email protected]

Tel: +852-2788-9635

Tel: +852-2788-8514

Tel: +852-2788-9599

Abstract Group decision making methods such as Multi-Criteria Decision Making (MCDM) have been developed extensively, but their organizational use for group decision making has been difficult. According to Arrow’s Impossibility Theorem, one possible reason is that a group decision outcome could never satisfy every decision maker’s individual preference. This paper proposes a fuzzy GSS (Group Support System) approach to improve the quality of the group decision outcome. The fuzzy GSS approach integrates a fuzzy MCDM model and a structured group decision making process with a GSS. The fuzzy MCDM model includes fuzzy individual preference generation and group aggregation. Supported by the GSS, the structured decision making process makes group participation effective. The proposed approach aims at providing more decision information, and enhancing group consensus, satisfaction and understanding of the decision outcome. This paper also postulates that the fuzzy GSS approach enhances group consensus, satisfaction and understanding of the group decision outcome. An empirical study was conducted to test the research predictions. Index Terms-- Fuzzy sets, group support system, multi-criteria decision making

1

I. INTRODUCTION According to Arrow’s Impossibility Theorem [2], no group decision making method is perfect. Being regarded as a very important work in modern social choice theory, the theorem shows that a group decision outcome can never satisfy every member of a group. It ends the search for an ideal democracy. However, there is still a growing amount of research on group decision making methods towards better quality and democracy [4, 9, 23, 53]. Ranking alternatives with multiple criteria is a common and important task in business organizations [24]. Several group decision making methods have been proposed to rank alternates among group members, one of them is the Multi-Criteria Decision Making (MCDM) method [16, 35]. Current MCDM methods tend to reflect an ideal decision making environment in which decision makers can rationally consider all aspects of the problem, think through these aspects, get all precise information, and then come up with a consensus solution that can satisfy most of the decision makers [38, 47]. However, in spite of the great potential of MCDM methods for group decision making, the organizational use of these methods has been difficult [11, 21]. A study indicates that out of many MCDM methods proposed in 78 research articles, only two methods were used regularly in organizational settings [24]. In practical applications, decision making is often a complicated process with imprecise judgments [6, 33, 61, 66]. For example, project teams often use "good", "excellent", and "poor" to evaluate the quality of software projects with respect to a number of decision criteria, e.g. functionality, reliability, and usability. But what do the terms "good", "excellent", and "poor" really mean? These terms do not constitute

2

a well-defined boundary. Fuzzy set theory [65] has been developed to solve decision making problems in which descriptions of observations are imprecise, vague, and uncertain. The term “fuzzy” refers to a situation where there are no well-defined boundaries for the set of observations. Bellman and Zadeh [6] note that much of the decision making in the real world takes place in an environment in which the goals, the constraints and the consequences of possible actions are not known precisely. They also report that the fuzzy set theory can be used to deal with imprecision, uncertainty and fuzziness in group decision making. As more and more decisions in real organizational settings are made by groups, applying fuzzy set theory into MCDM methods for dealing with imprecision, uncertainty and fuzziness in group decision making has become a hot research topic in current fuzzy set and MCDM research [13, 14, 64, 68]. However, current fuzzy MCDM methods overlook an important element of group decision making: group interaction between decision makers. Based on the small group literature [48, 49, 58], group interaction is regarded as an important element for the success of group decision making, it can be enhanced by introducing computer and communication technologies in group decision making tasks. For more than a decade, numerous GSS (Group Support System) researchers have been studying the potential of GSS in supporting group decision making. Results are summarized in a number of articles, e.g., Dennis et al. [18], McGrath & Hollingshead [49], and Fjermestad & Hiltz [27]. Dennis & Gallupe [19] suggest that although the research findings paint a rather unclear picture, much of the literature on GSS rests on the assumption that the addition of the electronic medium to verbal

3

information exchange would lead to better group interactions and decisions, and to higher productivity. The electronic medium allows greater group interaction in decision making process and increases access to the information. GSS is also appropriate for complex and imprecise tasks [36]. Although GSS improves group performance in terms of group interaction, idea generation, and equality of participation, the research shows that it fails to increase group consensus and satisfaction to both decision process and outcome [7, 37, 50]. Either using a fuzzy MCDM method or a GSS alone can hardly facilitate a complex and imprecise group decision making process. As shown in Figure 1, this paper proposes a fuzzy GSS approach which integrates a fuzzy MCDM model and a structured group decision making process with a GSS to improve the quality of group decision making tasks. The proposed fuzzy GSS approach supports the whole process of group decision making, which aims at enhancing the group consensus on the decision outcome. Supported by the GSS, the structured decision making process makes group participation effective, it provides more decision support information, and enhances group satisfaction and understanding of the decision. Section 2 of this paper presents the proposed fuzzy GSS approach. The empirical study designed to test the predictions of the approach is described in section 3. Section 4 analyses and discusses the results, lists the limitations and suggests the future research topics. Section 5 summarizes the findings.

4

Fuzzy GSS Approach

Group Decision Outcome

Fuzzy Multi-Criteria Decision Model

Group Consensus

Group Support System

Group Satisfaction

Structured Group Decision Making Process

Group Understanding

Figure 1. Effects of the fuzzy GSS approach II. FUZZY GSS APPROACH The proposed fuzzy GSS approach includes a fuzzy MCDM model, a structured group decision making process and a GSS, as shown in Figure 1. It is designed to enhance the group consensus, satisfaction, and user understanding of the decision outcome. A. Fuzzy MCDM Model The proposed fuzzy MCDM model for group decision making integrates non-ranked voting methods, particularly the approval voting method [26], with the fuzzy set theory [65]. It aims at enhancing group consensus on the group decision outcome. The model includes fuzzy individual preference generation and group aggregation. 1) Individual Preference Generation: Let A={A1, A2, ... , Am}, m>=3, be a given finite set of alternatives; C={C1, C2, ... , Ct} be a given finite set of attributes; P={P1, P2, ... , Pn}, n>=2, be a given finite set of decision makers. The steps of generating individual preference are shown below: Step 1: Considering the different importance of attributes C, the different weights

5

to the attributes are determined. The Analytic Hierarchy Process (AHP) method [54] is used here to get the attribute weights. By pairwise comparison of the relative importance of attributes, the pairwise comparison matrix E = [eij ]t ×t is established, where eij represents the quantified judgments on pairs of attributes Ci, and Cj. The comparison scale ranges from 1 to 9, each representing the concepts of: 1: equally important; 3: weakly more important; 5: strongly more important; 7: demonstratively more important; 9: absolutely more important; 2, 4, 6, and 8 are intermediate values between adjacent judgments. For example, eij = 5 means Ci is strongly more important than Cj. The consistent weights for every attribute can be determined by calculating the normalized principal eigenvector. The weights are denoted as w1, w2, ... , wt , where wi ∈[0,1] and

t

∑ w = 1. i

i =1

Step 2: Against every attribute Cj (j = 1, 2, …, t), assign 1 to preferred alternatives and 0 to unwanted alternatives. Because the choice is decision makers' subjective judgments, decision makers may often meet the situation where it is difficult for them to choose or reject alternatives. Thus only the yes/no method needs to be improved. One alternative method is for the decision maker to give a belief level to the selected alternatives. The belief levels belong to a set of linguistic terms that contains various degrees of preference required by the decision makers. Linguistic terms are words or sentences in natural or artificial languages. For example, "very low", "low", "medium", "high", "very high" are linguistic terms. Linguistic terms are illdefined and can hardly be described by single numerical values. In this paper, we use linguistic terms Z(Belief) = {very sure, sure, not very sure, not sure}, which are depicted in Figure 2. 6

µ

µ

1

1 “Very sure”

“Sure”

0.5

0.5 0

0.8 1

x

0

µ

µ

1

1 “Not very sure”

0.5

0.6 0.8 1

x

“Not sure” 0.5

0

0.3 0.5 0.7

1

x

0 0.1 0.3 0.5

1

x

Figure 2. The linguistic terms The individual selections are denoted as two matrices: alternative selection matrix ( vij ) and belief matrix ( bij ) respectively, were vij ∈ {0,1}, bij ∈ Z ∪ {0} (i=1, 2, … , t, j=1, 2, …, m). Step 3: The alternative selection matrix ( vij ) is aggregated to alternative selection vector ( v 'j ) (j = 1, 2, … , m). v 'j = w1 * v1 j + w2 * v 2 j + ... + wt * vtj

(1)

Step 4: The belief matrix ( bij ) is aggregated to belief vector ( b 'j ) (j = 1, 2, … , m). b 'j = w1 • b1 j ⊕ w2 • b2 j ⊕ ... ⊕ wt • btj

(2)

where the method for aggregation is similar to that of Baas and Kwakernaak [3]. Step 5: Based on the alternative selection vector ( v 'j ) and belief matrix ( b 'j ), the decision maker again makes overall judgment on alternatives. The process is similar to that in step 2, i.e., assign a belief level to the preferred alternative and 0 to unwanted alternative. The results are called the individual selection vector.

7

2) Group Aggregation: All individual selection vectors can be composed by group selection matrix ( bijg ). Similar to step 4, the selection matrix is aggregated to group preference vector ( r j ), j = 1, 2, … , m, where each decision maker has an equal weight of 1/n. By ranking the group preference vector ( r j ), the group can reach an agreement on the preferred alternatives. B. Application The proposed fuzzy MCDM model was used to select a list of assessment criteria by a group of students and lecturers in our experiment. It was to formulate a marking scheme for evaluating the student projects. For example, in a project assessment, five assessment criteria (A={A1, A2, ... , A5}) need to be ranked. Three decision makers (P={P1, P2, P3}) involved in the decision task, and the attributes set is C={ C1 (student interest), C2 (student ability), C3 (learning objective)}. 1) Individual Preference Generation: Following is the individual preference generation process of decision maker P1. Step 1: By pairwise comparison of the relative importance of the attributes, the pairwise comparison matrix E is established. 1 E = 1  3

1 1 3

1/ 3 1/ 3 1

   

The element of the matrix eij represents the relative importance of attribute Ci over attribute Cj. For example, the first row of the matrix means student interest is as equally important as itself, student interest is as equally important as student ability, learning objective is weakly more important than student interest.

8

By calculating the normalized principal eigenvector of the matrix, the attribute weights, W = (0.2, 0.2, 0.6), is obtained. Step 2: Against every attribute C1, C2, C3, assign 1 to the preferred assessment criteria and 0 to the unwanted one. Meanwhile, assign belief levels to the preferred assessment criteria. The selection matrix ( vij ) and belief matrix ( bij ) are obtained (i=1, ..., t; j=1, ... ,m). 1 1 0 0 1 (vij ) = 1 0 1 1 0  0 1 0 1 0  very sure (bij ) = very sure  0

very sure 0 0 very sure sure 0

0 sure  sure 0  not very sure 0 

Step 3: According to equation (1) and (2), the matrix ( vij ) and matrix ( bij ) are aggregated to selection vector ( v 'j ) and belief vector ( b 'j ) respectively, where ( v 'j ) = (0.4, 0.8, 0.2, 0.8, 0.2) and ( b 'j ) is shown in Figure 3. b5′ b3′

b1′ b4′

b2′

1

0.5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Figure 3. The values of b1' , b2' , b3' , b4' , b5'

9

0.9

1

Step 4: According to the values of vectors ( v 'j ) and ( b 'j ), the decision maker P1 again makes overall judgment on the projects. The individual selection vector is (not very sure, very sure, 0, sure, 0). 2) Group Aggregation: By combining individual preferences of another two decision makers (i.e., P2, P3), the group selection matrix ( bijg ) can be obtained. not very sure (b ) =  sure very sure

very sure 0 sure not very sure very sure sure

g ij

sure 0  very sure 0  0 sure 

Similar to step 4 in section A, the group selection matrix ( bijg ) are aggregated to group selection vector ( r j ) (j = 1, 2, … , m), as shown in Figure 4. After ranking the group preference vector, the result is: A2 > A1 > A4 > A3 > A5

r5

r3

r4

r1

r2

1

0.5

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 4. The values of r1 , r2 , r3 , r4 , r5

C. Group Support System A Group support system (GSS) is an interactive computer-based system that combines computing, communication, and decision technologies to facilitate

10

problem formulation and solution in collaborative work [20]. Its goal is to ease the cognitive load of groups on particular decision making tasks so as to improve the productivity, efficiency, and effectiveness of group meetings. Groups can also use GSS to work together while separated by space and time [62]. Features of GSS such as simultaneous & parallel processing and anonymity permit students to work interdependently as well as in a group [10]. During an electronic GSS meeting, students can type their ideas simultaneously into a network of computer workstations. The GSS immediately makes all these contributions available for other students to read on their individual screens. Sometimes students can use GSS anonymously to eliminate fear of reprisal from tutors or peers when contributing unpopular or sensitive ideas. The anonymity helps students to focus on the merits of the contributed ideas rather than its source. Furthermore, GSS helps to ease the cognitive load of groups on particular decision making tasks so as to improve the productivity, efficiency and effectiveness of group meetings. Anonymity has better effect on groups with a hierarchical structure whose members have different power and status. The ability for group members to work in parallel is a significant benefit that accounts for much of the success of GSS technology. Consequently, group performance and reactions to GSS technology may change over time, as the group gains experience with it. Nunamaker et al. [52] propose that the use of GSS can improve group processes in many cases. GSS are expected to reduce group process losses associated with information overload [34], social pressure [55], attention blocking [22] and other difficulties commonly encountered in group meetings. GSS is also claimed to increase process gains by enhancing participation [15] and generating more information [55]. 11

In this study, an Internet-based GSS is developed for the implementation of the fuzzy GSS approach [44, 67]. The GSS integrates the proposed fuzzy MCDM model with the structured decision making process. It was developed using a Windows NT server, SQL server, Internet Information Server (IIS) and Internet Database Connector (IDC) on a N-tier client-server architecture as shown in Figure 5. The architecture allows participants to access the system through an Internet browser, e.g. Netscape Navigator or Internet Explorer.

Client Browser

Network Connectivity IDC Request and Result

GSS Server Internet Information Server SQL Server Microsoft NT Server

Figure 5. A GDSS Server architecture For the portability reason, the system uses Open DataBase Connectivity (ODBC) together with the IDC to send and retrieve information from SQL Server.

12

D. Structured Group Decision Making Process In this study, a structured group decision making process is proposed to facilitate the use of GSS and the fuzzy MCDM model, as shown in Figure 6. The structured group decision making process consists of four steps, i.e. (1) brainstorm the basic alternatives; (2) evaluate the basic alternatives with reference to the decision criteria; (3) generate individual fuzzy preference on the basic alternatives; (4) aggregate individual preference to obtain decision outcome. If all decision makers agree with the evaluation result, then the whole decision process ends. Otherwise, decision makers may repeat the above steps in order to reach an appropriate level of group consensus.

Step 1: Brainstorm the basic alternatives

Step 2: Evaluate the basic alternatives with reference to the decision criteria

Step 3: Generate individual fuzzy preferences on basic alternatives

Step 4: Aggregate individual preference to obtain decision outcome

Negotiate

No

All agree? Yes End task

Figure 6. The structured group decision making process

13

Within the structured group decision making process, steps 1 and 2 are supported by the GSS for brainstorming and evaluating alternatives, while steps 3 and 4 are facilitated by the fuzzy MCDM model for generating individual preferences and group aggregation. This process structure keeps the group interaction on track so that the decision model can be effectively applied to the group decision making task [51]. A number of researchers have also indicated that providing process structure for group interaction can lead to enhance group performance in a number of dimensions, e.g. information sharing [59], task quality [17], conflict resolution [32] and group satisfaction [31]. Having an appropriate process structure, the group decision making process ensures that all decision makers have opportunities to participate [51]. III. EXPERIMENTATION: AN EMPIRICAL STUDY

A. Hypothesis This paper postulates that the fuzzy GSS approach improves the quality of group decision

making

by

enhancing

group

consensus,

satisfaction,

and

user

understanding on the group decision outcome. Three hypotheses are formulated to test the predictions. 1) Hypothesis on Group Understanding: Group decision making involves group processes where people work together to accomplish a decision making task. The process of decision making can be considered as a collaborative learning process, where decision makers share their understanding about the task domain. GSS are expected to make collaborative learning environments more effective [56]. The

14

literature [20, 52] suggests that GSS enhances group processes by increasing group process gains and/or reducing group process losses. Their effects on group process gains/losses contribute to the two attributes of effective learning attributes (i.e., active engagement and cooperation) which can enhance group understanding about the task domain [1]. With GSS support, decision makers can become more active, autonomous and confident in idea construction. Within an anonymous GSS environment, the decision makers are not intimidated and are less inhibited [28, 29]. Furthermore, as it is difficult to preclude others from contributing ideas, domination of the meeting can be reduced. Also, free riding might be diminished, as decision makers can contribute simultaneously and no longer need to compete for air time. All of these effects contribute positively to the “active engagement” attribute of effective learning. GSS also encourages decision makers to provide feedback that is useful for reinforcing understanding of the task through information sharing [40, 43, 57], individuals’

participation

and

objective

evaluation.

By

reducing

individual

domination, meeting time fragmentation and fear of reprisal, GSS encourages positive interpersonal communication and relationships [63]. These effects contribute positively to the “cooperation” attribute of effective learning. A GSS-supported meeting helps the decision makers better understand the task-related decision information by structuring the task. Integrated with the structured decision making process (see Figure 3), the GSS enables decision makers to perform a deeper analysis of the problem, resulting in a better understanding of the task. Decision makers can benefit from sharing information for successful problem solving. In addition, the anonymous GSS environment encourages objective evaluation and error catching in problem analysis. 15

This paper postulates that integrating GSS with the structured decision making process can enhance the group understanding of the group decision outcome. The first hypothesis is: H1: Subjects in a fuzzy GSS supported decision making environment are higher on group understanding of the decision outcome than subjects that are not supported by the fuzzy GSS. 2) Hypothesis on Group Consensus: Bellman and Zadeh [6] note that much of the decision making in the real world takes place in an environment in which the goals, constraints and consequences of possible actions are not known precisely. Fuzzy set theory can be used to deal with imprecision, uncertainty and fuzziness in group decision making. On the other hand, fuzzy set theory can also be used to enhance group consensus. In particular, researchers [25], [41] investigate the application of fuzzy set theory in supporting group consensus reaching. They suggest that use of the fuzzy set theory improves the quality of group decision making, and leads to a high degree of group consensus on the group decision outcome. Their findings suggest that group consensus reaching can be enhanced by the application of fuzzy set theory. It can be seen from the following example that the fuzziness of the linguistic terms (e.g. very sure, sure, not very sure, not sure) used in the individual preference generation can lead to a higher level of group consensus. Suppose three persons, P1, P2, P3, are to select one alternative from alternatives a, b, c using a classical voting method, e.g. Condorcet method [42]. If P1, P2 vote for a, and P3 votes for b, then the decision outcome is simply a. By matching the decision outcome with the preference made by each person, the group consensus is only 66.67% (see Table 1). 16

Table 1. Voting results in classic group decision method

A

P1

P2

√

√

P3

Decision Outcome √

√

B

Group Consensus 66.67%

C

On the contrary, suppose P1, P2, P3, are to select one alternative from alternatives a, b, c using the proposed fuzzy group decision method. If P1, P2 vote for a in very sure, and P3 votes for b in very sure, then the decision outcome is still a. By matching the decision outcome with the fuzzy preference made by each person, the group consensus is 100% (see Table 2) because the belief levels of the fuzzy linguistic terms overlap with each other (see Figure 2). Table 2. Fuzzy group decision method P1

P2

P3

A

Very sure

Very sure

Sure

B

Sure

Sure

Very sure

C

Not sure

Not very

Not sure

Decision Group Outcome Consensus √ 100%

sure

Based on the above justification and the previous research findings, this paper postulates that the proposed fuzzy set model enhances the group consensus on the group decision outcome. The second hypothesis is:

H2: Subjects in a fuzzy GSS supported decision making environment are higher on group consensus than subjects that are not supported by the fuzzy GSS.

17

3) Hypothesis on Group Satisfaction: Huang et al. [37] suggest that more equal information exchange and participation in group interactions would increase satisfaction with the decision process. Equal participation also encourages equal contribution to the group decision. As a result, satisfaction to decision outcome would be increased as well. Within an anonymous GSS environment, the decision makers are not intimidated and are less inhibited since they can contribute their ideas simultaneously and no longer need to compete for air time. Thus, equal participation and contribution to group decision are expected in a GSS meeting. With GSS support, the decision makers can achieve a high level of group satisfaction with the decision outcome. Furthermore, H1 posits that the GSS, integrated with the structured group decision making process, could enhance group understanding. Also H2 postulates that the fuzzy MCDM model could enhance group consensus. With high group understanding and consensus, high group satisfaction is also expected. This paper postulates that the fuzzy GSS approach can enhance the group satisfaction with the group decision outcome. The third hypothesis is: H3: Subjects in a fuzzy GSS supported decision making environment are higher on group satisfaction than subjects that are not supported by fuzzy GSS. B. Group Decision Making Task The group decision making task involved the development of an evaluation scheme for a semester long project in a 2nd year course of Distributed Information Systems. The aim of the project is to educate students to become Information Systems (IS) professionals through the application of knowledge and skills required by the course to a specific application. In undertaking the project, the student should demonstrate

18

a clear grasp of the chosen subject matter, a full understanding of the principles and concepts, and the ability to manage, evaluate and present the project in a coherent and precise manner. The course invites lecturers as well as students to decide the assessment criteria of the marking scheme. It encourages students to take an active and responsible role in the whole assessment process [46]. Based on the proposed structured group decision making process, the group decision making task was carried out in the following three stages: Stage 1: The students were provided with a description of the basic set of assessment criteria which were designed and drafted by the lecturers. Under the guidance of the lecturers, the students were asked to brainstorm and evaluate these criteria with reference to the course objectives while taking into account their interests and abilities, the learning resources and assessment policy of the institution (similar to the steps 1 and 2 of proposed structured group decision making process). As a result, some new criteria were added and existing ones modified. Stage 2: The students were then required to select six criteria to be used in the formulation of the evaluation scheme through an iterative process of voting, discussion of the outcome and re-voting until they reached a consensus (similar to steps 3 and 4 of the proposed structured group decision making process). The number of criteria was 6 so as to allow students to focus their attention on the most important aspects of learning. Stage 3: The students had to assign weights reflecting the relative importance of the selected criteria based on an iterative process similar to the one used in the second stage.

19

C. Experimental Design and Procedures To test the proposed three hypotheses mentioned earlier, an empirical study was conducted to investigate the effects of the fuzzy GSS approach on quality of group decision making. The subjects were asked to perform the group decision making task. 81 subjects were recruited from 2nd year undergraduates enrolled in the Distributed Information Systems course. Although participation was voluntary, all students opted to participate because the outcome of the task would be used to evaluate their projects. The participants were about half males, half females with an average age of 21.6 years. A pre-experiment analysis indicated that there was no significant difference in the background of subjects in terms of prior familiarity with the GSS tool used in the experiment. The participants were randomly assigned to two groups: an experimental group (with fuzzy GSS support) and a control group (without fuzzy GSS support). Each group was divided into two classes consisting of about 20 students. Participants of the experimental group completed the group decision making task with supports of the proposed fuzzy GSS while participants of the control group completed the task in a traditional face-to-face setting. This study involved two lecturers who were trained in general facilitation techniques. To enhance the external validity regarding facilitation in real settings, the two lecturers were both allocated to the two groups (experimental group and control group) and flexibly intervened in the group decision making processes based on their judgment of how to best assist the student groups. Having completed the group decision making task, each student was interviewed separately for 20 minutes and his/her answers were taped. During the 20

interview, each was asked to explain the selected project marking scheme. The question was open-ended so that students could demonstrate their level of understanding of the marking scheme. On the other hand, each student was asked to illustrate their level of consensus and satisfaction of the project marking scheme selected by his/her group. After the interviews, the audio taped responses on students’ understanding of the project marking scheme were analyzed and scored by two different course instructors. In order to ensure their familiarity with the scoring scheme, the two instructors performed a pilot test together with 10 sets of audio data. Afterwards, the two instructors analyzed the data and scored the students’ responses separately. For cases where the scores did not match, the instructors were asked to check their scoring without being told of the other’s score. If, after the second round, the two scores still did not match, the case was discarded. Only 2.33% of the cases were discarded in the experiment. 1) Fuzzy GSS Group Treatments: At stage 1 of the group decision making task, the fuzzy GSS groups were provided with the “Brainstorming” tool. The students used the tool to communicate with the lecturers and other students. They could type their ideas of the assessment criteria and read the ideas of other students and lecturers while separated in different places or at different time. The students could also use the fuzzy GSS anonymously so that they could contribute unusual ideas without having to worry about negative reactions from the lecturers and other students. Anonymous fuzzy GSS groups were expected to have lower levels of unanimity and member influence. A large number of alternatives and comments were generated while the normative influence of the majority or a powerful minority was eliminated [30]. 21

At stages 2 and 3, the fuzzy GSS groups were provided with the “Voting” and “Weighting” tools. The students used the tools to formulate the assessment criteria and agree on the final evaluation scheme. In order to improve the decision making process in the group decision making task, the proposed fuzzy MCDM model was used for the selection of the assessment criteria in the fuzzy GSS. Students could simply enter the fuzzy rating for each alternative under every decision criterion, and the corresponding weight for each decision criterion. The fuzzy GSS then processed the data using the built-in fuzzy MCDM model. The results of the voting were then ready for retrieval by students and lecturers. 2) Non Fuzzy GSS Group Treatments: Throughout the whole group decision making process, the non fuzzy GSS groups used the traditional face-to-face method to communicate with lecturers and other students. At stage 1, the students discussed the assessment criteria with their lecturers through verbal communication at normal classroom setting. At stages 2 and 3, the students selected their own sets of assessment criteria and then assigned weights to the selected assessment criteria. Similar to stage 1, the voting and weighting process were conducted manually and facilitated by the lecturers. IV. RESULTS AND DISCUSSION

A. Analysis of Pre-Experimental Results Among 81 students (41 males, 40 females) participated in the study, 39 were randomly assigned to the experimental group and 42 to the control group. Tables 3 and 4 confirmed that the two groups were balanced in age and sex. The average age of the students participating in the experiment was 21.6. In order to prevent biased subjects, each participant was asked about their prior experience with the 22

GSS tools. No student reported that he or she had prior experience with the GSS tool used in the experiment. Table 3. Sex distribution of the students Non Fuzzy GSS

Fuzzy GSS

Total

Male

22

19

41

Female

20

20

40

Total

42

39

81

Table 4. Age distribution of the students Measure

Non Fuzzy GSS Mean (standard deviation )

Fuzzy GSS mean (standard deviation )

t-value

p-value (two tailed)

Age

21.33 (1)

21.76 (1.47)

1.64

0.104

B. Effect of GSS on Group Understanding Burns et al. [12] identify two orientations toward understanding: the recognition of order within the subject matter (coherence orientation) and the ability to recall relevant information (knowledge orientation). The coherence orientation aspects of understanding were considered in this paper. The Structure of the Observed Learning Outcome (SOLO) taxonomy [8] was used to measure understanding. It is a research-based measure that is extensively used in educational research. Imrie [39] showed that SOLO was a widely applicable framework for judging the structure of essays, answers to technical questions, medical diagnoses, and open-ended problems. 23

In the SOLO taxonomy, there are five structural levels of understanding [39]: 1): Pre-structural: The task is engaged, but the student is distracted or misled. The use of irrelevant information provides no correct knowledge about the question. There is no meaningful response. 2): Unistructural: The student focuses on the relevant domain and concentrates on one aspect. The answer contains one correct feature or item of information. 3): Multistructural: The student picks up more and more relevant or correct information, but does not integrate it. The answer is list-like, containing a number of unconnected items. 4): Relational. The student integrates the related items with each other so that the answer has a coherent structure and meaning. The answer synthesizes connected information to make a case or logical whole. 5): Extended Abstract. The student generalizes the structure to take in new and more abstract features, representing a new and higher mode of operation. The answer not only relates items to make a logical whole, but it also contains a connection to a related area of knowledge beyond the demand of the question. These categories have an intuitive relation to different levels of understanding. Higher levels in the SOLO taxonomy correspond to higher levels of understanding. As illustrated in Table 5, the results of the statistical analysis of the subjects’ SOLO scores support the main hypothesis of this study. The average score of the GSS-supported group is significantly higher than the average score of the control group, as determined by a t-test at the 5% significance level. These results indicate that GSS enhances the group understanding of the group decision outcome, as measured by the SOLO taxonomy. These results can be interpreted as evidence for the significant positive contribution of the fuzzy GSS approach to group 24

understanding. Table 5. Results on Group Understanding Measure

Level of Understanding (SOLO taxonomy)

Non Fuzzy GSS Mean (standard deviation )

Fuzzy GSS mean (standard deviation )

t score

p score (one tailed)

2.42 (0.72)

2.89 (0.96)

1.82

0.038

C. Effect of GSS on Group Consensus Each student was asked to rate his/her consensus level on the project marking scheme selected by his/her group using a six point Likert-type scale (Strongly Disagree = 1, Quite Disagree = 2, Slightly Disagree = 3, Slightly Agree = 4, Quite Agree = 5, Strongly Agree = 6). The means and standard deviations for the level of perceived consensus of the selected marking scheme of the fuzzy GSS supported and the non fuzzy GSS groups were obtained. The means and standard deviations for the level of satisfaction of the selected assessment scheme of the two experimental groups were obtained. As shown in Table 6, the mean score of the consensus level of the GSS-supported group was significantly higher (p=.045). These results can be explained as evidence for the significant positive contribution of the fuzzy GSS approach to group consensus.

25

Table 6. Results on Group Consensus

Measure

Non Fuzzy GSS Mean (standard deviation )

Fuzzy GSS mean (standard deviation )

t score

p score (one tailed)

Level of Consensus

2.13 (0.85)

2.56 (0.70)

1.74

0.045

D. Effect of GSS on Group Satisfaction In order to measure the level of satisfaction of the selected marking scheme, each student was required to rate his/her satisfaction level using a six point Likert-type scale (Strongly Dissatisfactory = 1, Quite Dissatisfactory = 2, Slightly Dissatisfactory = 3, Slightly Satisfactory = 4, Quite Satisfactory = 5, Strongly Satisfactory = 6). The means and standard deviations for the level of satisfaction of the selected assessment scheme of the fuzzy GSS supported and the non fuzzy GSS supported groups were obtained. As shown in Table 7, the mean score of the satisfaction level of the GSS-supported group was significantly higher (p=.037). These results can be interpreted as evidence for the significant positive contribution of the fuzzy GSS approach to group satisfaction. Table 7. Results on Group Satisfaction Measure

Non Fuzzy GSS Mean (standard deviation )

Fuzzy GSS mean (standard deviation )

t score

p score (one tailed)

Level of Satisfaction

2.29 (0.75)

2.72 (0.75)

1.84

0.037

26

E. Discussion Current research indicates that GSS enhances group decision making in terms of decision quality, idea generation, and equality of participation, but it fails to enhance group consensus and satisfaction to both decision process and outcome [7, 50]. But it is still not known why GSS fails to increase group consensus and satisfaction. In this paper, the experimental results showed that decision makers who used the fuzzy GSS in support of their group decision making activities had a higher level of group consensus and satisfaction than those students who did not use fuzzy GSS. Moreover, they also achieved higher group understanding of the decision outcomes. Although the specific mechanisms by which the fuzzy GSS approach may improve the quality of group decision making were not investigated in this study, some of the students’ comments collected during the interviews provided further insight into a possible correlation between the fuzzy GSS features and the quality of the group decision making. For example, one student commented that the fuzzy GSS not only enhanced his understanding of the marking scheme through electronic interaction with other students and lecturers, it also made him feel a high degree of ownership of the marking scheme. He liked the selected marking scheme as well as agreed with it because he had contributed to the scheme. On the other hand, he also pointed out that the fuzzy GSS approach allowed him to enter soft and fuzzy individual preferences in the selection process in which he was not really sure about what assessment criteria could fit him. He was really pleased that he could just enter soft and fuzzy individual preferences that he was not really sure, and the system would then generate a concrete and clear group decision outcome which was close to his preference. 27

Another student commented that she felt the communication barrier between her and her lecturer was minimized when using the fuzzy GSS approach because they both shared a good mutual understanding of the marking scheme. These comments suggest that the fuzzy GSS approach improves the quality of group decision making by facilitating individual understanding as well as mutual understanding of the marking scheme among students and lecturers. Several studies [60], [45] report that establishing mutual group understanding among decision makers is an important factor for ensuring the quality of group decision making. For example, in an information system development project, the development of a requirement specification is conducted by system analysts through a series of meetings with users to decide the system requirements. When a person, no matter the system analyst or the user, misinterprets the group decision outcome and assumes that they are in agreement, errors in the systems building process can occur. In the experiment, students were encouraged to develop an appreciation and better understanding of the decision outcome through discussion and negotiation of the marking scheme with their lecturers following the structured fuzzy group decision making process (see Figure 3). By increasing group process gains and reducing group process losses, GSS improves the quality of group decision making process. Following the structural group decision process, GSS also helps the decision makers better understand and analyze task-related information, and enable decision makers to perform a deeper analysis of the problem, thus resulting in a better decision outcome. This study is not without limitations. Two factors on the settings of the empirical study limit the generalisability of the results. The first factor relates to the treatment 28

conditions and the other associates with the interaction of group members. First, the two treatment conditions were designed to test the overall effects of the fuzzy GSS approach on group consensus, satisfaction and understanding of the group decision outcome. Future studies could be conducted to investigate the main and interaction effects of the three individual elements of the fuzzy GSS approach (the fuzzy MCDM model, GSS, and the structured group decision making process) on group decision making under more treatment conditions. The second limitation of the study relates to the interacting effects between the subjects of different groups. Subjects of the experimental group may have interacted with those of the control group. Group consensus, satisfaction and understanding of the decision outcome might have been influenced by these interactions. Future studies should attempt to reduce the potential effect of subject interactions. V. CONCLUSION This paper presents a fuzzy GSS approach to group decision making. The proposed approach consists of a GSS, a fuzzy MCDM model and a structured decision making process. The fuzzy MCDM model includes fuzzy individual preference generation and group aggregation. Supported by the GSS, the structured decision making process makes group participation effective. An empirical study was conducted to test the effectiveness of the proposed approach. The study involved 81 second year undergraduate students and was conducted in a course of Distributed Information Systems. Students in the course were classified into two randomly selected groups, the experimental group and the control group. The experimental group used the fuzzy GSS approach to complete the experimental task, while the control group completed the experimental tasks in a

29

traditional face-to-face setting. The results of the empirical study indicate that subjects in a fuzzy GSS supported decision making environment are higher on group consensus, satisfaction, and understanding than subjects that are not supported by the fuzzy GSS approach.

ACKNOWLEDGEMENT This research was partly supported by the 1996 Industrial Support Fund (Project No.: AF/7/96) and the 1998 Competitive Earmarked Research Grant (Project No.: 9040375), Hong Kong. REFERENCES [1] M. Alavi, “Computer-mediated collaborative learning: An empirical evaluation,” MIS Quarterly, 18, pp. 159-174, June 1994. [2] K. J. Arrow, Social Choice and Individual Values. Second edition, New York: Wiley, 1963. [3] M. S. Baas and H. Kwakernaak, “Rating and ranking of multiple aspect alternatives using fuzzy sets,” Automatica, 13, pp. 47-58, 1977. [4] A. Banerjee, "Fuzzy preferences and Arrow-type problems," Social Choice and Welfare, 11, 1, pp. 121-130, 1994. [5] A. Banerjee, "Fuzzy choice functions, revealed preference and rationality," Fuzzy Sets and Systems, 70, pp. 31-43, 1995. [6] R. E. Bellman and L. A. Zadeh, “Decision making in a fuzzy environment,” Management Science, 17, 4, pp. 141-164, 1970. [7] I. Benbasat and L. Lim, “The effects of group, task, context and technology variables on the usefulness of group support systems,” Small Group Research, 24, 4, pp. 430-462, 1993.

30

[8] J. B. Biggs and K. F. Collis, Evaluating the Quality of Learning: The SOLO Taxonomy. New York: Longman, 1982. [9] G. Bordogna, M. Fedrizzi and G. Pasi, “A linguistic modeling of consensus in group decision making based on OWA operators,” IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 27, 1, pp. 126132, Jan. 1997 [10] S. Brandt and M. Lonsdale, “Technology-supported cooperative learning in secondary education,” Proceedings of the Twenty-eighth Hawaii International Conference on Systems Science, pp. 533-542, 1996. [11] T. Bui, “Building effective multiple criteria decision models: A decision support system approach,” Systems, Objectives, Solution, 4, pp. 3-16, 1994. [12] J. Burns, J. Clift, and J. Duncan, “Understanding of understanding: implications for learning and teaching,” British Journal of Educational Psychology, 61, pp. 276-289, 1990 [13] S. J. Chen and C. L. Hwang, Fuzzy Multiple Attribute Decision Making: Methods and Applications. Berlin: Springer-Verlag, 1992. [14] T. J. Crowe and J. P. Nuno, “A fuzzy parametrical forward and backward chained two dimensional attribute decision model,” IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 27, 2, pp. 186194, March 1997 [15]

B.

Daly,

“The

influence

of

face-to-face

versus

computer-mediated

communication channels on collective induction,” Accounting, Management & Information Technology, 3, 1, pp. 1-22, 1993. [16] M. A. P. Davies, “A multicriteria decision model application for managing group decisions,” Journal of the Operational Research Society, 45, 1, pp. 47-58, 1994. 31

[17] A. L. Delbeq, A. H. Van de Ven and D. H. Gustafson, Group Techniques for Program Planning. Glenview, IL: Scott-Foresman, 1975. [18] A. R. Dennis, J. F. George, L. M. Jessup, J. F. Nunamaker and D. R. Vogel, “Information technology to support electronic meetings,” MIS Quarterly, 11, 4, pp. 591-624, December, 1988. [19] A. R. Dennis and R. B. Gallupe, “A history of group support systems empirical research: Lessons learned and future directions,” in L. Jessup and J. S. Valacich, Group Support Systems. New York: Macmillan, pp. 192-213, 1993. [20] G. DeSanctis and R. B. Gallupe, “A Foundation for the study of group support systems”, Management Science, 33, 5, pp. 589-609, 1987. [21] G. Dickson, J. E. Lee-Partridge, M. Limayem and G. L. Desanctis G.L, “Facilitating computer-supported meetings: A cumulative analysis in a multiplecriteria task environment,” Group Decision and Negotiation, 5, pp. 51-72, 1996. [22] M. Diehl and W. Stroebe, “Productivity loss in brainstorming groups: Toward the solution of a riddle,” Journal of Personality and Social Psychology, 53, 3, pp. 497-509, 1987. [23] B. Dutta, "Fuzzy preferences and social choice," Mathematical Social Science, 13, pp. 215-229, 1987. [24] G. W. Evans, “An overview for techniques for solving multiobjective mathematical programs,” Management Science, 30, pp. 1268-1282, 1984. [25] M. Fedrizzi, J. Kacprzyk and S. Zadrozny, “An interactive multi-user decision support system for consensus reaching processes using fuzzy logic with linguistic quantifiers,” Decision Support Systems, 4, pp. 313-327, 1988.

32

[26] P. C. Fishburn, "Multiperson decision making: a selective review," in J. Kacprzyk, and M. Fedrizzi, Eds., Multiperson Decision Making Models Using Fuzzy Sets and Possibility Theory. Dordrecht, The Netherlands: Kluwer, 1988. [27] J. H. Fjermestad and S. R. Hiltz, “Experimental studies of group decision support systems: An assessment of variables studies and methodology,” Proceedings of the Thirtieth Hawaii International Conference on System Sciences, 2, pp. 45-53, 1997. [28] R. B. Gallupe, L. M. Bastianutti and W. H. Cooper, “Unblocking brainstorms”, Journal of Applied Psychology, 76, 1, pp. 137-142, 1991. [29] R. B. Gallupe, A. R. Dennis, W. H. Cooper, J. S. Valacich, L. M. Bastianutti and J. F. Nunamaker, “Electronic brainstorming and group size”, Academy of Management Journal, 35, pp. 350-369, 1992. [30] J. George, G. Easton, J. F. Nunamaker and G. Northcraft, “A study of collaborative group work with and without computer-based support,” Information Systems Research, 1, 4, pp. 394-415, 1990. [31] S. G. Green and T. D. Taber, “The effects of three social decision schemes on decision group process,” Organizational Behavior and Human Performance, 25, pp. 97-106, 1980. [32] J. Hall and W. H. Watson, “The Effects of a normative intervention on group decision performance,” Human Relations, 23, pp. 299-317, 1970. [33] Z. X. Han, G. Feng and J. Ma, “Dynamic output feedback controller design for fuzzy systems,” Accepted for publication in the IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 1999

33

[34] S. R. Hiltz and M. Turoff, “Structuring computer-mediated communication systems to avoid information overload,” Communications of the ACM, 28, 7, pp. 680-689, 1985. [35] K. W. Hipel, K. J. Radford and L. Fang, “Multiple participant-multiple criteria decision making,” IEEE Transactions on Systems, Man, and Cybernetics, 23, 4, pp. 1184-1189, July/August 1993 [36] W. Huang, K.S. Raman, and K.K. Wei, “Effects of group support system and task type on social influences in small groups,” IEEE Transactions on Systems, Man, and Cybernetics – Part A: Systems and Humans, 27, 5, pp. 578-587, September, 1997. [37] W. Huang, K. K. Wei, C. Y. Tan and K. S. Raman, “Why does a GSS fail to enhance group consensus and satisfaction? An investigation from an influence process perspective,” Proceedings of the Twenty-ninth Hawaii International Conference on Systems Science, CDROM version, 1997. [38] C. L. Hwang and K. Yoon, Multiple Attribute Decision Making: Methods and Applications. New York: Springer-Verlag, 1981 [39] B. W. Imrie, “Assessment for learning: quality and taxonomies”, Assessment & Evaluation in Higher Education, 20, 2, pp. 175-189, 1995. [40] L. M. Jessup, T. Connolly and J. Galegher, “The effects of anonymity on GDSS group process with an idea-generating task”, MIS Quarterly, 14, pp. 312-321, 1990. [41] J. Kaeprzyk, “Supporting consensus reaching under fuzziness via ordered weighted averaging (OWA) operators,” Soft Computing in Intelligent Systems and Information Processing Proceeding of the 1996 Asian Fuzzy Systems Symposium, pp. 11-14, 1996. 34

[42] J. S. Kelly, Social Choice Theory: An Introduction.

Berlin: Springer-Verlag,

1988. [43] C. W. Kwok and M. Khalifa, “Effect of GSS on knowledge acquisition,” Information and Management, 34, 6, pp. 307-315, 1998. [44] C. W. Kwok and J. Ma, "Use of group support system for collaborative assessment", Computers & Education, 32, 2, pp. 109-125, 1999 [45] Y. I. Liou and M. Chen, “Using group support systems and joint application development

for

requirements

specification,”

Journal

of

Management

Information Systems, 10, 3, pp. 25-41, 1994. [46] J. Ma, “Group decision support systems for assessment of problem-based learning,” IEEE Transaction on Education, 39, 3, pp. 388-393, August 1996. [47] J. Ma, Z. Fan and L. H. Huang, “A subjective and objective integrated approach to determining attribute weights,” European Journal of Operational Research 112, pp. 397-404, 1999. [48] J. E. McGrath J.E, Groups: Interaction and Performance, New Jersey: Prentice Hall, 1984. [49] J. E. McGrath and A. B. Hollingshead, Groups Interacting with Technology. Thousand Oaks, CA: Sage Publications, 1994. [50] P. L. McLeod, “An Assessment of the experimental literature on electronic support of group work: Results of a meta-analysis,” Human-Computer Interaction, 7, 3, pp. 257-280, 1992. [51] P. L. McLeod and J. K. Liker, “Electronic meeting systems: Evidence from a low structure environment,” Information Systems Research, 3, 3, pp. 195-223, 1992.

35

[52] J. F. Nunamaker, A. R. Dennis, J. S. Valacich, D. R. Vogel and J. F. George, “Electronic meeting systems to support group work,” Communications of the ACM, 34, 7, pp. 41-60, July 1991. [53] G. Richardson, "The structure of fuzzy preferences: social choice implications," Social Choice and Welfare, 15, 3, pp. 359-369, 1998. [54] T. L. Saaty, Fundamentals of Decision Making and Priority Theory with the Analytic Hierarchy Process. Pittsburgh, PA: RWS Publications, 1994. [55] M. Shaw, Group Dynamics: The Psychology of Small Group Behavior. 3rd edition. New York: McGraw-Hill, 1981. [56] R. E. Slavin, Cooperative Learning: Theory, Research and Practice. Boston, Mass.: Allyn and Bacon, 1995. [57] D. M. Steiger, “Enhancing user understanding in a decision support system: A theoretical basis and framework,” Journal of Management Information Systems, 15, 2, pp. 199-220, 1998. [58] I. D. Steiner I.D, Group Process and Productivity. New York: Academic Press, 1972. [59] G. Stasser, L. A. Taylor and C. Hanna, “Information sampling in structured and unstructured discussions of three- and six-person groups,” Journal of Personality and Social Psychology, 57, pp. 67-78, 1989. [60] M. Tan, “Establishing mutual understanding in systems design: An empirical study,” Journal of Management Information Systems, 10, 4, pp. 159-182, 1994. [61] M. Tong and P. P. Bonissone, “A linguistic approach to decision making with fuzzy sets,” IEEE Transactions on Systems, Man, and Cybernetics, 10, pp. 716723, 1980

36

[62] E. Turban, Decision Support and Expert Systems. London: Prentice Hall, 1995, 336-373. [63] J. B. Walther and J. K. Burgoon, “Relational communication in computermediated interaction,” Human Communication Research, 19, 1, pp. 50-88, 1992. [64] R. R. Yager, “On ordered weighted averaging aggregation operators in multicriteria decision making,” IEEE Transactions on Systems, Man, and Cybernetics, 18, pp. 183-190, 1988 [65] L. A. Zadeh, “Fuzzy Sets,” Information and Control, 8, pp. 338-353, 1965. [66] L. A. Zadeh, “Toward a theory of fuzzy information granulation and its centrality in human reasoning and fuzzy logic,” Fuzzy Sets and Systems, 90, 2, pp. 111127, 1997. [67] D. Zhou, J. Ma, Q. Tian and C. W. Kwok, “Fuzzy Group Decision Support System for Project Assessment,” Proceedings of The 32nd Hawaii International Conference on Systems Sciences, CDROM version, 1999 [68] H. J. Zimmermann, Fuzzy Sets Theory and Its Applications, Second Edition, Boston: Kluwer Academic, 1991.

37