Using GSSs to Support Error Detection in Software Specifications

Using GSSs to Support Error Detection in Software Specifications

Arno Vermunt, Martin Smits, Gert van der Pijl Tilburg University, School of Economics PO Box 90153, 5000 LE Tilburg, The Netherlands Phone: +31.13.4662188, Fax: +31.13.4663377 [email protected], [email protected], [email protected] Rini van Solingen Eindhoven University of Technology Eindhoven, The Netherlands [email protected]

Abstract Fagan inspections can be used to find errors in documents used during systems development. In the practice of Fagan inspections it has been found that Group Support Systems (GSSs) can significantly enhance error detection [1]. This paper describes our findings on the use of a GSS by Fagan inspection teams in an experimental set-up. In this study, 24 students and 24 managers participated; they looked for defects in a standardized four-page document. In the preparation phase the participants, searching individually without GSS support, found within one hour between 12 and 40 defects. Prior to the second (or ‘logging’) phase of the inspection we formed 16 groups, each consisting of three students or three managers. Eight groups were selected to work with GSS support, and eight groups without GSS but with a facilitator. We found that only 3 to 9 new defects were found in the second phase. The performance of the GSS groups did not differ significantly from the non-GSS groups, but the GSS participants evaluated their sessions significantly lower than the non-GSS participants.

1. Introduction Major problems in the development of information systems (IS) have been extensively reported: the IS development process takes too long and costs too much, and the quality of the resulting IS is too often not satisfactory [2,3,4,5]. Since information systems are of great importance to con-

temporary organizations, something must be done to solve these problems. Possible causes of these problems are reported to be (i) errors made during the software coding process, and (ii) insufficient or incorrect specification of user needs. When looking at the traditional IS development process, several phases can be distinguished in a systems lifecycle, such as systems analysis, systems design, programming, testing, conversion, and production and maintenance [5]. Although testing is identified as a phase on its own, testing is an important activity throughout the complete development process, and must not be restricted to one phase only. Checking of intermediate results during the process, allows immediate adjustments in case of perceived errors. This contributes to a decrease of process length and costs, and to better IS quality. After all, when an error is detected early, its impact will be less since it is easier and less expensive to correct it without having to go back several phases in the process [2]. Errors in software specifications, software code, and other intermediate results in a software development process can be detected by performing quality control. The Fagan inspection is a well known type of quality control. Originally a Fagan inspection was a group process without the use of specific software tools. But in recent years Group Support Systems (GSSs) have been introduced into software development, and into Fagan inspections. Van Genuchten et al.[1] report on the use of a GSS in 14 electronic inspections in two software firms in the Netherlands, and observe a considerable gain in the number of defects found in software code. Reports on the use of GSSs for inspections are generally derived from the practice of systems development.

1060-3425/98 $10.00 (C) 1998 IEEE

Such field or case studies are of the type ‘comparing notusing-GSS-in-the-past with using-GSS-in-the-present’. By definition, in this type of research no clear distinction can be made between effects of changing the old process into the new one, versus effects of the use of GSSs. The aim of this paper is to expand the knowledge of the role of GSSs in systems development (like [6]). More specifically, we study the GSS-effects on the performance of Fagan inspection teams. We report the results of two ‘laboratory’ experiments with Fagan inspections. We used one document as the object of inspection, and had it inspected by sixteen groups with or without the support of GSSs. In this way we studied the effects of using GSSs in Fagan inspections. All differences found between the groups with and without GSS support can in principle be attributed to a single variable: use of GSSs. We first give a short description of Fagan inspections (section 2), and GSSs (section 3). In section 4 and 5 the rationale and design of the experiments is described; section 6 contains our findings; and we conclude with a discussion, and conclusions and recommendations in sections 7 and 8.

2. Fagan inspections A Fagan inspection is a structured review process, developed by Michael Fagan at IBM in the 1970s, based on the basic ideas on statistical quality control of Deming [7] and Juran [8], applied in information systems development (e.g., [9]). Inspections are reviews whose objective is program defect testing [3]. Fagan [10] reported that more than 60% of the errors in a computer program can be detected by using informal program inspections. Fagan’s inspection can be used to inspect any item produced during the software project life cycle. In fact, there is no reason why it should be confined to software [11]. A Fagan inspection is organized by a moderator (facilitator), who receives (from the author) a copy of a document to be inspected (this is called the low-level document), and a copy of its source (the high-level document). The moderator forms an inspection panel of no more than five people, including the moderator. The inspection panel first goes

through a ‘kick-off’ meeting (for about 30 minutes), defining the objectives of the inspection. After this meeting, each inspector studies the material individually, and makes a list of the defects he/she finds (typically taking about 15 minutes for each page to be inspected). The inspection panel then gets together for a ‘defect-logging meeting’, in which every defect detected (both on one’s own and during the meeting itself) is logged. The ‘logging meeting’ is relatively short (not more than two hours; taking about 15 minutes per page to be inspected), and is exclusively concerned with identifying defects in the low-level document, anomalies, and non-compliance with standards. The communication between the panel members must be limited to the reporting of defects, with neither discussion, nor recommendations for improvement of the document. Sometimes the logging meeting is followed by a discussion meeting (looking for possible solutions of defects), and a causal analysis meeting (looking for the causes of the defects) [12]. Finally the list of defects is handed to the author for rework. The moderator checks that, for each defect, some remedial action is taken [11]. In summary, the following roles are distinguished in an inspection process [3]: & author/owner, who will use the defects found to improve his/her program or document; & inspector, whose task is to find errors, omissions, and inconsistencies; & scribe, who records the defects reported; & chairman/moderator, who manages the process, facilitates the inspection, updates checklists, and develops standards to improve the inspection process. The Fagan inspection method is based on (i) the assumption that it is easier to detect errors than to check whether everything is correct, and (ii) static verification, meaning that each error can be considered in isolation [3]. The performance of a Fagan meeting is measured quantitatively, in terms of the number of defects found. Effectiveness is defined as the number of defects found per inspected page. Efficiency is defined as the number of defects found per person hour invested in the inspection. The yield of a meeting is defined as the percentage of the defects found in that meeting with

Table 1. Variety in Fagan inspections in practice (as reported in Doolan [11]), compared with the standardized inspection set-up in the present experiments Aspects of a Fagan inspection Number of inspected pages in an inspection Number of pages studied in the preparation phase (per hour) Number of pages covered in the logging phase (per hour) Time spent on logging the defects (per page, in minutes) Number of defects logged (per page) Total man hours invested in the inspections (per page)

Doolan [11]

Present experiments

3 - 126 1 - 19 1 - 30 2 - 48 1 - 17 1-9

4 4 4 10 - 15 8 - 20 1.5 - 2

1060-3425/98 $10.00 (C) 1998 IEEE

respect to the total number of defects known to be present [1]. It is generally accepted that fixing problems in released software can be as much as 80 times more expensive than fixing problems at the specification stage [11], and that the payback for every hour invested in inspection is a factor 30. Doolan [11] reports on 11 Fagan inspections, held in the IS practice at Shell Research. The inspections varied considerably, as shown in Table 1. In the present experiments we standardized the inspection process, according to the values listed in the last column.

3. Group support systems Group Support Systems (GSSs) are a set of methods, techniques, and (software) tools, to support groups handling complex problems or tasks [13,14]. A wide range of software packages have been developed in the past decade; for example, GroupSystems (Ventana), Decision Explorer (Banxia), Notes (IBM), and Powersim (AS). These GSSs provide new opportunities for groups to deal with complex tasks. Several types of GSSs can be distinguished, ranging from GSSs for groups that work together ‘at the same time, in the same place’ to GSSs for working together ‘at different times and different places’ [15]. In the present experiments we used the GroupSystems software to support groups of three or four people ‘same time, same place’. A GSS can support group activities in several ways. It is often used to improve the interaction and communication between group members during a team process, in order to reach deep insight in the problem at hand, to clarify the stakeholder positions, and to obtain better chances for shared understanding and a shared basis for implementation of decisions made [14,16]. A GSS can be used for several types of group tasks. McGrath [17] distinguishes eight types of tasks, based on task properties, such as conceptual versus behavioral tasks, and conflict versus cooperation tasks. Typical tasks in software development are ‘creativity and model building tasks’ (for instance, in the phases of requirements specification) and ‘performance tasks’ (for instance, finding errors during testing). Three modes of GSS use can be distinguished: (i) chauffeured: one group member uses a software package with central (large) screen projection (chauffeured is also called ‘single user group support’); (ii) supported: all group members use a software package simultaneously, and mix verbal and electronic communication; (iii) interactive: supported mode without verbal communication. In the present experiments we used GroupSystems in the supported mode. An important factor in the use of GSSs is the facilitation role, fulfilled by one or more persons who guide the group through the meeting. The role ranges from technical support to process guidance and content facilitation. The facilitator

can use the GSS technology to design a meeting, to prepare a meeting together with other participants, and to guide the group through the meeting agenda [18]. In this way the facilitator can have great influence on the group performance. The GroupSystems software aims to make meetings more effective and efficient. GroupSystems is based on the meeting model [19], in which the present state (with problems) changes into a desired state (with solutions) by performing actions (stepwise, following an agenda) and using resources (people, technology). The outcome of a (series of) meeting(s) depends on characteristics of the task, the group, and the technology used in the group process [20]. With respect to opportunities for improving the performance of Fagan inspections, some relevant features of GroupSystems are [19,20]: (i) the simultaneous input of data, (ii) the (relative) anonymity of the participants, (iii) the opportunities for structuring the meeting process by preparing an agenda, (iv) the electronic recording and display of the data entered by the participants, (v) extended information processing capacity, and (vi) reporting capabilities. It is expected that these features of GSS technology can enhance the effectiveness and efficiency of Fagan inspections, especially by improving the group processes during the logging meeting.

4. Rationale The present experiments fit in a series of (generic as well as specific) developments: (i) the continuously improving performance/price ratio of electronic meeting systems and GSSs provides more and more opportunities for practical applications, and (ii) the growing interests in applications of GSSs in the field of IS development in research and in practice [1,2,6]. The initiative to conduct these experiments was triggered by a paper at HICSS’97 by Van Genuchten et al.[1]. In that paper, GSSs are reported to improve inspection meetings in practice. It is peculiar however, that they found strong improvements in the preparation phase, i.e., in the phase before the actual use of the GSS (see Table 2). GSSs provide several opportunities for redesigning inspection processes [1,6]. Bostrom et al.[19] state that ‘one of the biggest benefits of the introduction of GSS is that it forces people to pay extra attention to meeting design’. What will be the effect on the performance of inspection teams, if the document that is to be inspected is presented electronically? What happens if the preparation and the logging phase are mixed? What are long-term results, after the group has become familiar with the new tool? What is the influence of the facilitator? Sengupta and Te’eni [21] demonstrated that a GSS can enhance the performance of a group by providing cognitive feedback to the group members performing a group task. Feedback can focus the attention of the group on specific details (or defects) found, thus providing opportunities for

1060-3425/98 $10.00 (C) 1998 IEEE

Table 2. Effectiveness (defects per page) and efficiency (defects per person hour) of 14 GSS supported Fagan inspections, compared with historical data on inspections without GSS support (based on [1]). Non-GSS inspections

GSS inspections

Variables of interest Majors

Minors

Total

Majors

Minors

Total

Effectiveness: preparation phase Effectiveness: logging phase

0.39 0.03

2.88 0.11

3.27 0.14

2.27 0.81

6.90 3.00

9.17 3.81

Efficiency: preparation phase Efficiency: logging phase

0.81 0.09

6.34 0.49

7.15 0.58

2.70 1.45

8.31 3.22

11.01 4.67

enlightening and solving cognitive conflicts. Amason [22] mentions cognitive conflict and emotional conflict as factors influencing group performance. Anonymity in the exchange of information might provide opportunities to avoid emotional conflicts among group members. In the present experiments we used relatively homogeneous groups, and did not expect many emotional conflicts with respect to the Fagan inspection task, and therefore not much effect of GSS use. However, in our experiments we might find effects of the GSS with respect to solving cognitive conflict, because the GSS provides group focus on the defects logged. In the present experiments we used GroupSystems exclusively in the logging phase, without changing the inspection process in the individual phase. Based on previous findings [1], we expected improved effectiveness and efficiency in the logging phase. More reliable conclusions on the use of GSSs in inspections can be drawn, if consistent results are obtained from different kinds of research methods [17,23,24]. In GSSresearch, these methods are laboratory experiments and field/case studies. Each of these types of research has it own contribution. Field studies, such as those reported by Van Genuchten et al.[1], have the advantage of describing a phenomenon in the real world. However, it is very difficult to control all variables in real world experiments. Therefore it is better to combine field studies with laboratory experiments. The price paid in many laboratory experiments is the level of experience of the participants: experienced practitioners are not available in this type of experiment. Therefore many experiments (including our case) use volunteers participating in training programs. Thus there is a loss of comparability with the real life situation, which in turn might be a variable influencing the outcome of the experiments. The experiments described in this paper follow an experiment conducted at Eindhoven University several months earlier [25]. The latter experiment did not confirm the findings of Van Genuchten et al.[1], i.e., it was found that groups without GSS support seemed to perform better than groups with GSS support. With Van Genuchten and Van

Solingen [25], we think that one of the main explanations for these findings could be the high amount of errors in the document that was inspected (the groups found about 250 errors in four pages). Therefore we used an improved document that resulted after removing approximately 50% of the errors from the original document. As much as possible, the other variables in the experiments were kept the same as in the experiment described in [25].

5. Design of experiment We conducted two similar experiments, both with 24 participants. In the first experiment the participants were second-year students (age approximately 22) in MIS and Business Economics at Tilburg University. The students participated in the experiment on a voluntary basis for two reasons: (i) to get some hands-on experience in software inspections, and/or (ii) because this experiment fitted well in their study program (the experiment provided them with some study credit points). In the second experiment, the participants were managers (age approximately 35), also volunteering to participate. All these managers are part-time Master’s students (MBA, Controlling or EDP auditing) at Tilburg University. All 48 participants had good basic knowledge of IS and IS development, but had no previous experience with either Fagan inspections, or GroupSystems. For the experiment we used a four-page text document (the low-level document), describing the flows of materials and data in an anonymous petro-chemical factory. The highlevel document consisted of one page of text and a figure describing the factory. A checklist of ten items and a list of nine standards were given to the participants as criteria for the inspection. Participants were also provided with forms to write down the defects they detected. Both experiments consisted of the following four steps, taking about three hours in total: 1. Kick-off meeting (30 minutes): a plenary lecture on Fagan inspections in general (since no participants had any experience in this task) and on specific instructions con-

1060-3425/98 $10.00 (C) 1998 IEEE

cerning the experiment; 2. Preparation phase (60 minutes individually): all participants inspected the low-level document for violations of the list of standards and the checklist; the participants recorded the defects detected on a ‘defect report form’. They did this manually, so without GSS support. Every inspector worked individually; there was no communication among the participants. After this phase eight groups were formed, each consisting of three inspectors; 3. Logging phase (60 minutes, groupwise): the groups reported the defects they found individually. Working through the document page by page, the defects were logged together in one ‘defect report form’. The reporting of defects is supposed to trigger participants to find new defects, which are added to the list; and 4. Evaluation phase (15 minutes): each participant answers a number of questions concerning his/her view and opinion on the experiment. After the preparation phase, groups were formed consisting of three persons each. Four groups proceeded with GSS support, whereas the other four groups proceeded without GSS support. To form the groups, we did not assign the individuals to the groups (and the treatments) at random, as this can lead to undesirable unbalanced situations. For instance, in the Fagan experiment at Eindhoven University [25], the individuals were assigned randomly to the groups, which unfortunately resulted in significant differences between the groups before entering the logging phase. We have tried to avoid such problems. Therefore, we first ranked the 24 individuals, according to the number of errors they found individually in the preparation phase, and then we assigned them to eight groups as indicated in Table 3. In this way we created eight relatively homogeneous groups, denoted A through H, meaning that all members of a group had found approximately the same number of errors during the preparation; that is, we deliberately formed ‘productive’ (e.g., G and H) and ‘less productive’ groups (e.g., A and B). In order to obtain comparable/similar groups for the GSS and non-GSS treatments we assigned the groups according to Table 3. The GSS procedure was as follows. The four groups were brought in one (big) electronic meeting room, and assigned to four sets of three networked PCs, each running GroupSystems (using the Topic Commenter tool). For each

page in the low-level document, one topic was created in the tool to report defects. The participants were asked to start typing the defects and inspect the document for up to 60 minutes. While typing, the tool provided each inspector with the defects found by his/her two group members simultaneously and anonymously. Verbal communication in the group was restricted to short comments and explanations of the defects listed. No in depth discussions were allowed. Only two technical facilitators were present for the support of all four groups, and to prevent groups entering discussions in stead of looking for defects. Thus, each GSS logging team consisted of 3.5 members: 3 inspectors and 0.5 facilitator. The GroupSystems software package guided the team in a stepwise, page by page mode through the document. All inspectors were first time users of the GroupSystems software. In the non-GSS procedure each group went through a ‘classic’ logging phase: during up to 60 minutes the three inspectors reported the defects verbally, while working through the document, guided by a moderator/facilitator. The defects were listed by a scribe. Thus, each non-GSS logging team consisted of 5 members (3 inspectors, 1 facilitator, and 1 scribe). The role of the facilitator can be described as (i) guarding the progress of the process, watching the time spent on each page, and (ii) stimulating the participants to find more defects. After the experiments, all error report forms were analyzed by counting the total numbers of distinct defects found individually and by the groups (per page and in the whole document). We also categorized the defects as ‘major’ or ‘minor’. A major is an error indicating a need for fundamental revision of the low-level text, including further analysis of the high-level document. A minor error can be a typo, bad use of language, or unclear phrasing. The (time consuming) categorizing was done double-blind by two of the four authors of this paper. For about 90% of the defects, both authors categorized identically. The other 10% needed brief discussion to end up with consensus on the category.

6. Results of experiments The findings of the experiments are presented in Table 4, showing the numbers of defects found by the GSS suppor-

Table 3. Assignment of participants to groups (and treatment). Participants are ranked on number of defects found in preparation phase (1 = lowest number of defects found). Group

A

B

C

D

E

F

G

H

Treatment

GSS

non GSS

non GSS

GSS

GSS

non GSS

non GSS

GSS

Participant

145

236

7 10 11

8 9 12

13 16 17

14 15 18

19 22 23

20 21 24

1060-3425/98 $10.00 (C) 1998 IEEE

Table 4. The average total number of defects, and the average number of majors and minors found during the preparation and logging phase, with and without GSS-support. The numbers between brackets denote the standard deviations. Experiment 1 (24 students)

Net total errors

Preparation phase (8 ‘groups’)

32 (5.3)

Experiment 2 (24 managers)

Minors

Losses

11 (4.1)

21 (1.4)

n.a.

+ 9 (5.9)

+ 2 (3.0)

+ 7 (3.0)

1.75 (1.8)

+ 4.5 (5.0)

+ 2.5 (2.3)

+ 2 (4.5)

2.25 (1.5)

Logging phase: GSS (4 groups) Logging phase: non-GSS (4 groups)

Majors

Net total errors

Majors

Minors

Losses

Preparation phase (8 ‘groups’)

61 (15.8)

22 (4.6)

39 (13.2)

n.a.

Logging phase: GSS (4 groups)

+ 3 (6.5)

+ 1.5 (1.1)

+ 1.5 (5.7)

2.5 (2.6)

+ 7.5 (6.3)

+ 3.0 (4.1)

+ 4.5 (2.7)

2.75 (1.6)

Logging phase: non-GSS (4 groups)

Table 5. Overview of results (averaged over participants) of the questionnaire, collected immediately after the logging phase. Responses are on a five-point scale (1 = disagree/low score; 5 = agree/high score).

Students

Managers

Question/proposition: GSS (n=12)

non-GSS (n=12)

1. How do you rate the introductory lecture?

3.92

4.09

3.55

3.30

2. I found most defects in the preparation phase

3.83

3.82

4.09

3.90

3. I found the most important defects in the preparation phase

3.33

3.73

4.09

4.00

4. I found the preparation phase very efficient and effective

3.83

3.45

3.82

3.50

5. I found the logging phase very efficient and effective

2.67

4.00

1.82

3.40

ted and non-GSS supported groups, and Table 5 showing the opinions of the participants after they had completed the logging phase. The terminology in Table 4 is defined as follows: & preparation phase. The net total number of defects found in the low-level document (4 pages) after the preparation phase by the three group members. ‘Net’ means that a defect is counted only once, if it is found by more than one inspector (of the group). In these experiments the net number of defects was approximately 75% of the gross number of defects (the straight sum of the numbers of defects found by the individual inspectors); & logging phase: GSS/non-GSS. A positive value means that the logging phase resulted in extra errors found, compared to the preparation phase; & majors: the net total number of major defects found;

&

GSS (n=12)

non-GSS (n=12)

minors: the net total number of minor defects found; losses: defects that were found in the preparation phase, but not reported in the logging phase. The results of Table 4 can be summarized as follows: & a large number of defects (up to approximately 95% of all defects found) are found in the preparation phase. Only few new defects (between 5 and 9% of all defects) are found during the logging phase. This confirms findings in practice, in which the usefulness of the logging phase is often questioned [1]. & most defects found in the preparation phase were also reported in the logging phase. Only about 2.5 defects were ‘lost’. & the students found considerably (approximately 50%) fewer defects, major as well as minor defects, than the managers did. This might be due to different levels of &

1060-3425/98 $10.00 (C) 1998 IEEE

experience with respect to checking documents for errors. & GSS support worked relatively well for the student groups: on average, 9 new defects were found, instead of 4.5 in the non-GSS group. In contrast, the GSS support did not work well in the manager groups: the non-GSS groups found twice as many new defects as the GSS groups did. These differences, however, are not statistically significant. Table 5 shows an overview of the responses to the questionnaire, given by participants individually, immediately after the logging phase. It shows that the GSS groups rated the efficiency and effectiveness of the preparation phase higher, and the logging phase lower than the non-GSS groups. This indicates that the use of the GSS did not contribute to feelings of trust in the outcome of the meeting. These feelings, however, did not reflect the objective, quantitative results of the meetings, as Table 4 shows. Especially the managers had expected to see more benefits of the electronic system. After the experiment some managers voiced their disappointment. The student GSS groups seemed to be less critical and more enthousiastic when listing defects than their non-GSS counterparts. This could be a special effect of the role of the facilitator (a teacher) in the non-GSS groups, whereas the GSS groups did not have a full time facilitator. The number of defects in the document was reduced in comparison with the Eindhoven experiment [25]: about 100 defects were removed, so we expected to have a document with about 40 defects per page. However, new defects were detected: the participants found about 20 new defects in

each page. This suggests that no ‘absolute number of defects’ can be determined in text documents. This is in accordance with other authors, who report that the yield of an inspection is always a number that is relative to other groups or other procedures [26].

7. Discussion Figure 1 shows hypothetical relations between use of GSS and the numbers and types of defects found in a Fagan inspection. The use of GSS (1) can influence the role and attitude of the facilitator (3)(e.g., reduce the need for his or her presence), the Fagan working procedures (4)(might be redesigned into a shorter preparation phase, for example), the documents to be inspected (5)(these might be presented in electronic format; maybe larger, maybe more frequent inspections in the systems development process). Also, the individual attitudes and experiences of the inspectors and other Fagan inspection personnel (2) influence the number and types of defects found after the preparation phase (6), and the group’s attitude towards the task (7), ultimately affecting the errors found after the logging phase (8). In this perspective, Adaptive Structuration Theory (AST) is of interest [27]. AST describes the dynamic interaction between a group and its environment. Attitudes of groups change over time, as does the way groups make use of technology [28]. In the present experiments the independent variables were (1), (3), and (4). The dependent variable was (8), and the controlled variables were (2), (5), (6), and (7). We used GSSs to slightly change the group facilitation (3)(0.5 versus

3. facilitation 1. use of GSS/non GSS in a Fagan Inspection

4. working procedures

5. document 2. individual attitudes & experiences

8. # defects after logging

6. defects after preparation 7. group attitudes & composition

Figure 1. Factors influencing numbers and types of defects found in Fagan inspections.

1060-3425/98 $10.00 (C) 1998 IEEE

2 people present as facilitator/scribe), and the working procedures in a Fagan meeting (4)(verbal mode in the non-GSS groups versus the electronic mode {namely GroupSystems’ Topic Commenter tool} in the GSS groups, only during the logging phase), and looked for changes in the numbers and types of defects (8). The other factors were held constant in the experiment: standardized document (5), the numbers of defects after the preparation phase (6)(by selectively grouping the participants according to the numbers of errors found), the attitude and experience of the participants (2), and the group attitudes and composition (7)(e.g., no authors were part of the groups). In the present experiments we used relatively inexperienced inspectors, facilitators, and scribes. To compensate this feature, we standardized the document, and we used a document that fitted the knowledge and experience of all inspectors, facilitators, and scribes.

8. Conclusions and recommendations We conducted two experiments to investigate the effects of GSSs on Fagan inspections. More specifically, we used a GSS to support the logging phase of inspections; we expected improved performance. We found that only 3 to 9 new defects were found during the logging phase (a gain of about 5%), without statistically significant differences between the groups with and without GSS support. We also asked the participants what they thought of the efficiency and the effectiveness of the logging phase. Although GSS groups showed similar performance as non-GSS groups, the former groups were less satisfied with the logging session than the latter groups. The relatively low results of the logging meetings of the manager groups might be explained by the relatively negative attitudes of these managers. From the experiments, we derive the following conclusions: & a facilitator and a scribe in the logging session can sometimes be replaced by 0.5 technical facilitator (and a GSS), which improves efficiency in the logging meeting, without losing session effectiveness; & more research must be done to investigate the impact of GSS technologies on Fagan inspections, and the possibilities of redesigning (Fagan) inspection procedures. The inspection process might be redesigned, based on the basic needs and goals of the process, given the opportunities provided by GSS technologies. When redesigning the inspection procedures or processes, we recommend to take the following into account: & the use of GroupSystems in the individual phase, probably remotely (participants log in from their normal working place) and possibly using the ‘private list’ option (participants cannot see each other’s input, in order not to influence them too early). GSSs might also be used during the kick-off meeting and in the discussion phase

with the author(s) of the inspected document; in the present experiments we had the logging phase follow almost immediately (about 15 minutes) the preparation phase, thus giving little time for reflections or other cognitive processes in the inspectors’ minds. The time between the phases can be extended; & we have chosen to reduce the facilitator role in the present experiment. Carmel et al.[6] found that in creative tasks (in JAD group sessions), the group facilitator role is more important than the technical facilitator role. This conclusion seems not to be fully supported by the findings in our experiments, since we found similar results from groups with technical facilitation and groups with a group facilitator. This might be explained by the Fagan inspection tasks being of another type (less creative), or the effectiveness of the logging meeting as such not being very high. The role of the facilitator on the performance of inspection teams needs further investigation. Basically, Fagan inspections consist of the individual search for defects in a document, followed by the logging of these defects in a small team, to provide the author of the document with suggestions for improvements. The issue is how to design the inspection team and the inspection process, making appropriate use of technical opportunities, in order to improve the system development process. We suggest further research into (i) the effects of using GSSs to support more experienced teams and facilitators in an experimental setup, and (ii) to experiment with alternative designs of test procedures, making better use of the opportunities provided by GSS technology. GSSs might be especially useful when discussion meetings and causal analysis meetings are included in the process. &

References [1] M. van Genuchten, W. Cornelissen, C. van Dijk, “Supporting inspections with an electronic meeting system”, Proceedings HICSS, January 1997. [2] W.S. Humphrey, Managing the software process, AddisonWesley, Reading, Mass., 1989. [3] I. Sommerville, Software engineering, Addison-Wesley, Harlow, England, 1995. [4] P. Loucopoulos, V. Karakostas, System requirements engineering, McGraw-Hill, London, 1995. [5] K.C. Laudon, J.P. Laudon, Management information systems: organization and technology, Prentice Hall, Upper Saddle River, New Jersey, 1996. [6] E. Carmel, J.F. George, J.F. Nunamaker, “Supporting joint application development (JAD) with electronic meeting systems: a field study”, in: J.I. DeGross, J.D. Becker, J.J. Elam (eds.), Proceedings of ICIS, December 1992, pp.223-232. [7] W.E. Deming, Quality, productivity, and competitive position, MIT Center for Advanced Engineering Studies, Boston, 1982. [8] J.M. Juran, Quality control handbook, McGraw-Hill, New York, 1974.

1060-3425/98 $10.00 (C) 1998 IEEE

[9] ISO 9000-3, Quality management and quality assurance standards, part 3: Guidelines for the application of ISO 991 to the development, supply, and maintainance of software, International Organization for Standardization, Geneva, 1991. [10] M.E. Fagan, “Advances in software inspections”, IEEE Transactions on Software Engineering, vol.12, no.7, July 1986, pp.744-751. [11] E.P. Doolan, “Experience with Fagan’s inspection method”, Software - Practice and Experience, vol.22, no.2, February 1992, pp.173-182. [12] M. Pol et al., Testen volgens Tmap (in dutch, Testing according to Tmap), Tuteijn Nolthenius, Den Bosch, 1995. [13] L.M. Jessup, J.S. Valacich (eds.), Group support systems: new perspectives, Macmillan, New York, 1993. [14] C. Eden, “On evaluating the performance of wide-band GDSSs”, European Journal of Operational Research, vol.81, no.2, 1995, pp.302-311. [15] A.R. Dennis, J.F. George, L.M. Jessup, J.F. Nunamaker, D.R. Vogel, “Information technology to support electronic meetings”, MIS Quarterly, vol.12, December 1988, pp.591624. [16] M.T. Smits, “Group decision making for messy problems: stakeholders, mental models and system dynamics”, Proceedings of ECIS, 1995, pp.683-694. [17] J. McGrath, Groups: interaction and performance, Prentice Hall, Englewood Cliffs, New Jersey, 1984. [18] F. Ackermann, C. Eden, “Issues in computer and non-computer supported GDSSs”, Decision Support Systems, vol.12, 1994, pp.381-390. [19] R.P. Bostrom, R. Anson: “The face-to-face electronic meeting: a tutorial”, in: R.P. Bostrom, R.T. Watson, S.T. Kinney (eds.), Computer augmented teamwork: a guided tour, Van

Nostrand Reinhold, New York, 1992. [20] J.F. Nunamaker, A.R. Dennis, J.S. Valacich, D.R. Vogel, J.F. George, “Electronic meeting systems to support group work”, Communications of the ACM, vol.34, no.7, July 1991, pp.40-61. [21] K. Sengupta, D. Te’eni, “Cognitive feedback in GDSS: improving control and convergence”, MIS Quarterly, vol.17, March 1993, pp.87-109. [22] A.C. Amason, “Distinguishing the effects of functional and dysfunctional conflict on strategic decision making: resolving a paradox for top-management teams”, Academy of Management Journal, vol.39, no.1, 1996, pp.123-148. [23] R. Galliers, “Choosing information systems research approaches”, in: R. Galliers (ed.), Information systems research: issues, methods and practical guidelines, Blackwell, Oxford, 1992, pp.144-162. [24] G. Walsham, “Interpretive case studies in IS research: nature and method”, Operational Research Society, 1995, pp.7481. [25] M. van Genuchten, R. van Solingen, “...”, paper accepted for HICSS ‘98. [26] A.A. Porter, L.G. Votta, V.R. Basili, “Comparing detection methods for software requirements inspections: a replicated experiment”, IEEE Transactions on Software Engineering, vol.21, no.6, June 1995, pp.563-575. [27] M.S. Poole, G. DeSanctis, “Understanding the use of group decisions support systems: the theory of adaptive structuration”, in: J.Fulk, C.Steinfeld (eds.), In organizations and communication technology, Sage, Beverly Hills, CA, 1990, pp.173-193. [28] L. Chidambaram, R.P. Bostrom, “Group development: a review and sysnthesis of development models”, accepted for publication in Group Decision and Negotiation, 1994.

1060-3425/98 $10.00 (C) 1998 IEEE