Sep 2, 1991 - We have developed a prototype Collaborative Software Inspection. (CSI) tool that allows geographically distributed participants to col- laborate ...
A Distributed Collaborative Software Inspection Tool: Design, Prototype, and Early Trial Janet Drake
Vahid Mashayekhi Wei-Tek Tsai
John Riedl
Computer Science Department University of Minnesota Minneapolis, MN September 2, 1991
Contents
1 Introduction
1.1 Previous work 1.2 Overview
4 : : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
2 Process 2.1 2.2 2.3 2.4
Goals of CSI Software Inspection Process CSI Meeting Model CSI Media
6 7
7
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : : :
7 8 8 9
We gratefully acknowledge the support of the National Science Foundation (grant number N SF=IRI ? 9015442) and the research funds of the Graduate School of the University of Minnesota.
1
3 Suite - Basic Support for CSI 3.1 Object Model 3.2 Components 3.3 Coupling
9
: : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
4 Design and Use of CSI
4.1 Synchronous vs asynchronous 4.2 Design of CSI 4.3 Components 4.3.1 Browser 4.3.2 Inspection Summary 4.3.3 Note-pad 4.3.4 Action List 4.3.5 Other objects 4.4 Coupling attributes
14 : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : :
5 Experience 6 Conclusion and Future Work A First Trial of CSI A.1 Description of CSI Test A.1.1 Day one A.1.2 Day two
: : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : :
B Target Material Objective Description Calculations Output Data
14 14 19 19 19 21 22 22 22
25 30 33
: : : : : : : : : : : : : : : : : : : : : : : : : :
B.1 B.2 B.3 B.4 B.5
13 13 13
33 33 33
35
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
C Software Inspection Meeting Model
35 35 36 37 37
38
List of Figures 1
Inspection process
: : : : : : : : : : : : : : : : : : : : : : : : :
2
10
2 3 4 5 6 7 8 9 10
An instance of Suite distributed environment Object Hierarchy Design of CSI Browser window Inspection summary window Note pad window Action list window History log window Eval criteria window
: : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
12 16 18 20 21 22 23 23 24
List of Tables 1 2 3
Inspection material Some Suite attributes Windows used in inspection process
: : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : : :
3
: : : : : : : : : : : : : : :
11 15 28
Abstract
Software inspection can be an e ective method for detecting faults throughout the software life cycle. Traditionally, software inspection has been done with all the participants present in a meeting room and the inspection done on paper forms and documents. This approach is expensive in terms of traveling, scheduling, human resources, premeeting preparation of material, and post-meeting recording of the inspection results. We propose a distributed collaborative software inspection, relaxing the requirement that all the participants be in the same room, providing automated support, and adding structure to the process. We have developed a prototype Collaborative Software Inspection (CSI) tool that allows geographically distributed participants to collaborate in the inspection and provides automated support for all inspection activities. In its trials, CSI has performed well and the participants have been able to use it comfortably to complete the inspection. This paper describes the inspection process, the CSI design and implementation, and our experience with CSI.
1 Introduction Software inspection is a successful method for detecting faults in documents and code produced in software development [MT90]. Barry Boehm includes software inspection in his list of the ten most important issues for improving the quality of software, saying, \Walkthroughs catch 60 percent of the errors (faults)." [Boe87] Inspection can be used on many types of software products including analysis documents, design documents, code, test plan, and test cases. Proposals, technical reports, and business reports can also bene t from inspection. Cost and time are the limiting factors for inspection. It is costly because many people are involved and each inspection covers only small portion of the product. The time required from technical sta and the problem of scheduling inspections can make inspection a bottle-neck in the software engineering process. Nevertheless, inspection has proven cost-ecient because faults are uncovered earlier in the software life cycle and are corrected before they can propagate to the next phase. \Finding and xing a software problem after 4
delivery is 100 times more expensive than nding and xing it during the requirements and early design phases."[Boe87] In software inspection a group of peers review a portion of the target material with the goal of improving the quality of the product. Peer review nds more faults than the producer alone can nd. When a group of peer work together and reaches consensus on the faults, the quality of the faults found improves. [FW90]. The objectives of inspection according to [Hum89] are: 1. \To nd errors (faults) at the earliest possible point in the development cycle. 2. To ensure that the appropriate parties technically agree on the work. 3. To verify that the work meets prede ned criteria. 4. To formally complete a technical task. 5. To provide data on the product and the inspection process" [Hum89] also points out secondary bene ts such as team building, product awareness, and development of skills. [CEG+ 87] describes face-to-face meetings as a \communication activities in which all participants are physically and simultaneously present". Bringing all participants together at the same time and space is expensive, but face-to-face meetings have been the only method for e ective collaboration. The addition of distributed collaborative meeting environments changes these constraints. A distributed collaborative environment completely relaxes the space constraint and provides structure to partially relax the time constraint. The bene ts of distributed collaborative software inspection are:
Distribution - People who are geographically distributed can participate in
the inspection. The participants can use a workstation at their desks or in their oces and meet with people in other cities. This reduces travel cost and increases the feasibility of inspections when the development team is not located in the same city. Meeting times do not depend upon scheduling a conference room. 5
On-line Capabilities - The current version of all material is accessible
on-line. The results of the inspection are created on-line and therefore no secondary data entry into permanent records is necessary. The inspection information is available for metric collection and the use of paper is decreased. Added Structure - Automated support adds structure and consistency to the software inspection process. [HSW91] says that a consistent and uniform review is a criteria for developing a high quality software process. Enforcing the structure of the inspection process results in repeatable and measurable results.
1.1 Previous work
Other projects have explored facets of the distributed meeting problem. [CEG+ 87] studied the theory of meetings and de nes the meeting types and classes. Project Nick focuses on small face-to-face meetings specializing in exploration activities such as brainstorming, de ning design structure, analyzing issues, and problem resolution. Meeting-aids include an electronic blackboard, interconnected PCs, and recording apparatus. The Colab experimental meeting room developed at Xerox PARC is designed for facilitating interaction in small working groups [SFDB+87]. The room is equipped with workstations linked together over a LAN, a large touch-sensitive screen, and a stand-up keyboard. Cognoter is a Colab tool used to prepare presentations collectively. It provides support for brainstorming, organizing, and evaluating. In general, Colab tools support the sharing and joint revision of information in a meeting. Projects such as Nick and Colab demonstrate that computer support facilitates group meetings. There are many desktop conferencing systems. For example, Mermaid is a multimedia desktop conferencing system [Sak90] . This project is concerned with network performance issues when dealing with multimedia. Mermaid is a general purpose distributed meeting tool using PCs, a LAN, and a PBX (telephone for sound). Mermaid supports a wide range of media including pointing, drawing, sound, and graphics. Mermaid demonstrates that distributed systems for desktop conferencing can provide sucient performance for practical oce use. [Ste89] on the Advanced Learning Technology Project at the Software Engineering Institute used the software inspection process for experiment6
ing with educational techniques. They created a computer interface that allows a student to participate in a simulated software inspection as part of a training course. Using Digital Video Interactive (DVI), hypertext, hypermedia, and an intelligent tutoring system the computer simulation presents the inspection material and plays the roles of the other team members. This system demonstrates the bene ts of a structured tool for learning software inspection skills. Our project di ers from the general meeting tools because we are examining a speci c, well-structured meeting application. Our tool is speci cally structured for software inspection, facilitating e ective use of team resources for fault nding and analysis. Our work also di ers from [Ste89]. Their system provides an enhanced learning environment for individual students. Our tool provides a distributed structured environment for actually performing inspections. We do plan to build on [Ste89] in that the structure of our tool should help team members learn e ective inspection. The study of behavioral aspects is out of the scope of our project.
1.2 Overview
In this paper, we describe the inspection process, brie y introduce Suite (a UNIX-based toolkit used to develop our Collaborative Software Inspection (CSI) tool), describe CSI, and present the results of one of our trials. The appendices supply the target material used for the trial, the software inspection method used in the trial, and additional detail on the meeting model.
2 Process
2.1 Goals of CSI
The goal of the Collaborative Software Inspection (CSI) project is to determine if software inspection can be as e ective in a distributed environment as in a face-to-face meeting. The measure of the success of an inspection is the number, type, and the quality of faults detected. We are currently establishing an adequate and e ective inspection tool. In future experiments, the e ectiveness of fault detection in manual inspections will be compared with distributed collaborative inspections. 7
2.2 Software Inspection Process
There are several di erent type of meeting use for review in software development. They di er in both structure and goals. Management review, technical review, and software inspection (also known as walkthrough) are all types of reviews. Management reviews judge progress and proper use of resources. Technical reviews are high-level reviews by technical leaders to ensure that the technical decisions are represented consistently. Software inspection is the category of review we are concerned with. Software inspection is a detailed technical review of a small amount of material by technically competent peers with the goal of detecting faults. [You89] and [Hum89] have both developed widely used methods for inspection. In both approaches the team members have speci c duties, the target material is the same, the participants prepare for the inspection, and the inspection results are action items and the inspection status report. Yourdon treats the preparation for inspection di erently than Humphrey. In Yourdon's approach, the reviewers read the target material and informally note faults and concerns. Reviewers are also encouraged to note positive aspects of the target material. In Humphrey's method, the reviewers each create a fault list and give this list to the producer before the meeting. The producer correlates the faults and prepares to address the faults in the inspection meeting. Humphrey also add an optional introductory meeting to his model. This introductory meeting is used to introduce unusual material or inspection criteria. We follow Humphrey's model because the it is more structured and lends itself better to computer support. It also provides more metrics because of the individual fault lists and correlated fault list.
2.3 CSI Meeting Model
In the next sections we introduce the software inspection model, requirements of this model, and the activities and products that are supported by CSI. Software inspection has a well-de ned model. The inspection team is a group of peers with the technical knowledge required for detailed inspection. The quantity of target material addressed in one inspection is small because the level of review is detailed. The participants play speci c roles: moderator, recorder, producer, and reviewers. The four phases of software inspection are planning the inspection, preparation for inspection, the inspection meeting, 8
and post inspection clean-up. The roles of the participants and the phases of inspection are described in Appendix C. Figure 1 shows the phases of inspection and the activities that occur in each phase. Both synchronous and asynchronous activities occur and some activities can be either. For example, if the target material has multiple producers, the creation of the correlated fault list may be a collaborative e ort.
2.4 CSI Media
The material used in inspection are target material, inspection criteria lists, fault lists, an action item list, and a status report. The target material is an analysis product, design product, or code product. For analysis and design the target material may be a data ow diagram, entity relationship model, state transition diagram, object model, or a textual speci cation. The goal of inspection is to nd faults so they can be corrected later. Therefore the target material is only read, but never updated during inspection. The inspection criteria list is used as reference material to help identify faults. The fault lists, action item list, and status report are created in the inspection process. The reviewers use the report forms for their initial fault lists. The action item list and status report are used by the recorder. Table 1 shows the activities that occur in each phase, the material required, the use of the material, the collaborative requirement of the activity, and the participants in the activity.
3 Suite - Basic Support for CSI
We used Suite to develop our application. Therefore, before describing CSI, we will give a brief overview of Suite. Suite is a software system designed and developed at Purdue University by Prasun Dewan and his research team [Dew90]. Suite supports remote procedure calls (RPC), active persistent data, and management of \direct manipulation" user interfaces. A prototype of Suite has been implemented on top of UNIX, TCP/IP, NFS, and X [DV90].
9
Activities in Inspection Phases
Phases of Software Inspection Producer and Manager Ready
Initiate Inspection Identify Moderator
PLAN
Identify Reviewers and Recorder - agree upon reviewers - invite participants Identify Inspecton Criteria
INSPECTION
Participants Ready
Introductory Meeting - introduction or tuttorial Provide Inspection Material Individual Review - list faults and ambiguities - submit fault list Correlate Faults
PREPARE FOR INSPECTION Producer Ready (correlated faults)
Inspection Meeting Discuss Correlated Faults Get Consensus on Fault Uncover Faults Through Discussion Record Action Items
CONDUCT INSPECTION
Repeat Inspection
Inspection Meeting Closed Action Items Complete
Determine Status of Inspcetion
Close Inspection
POST INSPECTION
Review Status of Inspection Update Records and Reports
Inspection Complete
Figure 1: Inspection process
10
Material Target Material Inspection Criteria
Role
Moderator Producer Faults Reviewer Correlated Faults
Inspection Status
Process Initail Inspection
Recorder
Action Items I
I C
Timing Synchronous Asynchronous
X X
X
Introductory Meeting
R R U
Individual Reviews
R R U C
Correlate Faults
R A U R C
Inspection Meeting
R A U A R C
X X X X
X
Close Meeting
R A U A A C
X X
X X
Process Use of Material:
X X X X
X X
X
X X
A - available C - create I - initialize R - read U - update
Table 1: Inspection material
11
Purdue University Browser Public Note Honeywell, Minneapolis Browser Public Note
Dialogue Mgr Browser Dialogue Mgr Public Note
Dialogue Mgr Browser Dialogue Mgr Public Note
TCP/IP
Browser
Possible Additional Sites
Internet
University of Minnesota Browser Public Note
Public Note
Dialogue Mgr Browser Dialogue Mgr Public Note
Possible Additonal Workstations
UNIX/X Workstation
Figure 2: An instance of Suite distributed environment
12
3.1 Object Model
Suite object model is an extension of UNIX allowing distributed, shared, protected, and persistent objects (see gure 2). Suite objects coexist with existing components of the operating system, access conventional resources, and can be invoked by conventional processes [DC91]. Suite has an object compiler that takes object programs (C is currently implemented), scans and interprets the speci cation of an object module in the object program, and generates an interface le and RPC stubs. Similarly, a dialogue manager compiler compiles the object program and generates object editors that allow the interaction between the user and the object.
3.2 Components
The components of Suite are:
RPC - Suite RPC allows for applications executing in di erent address
spaces and possibly on di erent hosts to name and communicate with each other by calling high-level remote procedures. Persistence - Suite objects are persistent in that their data structures can be checkpointed onto disk and later restored to memory. User Interface - The Suite user interface treats all objects as data that can be edited. Interaction with \editable objects" is made possible by dialogue managers (DMs). DMs display a presentation of selected data structures, allow users to edit the presentation in a syntactically and semantically consistent fashion, and communicate these changes with the object. The object in turn ensures that the other displays also update their values.
3.3 Coupling
An important feature of Suite is that it allows exible coupling among the windows[DC91]. Flexible coupling allows users to choose an appropriate degree of collaboration with other users, specifying when changes to the shared data structures are transmitted. Users can also choose what degree of correctness (i.e., raw, parsed, validated, committed) is required before 13
transmitting or receiving takes place. Users can also couple properties of the shared windows such as the scroll bars, the cursor, or the key pressed (table 2 lists some Suite coupling attributes).
4 Design and Use of CSI
4.1 Synchronous vs asynchronous
Collaborative software is often categorized by the two dimensions of space and time [NDV+91]. The four meeting types in this category are:
same-time-same-place - This meeting type happens in the a single meet-
ing room of an organization, with the simultaneous participation of all involved. same-time-di erent-place - This meeting type happens in di erent meeting rooms of an organization, with the simultaneous participation of all involved. di erent-time-same-place - This type happens in a single meeting room with the participants attending at a time of their own choosing. di erent-time-di erent-place - This type happens in di erent meeting rooms of an organization with the participants attending at a time of their own choosing. The rst two correspond to the synchronous activities of software inspection such as the discussion and categorization of faults, and making a decision about the product. The second two correspond to the asynchronous activities of software inspection such as the individual review of the target material for the purpose of listing and categorizing the faults, and integration of the lists into a single list by the producer.
4.2 Design of CSI
In this subsection we describe a hypothetical CSI session to demonstrate how the tool supports the synchronous and asynchronous activities of software inspection (Figures 3 and 4). 14
Attribute
ValueCoupled TransmitOn TransmitCorrectnes ListenOn ListenCorrectness ViewCoupled FormatCoupled ScrollCoupled ActionCoupled PositionCoupled SizeCoupled
Description
Determines whether changes made by one user should be transmitted to other users. Determines which operation on the data structures triggers transmission of changes to other displays. Determines how correct the changes must be before they can be transmitted. Same as its TransmitOn counterpart, but has to do with receiving messages. Same as its TransmitCorrectness counterpart, but has to do with receiving messages. Determines whether users see the same or di erent views of the data structures. Determines whether users share the formatting properties of the data structures. Determines whether scroll bars of the windows are coupled. Determines whether actions performed in a window for invoking commands are coupled. Determines whether positions of the windows are coupled. Determines whether the sizes of the window are coupled. Table 2: Some Suite attributes
15
History log
Note-pad
Record all activities
Add a note
Identify faults/Discuss faults
Inspection summary Make a decision about the document
Action-list Annotate the line containing the fault
Generate an action item
Enter general note
Add an action item
Criteria Annotation Criteria used by the participants
Annotate the line
Browser Target material
Inegrated fault list faults
Uses object Control
Figure 3: Object Hierarchy
16
Asynchronous - The asynchronous activities of software inspection include
the distribution of target material and the creation of reviewer's fault lists. The target material is on-line and accessible by reviewers who can use the browser to view it and annotate the lines of the target material appropriately. These annotations help the producer in the creation of the integrated fault list that will be used during the meeting. Synchronous - The synchronous activities of software inspection include discussion of the correlated faults, reaching a consensus on the faults, recording the action items, and nally determining the status of inspection. The target material is brought on-line in a window on the screen of all the participants. Associated with each line of the document, there is a tag eld that allows the annotation of that line. Once the meeting gets underway, if there are issues that need to be addressed concerning a particular line of the document the participants can annotate that line by changing the value of the tag eld. Once the annotation tag eld is changed to true by a participant, a new window comes up allowing that participant to enter comments concerning the line in question. Any other participant who chooses to see or extend these comments can bring up the same window, or has the freedom to annotate another line of the document. The recorder can also enter action items into an action list that is on-line and visible to all the participants. Any general comments that do not correlate with a single word or line of the document and cannot be entered using the annotation windows, can be entered into a public note pad that is also on-line and accessible by all the participants. The recorder has a form that allows recording of information on the attendees, status of the meeting, and the nal decision made concerning the target material being inspected. For the convenience of the participants, the evaluation criteria are available on-line as reference material. The system activity (such as loading of les and objects, updating of data structures in objects, unloading of objects) is recorded automatically and is available for future statistical studies. Figure 4 depicts CSI activities.
17
Annotation window Note on line two
Public note pad notes on the session
Action list
Browser
1. action item 1 2. action item 2 . . .
1. line one 2. line two 3. line three . .
History log
Inspection summary
object loaded object updated object closed . . Criteria List of criteria Flow of control System Activity
Figure 4: Design of CSI
18
product inspected date participants . . meeting status
4.3 Components
The components of the CSI tool are all Suite objects (i.e., implemented in Suite environment and therefore have the common characteristics of persistence, user-interface, and collaboration). We introduce these components individually now:
4.3.1 Browser
The browser supports perusal and annotation of a document (Figure 5). The document may be speci cation, design, or code. In the prototype, the target material is a single le. Associated with each line of the le is a tag eld that is originally set to N (for no), meaning that the line does not have any comments associated with it. During the course of software inspection, participants may enter their comments for a particular line by changing the tag eld to Y (for yes). This will pop-up an annotation object window on the screen, which can be used to record the comments and concerns of the participants. The document itself is not modi ed. A link is made from the appropriate line of the document to the annotation object. The process of adding comments can be done synchronously (during the course of the meeting) or asynchronously (i.e., before the meeting takes place, so that the reviewers' comments are on-line). Most comments should be given to the producer before the meeting so an integrated fault list can be prepared.
4.3.2 Inspection Summary
The inspection summary object is the form used by the coordinator during software inspection (Figure 6). The coordinator lls out various elds such as the time and place of the meeting, the name of the target material being inspected, attendees' information, and the decision made on the module during the inspection process. For example, after the inspection, the coordinator asks the reviewers to make their recommendations concerning the target material (with a decision box). The possible decisions are: 1. The group may decide to accept the product in its present form. 2. The group may vote to accept it with the revisions and corrections suggested during the inspection. 19
Figure 5: Browser window
20
Figure 6: Inspection summary window 3. Another inspection may be scheduled to make sure new faults are not introduced during the present fault correction. 4. The product must be thrown away because of the high density of faults. 5. Review is incomplete. Initially this window is only visible to the coordinator and not the other participants. At the end of the meeting all participants can bring this window up on their screens to sign the form.
4.3.3 Note-pad
The note-pad supports the recording of annotations for the document being inspected (Figure 7). It consists of two parts: the template part for entering a new note, and the committed part for all the notes that have been entered thus far. 21
Figure 7: Note pad window
4.3.4 Action List
An action list is a detailed list of faults and suggested changes to the product (Figure 8). Each comment has a disposition code indicating how the reviewers view the fault.
4.3.5 Other objects
There are two other objects in the prototype. The history log is used to record all the activities during the inspection (i.e., loading of an object, updates, and unloading of objects) (Figure 9). This information is time-stamped and can be used for metrics collection. The evaluation criteria object is used to display the evaluation criteria on-line as the participants need it (Figure 10).
4.4 Coupling attributes
We support the transmission of the changes made by one user to the other users for all the objects in CSI except for the public note pad. This decision supports a di erent level of collaboration between two users who are entering a note into the public note pad at the same time. Notes are transmitted only after they are committed, not while they are being composed. This allows for users to work independently. For the other objects, the collaboration is tighter and the users can see one another's work immediately. We choose this scheme for the other objects because we want the work done in the browser, 22
Figure 8: Action list window
Figure 9: History log window 23
Figure 10: Eval criteria window 24
action list, and the inspection summary to be visible to all the participants at the same time. This decision is due to the fact that these objects are mostly used in the synchronous type of meetings that require immediate propagation of the changes to all users. The attribute used to accomplish this scheme is the ValueCoupled attribute. This is an example of a propertycoupling attribute that determines whether some set of properties of a data structure is shared by its copies displayed to di erent users. Users can change the settings of these attributes if they prefer. We choose to communicate changes made in a display to the other users after they have been syntactically, and semantically checked, and the user has explicitly demanded their transmission. This is true in case of all the components of CSI. This choice is made because the users involved in CSI are peers and have the same task. Therefore, we choose for all the users to see the changes at the same time (i.e., certain level of correctness and explicit demand to transmit). If the users had di erent roles, we could have chosen to receive/transmit the changes at di erent times. The scroll bars are not coupled in any of the objects to allow individuals to freely visit other parts of the les without starting \scroll wars" among the users. The windows are not size and position coupled so that the users can feel comfortable with their choice of size and position on the screen. Our aim is to support individual choice and taste without compromising the objectives of the group inspection meeting.
5 Experience
This section describes one of our trials with CSI using a requirement speci cation for target material. The target material was a FORTRAN laboratory assignment (Appendix A). It was chosen since the team members were peers when dealing with this material (because of their experience, they would know the faults). The review team was made up of seven members. One author served as producer/moderator and another as CSI system administrator and recorder. Of these seven team members, four have substantial industrial experience, two are teaching assistants, and all are graduate students in computer science. The next seven sub-sections describe the lessons learned in our experiences with CSI with examples from the inspection of the FORTRAN labo25
ratory speci cation.
Automate as much as possible.
The inspection material should be used on-line for all phases of the inspection. The advantage is consistency of material because the same copy of the target material and inspection criteria are available to everyone. The on-line creation of the fault lists eliminates transcription time and errors. Paper is not necessary and not desirable in this process because modi cations on paper are time-consuming and dicult to share and merge. Both the collaborative and independent work should be done through the same interface. Initiating the inspection and creating individual reviews are asynchronous activities. The introductory meeting and inspection meeting are synchronous activities. CSI supports both asynchronous and synchronous activities. The creation of the correlated fault list may be either a collaborative activity or an individual one, depending upon the producer's preference. CSI provides the reviewer a complete work-space for the individual review. The individual fault list can be created on-line and the CSI forms assist by providing pop-up windows containing valid values for fault classi cation elds. A valid classi cation must be chosen by the reviewer before continuing. This enforces the structuring of the inspection process.
Faults occur at many granularities.
The granularity of faults found includes words, phrases, sentences, paragraphs, general comments, and missing material. We used criteria from [FW90] that stresses identifying ambiguities. The reviewers followed the criteria well and many word and phrase faults were found. The reviewers with industrial experience spotted \weak" wording. For example \something like" needs to be replaced by a stronger term such as \shall be". The inspection criteria should include the standards for the target material. In some military product speci cations, for example, the term \shall" means that the product is not acceptable without the speci ed item; the term \will" means that the customer would very much like 26
to see the item; and the term \may" means that the implementation of the item is at the developers' discretion. The Data Item Descriptions (DID) associated with [Dep85] are another example of documentation standards. The DIDs describe the ordering and subject matter of each document section. Standards address the consistent presentation of material rather than the quality and correctness of the material (such as [FW90] criteria). CSI provides the capability to present such reference material.
Tracking faults to target material is mandatory.
Tracking faults to the target material is a mandatory function of inspection. Without tracking, the inspection team wastes times nding the fault under discussion. Word, phrase, and sentence level faults must track to their location in the text. Paragraph level faults should track to the paragraph. General comments and faults of omission should track to the appropriate level: paragraph or target material. CSI supports tracking fault through line numbers in the target material. To better track the faults to the target material, we need a combination of the line number and the granularity of the faults.
Correlation of individual fault lists requires automated support.
Correlation of the individual reviewer's fault list by the producer is an important task. The correlated fault list must be accurate and address faults in order of their importance. Recognizing duplicate faults is not easy because no comments are exactly alike and faults can con ict, especially when they are at di erent granularities. Some proposed xes, (e.g., violate standards, spelling, and grammar) can be accepted without further review. The system must assist the producer in fault list correlation by providing the capability to merge the individual fault lists, sort the lists based upon location, granularity, and classi cation, and support manual reordering of the list items.
CSI is a prototype development. As described in sections 4 and 5, CSI is built upon Suite components. Suite supports the prototype CSI design well by providing a toolkit 27
Object
target material note for fault list note for correlated fault list inspection criteria action item list inspection status report history log
IM
x o o x o o x
IM : initial meeting IR : individual review FC : fault correlation I : inspection
IR
x x o x o o x
FC
x x x o x
I
x x x x x
x : necessary - : not necessary, but available o : not yet created
Table 3: Windows used in inspection process approach for creating and modifying objects. The objects are wellencapsulated and modifying one object does not a ect the other objects. As each object is created, we can test and exercise it. We used ve workstations in the FORTRAN lab exercise and this was the reviewers' rst hands-on experience with CSI. After a ve minute introduction to the keys, buttons, and windows, they were able to manipulate CSI. They managed to use CSI successfully for the inspection. CSI objects used at each phase. During a phase, some objects are essential, some have not yet been created, and some are available at the user's discretion. Table 3 shows the state of the objects in each phase. Because all the objects are persistent, they are available for metrics collection after the inspection is complete.
Inspection results were good. 28
The FORTRAN lab inspection took approximately 90 minutes including the introduction to CSI. We did not cover all the faults. One half were individually discussed and a consensus was reached. Another one third were covered generally. We recognized them as being similar to previous faults. The remaining faults were not covered because of the lack of time. There were three major faults in the speci cation of the FORTRAN lab. The lab was used the previous quarter and the producer had seen forty implementations of it. The exercise was to support long integers with arrays of integer. The major faults re ected in the student implementations were: 1. The description of over ow from addition was ambiguous. 2. The description of the input did not specify \positive integer" or \signed integer". 3. The output was not speci ed. Our reviewers caught these three major faults along with many others. The individual reviewers found 143 faults. The number of faults after correlation was 60, and the number of action items was 23. The nal status of the inspection was \Accepted with minor revisions". The team consensus was that the general idea of the lab was good, but the speci cation contained many ambiguities and that it would be acceptable after correcting the faults listed as action items.
29
6 Conclusion and Future Work We are working on enhancements to make CSI a more complete tool. We have learned that we should use CSI in all steps of the software inspection process, namely, the introductory meeting, the reviewer's individual inspection, and the group inspection. This approach assists the producer in creating the correlated fault list and helps in tracking faults to the target material. We have demonstrated the feasibility of a distributed collaborative software inspection tool. The tool supports collaboration of geographically distributed individuals in the inspection, providing on-line capability for recording the meeting status and proceedings, and collection of metrics. CSI provides a consistent structure for software inspection. Each CSI object has a wellde ned structure and supports semantic checking of its data structures. In future experiments we intend to take advantage of this consistency by gathering metrics to compare distributed collaborative inspection with traditional face-to-face inspection meetings. Our future work includes adding a teleconferencing tool called teleconf. This is a stand-alone application written in Suite that will allow for inclusion of spoken conversations in all of our applications. For instance, during a CSI session, participants will wish to discus potential faults and possible xes. Users speak into microphones attached to their workstations. The sound is digitized, transmitted on ethernet to other workstations, and played through their built-in speakers. teleconf will enhance any collaborative application by making participation more natural to its users. We are currently implementing oor control policies in teleconf to help the participants have more e ective discussions. We plan to further investigate the issues involved in distributed collaborative software inspection. We believe that issues such as concurrency control and access control are best understood through real world use. We also believe that software inspection will support quantitative evaluation of the distributed collaborative meeting model. We expect that the distributed collaborative meeting model will be the basis for many future applications.
30
References [Boe87]
B. Boehm. Industrial software metrics top 10 list. In IEEE Software, September 1987. [CEG+ 87] P. Cook, C. Ellia, M. Graf, G. Rein, and T. Smith. Project Nick: Meeting augmentation and analysis. ACM Transactions on Oce Information Systems, 5(2), April 1987. [DC91] P. Dewan and R. Choudhary. Flexible user interface coupling in a collaborative system. Proceedings of the ACM CHI's 91 Conference, April 1991. [Dep85] Department of Defense. Military Standard - Defense System Software Development - DOD-STD-2167A, July 1985. [Dew90] P. Dewan. A guide to Suite. Technical report, SERC-TR-60P, Software Engineering Research Center at Purdue Univesity, February 1990. [DV90] P. Dewan and E. Vasilik. An object model for conventional operating systems. Usenix Computing Systems, December 1990. [FW90] D. Freedman and G. Weinberg. Handbook of Walkthroughs, Inspections, and Technical Reviews. Dorset House, 1990. [HSW91] W.S. Humphrey, T.R. Snyder, and R.R. Willis. Software process improvement at Hughes aircraft. IEEE Software, 8(4), July 1991. [Hum89] W.S. Humphrey. Managing the Software Process. Addison Wesley, 1989. [MT90] J. Martin and W.T. Tsai. N-fold inspection: A requirements analysis technique. Communications of the ACM, February 1990. [NDV+91] J. Nunamaker, A. Dennis, J. Valacich, D. Vogel, and J. George. Electronic meeting systems to support group work. Communications of the ACM, July 1991. 31
[Sak90]
S. Sakata. Development and evaluation of an in-house multimedia desktop conference system. In IEEE Journal on Selected Areas in Communications, volume 8, April 1990. [SFDB+87] M. Ste k, G. Foster, K. Kahn D. Bobrow, S. Lanning, and L. Suchman. Beyond the chalkboard: Computer support for collaboration and problem solving in meetings. Communications of the ACM, January 1987. [Ste89] S.W. Stevens. Intelligent interactive video simulation of a code inspection. Communications of the ACM, July 1989. [You89] E. Yourdon. Structured Walkthrough. Prentice Hall, 1989.
32
APPENDICES
A First Trial of CSI
A.1 Description of CSI Test
The test was done over two consecutive days, two hours on each day. The participants were the moderator, producer, and reviewers.
A.1.1 Day one
1. Introductory meeting in a normal meeting room. 2. Describe the material to be inspected. 3. Hand out hard copy of target material, evaluation criteria, and fault forms. 4. Individual evaluation by reviewers. 5. Fill out fault/concern form. Number, classify, and give a short description of all the faults. 6. Turn in forms to producer This meeting took approximately two hours.
A.1.2 Day two
On this day we used the tool to perform Software Inspection. We used workstations with a maximum of two people per machine. The di erent activities of this meeting are: 1. Short introduction to the use of the CSI tool. 2. Fill out status report form, subject, attendance. 3. Address each item on the correlated fault list. Possible outcomes are to nullify the fault, generate an action item, or to relegate the fault to the bottom of list. 33
4. End meeting when all points are covered or when time runs out. 5. Complete status report by assigning a status to the review following which everyone signs the form. This meeting took approximately 90 minutes.
34
B Target Material NOTE: This lab writeup is written as a program speci cation. It is organized so that your program can just about follow the structure of the writeup. Be sure that your program does EXACTLY what the speci cation requires (nothing more, nothing less).
B.1 Objective
This lab introduces the use of simple (one-dimensional) FORTRAN arrays. It also illustrates the use of DO loops to process arrays.
B.2 Description
In any kind of programming language like FORTRAN, integer variables can be used to represent integer numbers and to perform operations. But when the number is too large, e.g. 1,000,000,000,000 , for a standard integer variable to represent it, we need some special method to store the number. We also need to design some algorithms to perform operations. This lab uses one-dimensional FORTRAN arrays to represent large integers. Each digit is stored in an element of the integer array. For example, the number 9346571 could be stored in a FORTRAN array called X as follows: INTEGER X(10) X(1) = 1 X(2) = 7 X(3) = 5 X(4) = 6 X(5) = 4 X(6) = 3 X(7) = 9 X(8) = 0 X(9) = 0 X(10)= 0 So the maximal number that can be represented in array X is 1010 ? 1. You will need two arrays to store numbers, called X, Y. Two other arrays are also needed to store the result. These should be declared large enough 35
to hold up to ten digit numbers. Use a PARAMETER called MAXLEN to hold this value. Use MAXLEN as necessary in your program to avoid scattering the \magic number" `10' throughout your code. Your program should prompt the user for the length of the number to be entered. If this value is less than one or greater than MAXLEN, print an error message and loop back to ask for the value again. Once you know the actual length for this number, prompt the user for each digit of the number X, start from the most signi cant digit. For example, to input number 35124 to X, your prompts should look something like: Dimension : 5 Digit 5 for X is : 3 Digit 4 for X is : 5 Digit 3 for X is : 1 Digit 2 for X is : 2 Digit 1 for X is : 4 Use the same format to input Y. After reading in two numbers, your program will calculate X + Y and X - Y, and print these values. Refer to the \Calculations" and \Output" sections below. Once your program has completed its calculations and output for the sum and di erence, it should ask if the user wants to carry out another set of calculations, and read a single character response. If the user responds with 'y' or 'Y', loop back to the beginning of the program; otherwise end the program (you need a condition-controlled loop). Note that since the calculations can possibly be executed several times in a single run of the program, you should arrange for appropriate variables to be initialized before each calculation. Do not rely on DATA statements to do this. You should use DO loops to control reading and printing the arrays, as well as controlling the calculations and the output of the SUM array.
B.3 Calculations
In addition to the arrays X, Y, SUM, and DIFF, you will also need an INTEGER variable CARRY for computing the sum, and a CHARACTER variable SIGN for the di erence. The sum of two numbers is computed as follows: 1. add X(I) and Y(I) and CARRY, put the result into SUM(I). 36
2. if SUM(I) is larger than 10, subtract SUM(I) by 10 and put 1 into CARRY; otherwise, put 0 into CARRY. 3. repeat 1 and 2 from least signi cant digit to the most signi cant digit. CARRY is set to 0 initially. If CARRY = 1 after the most signi cant digit has been calculated, that means an over ow has happened. You should print out a warning instead of the number. The subtraction is almost the same as the sum, except that you have to compare which number is bigger. Use the bigger one to subtract the smaller one and store the result in DIFF. Be sure to record a \-" sign if X is less than Y so that you can print it out later.
B.4 Output
Label all output appropriately. For each set of calculations, output the sum and di erence. For the sum, output an error message if there is an over ow. For the di erence, output the sign.
B.5 Data
Submit a single run of the program, using the following test data sets: Set 1 Dimension = 4; X = 3414, Y = 1587 Set 2 Dimension = 10; X = 1732564891, Y = 9338360199 Set 3 Dimension = 2; X = 23 , Y = 93 Set 4 Dimension = 20 Set 5 Dimension = 0
37
C Software Inspection Meeting Model The software inspection meeting model [Hum89] has well de ned roles for participants and phases. The roles are: Moderator - The moderator organizes and leads the inspection. The moderator is responsible to invite participants, schedule the meetings and meeting rooms, maintain order in the meeting, and ensure that the action list item are completed. Recorder - The recorder ensures that the issues, action items, and nal meeting status are properly recorder. Producer - The producer is the author of the target material. If a tutorial on the target material is necessary, the producer gives the tutorial. After the reviewers have completed their individual fault lists, the producer correlates these faults into a single correlated fault list. At the inspection meeting the producer addresses these faults and any other problems that are uncovered during discussion. Finally the producer is responsible for xing the faults that were uncovered by the inspection. Reviewer - The reviewers individually examine the target material and reports any faults or confusing points. In the inspection meeting, the reviewers should receive assurance that the faults they located are adequately addressed. They also discuss any other faults or questions uncovered in the meeting. There are four steps in the software inspection process [Hum89]. 1. Plan the inspection. In this phase the target product is identi ed and the participants are chosen and invited. If a tutorial is necessary or there are special concerns about the target material, the producer and moderator can decide to schedule an introductory meeting. The inspection meeting is scheduled and criteria for the inspection is chosen. 2. Preparation for inspection. There are four activities in this phase. These activities are sequential. (a) The introductory meeting is held. The producer gives tutorial or introduction to the target material. This meeting is optional. 38
(b) The target material and inspection criteria is distributed to the reviewers. (c) The reviewers examine the target material and create a faults list. The reports are given to the producer when the examinations are complete. (d) The producer uses the individual fault list to create a correlated fault list. 3. Inspection meeting. The producer and reviewers discuss the issues on the correlated fault list. When consensus is reached, required actions are recorded on an action item list. Any other points uncovered in the discussion are addressed and possibly become action items. Finally the status of the target material is established by consensus. 4. Post inspection activities. After the producer xes the problems and the action items are complete, the moderator re-evaluates the status of the target material. If another inspection session is required, the inspection process begins again. The inspection records and reports are completed.
39
