A computational framework for supporting software inspections ...

A Computational Framework for Supporting Software Inspections Marcos Kalinowski Systems Engineering and Computer Science Program – COPPE/UFRJ

Guilherme Horta Travassos Systems Engineering and Computer Science Program – COPPE/UFRJ

[email protected]

[email protected]

Abstract

improve the efficiency of inspectors in detecting defects, and guidelines supporting tasks of the inspection process that involve decision making. Some of those theories and techniques have been evaluated by empirical studies, and can be considered a body of knowledge in the software inspection field [5][6]. For instance, in [4], a reorganization of the software inspection process, based on results of empirical studies has been described. Moreover, results of a survey [7] show that although many software companies perform reviews they do it unsystematically and few knowledge about software inspections is used. Thus the full potential of reviews is seldom exploited. Many tool support proposals sprouted to address problems concerning software inspections [8], most of these tools focused on defect detection or on the inspection of specific artifact types. Recently, two tools were used in empirical studies and deserve particular attention, IBIS [9] and GRIP [10]. Both consider aspects of the inspection process reorganization proposed in [4]. However, analyzing research knowledge concerning software inspections, it seems possible to explore empirical evaluated information to offer more support to some decision points of the inspection process. For instance, knowledge described in [6][15][17] and [20] has not been used in any of these approaches. Based on this scenario, we present a computational framework for supporting the software inspection process, which can be used to inspect different artifacts produced throughout the software development life cycle. Many of the requirements for this framework are derived from knowledge acquired by empirical studies. There are two main expectations for this framework: (1) make more systematic reviews possible in practice, and (2) provide support for correct decision making during the software inspection process, obtaining more effective inspections. To evaluate the first of these expectations a case study was performed. The evaluation of the second expectation is a more complicated task, however, an empirical study to evaluate the support for the decisions involved in the planning activity was planned and conducted.

Software inspections improve software quality by the analysis of software artifacts, detecting their defects for removal before these artifacts are delivered to the following software life cycle activities. Some knowledge regarding software inspections have been acquired by empirical studies. However, we found no indication that computational support for the whole software inspection process using appropriately such knowledge is available. This paper describes a computational framework whose requirements set was derived from knowledge acquired by empirical studies to support software inspections. To evaluate the feasibility of such framework two studies have been accomplished: one case study, which has shown the feasibility of using the framework to support inspections, and an experimental study that evaluated the supported software inspection planning activity. Preliminary results of this experimental study suggested that unexperienced subjects are able to plan inspections with higher defect detection effectiveness, and in less time, when using this computational framework.

1. Introduction The cost of software defect correction increases exponentially throughout the software life cycle. Therefore, tasks should be scheduled to find defects as soon as possible. Software reviews have shown themselves as an efficient and low cost approach to find defects, reducing rework and improving product quality. Software inspection [1] is a particular type of review, with a rigorous and well-defined process. The importance of software inspections in improving software quality has been reported elsewhere [1][2][3][4]. Over the years, many theories and techniques regarding software inspections have been proposed, including alternative inspection processes, estimation techniques for number of defects in documents and defect detection effectiveness, reading techniques aiming to

Proceedings of the 19th International Conference on Automated Software Engineering (ASE’04) 1068-3062/04 $ 20.00 IEEE

The remainder of this paper is organized as follows. Section 2 presents some characteristics of the software inspection process and the framework’s proposal. Section 3 presents its architecture and current configuration at COPPE/UFRJ. After that, in section 4, the support provided by the framework to the software inspection process is shown in details. Section 5 summarizes an empirical study and a case study concerning the framework use. Section 6, concludes this paper.

2. Software Inspection Support Proposal Since the definition of the software inspection process, many research aimed to evaluate or to question its structure. Votta [11], for instance, argues that by avoiding synchronous inspection meetings, costs, and scheduling conflicts can be reduced without sacrifying inspection effectiveness. Some empirical studies, like [12], reinforce this argument. Based on these results, amongst others, a reorganization of the inspection process was proposed [4]. This reorganization is shown in Figure 1.

2.1. Planning In the planning activity a user, assuming the inspection moderator role, defines the inspection context, selects the inspectors, and distributes the material for the inspection. The framework should provide support for the decision of which inspectors to select, since this decision affects the effectiveness of the whole inspection. Our tool support proposal uses (1) the inspectors characterizations, and (2) their performance in past inspections over the same artifact type. The inspector’s characterization could then be used to apply a result of Carver [15]. In his work, the impact of different characterizations of inspections and inspectors on their defect detection performance was evaluated. This was done elaborating and using a methodology to obtain grounded hypotheses (a grounded hypothesis is a formulated hypothesis which was confirmed in results of empirical studies) by the exploration of historical data from past experiments concerned with software inspections. Using this set of grounded hypothesis, a sorted most appropriated inspector list for defect discovery can be provided according to how characterizations match the desired characteristics for the inspection being planned. We consider that an inspector owns a desired characteristic for the inspection if its value in his characterization is near to the inspection’s desired value. The proposal’s sorting criteria, for each inspector, is the number of desired characteristics for the inspection that s/he owns.

2.2. Discovery

Figure 1. Reorganization of the inspection process, adapted from [4]. The inspection process reorganization keeps the rigid structure and the collaborative aspects of the traditional inspection process, where roles, activities and the relationships among them are well defined. Some guidelines that can be followed to instantiate inspections are described in [13]. In the following subsections an overview of the tasks of each inspection process activity is given. Moreover, for each activity our tool support proposal is described. This proposal was formulated based on theories about software inspections that have shown themselves adequated in empirical studies [6][15][16][17][20].

In the discovery activity each inspector selected by the moderator performs a defect detection. The main task of this activity consists in finding defects in the artifact to be inspected and produce a discrepancy list (a discrepancy is a possible defect). The framework should provide inspection techniques and support in its application. The provided technique depends on the inspection context, defined in the planning activity. Among the most popular techniques are ad-hoc and the use of checklists. More specific reading techniques, such as PBR (Perspective Based Reading) [5] and OORT’s (Object Oriented Reading Techniques) [18] are also available. PBR supports defect detection on requirements documents and the OORT’s support defect detection on high level designs described in UML. The framework should be extensible to make possible support for different inspection techniques.


2.3. Collection In the collection activity the moderator merges the inspectors individual discrepancy lists, removing duplicates. In the collection activity the framework should use the results of the empirical study presented in [16]. These results show that the defect discrimination activity should be restricted to discrepancies that were discovered by only one inspector. Therefore, the moderator should be able to set identical defects found by multiple inspectors as duplicates. One of the duplicated discrepancies should be selected and stored as a defect, being forwarded directly to the rework activity.

2.4. Discrimination In the discrimination activity the moderator, document author and inspectors discuss the discrepancies in an asynchronous way. During this discussion, some discrepancies will be classified as defects and others as false positives. The false positives are discarded and the defects are registered in a defect list. The framework should make this discussion possible for geographically-distributed participants. Moreover, results of an empirical study [17] revealed an interesting characteristic of the defect discrimination activity: the anonymity of the participants helps to correctly classify discrepancies as defects or false positives. Therefore the framework should not reveal the names of the participants, replacing it by the role performed by the participant in the inspection process.

To calculate the current defect detection effectiveness on a document it is necessary to know its total number of defects, what in practice doesn’t happen. Biffl et al. [6], present an equation to estimate the number of defects on documents. It uses the Weighted Average of Individual Offsets (WAO), where subjective individual estimates of the number of defects are combined to obtain a subjective estimate for the whole inspection team. In an empirical study presented in this paper, the subjective team estimate using the WAO estimator had better performance than the objective estimation model Jack Knife and than individual subjective estimates. According to [19], the Jack Knife estimator is a model that, between objective estimation models, has shown the best results in empirical studies. To calculate the defect detection effectiveness for the next inspection, the Improved Linear Model (ILM) [20] can be used. In an empirical study with 169 subjects [14], the ILM model showed more accurate results than the trivial models (always or never re-inspect), than the Original Linear Model (OLM) [20], and than the reliability growth Model proposed in [14].

Figure 2. The ILM model. Adapted from [20].

2.5. Rework The rework on the inspected artifact itself is out of the software inspection process scope. However, in this activity, the framework should make the elaboration of a defect correction report possible. The defect correction report could then be used in the following activity to evaluate the quality of the document after defect correction.

2.6. Follow-up In the follow-up activity the moderator decides whether another inspection on the artifact should occur or not. The framework should support this decision, using the following data: (1) defect detection effectiveness estimates for the current and for the next inspection, and (2) historical data about the number of defects found in past inspections on the same artifact type.

The ILM model estimates the defect detection effectiveness according to the equation shown in Figure 2. This model supposes that in a new inspection the defect detection effectiveness will be increased by the defect defection effectiveness of the previous inspection (normalized by the effort involved in each of the inspections) multiplied by the percentage of remaining defects in the document.

3. Computational Framework Based on this scenario, an architecture to allow the instantiation of a computational framework to support software inspections, on diferent artifact types, with geographically distributed teams, and that fullfill the requirements described in section 2, was elaborated. The following subsections present the designed architecture and its instantiation through the implementation of research projects of COPPE/UFRJ`s experimental software engineering team (http://www.cos.ufrj.br/~ese).


3.1. Architecture Three types of components where identified for the architecture: (1) a main component for the framework, (2) external tools to support specific defect detection techniques for particular artifact types, and (3) a tool integrator to make data interchange possible among the other components possible. Figure 3 depicts the proposed architecture. In this architecture, the main component for the framework turns it accessible over the web and assures that the inspection process is followed in a systematic way. Moreover, it sends notifications to the participants when an activity is reached. Its control module knows the existing external tools and provides the proper tools to the user, according to the technique that should be used for defect detection. The data from the external tools are then returned to the main component using transformation drivers generated by the integrator component.

and ontologies to centralize structural and semantical information, making the automated generation of transformation drivers between artifacts possible. ISPIS. Allows user access to the system. Specific tasks are then requested according to the user’s role(s) in the inspection process. Based on the software inspection processes rigid structure and collaborative aspects, the use of a workflow tool was intended for its automatization. However, ISPIS needs to redefine the process structure dynamically when certain decisions are made. Therefore, instead of using a workflow tool, it was developed as the extension of one of those tools. The selected workflow tool to be extended was PatternFlow [22]. To provide customized support for the inspection of specific artifact types ISPIS integrates external tools for the defect discovery activity. The support provided to each of the software inspection activities is shown in further details in section 4.

Figure 3. Proposed architecture.

3.2. Framework Instantiation The framework was implemented based on the architecture presented in the previous subsection. The main component for the framework received the name ISPIS. Between other functionalities, ISPIS automatizes the control of the software inspection process, and in the defect detection activity provides external tools depending on the context defined for the inspection in the planning activity. An external tool to support PBR [21] is currently available at COPPE/UFRJ, and the implementation of a tool that supports OORT`s is being finished. The data generated by the use of these tools can be loaded in ISPIS by using a transformation driver generated by the integration tool. The current configuration of the framework is shown in Figure 4. Presented this overview, in the remainder of this section the tools that represent the frameworks current configuration are described. Integration Tool. Tools embedded in the same application domain can represent semantically equivalent data in different ways. The integration tool assumes all artifacts to be stored in XML format and uses schemes

Figure 4. Framework’s current configuration. External Tools. The objective of the external tools is to complement the computational framework providing support for the application of specific reading techniques, since ISPIS supports only ad-hoc inspections in the defect detection activity. The external tools whose development is currently being finished are: x The PBR Tool. This tool provides a guide for the accomplishment of PBR specified tasks for the user perspective and facilitates the reporting of identified discrepancies. x The OORT`s Tool. This tool can be configured to allow the customized application of OORT’s. Thus, it aims to provide automated support for the configuration and application of OORT`s.


4. Framework’s Tool Support Overview In this section, the enactment of an inspection process in the framework is presented to illustrate the support to each of the software inspection process activities. The moderator can monitor each inspection instance. This feature helps the moderator to manage the process. Moreover, when an inspection instance reaches an activity, e-mails are sent to notify the activity’s participants.

4.1. Planning Following the proposal described in section 2.1, ISPIS supports the inspector selection providing a sorted list of the most indicated inspectors for the defect detection activity (Figure 5) of the current inspection, using some of the grounded hypotheses of Carver (2003) [15]. The selected hypotheses (here described using expressions in first order logic that show the characteristics of the inspector and the inspection that combined imply in more defects found by the inspector) are: x Grounded hypotheses implemented in ISPIS for requirements inspections: o R1: Inspector experience with software development; o R2: Inspector experience writing requirements; o R3: (Inspector experience writing use cases) ^ not (Inspection uses a procedural, well defined checklist); o R4: (Inspector experience with software design) ^ not (Inspection uses a procedural well defined checklist); o R5: (Inspector domain knowledge) ^ not (Inspection uses a procedural technique); o R6: (Inspector experience reviewing requirements) ^ (Inspector familiar to documents language) ^ ((Inspection of a non familiar document) ^ (Inspection of a document from a non familiar application domain). x Grounded hypotheses implemented in ISPIS for design documents: o P1: Inspector experience with software development; o P2: Inspector domain knowledge; o P3: Inspector experience reviewing object oriented designs. To each of the characteristics the inspectors receives a rating from 1 to 5. Beyond the sorted inspector list, the moderator can visualize the profile of each available inspector to perform a more subjective analysis. The profile of an inspector is given by its characterization and by its performance on past inspections over the same artifact type of the one to be inspected.

Figure 5. Sorted inspector list. In the inspector characterization, shown in Figure 6, for each of the characteristics the inspector’s value, desired value and rating can be consulted. Some of the characteristics have an unknown desired value for the inspection, and for them no rating is given.

Figure 6. Inspector profile characterization part. In the performance on past inspections on the same artifact type, shown in Figure 7, it is possible to consult the mean efficiency of the inspector and the efficiency in his last inspection. The following information about past inspections is shown: number of defects, number of false positives, time to perform defect discovery activity, and efficiency (defects/hour). Moreover, the current inspections defect type emphasis is shown and a graph with the distribution of defect types found by the inspector can be consulted. It is important to notice that ISPIS only makes suggestions aiming to support the inspector selection. The final decision about the inspection team is still responsibility of the moderator. To address the distribution of material, ISPIS allows the controlled attachment of several document types.


document and loads the discrepancy list data. The transformation occurs using a XSLT driver generated by the integration tool.

Figure 7. Inspector profile performance part. When the planning activity is finished, ISPIS has suficient information to define the inspection process structure defined in [4]. At this moment the following workflow elements are defined in PatternFlows metadata: a discovery activity for each selected inspector; a collection activity for the inspection, synchronizing the data from the selected inspectors discovery activities; a defect discrimination activity for the inspection; a rework activity for the document author; a follow-up activity for the moderator; conditional transitions between the activities so that they are linked as described in [4]; proper user accesses to the activities. This dynamically process definition enables ISPIS to coordinate the accomplishment of several inspections being performed at the same time. After defining the process structure, ISPIS forwards the inspection to the defect discovery activities of its selected inspectors.

Figure 8. Ad-hoc defect discovery. To finish the discovery activity the inspector must inform the time he spent inspecting the document and perform a estimate of the documents number of defects. This data will be used respectively in the planning activity to calculate the inspectors efficiency and in the follow-up activity to estimate the number of defects in the document using WAO [6] and estimate the defect detection effectiveness of a new inspection on the document, using ILM [14].

4.2. Discovery To enable the inspection of several artifact types, ISPIS, in its basic configuration, allows ad-hoc inspections. A more specific support for particular artifact types can be obtained by using external tools. In an ad-hoc defect discovery, shown in Figure 8, the inspector’s tasks can be resumed in reading the attached document and register discrepancies. Each discrepancy found is registered with information on its location, defect type, defect severity and its description. In a discovery using an external tool, shown in Figure 9, ISPIS provides the appropriate tool for the inspection for download, the inspector then uses this tool to perform the defect discovery. During the discovery, the tool produces a XML document, describing the discrepancies found. This document can then be loaded in ISPIS. After loading the documents, they can be activated. The document activation makes its discrepancy list to be the current discrepancy list of ISPIS. The activation occurs internally in ISPIS in two steps. First ISPIS transforms the XML of the document to be activated in a format that ISPIS understands. After that, ISPIS reads the XML

Figure 9. Defect discovery using the external PBR Tool. When the discovery activities of the inspectors are concluded, its data is synchronized in the defect collection activity. The inspection is available in its collection activity when the last defect discovery activity is accomplished.

4.3. Collection In the defect collection activity the moderator has access to all discrepancy lists produced during defect discovery, as shown in Figure 10. He is then enabled to select discrepancies of these lists and to discard them or set them as duplicated. When a discrepancy is discarded,


it is classified as a false positive and disappears from the discrepancy list.

Following the proposal described in section 2.4, the participants of the discrimination activity are anonymous. In the commentaries only the role of the participant is shown and not his name. Only the moderator can edit, discard and classify discrepancies. The remaining participants are limited to add commentaries. Moreover, only the moderator can determine the conclusion of the activity.

Figure 10. Discrepancy lists of the inspectors. The setting of discrepancies as duplicated is shown in Figure 11. In this case, the moderator has to select one of the versions of the discrepancy. The selected version can then be edited so that important data contained in the duplicated version can be added. Following the empirically evaluated suggestion of [16], the selected version is directly classified as a defect and forwarded to rework, the replications will not be shown anymore in the discrepancy list.

Figure 12. Defect Discrimination in ISPIS. Once the moderator defines the discrimination activity as concluded the inspection is forwarded to the rework activity of the document’s author. Moreover, participants’ access to the discrimination activity of this inspection is removed.

4.5. Rework In this activity, the corrections of the defects can be described in ISPIS, creating a defect correction form and the corrected document can be attached. When the author defines the rework as concluded the inspection is forwarded to the follow-up activity. Figure 11. Setting a discrepancy as duplicated. The conclusion of the defect collection activity implies in the forwarding of the inspection to the defect discrimination activity. Moreover, a process definition change occurs, removing the moderator’s access to the defect collection activity of this inspection.

4.4. Discrimination The moderator, the author and the inspectors participate in this activity. During the discrimination, the discrepancies are treated like discussion topics. Each of the participants can add commentaries to the discrepancies. The discrepancies are available as a discussion topic until the moderator decides if it represents a defect or a false positive. Figure 12 shows the support of ISPIS to the discrimination activity.

4.6. Follow-up In this activity the moderator has access to the inspection context (defined in the planning activity), the old and the corrected version of the document, and the defect correction form created in the rework activity. The moderator’s task is to decide if a new inspection on the document should occur. Therefore, the proposal described in section 2.6 is used to estimate the current defect detection effectiveness and of the defect detection effectiveness of the next inspection. ISPIS considers the subjective variable effort of the ILM model to be the same in both inspections. Based on the results of these estimations ISPIS suggests a new inspection if the defect detection effectiveness of the current inspection is lower than 0.7 (less than 70% of documents defects where found), and


the defect detection effectiveness after the next inspection is higher than 0.7. ISPIS presents to the moderator: the estimated value for the current inspection, the estimated value for the next inspection and the suggestion. Figure 13 shows ISPIS suggesting a new inspection for an inspection with current estimated defect detection effectiveness 68.96% and estimated defect detection effectiveness for the next inspection of 90.36%.

Figure 13. Re-inspection decision. Moreover, ISPIS presents the number of defects found in the current inspection and the mean number of defects found in past inspections on the same artifact type. Following a guideline [1], ISPIS suggests a new inspection if the number of defects found in the current inspection is 5% higher than the mean. This suggestion aims to increase quality avoiding that documents containing an initially very high number of defects are forwarded to the next software life cycle activities. However, ISPIS only makes suggestions, the final decision about accomplishing another inspection is still responsibility of the moderator. When the moderator defines the follow-up activity as concluded, the current inspection is finalized in ISPIS. If the moderator decided to accomplish another inspection on the document, a new inspection instance for the document is created and moved to the planning activity. Creating a new instance makes it possible to distinguish between the different inspection occurrences to elaborate the historical data.

5. Experiment and Case Study 5.1. Experiment: Evaluating Planning in ISPIS The experiment aims to evaluate ISPIS’s support for the planning activity, since this activity uses a new support proposal, elaborated in this work. Therefore, three hypotheses where stated in the experiment plan: x H1: The use of ISPIS helps to elaborate inspection plans for higher defect detection effectiveness, when compared to manual elaboration of inspection plans.

x H2: The use of ISPIS helps to elaborate inspection plans in less time, when compared to manual elaboration of inspection plans. x H3: The use of ISPIS in elaborating an inspection plan implies in a higher user satisfaction degree, when compared to manual elaboration of inspection plans. The experiment design uses one factor (the elaborated inspection plan) and two treatments: (1) subjects elaborating an inspection plan using ISPIS and (2) subjects elaborating an inspection plan manually. To accomplish the experiment, data from seven inspectors that performed two consecutive defect discoveries on different requirement documents using PBR were used. They performed, respectively an inspection of a requirements document for a parking garage control system (PGCS) and an automated teller machine system (ATM). Thus, to instrument the experiment, the characterizations of the seven inspectors where registered in ISPIS, and the defect discovery on the PGCS document was used to make historical data available. For the manual treatment, the characterizations of the inspectors together with their discrepancy lists (with the real defects highlighted) on the PGCS document were prepared in paper format. The experiment was carried out by 12 volunteer subjects represented by graduate students at COPPE/UFRJ, partitioned in two balanced groups according to their experience. All of the subjects reported having practical software development experience. The subjects signed a consent form to take part in the experiment. Qualitative data were collected through a follow up questionnaire. Subjects of both treatments were asked to realize an inspection plan for an inspection on the ATM document, using the characterization of the inspectors, and their performance on the PGCS inspection. The context for the inspection was the following: PBR-user as inspection technique, focus on all defect types, up to three of the seven available inspectors, and two days for the defect discovery activity. When elaborating their inspection plan, the subjects did not know that data about the inspector’s performance in the ATM document inspection were already available. Therefore, they had no idea that data such as the defect detection effectiveness of their inspection plan could be measured. Some observations concerning the hypotheses and the experiments preliminary results could be made: x H1: The mean defect detection effectiveness was 14% higher for subjects using ISPIS. However, the most experienced subjects performed well in both treatments. We believe that the less experienced subjects where supported by the knowledge of [15], embedded in ISPIS.


x H2: All the subjects performed the planning in less than 40 minutes. However, the subjects using ISPIS performed their task in 22% less time. x H3: All subjects informed to feel themselves at least satisfied in realizing the task. However, the subjects using ISPIS related a very high user satisfaction degree, and the manual users related only a high satisfaction degree. The main problem associated to manual user satisfaction was paper shuffling, which in ISPIS does not occur. Despite of the good preliminary results, we realize that more subjects must be used to confirm these results, and that the experiment plan should be discussed with other researchers to obtain further improvements and conclusions. The complete experiment definition and plan, as well as the preliminary results of the experiment are available at http://ese.cos.ufrj.br:8081/ispisexperiment.

In the treatment with PBR, an external tool supporting PBR-user [21] was provided for supporting the discovery activity. Three of the 5 inspectors effectively used the external tool. After using the tool, the produced XML file was loaded and activated in ISPIS. The other two subjects applied the technique manually and then registered their discrepancy lists directly in ISPIS. In the treatment with ad-hoc, all of the inspectors registered their discrepancies directly in ISPIS, since no external tool could be provided. After that, the moderator performed the defect collection activity for each of the two parallel inspections. Then, the moderator, document author, and inspectors participated in the defect discrimination activity. Figure 14 shows part of a screenshot captured during the case study in this activity, showing the discussion of one of the discrepancies. Finally, the rework and the follow-up activity were accomplished. In the follow-up activity, the moderator decided to not realize another inspection.

5.2. Case Study: ISPIS Feasibility Study Subjects (12) represented by a professor and 11 graduate students at COPPE/UFRJ carried out this case study we conducted. All of the subjects reported having practical software development experience. The subjects signed a consent form to take part in the study. Qualitative data was collected through a follow up questionnaire. A tool demonstration was given to subjects in class. Five of them had prior experience with software requirements inspections using PBR-user perspective as they had participated in a previous study to analyze tool support for PBR [21]. The professor performed the role of the inspection moderator. One of the subjects performed the role of the document author. The document to be inspected was a real requirements document for the implementation of an OORT’s support tool. The remaining 10 subjects were grouped in two inspector teams, in one of them subjects had PBR-user perspective experience, and in the other one were subjects with no previous experience with inspections. To verify the feasibility to use the framework to support software inspections, with and without specific inspection techniques, the moderator was asked to manage two real inspections of the requirements document. One of the inspections should use PBR-user as detection technique and the other one should use ad-hoc. Therefore, the PBR-user experienced team and the inspection unexperienced team were used, respectively. Once the moderator planned the two inspections they were forwarded to the inspectors defect discovery activities.

Figure 14. Real discrimination in ISPIS. Both treatments successfully reached the end of the inspection process and participants’ feedback was very positive. Some suggestions considering usability facilities to treat discrepancy lists with a high number of discrepancies (in both treatments of the study more than 90 discrepancies were reported) were made and implemented in ISPIS, generating a new version.

6. Conclusions One of the ways to improve software quality and the productivity of software development is using software inspections on the artifacts being produced throughout the software life cycle. Even so, software inspections are unsystematically performed in industry and their potential is seldom exploited. Considering this, a framework to support the software inspection process exploring knowledge acquired from results of empirical studies and historical data from past inspections has been developed. Moreover, an experiment plan to evaluate the framework’s support


to inspection planning was elaborated and the experiments preliminary results were very satisfactory. A case study showed the feasibility of using this framework in real inspections and therefore we believe that making this framework available at COPPE/UFRJ (http://ese.cos.ufrj.br:8081/ispis) allows its use by other partners. Thus, we believe that organizations could accomplish their inspections in a more systematical way and take advantage of the benefits of the automated application of empirically evaluated knowledge and historical data.

Acknowledgements The authors would like to thank CAPES, CNPq, and the CNPq-NSF READERS project for financial support, COPPE/UFRJ’s experimental software engineering team (http://www.cos.ufrj.br/~ese) for their contributions in this work (specially Rodrigo Spínola, Wladmir Chapetta, Leonardo Nuñez, and Luis Felipe Silva), and all the subjects that participated in the evaluation studies.

References [1] Fagan, M.E., “Design and Code Inspection to Reduce Errors in Program Development”, IBM Systems Journal, vol. 15, no. 3, pp. 182-211, 1976. [2] Ackerman, A., Buchwald, L., Lewski, F., “Software Inspections: An Effective Verification Process”, IEEE Software, vol. 6, no. 3, pp.31-37, 1989. [3] Gilb, T., Graham, D., Software Inspection, Addison-Wesley, 1993. [4] Sauer, C., Jeffery, D.R., Land, L., Yetton, P., “The Effectiveness of Software Development Technical Review: A Behaviorally Motivated Program of Research”, IEEE Transactions on Software Engineering, 26 (1): 1-14, Jan. 2000. [5] Shull, F., Rus, I., Basili, V., “How Perspective-Based Reading Can Improve Requirements Inspections”, July, IEEE Software, pp. 73-79, 2000. [6] Biffl, S., Halling, M., Koeszegi, S., “Investigating the Accuracy of Defect Estimation Models for Individuals and Teams Based on Inspection Data”, Proc. of the 2nd ACM-IEEE International Symposium on Empirical Software Eng. (ISESE 2003), Roman Castles (Rome), Italy, September/October, 2003. [7] Ciolkowski, M., Laitenberger, O., Biffl, S., “Software Reviews: The State of the Practice”, IEEE Software 20 (6): 4651, 2003. [8] MacDonald, F., Miller, J., “A Comparison of Computer Supported Systems for Software Inspection”, Automated Software Engineering, An International Jornal, Volume 6, No. 3, pp.:219-313, July 1999.

[9] Lanubile, F., Mallardo, T., “Tool support for distributed inspection”, Proc. of the 26th Annual International Computer Software & Applications Conference (COMPSAC 2002), Oxford, England, August 2002. [10] Grünbacher, P., Halling, M., Biffl, S., “An Empirical Study on Groupware Support for Software Inspection Meetings”, Proceedings of the 18th IEEE International Conference on Automated Software Engineering, 2003. [11] Votta, L.G., “Does Every Inspectin Need a Meeting?”, Proc. First ACM Symp. Foundations of Software Eng., pp. 107114, 1993. [12] Johnson, P.M., Tjahjono, D., “Assessing Software Review Meetings: A Controlled Experimental Study Using CSRS”, Technical Report 96-06, Dept. of Information and Computer Sciences, Univ. of Hawaii, 1996. [13] Melo, W., Travassos, G.H., Shull, F., “Software Review Guidelines”, Technical Report ES – 556/01 – COPPE/UFRJ, August 2001. [14] Biffl, S., Gutjahr, W., "Using a Reliability Growth Model to Control Software Inspection", Empirical Software Engineering: An international journal; vol.7, pp. 257-284, Kluwer Academic Publishers, 2002. [15] Carver, J.C., “The Impact of Background and Experience on Software Inspections”, PhD Thesis, University of Maryland, USA, 2003. [16] Launubile, F., Mallardo, T., “An Empirical Study of WebBased Inspection Meetings”, Proc. of ISESE, Roman Castles (Rome), Italy, IEEE Computer Society Press, Sept./Oct. 2003. [17] Vitharana, P., Ramamurthy, K., “Computer Mediated Group Support, Anonymity, and the Software Inspection Process: An Empirical Investigation”, IEEE Transactions on Software Engineering, vol. 29, number 2, Feb. 2003. [18] Travassos, G. H., Shull, F., Fredericks, M., Basili, V. R., “Detecting Defects in Object Oriented Designs: Using Reading Techniques to increase Software Quality”, Acm Sigplan Notices. USA, v.34, n.10, p.47 - 56, 1999. [19] Biffl, S., Grossmann, W., “Evaluating the accuracy of objective estimation models based on inspection data from multiple inspection cycles”, Proc. ACM/IEEE ICSE, Toronto, Canada, IEEE Comp. Soc. Press, May 2001. [20] Adams, T., “A formula for the re-inspection decision”, Software Engineering Notes 24(3): 80, 1999. [21] Silva, L.F.S, Travassos, G.H., “Tool-Supported Unobtrusive Evaluation of Software Engineering Process Conformance”, Proc. of the 3rd ACM-IEEE International Symposium on Empirical Software Eng. (ISESE 2004), California, USA, 2004. [22] Kalinowski, M., “PatternFlow: um software de workflow para a implementação e execução de processos de negócio customizados”, Bachelor on Computer Science Conclusion Project, DCC/UFRJ, 2001.
