Virtual Software Inspections for Distributed Software Engineering Projects Henrik Hedberg
Lasse Harjumaa
Department of Information Processing Science University of Oulu P.O.Box 3000 FIN-90014 Oulun yliopisto, Finland
Department of Information Processing Science University of Oulu P.O.Box 3000 FIN-90014 Oulun yliopisto, Finland
[email protected]
[email protected] properly, distributed teams lost nothing as compared with colocated teams.
ABSTRACT Traditional software inspection is recognized method, but is poorly suited to modern distributed software engineering projects. Arranging inspections in a distributed and asynchronous manner is not a straightforward matter. Over and over again, researchers have constructed tools with unique sets of features to support paper-based inspections. Now the concept of virtual software inspection is introduced to overcome that problem. This is based on three important aspects – computer tools, flexibility and integration – and can be used as a reference model in distributed software projects. This paper briefly describes the concept, the tool supporting it, and ongoing experiments to evaluate these.
Research lasting almost ten years has led to the construction of computer support for software inspection and proved its feasibility. Until now, different tools have been produced with conventional inspections in mind, with unavoidable differences in paper-based approaches having to be reasoned out separately each time. The concept of virtual software inspection is meant to help to overcome this and provide a basis of all inspections to be performed in a distributed manner. In this paper we first define virtual software inspection and then introduce a new tool supporting it. Finally, we will also consider the utility of the idea and the tool in the light of experiments carried out with students. The results are used to develop the concept and tools futher with future uses in mind.
1. INTRODUCTION The idea of peer reviews is over 30 years old and the original concept of egoless programming put forward by Weinberg [13] and the more formalized inspections proposed by Fagan [3] have gained well-deserved positions in software engineering research. IEEE Software, for example, lists reviews and inspections among the factors having the best influence on software engineering [10]. Traditional software inspections are often considered too laborious and time consuming to arrange, however, as development teams are geographically distributed and team members are involved in many projects simultaneously. Finding a time and place for inspection meetings is often impossible. To overcome the problem, a variety of software tools for inspection collaboration have been developed.
2. VIRTUAL SOFTWARE INSPECTION Although the purpose of inspection tools is to provide assistance, they have reshaped the whole process to some extent. Inspector roles, tasks and phases in the process have been adapted to match up to distributed working. Thus, computer-supported software inspection is counted as a method of its own, separate from traditional software inspection. We have introduced the concept of virtual software inspection to emphasize this difference. Virtual software inspection is a process that conforms to a defined workflow and is performed in a distributed manner with the aid of an inspection tool. The process can be lightweight or all-inclusive, comprising both asynchronous and synchronous phases. It is usually carried out through network techniques, but traditional meetings can be included if necessary. Geographical distribution is typically achieved through the World Wide Web. An inspection tool is a software package particularly designed for inspection collaboration, and it should be capable of at least managing and delivering the inspection documentation on-line, enabling the effortless recording of defects and automatic gathering of metrics.
As Porter and Johnson state, face-to-face meetings do not make the defect detection process significantly less effective [11], and several other sources [8, 9] show that there should be no substantial differences in efficiency between traditional and computer-supported inspections. Thus, if tools are implemented The work carried out at the University of Oulu was supported by the Academy of Finland under grant 74780.
Some organizations wish to carry out inspections exhaustively, while others desire simple, yet efficient inspections, such as pair inspection [12]. Instead of a fixed process model, virtual inspection tools should provide capabilities for customizing the process for an individual organization or project.
1
In addition to the process, flexibility should encompass the documents reviewed. Code is not the only type of document that has to be inspected. Artefacts from every phase of development should be reviewed. Thus, inspection tools must deal with requirement documents, designs and other content-rich documentation including text, images and possibly hyperlinks. Clearly, emphasis must be given to complex sets of documentation. As checklists, metrics and even annotating methods may vary depending on the characteristics of the documents, managing them and the attached annotations is not straightforward from the point of view of the tool.
3. THE XATI TOOL Computer tools are the essence of the virtual inspection process. There are a number of tools and prototypes available for inspection support. Each research group has implemented a tool of its own, and some – like us – have implemented a whole bunch of tools, each stressing a specific aspect of the distributed inspection process. The Internet-Based Inspection System (IBIS) [1] and Web-based Inspection Process Support Environment (Web-IPSE) [6], for example, illustrate the fact that this area is still subject to active research. These tools utilize the modern technologies provided by current web infrastructure and the later also supports version management. However, there is still a need for a comprehensive inspection environment that combines the good ideas from several sources and supports the flexibility and integration aspects of the virtual software inspection.
As well as numbers and metrics, inspections always include implicit information to some extent. Although discussions about technical details and possible solutions to problems are discouraged in the original software inspection method, it must be acknowledged that such improvement ideas often emerge in the course of inspection meetings and that they are often useful. A straightforward way is to establish a general support system for virtual meetings in conjunction with the inspection tool.
As a start for an extensible inspection tool platform, we have implemented a new XML-capable annotation tool, XATI (XML Annotation Tool for Inspection) to support virtual software inspections. Emphasis in the development of this new tool has been placed on ease of use and deployment in addition to interoperability. XATI is based on experiences gathered during the development and use of our earlier tools, such as WiP [4], WiT [5] and SATI.
An easy-to-use inspection tool is not a synonym for an easy inspection process. In fact, making the reviewing procedure as easy as possible for the individual inspection team members may result in an extraneously troublesome form of tool and process administration, thus shifting the difficulties from the inspectors to the moderator or system administrator. Administration of an inspection tool may require a great deal of manual work: copying files, granting user access across servers and systems and cleaning them up after the inspection is completed. The manual handling of data exchange between several repositories is error-prone and strenuous. [5, 7]
The present web browser infrastructure provides a solid and extensive foundation for Internet-based tools, with a number of feasible features such as XML parsing, network transaction support and means for building graphical user interfaces. The Mozilla browser, for example, which is actually the open source version of Netscape Navigator 6, is an interesting and flexible collaboration tool platform that enables browser-based applications that are independent of the underlying operating system to be developed rapidly. A new tool can be installed as a browser module directly from a developer’s home page, for example, merely by clicking a given link [14].
The inspection tool must exchange information with other tools through a number of connection points. Document management is an obvious example of the interoperability need. The inspection tool must be aware of how the documents are produced, where their revisions are located and how they can be accessed.
The XATI tool uses Mozilla as a generic application environment and as a viewer for a variety of document types. Actually, many formats are evolving towards XML, enabling the viewing and modifying of documents in web browsers. XATI accesses the document under inspection through DOM (Document Object Model) [2] and can operate with any format that is supported by Mozilla. The focus is on document structure, which means that annotations are linked to the logical elements of the document, which also makes it possible to gather data on defect locations. This enables advanced analysis of the actual origins of defects in addition to the normal quantity-based calculations and metrics. It is possible, for example, to track the annotations in the requirement documentation that are attached to items of highest priority while defects attached to requirements of lower priorities are postponed. Initial work is needed, of course, to specify the structure of the document being inspected, but this is achieved by constructing a document template, which benefits the software development process in general.
There must be a mechanism for workflow control. As inspection tasks and resources (people, time) are initiated in the project plan, they should be imported from a project management system. Accordingly, the project plan should be updated as the inspection tasks are completed. Inspections also usually produce new action points (for reworking), which it would be convenient to read off directly from the inspection tool database and insert into the task list in the project management tool. In summary, there are three important aspects to virtual software inspection: Tools that enable efficient running of the process. Independence of time and place, on-line recording of issues and data management can be achieved through networked tools. Flexibility of the process and supporting tools, to ensure tolerable adoption effort and acceptance of the method. Interoperability of the processes and tools, to enable convenient everyday use of the method and improves the effectiveness of inspections. These characteristics should make inspections more attractive and widely used by easing the implementation effort required.
ª ª ª
The document under inspection, which contains hyperlinks to related material, is presented in the main browser window. Findings are reported using a simple dialogue, and after submission they are stored in a centralized database, thus allowing their distribution to all participants. Each annotation is connected
2
Single interconnectivity point
project management data
versioned file repository
XSQML XML + XSLT
Xfi
annotations, metric database
Figure 1. Integrating the XATI inspection tool with other development tools by means of Xfi. to a logical element in the document, or to an individual word. An inspector can decide on the scope while making a comment.
that almost all of them already had software engineering experience in industry.
Integration with other software development tools and repositories is carried out with an interoperability framework called Xfi (eXtensible framework for interoperability) [7]. This consists of an XML-based query and data manipulation language XSQML together with drivers and schema transformers for adapting the general interface to different situations. This prototype framework allows the inspection tool to fetch the necessary data from other applications and minimizes administrative and out-of-core tasks. No separate server is needed to support it, because the tool operates in co-operation with a generic interoperability framework configured to serve inspection-related data, taking advantage of the Xfi's feature of supplying data from a variety of sources through a simple XML-based interface and just one interconnectivity point. Data on inspection tasks are derived from the project plan, documents under inspection are fetched directly from a file repository, and annotations are referred back to the shared databases for the collection of metrics (Figure 1).
Over a period of one month, 74 students evaluated the XATI tool and filled in an on-line questionnaire. The material to be inspected was a fictional requirements definition document containing UML diagrams and many defects. It was structured using XML and visualized with a CSS stylesheet. The questionnaire was based on research by Laitenberger and Dreyer [9] with only slight modifications, chiefly that the middle alternative “neither” was omitted, as suggested in the original paper. The resulting item scale is shown as a table in Figure 2. The questionnaire also included additional statements about new features provided by XATI and free form questions, which, among other things, provided us with important feedback about the user interface and some feature requests. When analyzing the results, the students were divided into two groups based on their working experience with inspections or technical reviews. 35 students were marked as experienced and 39 as inexperienced (had performed inspections only once or not at all). The ratio is suitable, as the two groups can be interpreted as implying that the inspection was introduced either as a new process or else merely as a new tool to support an already familiar process. On the other hand, the numbers unfortunately indicates that inspections and reviews are still unpopular in industry.
4. EVALUATION Although the concept of virtual software inspection was formulated in co-operation with our partner companies, we are transferring the generalized results back to the industry by experimenting with the virtual software inspection tool. We started with a student experiment that focused on the usefulness and ease of use of the XATI tool. The students were taking a course in reviewing and testing at the University of Oulu, and they were all familiar with inspection techniques and had performed a conventional paper-based inspection as an exercise. In addition, they were 3rd year students or higher, which meant
Figure 2 depicts also the relative frequencies of answers to the four most interesting questions. These consisted of the following statements: Usefulness: I would find XATI useful in my job. Ease of use: I would find XATI easy to use.
ª ª
Figure 2. Relative frequencies (%) of answers to the four most interesting questions. 3
ª
The initial results of our evaluations of the key aspects of virtual software inspection were encouraging, as the students felt our first virtual inspection tool, XATI, to be both useful and easy to use. An obvious majority preferred it to paper-based forms for performing inspections and predicted that they would use it on a regular basis in the future if available. We are now testing the tool in a partner company, and focusing on integration and flexibility aspects. In addition, we intend to define the subprocesses of virtual inspection in more detail, although still expressing them in a tailorable manner.
Future usage: Assuming that XATI were available on my job, I predict that I would use it on a regular basis in the future. Preference over paper-based forms: I would prefer using XATI to paper-based forms for performing inspections. The answers to these questions summarise the general trend among the test users. 74% found XATI useful, and the same percentage found it easy to use, the mean values being 3 (slightly likely) and 4 (quite likely). When predicting future usage, 69% said that they would like to use the tool and 72% were ready to replace paper-based forms with on-line inspection. The mean value for both statements was 3.
ª
The role of virtual software inspection will be even more important in the future. To meet the challenges inherent in modern software development environments, the arranging of virtual reviews would seem to be the only competent solution. More research will be required, however, to refine distributed inspection in order to exploit its full potential.
It was interesting to note that the experienced students were a little more sceptical than unexperienced ones when predicting future usage of the XATI. Only 60% of the students making inspections or reviews in their job were ready to employ the new tool, whereas 77% of the inexperienced students predicted that they would use it on regular basis. It is not clear whether this is only a consequence of bugs in the tool or whether it represents natural resistance to change. Luckily, there was no difference when the students were asked whether they preferred using XATI to paperbased form for performing inspections, the proportions being 71% and 72% respectively. It seems that tool support is quite well accepted, but individual tools need more finishing before they can be adopted without resistance.
REFERENCES [1] Caivano, D., Lanubile, F. and Visaggio, G. Scaling up distributed software inspections. Proceedings of the Fourth ICSE Workshop on Software Engineering over the Internet, 2001. [2] Document Object Model Working Group, http://www.w3c.org/DOM/ [3] Fagan, M.E. Design and code inspections to reduce errors in program development. IBM Systems Journal, vol15, no 3, 1976, 182-211. [4] Harjumaa, L., and Tervonen, I. A WWW-based tool for software inspection. Proceedings of the 31st HICSS conference, vol III, 1998, 379-388. [5] Harjumaa, L., and Tervonen, I. Virtual software inspections over the internet. Proceedings of the Third ICSE Workshop on Software Engineering over the Internet, 2000. [6] Hazeyama, A. and Nakano, A. Web-based Inspection Process Support Environment for Software Engineering Education. Proceedings of the Fourth ICSE Workshop on Software Engineering over the Internet, 2001. [7] Hedberg, H. Integrating software engineering tools and repositories with XML and XSLT. Proceedings of the Fourth ICSE Workshop on Software Engineering over the Internet, 2001. [8] Johnson, P.M., and Tjahjono, D. Does every inspection really need a meeting? Empirical Software Engineering, no 3, 1998, 935. [9] Laitenberger, O., and Dreyer, H.M. Evaluating the usefulness and the ease of use of a web-based inspection data collection tool. Proceedings of the Fifth International Software Metrics Symposium, 1998, 122-132. [10] McConnell, S., The Best Influences on Software Engineering, IEEE Software, January, 2000. [11] Porter, A. A., and Johnson, P.M. Assessing software review meetings: Results of a comparative analysis of two experimental studies. IEEE Transactions on Software Engineering, vol 23, no 3, 1997, 129-145. [12] Tervonen, I., Iisakka, J., and Harjumaa, L. Software inspection – a blend of discipline and flexibility. Proceedings of the ESCOM-ENCRESS conference, 1998, 157-166. [13] Weinberg, G., M., The Psychology of Computer Programming, Van Nostrand Reinhold Company Inc., New York, 1971. [14] XPToolkit project, http://www.mozilla.org/xpfe/
The next step will be to evaluate the XATI and Xfi tools in industry projects to obtain feedback from real users. We are especially interested in how feasible the aspects of virtual inspections are and whether we have succeeded in implementing them in our tool. We thus aim to seek answers to at least the following questions: How efficient and easy to use is our tool? Especially, how well are the annotation, linking and commenting capabilities implemented? Are the virtual inspection process and tools supporting it as flexible as they should be? Especially, what changes should be made to the processes when using our tool? Is the organization able to make efficient use of the interoperability of the processes and tools? Especially, where are the bottlenecks when creating seamless operation between inspection and other phases? The experiment will be carried out during 2002, and will include couple of cycles in which opinions on virtual inspection and the tool itself are evaluated using questionnaires.
ª ª ª
5. CONCLUSIONS Distributed software engineering projects cannot make use of the same traditional methods as do co-located teams, although their communication and quality assurance needs are the same. This is especially apparent in software inspections that have a heavy accent on teamwork. It is not feasible to construct computer support for paper-based inspections, but rather distributed working, which has its own needs as fas as the roles, tasks and phases involved in the process are concerned, should be counted as a method of its own. We call this method virtual software inspection. Its characteristics, notably flexibility and integration, are important factors when introducing software inspections and new tools into organizations.
4
