This paper is an extract from Project Control for Software Quality, Editors, Rob Kusters, Adrian Cowderoy, Fred Heemstra and Erik van Veenendaal. Shaker Publishing, 1999. ISBN 90-423-0075-2.
Experiences of painless improvements in software inspection Juha Iisakka, Ilkka Tervonen and Lasse Harjumaa
Abstract An inspection is regarded by software developers as an efficient method for finding errors. Unfortunately companies do not necessarily have the power to implement inspection efficiently. Why we have started a project in order to find a suitable way to carry out inspections in four companies. This paper illustrates our plans and early experiments with two companies.
1. Introduction Formal inspection and other types of review have increased in popularity continuously during the last ten years. There hardly exists a company that dares to confess publicly that it does not use any inspections at all. Nevertheless, we personally do not know any company that would strictly follow conventional formal "Fagan" inspection. Although, fine guides exist (e.g. [1]), every company has modified the inspection procedures to suit its own purposes. Unfortunately most modifications are haphazard and do not emphasise features that are relevant for the companies. We believe that there are some reasons why this is the case. For one thing, there are companies that have proved formal inspections but the staff have quickly seen that the method is cumbersome relatively to its advantages. Installing the inspection is a normal process investment that has to be paid off for a long time before profit starts to accumulate. For another thing, there are companies which consider that they have business to do and have no time for academic snobbery. These companies nevertheless have time to correct their software. We presented in our earlier paper [2] the idea of a painless improvement in inspection. We believe that if a company takes small, planned steps when improving its inspection process it will be meet with better success. Perhaps in the future, these steps will have a "happy end" and the company will adopt a strict formal inspection. In order to put our ideology into practice, we started a project with four software engineering companies last autumn with the aim of improving their inspections. The project is financed by the Tekes Technology Development Centre and the companies and will last two years. The focus of this paper is on the first experiences we have gathered with two of the four companies. Process improvement in these companies is focussed on the inspection process, while for the other companies it is more important to improve the use of the results of inspections. We have found it motivated at the start to determine the phases where more efficient inspection could improve software development most relative to the costs. This paper will first present new types of inspection that we have constructed in our previous studies [see 3]. One of these new types is a virtual logging meeting. We will then shortly introduce Inspection Window, an application of virtual logging meeting. Finally, we will present our first experiments regarding inspection in two companies.
2. New types of inspection In our previous paper [3] we defined two aspects, flexibility and discipline, to characterize different inspection styles. The discipline dimension explains the formal aspect of inspection and has three key words: roles, procedures and metrics. A very disciplined inspection follows the roles and procedures of Gilb and Graham [1], for example. Furthermore, if the organisation (i.e. company) desires to shift inspections from plain error removal to the improving of quality, it will need metrics to measure inspections. Flexibility is harder to illustrate, so that we should perhaps adopt the viewpoint: "How easy is it to organize an inspection meeting?" A flexibility ratio can be measured with respect to four key items: • Place and/or time independence, it is very common to have difficulties in gathering the group in the same place at the same time. • Network, especially webtechnology; provides a base solution for place and/or time independence • Tailorable, the way of inspection may vary if the materials, participants etc. vary. • Varying numbers of participants, the conventional inspection has 4-8 participants. What if we cannot gather so many? The traditional walkthrough and conventional inspection are placed in terms of our dimensions in Figure 1, which also shows four other types of inspection. We will explain briefly these other new types of inspection. If you are not familiar with conventional inspection or walkthrough, please consult [1] or [4], for instance.
Inspection with a limited logging meeting
Discipline
Conventional inspection
Virtual inspection
Inspection with a logging meeting Walkthrough Pair inspection
Flexibility Figure 1. The Flexible and Discipline dimensions and six ways to inspect.
2.1. Virtual inspection Logging in virtual inspection may be implemented at the same time but in a different place, for example, or at a different time and in a different place. This physical and temporal distribution reduces the mutual time required and makes reconciliation of participants' timetables easie. It is this that constitutes the flexible dimension in a virtual logging meeting. Support is required for flexible communication between participants. This is an area of innovative research in computer-supported co-operative work (CSCW) and organisational memory. In order to manage distributed inspectors and their comments, the virtual logging meeting needs quite a fixed protocol for acting in the meeting. Moreover, CSCW tools demand a certain way of acting. Thus the virtual logging meeting has at the same time a highly disciplined dimension. The moderator's duties in particular are strictly laid down. Inspection Window, a public domain application for virtual inspection purposes, has been constructed by Lasse Harjumaa1. In Inspection Window reviewing is performed during a relatively short period of time settled by the moderator. In this time interval the material to be inspected is public to the inspectors to browse through and comment on. Unlike conventional inspection processes, which typically include about five separate phases, inspection process of the Inspection Window tool is mainly materialised by a single inspection step, the public phase. There are two additional actions that are taken by the moderator: Starting of the inspection and closing it after adequate agreement of the quality of the material is gained. This form of inspection is more of supervised debate than formal technical review, but, of course, as meaningful as any inspection.
Figure 2. An example of Inspection Window
1
You may like to check his homepage http://rieska.oulu.fi/~marcus
All annotations made to the document are recorded on-line in the inspector's WWW-browser. No extraneous client software is needed, and locations of findings are presented as vertical marker lines on the left margin of the document. Checklists are visible for the inspectors all the time during the reviewing. After saving annotations, they become available to the other inspectors, who can further comment on them. In addition to annotation data, the tool is capable of handling inspector information and inspection schedules as well as checklists of multiple languages. In Figure 2 there is an example of inspection screen in Inspection Window. If you are interested in knowing more about this product, you may like to read [5] or [6].
2.2. Inspection without a logging meeting If the virtual inspection is time-dependent, participants need some kind of notice board on which to have public discussions that everyone can take part in. If the comments from other inspectors regarding the discussion between the author and an inspector are irrelevant and the author is merely checking the issues, the virtual logging meeting has become a no logging meeting inspection. This style of inspection is semi-disciplined and semi-flexible. As there is no virtual meeting it is not so disciplined as the virtual logging meeting is, but as the networking demands some rules, it is not totally undisciplined, either. On the other hand, on the flexible dimension, the no-logging meeting has no physical demands (with respect to the capabilities of the inspection tool) or temporal demands (except a deadline).
2.3. Pair inspection Smaller companies in particular have difficulties in providing enough inspectors for a conventional inspection, and it would suit their capabilities if the inspection group were limited to two - the author and an inspector. Although we propose pair review for smaller companies, the first time we ourselves found this way of inspecting was in a large company. There this style was used because the software system included very many algorithmically complex but short program modules, and the company had noticed that a pair could give support and validity to each other. The inspection practices of such a pair will evolve step by step during the discussions, and it is meaningless to formulate any strict rules of action. Consequently pair inspection is a very flexible and undisciplined way for inspectioning. The recording of "temporary" issues (defects) found in informal meetings is not required. In the final (formal) meeting, however, the pair inspection needs a form or template of its own in which both the author and the inspector guarantee that the material has passed the pair inspection. This final document is the only disciplined aspect of pair inspection.
2.4. Inspection with a limited logging meeting Inspectors ought in any case to write issues down before the meeting, as this helps the scribe, and as the issues themselves may be very complicated. In this way they can be sent to the author by electronic mail to be collected and brought to the meeting. If it is hard to find a suitable meeting time for all the participants, only the most important and technically competent employees may need to participate, since they can check all the issues and decide what to do. The limited logging meeting is a specialisation of the virtual logging meeting in the same way as the no logging meeting is, but in this case the common notice board is replaced by a smaller group of inspectors. In principle, the limited logging meeting is a "normal" inspection meeting, and thus it should be quite disciplined. It can be a specialized case of a pair inspection, however, in which only a couple of people -- the author and another person (e.g. the chief designer) -- check the issues together. Since they can work iteratively, focus more on issues than on the document to be inspected. In this sense, it is sometimes a very flexible way of inspecting the material.
3. The companies Last autumn, we started a project with four partner companies aimed at improving the inspection process. This paper presents our experiences and proposals in the case of two of these companies. That is because we have in practise not yet started our project with the other two companies and their inspection problems differ from those of the companies presented here. The other two companies have a software development process that may be suitable for conventional inspection, and our interest with them is focussed on metrics and other statistical information regarding inspections. In addition, one of these companies is going to develop Inspection Window for commercial use and will merge it with its own Follow-up tool (supports the monitoring of quality control processes).
3.1. First Company Our first partner is working in a very well stabilized area of software development, so that its needs for new software are not significant, having mostly been already fulfilled. More important than new products is the maintenance old software. Maintenance is usually divided into four parts (c.f. Pfleeger [7]): corrective maintenance, where one makes correction because of failures, adaptive maintenanc, when a change made in on part requires changes to other parts, perfective maintenanc, improving the system for future use, and preventive maintenance, to prevent future failures. Adaptive and perfective maintenance are the usual cases in this company. (We would include in this the adding of new features to a system.) Because of the nature of maintenance-oriented development, we could not suggest a conventional inspection process for code reviews except in some minor cases. We could offer two reasons. Firstly, the conventional inspection process suits best for new software, but encounters problems in the case of old code to which some changes have been made. The suggested maximum number of lines inspected (literature) is 100. In maintenanced-oriented coding there might be only 100 new lines but these could be embedded in hundreds or thousands lines of older code. The problem with conventional inspection is that inspectors have to check both the new code and its effect on the older code. Secondly, in this company the typical work group for a certain change is two employers - the coder and his/her foreman. The inspection group should be a minimum of four persons and all of them from outside the project. Recruiting an inspection group for this purpose would be difficult. Besides, the author may sped less time on the item than four inspectors together. For these reasons we have proposed pair review for the code and design phases. Naturally, a conventional inspection would be suitable for a totally new module, although, we have decided not to recommend it at this stage, because we are following the "painless" procedure and are afraid of introducing too many new things at the same time.
Project 1 Requirements and specifications
Walkthrough for specifications with scenarios
Design and coding
Project 2 Implementation
Figure 3: The process of producing new software in the first company. A true project is established when there is a major change or even a totally new program. Actually, there will be two projects. The first will be to build a specification, after which there will be a decision as to whether the software is to be produced. The second project will then be to create the desired software. There may even be a year between the end of the first project and the start of the
second, and the staff employed in the second project may be totally different from the first. We have proposed a walkthrough at the beginning of the second project in order to verify the specification with representatives of the customer (see Figure 3). We do not doubt the capability of the first project, but the time interval may be so long that the customer's needs will certainly have changed slightly. Furthermore, the staff in the second project would profit from understanding the customers' needs better. As we have indicated [3], a walkthrough is the most informal review method. A customer's representatives are seldom computer professionals, and we consider the walkthrough method the best for "amateurs." Participants need not prepare for a walkthrough, and a computer professional can be the leader, presenter and scribe in the meeting. One method used in a walkthrough is a task scenario script describing how a user performs the task on one occasion (c.f. [8]). We have already noticed that the staff of the second project must consider one danger in walkthrough with the customer. They must not sell their own idea of the software for the customer but endeavour to check if the specification fulfils the customer's needs.
3.2. Second company By contrast with the above, our second partner company is working in a non-stabilized area of software development, and usually has to produce totally new software, so that maintenance programming is rarely sufficient for its purposes. Because of the highly dynamic software development area, the company endeavours to freeze specifications as late as possible. To make software development faster, they use the principles of concurrent engineering (see e.g. [9]). Between the specification and integration phases, the project is divided into subprojects that design and code concurrently. These subproject teams are largely autonomous and have relatively independent staff with equal rights to make changes and to get all data relevant to the current project. Requirements, specifications and integration of the results of subprojects are all considered together, since the project has one single product under construction at a time. The development process in the company is illustrated in Figure 4. The subproject threads start simultaneously after the specification phase. It is difficult or even impossible to divide specifications into equal fragments or to predict what change requests might be made. Consequently the subprojects end at different times and sometimes some of them may end after the integration phase has had to begin. Due to concurrent slices there will be a number of connections between slices. We have thus interfaces by which the subprojects are synchronized (see Figure 4). Change requests cannot be accepted for ever, of course. The subproject phase will have a deadline for freezing specifications and design plans. As an individual change request from outside the software development project usually focusses on a single subproject, the design plans differ at that time. We propose a virtual interface inspection for syncronizing, since it is important that every one takes part in the inspection and personal timetables differ. Besides, if it were a normal face-to-face meeting, the preparation would be very demanding as everyone would have to collect all the interfaces of interest. In a virtual inspection the process can be iterative. First the staff list interfaces of modules that they regard as important for themselves, and each unit then publishes those of its interface plans that others have ordered. After some iterations and some negotiations (when there is disagreement) there will be a predetermined deadline. No one can complain once the time has expired. Unfortunately we have not gathered any experiences as to how they have succeeded, as the subproject phase started too early this time. Anyway, the staff has recognized that the greatest risk will be self-discipline. Each participant should understand the meaning of a deadline.
Prestudy Feasibility Milestone Interface inspection
Integration Figure 4, The software development process in company 2. The design and coding threads starts simultaneously but end at different times.
4. Conclusion An inspection is regarded as a good way to detect problems in software development products. Unfortunately, people tend to feel that inspections are too strict for ordinary use. Consequently we have developed some new types of inspection, as we believe that it is possible to design a proper type of inspection for any occasion. There is no excuse for saying that an inspection has had to be abandoned because it does not match the software development process.
5. References [1] Gilb T.and Graham D. "Software Inspection", Workingham, Addison-Wesley, 1993 [2] Iisakka J.and Tervonen I. "Painless improvements to the review process," Software Quality Journal, 7, 1998, pp. 11-20 [3] Tervonen I., Iisakka J. and Harjumaa L. "Software Inspection - a blend of discipline and flexibility." ENCRESS-98, published in Project Control for 2000 and Beyond, Kusters R., Cowderoy A., Heemstra F., and Trienekens J., (ed.) Shaker Publishing B.V., 1998, pp. 157-166. [4] Freedman D. P. and Weinberg G. M. "Handbook of walkthroughs, inspections and technical reviews : evaluating programs, projects and products." Boston, Little, Brown and co. 1982. [5] Harjumaa L. and Tervonen I. "A WWW-based tool for software inspection." HICSS-98, vol III, 1998, pp. 379-388 [6] Tervonen I., Harjumaa L. and Iisakka J. "The virtual logging meeting: a web-based solution to resource problems in software inspection," Sixth ECSQ (European Conference on Software Quality), 1999, (to appear) [7] Pfleeger S. L. "Software engineering", London, Prentice Hall, 1998. [8] Redmond-Pyle D. and Moore A., "Graphical user interface design and evaluation," London, Prentice-Hall, 1995 [9] Linton L., Hall D., Hutchison K., Hoffman D., Evanczuk S. and Sullivan P. "First Principles of Concurrent Engineering - A Competitive Strategy for Product Development." Technical Report, CALS/Concurrent Engineering Working Group - Electronic Systems. 1992.
This paper is an extract from Project Control for Software Quality, Editors, Rob Kusters, Adrian Cowderoy, Fred Heemstra and Erik van Veenendaal. Shaker Publishing, 1999. ISBN 90-423-0075-2. This book represents the proceedings of ESCOM SCOPE 99 Conference, a joint event representing the 10th conference on European Software Control and Metrics, and the 2nd conference of the SCOPE network of European evaluators of software product quality. The conference was held on 27-29 May 1999, Herstmonceux, England. The paper has been downloaded from http://www.escom.co.uk/escom Permission is given for it to be printed, electronically stored and distributed and photocopied but may not be sold or included in a product for sale, and the document must be maintained complete and without modification of content. Copyright © Shaker Publishing, 1999. ESCOM Conference Office, 1 Walstead Cottages, Walstead, West Sussex RH16 2QQ, United Kingdom. Email
[email protected]
