How to inspect minor software projects - CiteSeerX

How to inspect minor software projects Juha Iisakka, Ilkka Tervonen and Paula Hiitola

Abstract Software inspection is advertised as the most effective way to find bugs in software documents. Unfortunately, practically all inspection/review methods focus on projects that are developing new software and involve more than two employees. Software companies also have small projects in which one or two employees are working on job lasting less than one man-month, e.g. maintenance-oriented changes to existing software systems. We have developed a principle for managing the quality process in the case of these minor software projects.

1. Introduction There are three ways to improve the quality of software documents1. The first and most important way is naturally for the authors of the document to do their work well, the second way is to review the document, i.e. that they ask another person to check what they have done, and the third, which is possible only if the software document is a code of some sort, is to use or test the document with a computer. As long as human beings make errors, we cannot rely on the first way alone. The third way - testing - has traditionally been more popular than the second, but software development can be a very long process and the testing is arranged at the end of it, so that an error arising at the beginning may prove costly if it is only found much later. That is why reviews are also needed. Inspection has been regarded as the most efficient way to organise reviews after Mr. Fagan introduced the idea in IBM in the mid1970's [1]. Unfortunately, it seems that inspections are not so popular with smaller organisations and projects as with larger ones. Most published case histories illustrating the success of inspections, for instance, are concerned with large companies. We have developed a way to handle inspections in small projects, and this principle will be explained in this paper. Although the principle has been developed for one particular Finnish company, we are sure that it has many parts that can be easily adapted for other companies. The principle has been developed with assistance from the employees of that company, and this study is part of a research project financed by the Finnish government and the partner companies. This paper is structured as follows. Section 2 will discuss the differences between inspection and review, and it will present some ways to of organising inspections. Section 3 will provide some relevant information on our company's way of producing software, and section 4 will introduce the principle we have defined for the company and recount some experiences.

2. Reviews and inspections This chapter will first discuss the use of the terms review and inspection, and will then present some ways of organising inspections. 1

We mean by "document" all kind of creations that people have made inside software development. These can be a code, a specification, a technical report, the painting of a program logo, or even a simple invoice, for instance.

2.1. Reviews versus inspections The IEEE Standard for a review is: An evaluation of software element(s) or projects status to ascertain discrepancies from planned results and to recommended improvement. This evaluation follows a formal process (for example, management review process, technical review process, software inspection process, or walkthrough process) [2]. As this standard defines it, the review is a universal term in this field. By contrast, the inspection is the most disciplined, defined efficient and formal version of all reviews. Moreover, the inspection has the best defined process of all; Gilb [3] limits it to determing, if the people and process have complied with the current recommended best practices for a particular kind of specification. Nonetheless, we are breaking these definitions. That is why we have found that many information professionals (at least in Finland) understand a review slightly wrongly, regarding it as a meeting with expected preparation in which people check documents (plural!) to find bugs. There is no problem with that definition, but they use "check" too extensively. Typically, a review meeting gathers together numerous project members often to consider documents from many authors. As Graham [4] states, a review meeting is difficult to arrange, so one may be held even though the documents are not yet worth reading. When it is noticed that some features are missing, the meeting agrees who will produce the missing part. This part may have originally been assigned to a different person, but some people may be behind schedule and others in advance. Also, if there are change requests these can be agreed on at same review meeting. This occasion is very close to Gilbs [3] "GO NO-GO review," but its major goal is still to find bugs. We have complied with this misunderstanding and have found that we are understood better when we use "review" in this GO NO-GO sense and "inspection" as a universal name for ways to find defects in documents by reading them. We gather under Inspection both "real" inspections and some other review types.

2.2. Ways of inspection We have presented in a previous paper [5] some new ways to organise an inspection meeting. A formal, strict inspection meeting is highly efficient, and unfortunately part of this efficiency may be lost when using the new ways. However, companies must sometimes make compromises between efficiency and the costs of inspection. Consequently, these ways have: ¥ Place and/or time independence. It is very common to experience difficulties in gathering the group together in the same place at the same time. A virtual inspection may be implemented at the same time but in different places, for example, or at different time and in different places. This makes reconciliation of the participants' timetables easier. A logging meeting can even be unnecessary when the comments from the other inspectors regarding the discussion between the author and an inspector are irrelevant. ¥ Network, especially webtechnology; provides a base solution for place and/or time independence that the virtual inspection needs in order to manage distributed inspectors and their comments. ¥ Varying numbers of participant. The conventional inspection has 4-8 participants. What if we cannot gather so many together? Pair inspection has the minimum number, that is, two members - the author and an inspector who is checking the author's document. When not every inspector takes part in the inspection meeting to discuss comments, we have a case of a limited logging meeting inspection.

The principle we have defined for handle inspections in minor projects benefits one of these types. Pair inspection is useful for small documents.

3. Our company As noted in our previous paper [6] our company is working in a very well stabilised area of software development, so that its needs for new software are not significant, having mostly been already fulfilled. More important than new products is the maintenance of old software. As we all know, typical duties in maintenance oriented programming require less manpower than those involved in programming totally new software. Maintenance-oriented software development is characteristically a matter of making some minor changes to existing software (unless a user finds a bug, of course). Software production in our company is based on customer orders. It is not permitted to do anything without a customer's written order. In fact the order is not usually written by the customer but by an employer of our company. First, there is a need for something to be done and the customer issues an unofficial order (telephone conversation, email etc.). Then, a quotation is drawn up in which work is detailed and the expenses are calculated. Lastly, the customer accepts that offer if the job is considered worth the price. Nowadays the quality process is hidden from the customer, who does not know what quality tasks are involved when the order is fulfilled. Naturally, the staff will always endeavour to do their best, but as we all know, to err is human. It is planned that customers in the future will be given two prices (and timetables) with the offer - one with a light quality process and the other with a heavier one. This means that customers should understand that quality has its costs. After the order has been established, it has to be decided whether it will be performed as an ordinary project or a minor one. An ordinary project contains everything that belongs to projects everywhere: a steering group, a project manager, a project plan etc, while a minor project (or as some people in our company call it, "an undertaking") is a lighter process. There are less reports, groups and controls, and there is typically no project plan. The order may be very easy (e.g. to change text in a user interface window), so that one employee can do the work right away. Although, it is a principle that a lighter order will have minor project routines and a more demanding order will lead to an ordinary project there is no fixed principle when an order needs a real project. It is normal practice, however, to establish a real project when fulfilling an order that • covers more than one phase in software development, i.e. the traditional requirements, specification and design phases etc. • is difficult to fulfil, so that many experts are required • is laborious and requires more than three man-months. The company does have some inspections nowadays, but it is very haphazard matter what documents are inspected, and far too many receive no review at all. The normal case in our company is to have only one review in a minor project, checking all the documents at the same time. The meeting lasts hours and there are normally hundreds (or even thousands) of pages to go through. Furthermore, the preparation is usually poor, so that in the end the documents are reviewed very shallowly. The company's information system is one department among others (even though the biggest). The other departments include security and telecommunications, for instance. If the document needs expertise that belongs to another department it is usual to send it to the

appropriate place for comments. Someone in that expert department then checks the document from an expert viewpoint, possibly using the company's official rules and checklist. Although the company's inspections have been poor, we have found this use of experts well justified, as we have met with companies in which one cannot ask outsiders for help without losing face.

4. The principle When, with an assistance of the company staff, we developed our principle at having inspections in minor projects, we had two aims in view. First, the use of experts should be included because it was a good habit. Second, implementation of the principle should be as painless as possible2. One idea was that not all documents need to be inspected. Once, the staff have become used to the principle, it will be fairly easy at some time in the future for the management to increase numbers of documents to be inspected. The principle is fairly simple and needs no big investments. Naturally, it is no guarantee of better quality, as the ultimate key to quality is always how well people do their jobs. The principle is presented in two parts. Firstly, the management and inspection levels are introduced, and secondly, the inspection level is presented in more detail.

4.1. Process levels After an order for a minor project has been established the person responsible for it should write a quality plan, a sort of project plan for the quality process aspect of the project. This means deciding how many quality tasks (inspections and/or tests) there will be in the project. Every project document must be listed, with type of inspection that should be used. One of our key ideas in the principle is that not all documents need to be inspected but that the decisions must be written down in the quality plan, since this will be written before the project itself starts. All documents should really be inspected, of course, but there are many reasons why not, and some reasons are better than others. We can list some reasons: • The author of a document does not want this, perhaps even being afraid of an inspection because the document is so poor. • There is no time, as the project is late. • There is no time, as the order has to be fulfilled promptly. • The document is very simple so that the chances of a severe defect are minimal • Possible defects in the document would not be dangerous All except the first two reasons can usually be known before the project starts, and these latter three are more acceptable reasons, to the extent that they can be written into the quality plan. Also, when one writes down that a document will have no inspection, the matter has been properly considered, and the quality plan constitutes a formal document that will be kept in the company's files. If someday there is a catastrophe because of the uninspected document, it will be known who is responsible. When a quality plan has to undergo inspections, a quality representative must be chosen. The company will name this person as auditor because the work concerned is to audi the project and check that it conforms to the quality plan and the company's quality standards. An auditor should generally be independent of the organisation that is being audited [2]. This should also be the case with the auditor of a minor project, who should be able "distance" 2

We have considered painless improvements in a previous paper [5].

him/herself from the problems encountered in it. The auditor should be an equal colleague for the project members - not a manager - a person who could just as well have been in that project. It follows that the purpose is not to check on employees but on quality. There is nothing abnormal in the prospect that a member of a project will some day be auditing the auditor's project. Not everyone can be an auditor, however. The company has to train people for this and they ought to be experienced workers in order to understand what project is doing. The quality process should not retard the project. Besides, it is not motivating to have to advise or teach one's auditor, who would anyway probably give only unrelevant comments. As mentioned above, an auditor is not a member of the project, and auditing can never be a main duty. Of course, auditing must be done well, and so auditors need deduction to their main tasks. After being appointed, they should take part in writing the quality plan with the person responsible for the order. Once the project is working the auditor will check that all the documents are inspected or tested as planned and will be the leader at the inspection meetings. Finally when the project has ended, the auditor will organise the acceptance review. Illustrated in figure the quality process for a minor project starts with a quality plan and ends with an acceptance review, between which lies the "real work" of testing and inspecting. The acceptance review has two purposes, to reach a conclusion regarding a quality and to collect together lessons learned. It is that meeting that checks whether the project followed its quality plan. It is not the purpose to improve the quality of the products on that occasion, since at all events the project is over and the customer will receive the order. The decision that the project is finished and the product can be delivered to the customer will have been made before the acceptance review. Nevertheless, the meeting has the following purposes: • To check all test and inspection records, to ensure there are no defects that the authors have forgotten to correct or mark as corrected. • to discuss whether it was a good decision to leave a certain document uninspected, or whether some of the inspections were efficient • To determine a quality of order as fulfilled for the customer • To collect together the lessons learned. • To be an occasion when employees can discuss what they would do differently in the next project. The meeting is informal and participants are all the project members, the person responsible for the order and the auditor (the person responsible for the order is a foreman or one of the authors). The meeting is relatively short, and "very minor" projects have even shorter meetings.

(management level) Acceptance review

Quality plan Experience knowledge

Inspection

Issue Log

Issue Log

Issue Log

(real work)

Inspection 1

Inspection 2

Inspection n

Our first idea was that the customer should receive a very honest evaluation of the project, "graded" on scale of "poor," "normal," "good" and "excellent," but the employees were strongly against that, as it would have been very difficult to evaluate a colleague's work in that way. Besides, a grade of "poor" for one's own work would have been extremely humiliating. On the other hand, the customers would have benefited from knowing what to expect from the product. As a compromise, the evaluation scale has three choices: "not evaluated," "fulfils company standards," and "fulfils company standards with exceptions." These exceptions have to be listed and they provide information for the customer about untested or uninspected parts, for instance. Sometimes the order is so prompt that a Beta-version is enough. Apart from evaluating, the acceptance review has another purpose. It is the occasion when the lessons learned are gathered together. It is possible to write comments for the acceptance review at the end of inspection records. These and other possible comments are then noted in the review and discussed. The auditors have a few meetings a year with the quality personnel in which they can revise the company's quality improvement rules.

4.2. Inspections When a minor project is begun, the quality plan indicates what project documents will be inspected and how. Of the four new types introduced in section 2.2 the pair inspection and a normal inspection were regarded as suitable for our company's purposes. Employees can be brought together fairly easily, so that virtual inspection is unnecessary. Similarly, the no logging meeting inspection was considered unsuitable for the company at this moment. This inspection type demands that inspectors do their duty well and autonomously, and a meeting is a good place to ensure this. Meetings may become unnecessary in the future when the inspection process is robust. A pair inspection is justified for small documents. This has iterative step by step discussions, and it is thus meaningless to formulate any strict rules of action. Due to the iterative nature of the process, the recording of "temporary" issues (defects) found in informal meetings is not required. The issues are typically marked on paper documents and corrected during the next correction cycle. In the final meeting however, the material that has passed the

pair inspection must be written down on a form. In addition, since the company is interested in continuously improving its quality process, the author and inspector together can also write down comments for the acceptance review. The pair inspector is usually named in the quality plan in order to be prepared for the task, and as a minor project may lack a project plan, the quality plan can contain a deadline for pair inspection. The inspector may be a companion from the information systems department (see section 3) or an expert from another department when special expertise would be useful. The inspector should use rules and checklists, as should be customary in all inspections. Perhaps the biggest danger with pair inspection is that the author may hand over the document to the inspector, say, "approve this" and stay there to wait for the reply. The inspector should always understand that it is necessary to take time over the document because of being partially responsible for it. If the document to be inspected is large3, very important or requires the expertise of many people, a normal inspection should be chosen. As in pair inspection, the quality plan should name the inspectors beforehand and give the deadlines for the normal inspection. As stated above, the auditor is the inspection leader. When the author of the document considers that it is ready for inspection, the auditor will be asked to call a meeting and the document will be sent to the inspectors. According to Gilb and Graham [7] the inspection leader (moderator) should approve the document for inspection so that time is not wasted on poor documents. In order to increase efficiency, however, every task in our company needs a payee. That means that the inspector has to bill the minor project for inspection of its document. Since the price is normally agreed with the customer, the budget is fixed and tight. Consequently, the author will scarcely want to have an unfinished document inspected, as this will not give full value from the use of inspectors. The role of company's experts in an inspection is exactly the same as described by Gilb and Graham [7]. In order to increase the efficiency of the experts the company should divide the checklist questions among their groups. If the checklist is long and has many questions that an inspector cannot consider, this may be omitted.

5. Conclusions The focus of this paper was on a principle for handling inspections in small projects. This principle is based on dividing the quality process into two levels. At the management level each project has a quality plan in the beginning and at the end an acceptance review which checks whether the quality plan was obeyed or not. At the real work level, all the project documents are assigned to be inspected, pair inspected or non-inspected. The overall idea is to decide the quality status when writing the quality plan. The company has had a bad habit of confusing reviews and project meetings together, the worst difficulty being to impress upon the inspectors that the meetings have to focus on the material to be inspected. As the author is responsible of quality of the document, this author will suffer if the inspection is not as good as possible. As the next step, the company has begun to apply this principle to large projects. Moreover, it is organising and standardising its inspection records so that it will be possible 3

The difference between "small" and "large" is a relative one, and should be decided by the auditor and the person responsibe for the order.

to analyse sources of defects in the future. If people classify defects as well as possible, the records will help them to analyse where "quality bottlenecks" exist.

References [1] Fagan M. E., Design and code inspection to reduce errors in program development, IBM Systems Journal, Vol. 15, No. 3, 1976, pp. 182-211. [2] IEEE Standard for Software Reviews and Audits, IEEE Std 1028-1988. [3] Gilb T., Software inspections are not for quality, but for engineering economics, draft article, available from http://www.Result-Planning.com, 1999. [4] Graham D., Inspection: myths and misconceptions, Presentation at: Nokia Review and Inspection Forum, Helsinki Finland, Nov 1th 1999. [5] Iisakka J. and Tervonen I., Painless Improvements to the Inspection Process, Software Quality Journal, 7, 1998, pp. 11-20. [6] Iisakka J., Tervonen I. and Harjumaa L., Experiences of painles improvements in software inspection, Proceedings of the ESCOM-SCOPE 99 Conference, Hermonceux, England, Appril 1999, Available as Project control for software quality, Kusters R. et all editors, Maastricht: Shaker Publishing, 1999. [7] Gilb T. and Graham D., Software Inspection, Wokingham:Addison-Wesley, 1993.