Reading Techniques for Software Inspection: Review and Analysis. Rufus O. Oladele
Department of Computer Science
University of Ilorin, Ilorin, Nigeria.
[email protected] Abstract
Software inspection is a proven method for software quality assurance. Proposed in the literature is an array of various kinds of reading techniques that can be applied to software artefact under inspection. In this paper, a taxonomy, classification and comparative analysis of these reading techniques are systematically discussed. The current and emerging trends regarding research in the area of reading techniques are enumerated.
Keywords: reading techniques, software artefact, software inspection, quality assurance.
1
1.
Introduction
The Problem
The entire software lifecycle is characterized by the production of deliverables and documents such as requirements, design, source code, test plan, and final application. In order to assure the quality of the final product, each software artefact must be placed under some form of review and examination at various stages in the life cycle. Consequently, software reading – the process of understanding a software document in order to accomplish a task, is an important activity in software development. Software reading is however done in an unstructured or ad hoc fashion. The reason being that developers are trained how to write artifacts but are seldom trained how to effectively read and analyze them. Developers are therefore left with the option of acquiring reading expertise by personal experience. This poses a problem to software inspection and tends to slow down inspection process for the following reasons:
(1) Reading expertise can only be acquired slowly (2) The expertise acquired by personal experience might be of little help (3) The expertise might be difficult to share among readers.
In summary, the lack of formal training in software reading coupled with the inherent problems of acquiring this training on the job make software inspection a daunting task for software developers.
2
The Solution
Recent work in software reading techniques seeks to improve software reading. A reading technique can be defined as a series of steps for the individual analysis of a textual software product to achieve the understanding needed for a particular task. This definition is made of three major parts. First, the series of steps guide the reader towards achieving the goal of the technique. Defining a concrete set of steps provide all readers with a common reading process which can further be enhanced depending on readers’ experience. In contrast, in an ad hoc , or unstructured, reading process, the reader lacks direction and guidance on how to read hence readers use their own tacit reading processes. Without a standardized process, improvement of the process is much more difficult. Secondly, a reading technique is meant for individual analysis, meaning that the aim of the technique is to support the understanding process within an individual reader. Lastly, the techniques seek to equip the reader with the understanding needed for a particular task, meaning that reading techniques have a particular goal and they strive to produce a certain degree of understanding associated with that goal. The series of steps given each reviewer consists of two major components: A formal procedure that can be followed to focus on only the information in the review document that is important for the quality aspects of interest, and the probing questions about the newly uncovered information, which can help in tracking defects.
Reading Techniques for Defect Detection
Software inspection aims to ensure, through defect detection and removal that a particular software artefact is complete, consistent, and correct. Typically, inspections require 3
individuals to review a particular artefact, then to meet as a team to discuss and record defects, which are sent to the document’s author to be corrected. Software reading techniques can be employed to improve the effectiveness of software inspection by improving the effectiveness of individual reviewers. Reading techniques provide procedural guidelines containing tested procedures for effective individual inspection – a step that is often neglected in the state-of-the-practice inspection process. Reading techniques combine and emphasize three “best practices” that are very helpful for effective inspection. Although any of the three practices can be useful in itself, their integration in a unified inspection approach has been shown to be particularly valuable. The practices are as follow: ·
Giving each reviewer a particular and unique focus (perspective) on the document under review. Empirical studies in the literature have shown that reviewers work better when they have a clear focus than when they feel themselves responsible for all types of defects, in the entire document. Besides, having a unique focus implies that each reviewer has clear responsibility for a certain aspect of the document and cannot rely on another reviewer tracking any missed defects.
·
Making individual review of a document an active (rather than passive) undertaking. Reviewers tend to make a more thorough review when they are actively engaged in working with the information contained in a document than when they can get by with merely reading it over. This is an important philosophy driving the development of certain inspection approaches such as Active Design Review.
·
Articulating a taxonomy of the types of defects of interest, and giving the reviewer an understanding how to look for those types of issues during the 4
individual review. Reviewers do a better job of reviewing documents when they have a good idea of what they are looking for. In tailoring reading techniques to specific project environments, defect taxonomies must be made explicit and reflected in the questions that are given to the reviewer.
In the rest of the paper, we focus on taxonomy of reading techniques in section 2 while the techniques are put under 2 broad classes in section 3. Section 4 presents a comparative analysis of the techniques. Emerging research trends in the area of software reading techniques are stated in section 5. Finally, section 6 summarizes and concludes the paper. 2.
Reading Techniques: Taxonomy
2.1
Ad hoc Reading
Typically, no technical support is provided. Inspectors only count on their own skills, knowledge and experience to identify defects. At most inspectors are taken through some sessions of training in program comprehension before the review process starts. 2.2
Checklist-based Reading (CBR)
The original inspection method includes the idea of using checklists in defect finding. In CBR the artefact is inspected by means of using a pre-defined list of questions – a checklist. This technique is more structured and is believed to offer more support to the reviewer compared to ad hoc reading. Owing to its simplicity, the method is believed to be one of the most accepted inspection methods in the industry.
5
2.3
Reading by Stepwise Abstraction
This is a technique proposed to correct some of the inadequacies of traditional checklist-based reading. The technique “Reading by Stepwise Abstraction” was described in the context of the verification-based inspection approach. This is an inspection variation used in conjunction with the Cleanroom software development method. Although this method requires the author(s) to perform various inspections of the work products, the inspection process itself is not well described in the literature. It is however clear that it consists of at least one step, in which individual inspectors examine the work product using “reading by stepwise abstraction”. This technique requires the inspector to read a sequence of statements in the code and to abstract the functions these statements compute. This procedure is repetitively carried out until the final function of the inspected code artefact has been abstracted and can be compared to the specification. The technique is only applicable to code artefact, it nonetheless provides a more formal approach for checking functional correctness.
2.4
Active Design Review (ADR)
This is another reading method aimed at fixing the weakness of traditional checklistbased reading. In the original CBR, all inspectors use the same checklist in finding defects. The authors in [3] criticized this approach and shared the task of finding defects among different roles in their Active Design Reviews. Another major idea in their method was to make reviewer’s role more active than in traditional checklist-based reading. Active Design Review is an important inspection variation as ADR inspectors are guided by a series of questions posed by the author(s) of the design in order to enhance a thorough defect detection step. Thus, inspectors get reading support when scrutinizing a design document. Although 6
little empirical evidence shows the effectiveness of this approach, other researchers have used it as the thrust of their inspection methods.
2.5
Defect-based Reading (DBR)
The main idea behind defect-based reading is for different inspectors to focus on different defect classes while reviewing a requirements document. For each defect class, there is a scenario consisting of a set of questions an inspector must answer while reading. Answering these questions helps the inspector primarily to detect defects of that particular class. The defect-based reading technique has been validated in a controlled experiment with students as subjects.
2.6
Perspective-based Reading (PBR)
When inspecting a document using PBR the reviewers adopt a certain perspective, for example designers, testers or a system user’s perspective. Inspections are performed by actively examining artefacts, for example when inspecting requirements specifications persons working from the designer perspective create a high level design, user perspective produces users manual and test perspective creates test cases. The idea is that by adopting a certain perspective the reviewers focus on issues relevant to this perspective which can increase the likelihood of finding defects. Reviewers using different perspective find faults that are relevant to the specific viewpoint they adopt. For instance, tester perspective may find faults related to testability of the requirement, user perspective may find missing requirements and designer perspective may discover incomplete or inconsistent requirements. 7
It is believed that the combination of various viewpoints will provide a better coverage. The studies performed on PBR have shown that PBR may require more effort compared to lessstructured approaches, but at the same time has a higher defect detection capability. In a typical case, developers perform PBR employing different viewpoints. 2.7
Usage-based Reading (UBR)
Many reading techniques focus on finding as many faults as possible, regardless of their importance. The inspection effectiveness is often measured in terms of number of faults found, without taking consideration of the severity of each fault. The principal idea behind UBR is to focus reading effort on detecting the most critical faults in the inspected object. Hence, faults are not assumed to be of equal importance, and the UBR method is aimed at finding the faults that have the most negative impact on the users’ perception of system quality. The UBR method focuses the reading effort guided by a prioritized, requirementslevel use case model. In order to specify the users’ perception of system quality, use cases are prioritized. The order of the use cases reflects what a user or a group of users regards as most important in the system under development. The prioritization can be made by, for example, pair-wise comparisons according to the Analytical Hierarchy Process [6]. UBR employs a set of use cases as a vehicle for focusing the inspection effort.
2.8
Traceability-based Reading (TBR)
Traceability-based Reading centres on how to perform consistency and correctness checks among various working products. The method is divided into vertical and horizontal 8
reading. Vertical Reading is to check the consistencies of a work product to its refinement, while horizontal reading is to detect the violations of the work product being reviewed to the others from the same software development phase. 2.9
Functionality-based Reading
This reading approach is based on Function Point Analysis (FPA). FPA defines a software system in terms of its inputs, files, inquiries, and outputs. The scenarios, that is, the Function Point Scenarios, are developed around these items. A Function Point Scenario consists of questions and directs the focus of an inspector to a specific function-point item within the inspected requirements document. 2.10
Task-driven Inspection
Kelly and Shepard have created a code inspection method, which they call task-driven inspection. It is not only a reading technique, but also includes some elements related to the inspection process. Their method includes the same kind of idea as the abstraction-driven technique by Dunsmore et al, but Kelly and Shepard have defined the task more specifically. The inspector has to create a data dictionary, a complete description of the logic and a crossreference between the code and the specifications
3.
Reading Techniques: Classification
All reading techniques proposed in the literature to date can be put under two (2) main classes’ namely passive reading approach and active reading approach. In passive reading
9
approach inspectors follow a sequence of individual steps (e.g. a given checklist). However, inspectors are given the freedom to figure out how best to proceed with the reading process.
In active reading approach, inspectors are not only equipped with what steps to follow they are also guided as to how to carry out the steps during the reading process. This approach among other things provides details on the inspection process (hoe to perform an inspection), includes a separation of perception (what to inspect) e.g. focus on different defect severity classes, defect types, etc. It also provides guidance through the most important parts of the document even as it supports inspectors in their defect detection process.
Passive reading approach
Active reading approach
CBR
DBR
Reading by Stepwise Abstraction PBR ADR
UBR TBR Functionality-based Reading Task-driven Inspection
Table 1: Classification of Reading Techniques
10
4.
Comparative Analysis of Reading Techniques
There is no general prescription about which reading technique to use in which circumstance. However, in order to compare them, we set up the following criteria : Application Context, usability, Repeatability, Adaptability, Coverage, Training, and Validation. The criteria are to provide answers to the following questions: ·
Application Context: To which software product is a reading technique applicable and to which software products has a reading technique already been applied?
·
Usability: Does a reading technique provide prescriptive guidelines on how to scrutinize a software product for defects?
·
Repeatability: Are the results of an inspector’s work repeatable, i.e., are the results such as the detected defects, independent of the person looking for defects.
·
Adaptability: Is a reading technique adaptable to particular aspects, e.g., notation of the document, or typical defect profiles in an environment?
·
Coverage: Are all required quality properties of the software product, such as correctness or consistency, verified in an inspection?
·
Training Required: Does the reading technique require some training on the part of the inspectors?
11
·
Validation: How was the reading technique validated, i.e., how broadly has it been applied so far.
The table below characterizes each reading technique according to these criteria. We use question marks in cases for which no clear answer can be provided.
Reading
Characteristics
Technique Application
Usabilit
Repeatabilit
Adaptabilit
Context
y
y
y
Coverage
Training
Validation
Require d
Ad –hoc
All products
No
No
No
No
No
Industrial practice
All products
No
No
Yes
Case
No
Dependen Checklist-
Industrial practice
t
based Reading Reading by
All
products
Yes
Yes
No
High
for
Yes
stepwise
allowing
correctnes
mainly
Abstraction
abstraction,
s defects
Cleanroom
functional code Active
Design, Design
Applied in
projects Yes
Yes
Yes
Design
?
?
Initial case study
Review
12
Defect-based
All
products,
Reading
Requirements
Perspective-
All
based
requirement,
Reading
design, code
products,
Yes
Case
Yes
High
Yes
validation
Dependent Yes
Yes
Experimental
Yes
High
Yes
Experimental validation & initial industrial use
Usage-based
All
products,
Reading
requirements,
yes
Yes
Yes
Yes
High
Experiment al validation
design, code Traceabilitybased
All products Yes
Yes
Yes
High
Yes
Experiment al validation
,requirement
Reading
s Functionality
All
products,
-based
requirements,
Reading
design, code
Task-driven
Code, Code
Yes
Case
Yes
?
Yes
dependent
Yes
Yes
al validation
No
Inspection
High
for
correctnes s defects
13
Experiment
Yes
Experiment al validation
5.
Emerging Research Trends in Reading Techniques
5.1
Object-Oriented Reading Techniques
There is active research going on in this area. Object oriented reading techniques are an interesting topic of research for a variety of reasons. As OO analysis and design are gaining popularity, it becomes imperative to tailor software development technologies to work in the object oriented world. Also, work in this area combined with previous research can help illuminate high level questions in reading technique research. 5.2
Reading Techniques for Web-based Systems
The lack of substantial work on web project inspection and the unstructured nature of most web development process informed the need to develop reading techniques for facilitating the inspection of web software application. Conte, et al [8] have done some work in this area. However there are still many open problems in this research area. 5.3
Architecture Reading Techniques
Very recently, a new set of reading techniques tagged Architecture reading techniques (ARTs) has been proposed in the literature. The authors provide an overview of these new techniques and report on initial feasibility study. The result of their feasibility study revealed that the techniques were useful and seen by the subjects to provide benefit over a checklist-based approach. More experimental studies are needed in this research area to fully investigate the strength and weakness of this new set of reading techniques.
14
5.4
Building a Body of Knowledge about Software Reading Techniques.
The above named research topic has become the concern of researchers in the area of reading techniques. Meta-analysis has been cited to be a promising vehicle for use in software engineering research. However, initial attempts to apply it to studies of software development technologies have not been successful [9]. The authors in [10] advocate a more informal approach to building a body of knowledge while the field of software engineering matures to the point of using meta-analysis. One of the main techniques used by the authors in building these bodies of knowledge is that of replications. 6.
Summary and Conclusion
In this paper, we have reviewed most of the commonly investigated software reading techniques in the literature. Despite the fact that many reading techniques have been and are still being proposed. Virtually all of the newer techniques have not found relevance in the industry. In particular, ad hoc reading or checklist-based reading still remains the standard practice in the industry. The possible reason for this trend is; either the industry is unwilling to change owing to costs (cost of initial training of their staff on the use of new promising technique) that will be involved or that the researches on many of these new reading techniques are inconclusive. We conjecture that the second reason has ninety-nine percent contributions to the non-use of the said reading techniques. We therefore hope that researchers will raise their work so that these newer techniques will gain wide acceptance in the industry.
15
References
[1]
Oliver Laitenberger. A Survey of Software Inspection Technologies, Handbook on Software Engineering and Knowledge Engineering, Vol. II, 2002.
[2]
Olalekan S. Akinola, Adenike O. Osofisan. An Empirical Comparative Study of Checklist-based and Ad hoc Code Reading Techniques in a Distributed Environment, (IJCSIS) International Journal of Computer Science and Information Security, vol. 5, No. 1, 2009.
[3]
David L. Parnas and David M. Weiss. Active design reviews: Principles and Practices. In Proceedings of the 8th International Conference on Software Engineering, pages 215-222, August 1985
[4]
Michael E. Fagan. Design and code inspections to reduce errors in program development, IBM Systems Journal, 15(3): 182-211, 1976.
[5]
Michael E. Fagan. Advances in software inspections, IEEE Transactions on Software Engineering, SE-12(7): 744-751, 1986.
[6]
.
T. Thelin, P. Runeson and C. Wohlin. Prioritised Use Cases as a Vehicle for Software Inspections. IEEE Software, 30-33, 2003.
[7]
T. Thelin, P. Runeson and B. Regnell. Usage-Based Reading – An Experiment to Guide Reviewers with Use Cases. Information and Software Technology, 43 (15): 925-938, 2001.
[8]
T.U Conte and G.H. Travassos. Reading Techniques for Web-based Systems.
16
[9]
J.
Miller.
Applying
Meta-Analytical
Procedures
to
Software
Engineering
Experiments. Journal of systems and Software 54 (1):29-39, 2000
[10]
F.Shull, J.Carver, G.H. Travassos, J.S. Maldonado, R. Conradi and V.R. Basili. Replicated Studies: Building a Body of Knowledge about Software Reading Techniques.
17
