Jul 3, 1984 - Jones [2] describes that software quality is the absence of defects or ..... [2] T. C. Jones, âMeasuring Programming Quality and. Productivityâ, IBM ...
Assessment of Reverse Engineering Tools : A MECCA Approach. Torbjgm Skramstad
Md. Khaled Khan
Department of Informatics, University of Trondheim, AVH, N-7055 Dragvoll, Norway.
Abstract Quality and productivity factors
It is a general requirement in the software engineering field that quality and productivity should be taken into consideration. Software development tools can have significant impacts in assuring the quality of a software system and productivity of the development process. In a rapidly evolving engineering field such as software engineering, it is therefore important to select appropriate development tools. In this paper we discuss reverse engineering tool as a quality software development tool. An introduction to the reverse engineering tools, their impact on the development environment and their functionality in general are cited before a MECCA-model for assessing specific tools is proposed. The main objective of this paper is to introduce to our reader how a reverse engineering tool can be assessed in terms of quality and productivity.
What do we really mean by quality of a software product? Cavano and McCall [5] claim that there is no widely accepted definition of software quality assurance. There are lots of quality factors being proposed so far. Jones [2] describes that software quality is the absence of defects or errors. Chow [3] observes that absence of defects is insufficient in quality assurance. McCall [4] proposes 11 factors defining software quality. A more modem approach is to define software quality as reaching customers and users satisfaction and flexibility in effective maintenance at the operation level. This is also in accordance with IS09004. What about productivity in relation to quality? It is a general excuse that if you ensure quality, the productivity has to be slowed down. This is a dilemma for the development managers in the data processing departments. This attitude is however, not always true. How both quality and productivity could be maximized in development and maintenance environment has remained an open question in the software engineering field. The factors having great influence in quality and productivity of a software system are generally divided into two categories: process oriented factors, and Droducf oriented factors. Process oriented factors comprise organisational mode, staff, resources, management skills, experience, work motivation and so on. On the other hand, the product oriented factors include product attributes like reliability, correctness, maintainability, reusability, flexibility, testability and so on. To satisfy quality criteria, the managerial aspect must be taken with a certain degree of importance. The structure of the organisation, the experience and education of the development staffs, their skills and motivation, the availability of resources, development techniques and methods used, and the leadership expertise are important factors in assuring quality of a
1. Introduction Software development tools can have significant impacts in assuring the quality of a software system and productivity of the development process. In a rapidly evolving engineering field such as software engineering, it is therefore important to select appropriate development tools. Quality factors are not only important in the production process, but even more important during the post development stages. Quality is not merely related to the finished product alone, but it also has certain degree of impact on the processes involved in producing the product, because the attributes of these processes have great influence in the maintenance stage. Quality Management and Quality Assurance Standards [ll defines quality as “The quality system should be an integrated process throughout the entire life cycle, thus ensuring that quality is being built-in as development progresses, rather than being discovered at the end of the process.”
0-8186-2620-8/92$3.00 Q 1992 IEEE 131
software system. It also includes the effectiveness of the internal quality system audits[ll, inspection criteria, and the constant monitoring of the development process.
Modern tools in development environments The use of modem development techniques, methods and tools can conmbute strongly in increasing quality and productivity. In this perspective, the CASE (Computer Aided Software Engineering) tools have been successful in getting some attentions from the practitioners in the past few years. For example, Chikofsky and Rubenstein [6] predict that developer can achieve greater system development productivity and reliability by introducing new development tools like CASE technology in developing information systems. In this paper, it is neither our intention nor our objective to discuss CASE tools in general. However, we are particularly interested in reverse engineering tools either delivered with CASE package, or as independent tools. Reverse engineering tools assist maintainer to understand an existing software system at different abstraction levels prior to, or in parallel with the maintenanc work. In this paper, we present our views on how reverse engineering tools can improve both quality of the product and productivity in the development as well as the maintenance process. In subsequent sections, we discuss the definition of reverse engineering tools in general, and describe their functionality. We finally propose some criteria based on MECCA-model (Multi-Element Component Comparison and Analysis) in assessing reverse engineering tools.
2. Reverse engineering (RE) tools During the development process the conceptual level description of the software system is transformed into the logical level description and the logical level description is transformed into the physical level description. In the reverse engineering process, the logical level is recreated from the physical level and the conceptual level of the software system is reconstructed from the logical level. The transformation of a software system from the physical level to the higher level of abstraction can be done by step-wise analyzing the source code. The reverse engineering technique begins at the “bottom” of the development process, and raises its processing to the successively higher levels of abstraction. The process generally starts from the existing implementation stage of the system life cycle and proceeds to the subsequent higher abstraction levels, and gradually recapturing and recreating the underlying system design abstraction which has been implemented
during the system development time[7]. A reverse engineering tool helps the maintainer to understand an existing program, its behavior, its external as well as its intemal design structure. Software maintenance, and reusability require the reproduction of underlying design artifacts in the existing source code. This detection and reproduction of design description of an existing system is the major concern of reverse engineering tools. One of the main objectives of these tools is to assist programmer in the program understanding process.
Functions of reverse engineering tools A prototype of a reverse engineering tool is under development by the authors [151. In the light of this experience, we describe some fundamental functions of a reverse engineering tool. At the architectural level, reverse engineering tools basically should have three categories of process as follows:
Analyzing the source code format, and recognizing the program structure and semantics of underlying components. - Save the information in a repository. - Present this information to the user. -
The reverse engineering tool typically provides the following information to the programmer: The graphic layout of module chart, structure chart, call graph, control structures, data flow diagram etc. and the &textual representation of globally declared data structures, local data structures, parameter list and associated source code of an object can also be produced. Generally, the reverse engineering tool first generates large structural components, for example the sub-system structure, fundamental data structures and the functional module structure of an existing software system. Then it abstracts the various design information and visualizes them into different forms like call graphs, control graphs, and formal data flow diagrams. The next task of the tool is to keep track of the impact between the different design structures and the code that corresponds with them, A list of functions typically offered by the reverse engineering tool is given below : -
Module Chartnntermodular data flow dagram.
- Procedure Call Graphflntramodular data flow diagram. - Control graph. - Global Data Structure. - Local Data Structure. - Parameter list of each Module Interface. - Parameter List of each procedure.
- A fraction of corresponding S o m e Code
product. In the maintenance process, the programmer has to spend a great deal of effort and time in understanding the source code. It is a very time consuming process to manually read and comprehend thousands of s o m e code lines. It is equally difficult to keep track of s o m e code implementation details for the human brain. This process decreases productivity of the programmer as well as the quality of the product. A reverse engineering tool can assist the programmer to understand the source code more easily and confidently. The module charts, for example, provides programmer with a clear picture of which modules are affected if any modification is introduced into a particular module. The procedure call graph and control graph can assist programmer to mace the direction of control flow in far less time with far more accuracy than manually possible. Reverse engineering tools generates cross reference information and different types of graphical representation of source code that provides more information quickly and correctly. The programmer can utilize these pieces of information in his maintenance work. These ultimately improve productivity of the programmer and also the quality to the product. The module information stored in the reverse engineering repository can be used by the involved staff more confidently during code reviews, inspections and step-wise-walkthroughs. So the scope of quality control could be increased. The structure charts and module graph can be used to determine overall software modularity, degree of cohesion and depth of control [8]. The procedure call graph representation, and module graph can provide a clear picture of the complex interface, the type of direction of information flow between the components, the depth of the hierarchy, and the level of coupling between modules. Programmers have more opportunity to cross-check whether the software system has reached its desired requirements. With the same token, the quality assurance personnel can easily verify whether requirements of code modularity, parameter passing and other similar system specifications are correctly satisfied
Representation. - Global data flow diagram. The maintainer can also zoom in depth of the derived structure through menus and selection key options. The tool can also show the static references between different functions and variables, and dynamically determine the call order of the functions.
3. Impact of RE tools in SW development envi ronments The reverse engineering tool has greater potential to ensure quality and productivity not only in the maintenance phase, but also over the entire life cycle. The reverse engineering tool analyzes source code and generates the corresponding higher level design information. This information is stored in a central repository. The repository contains information of different components of the program. The developer can have access to this integrated information to evaluate the product of a given phase to ensure consistency and completeness with the standards provided as input to that phase e.g. verification, validation. The information stored in the repository can be used to evaluate modules to ensure compliance with specified requirements defined at the initial phase. During the development process, reverse engineering environment can also provide opportunities to apply analysis techniques to assess the correctness and reliability of the system before it is delivered to the real users. The final product can be compared automatically with its design description for consistency and accuracy. Keith D. Gillis and David G. Wright [81 discuss how reverse engineering tools are able to increase software quality and productivity. We discuss briefly this aspect based on the work in [8]. In software maintenance, the information obtained through the reverse engineering tool can be used to trace down a bug or to point out in advance whether an intended change to the source code will introduce side effects. A reverse engineering tool provides the user with a facility to browse the program by following either the control or data flow. This allows the maintainer to understand and navigate the logical structure rather than the textual structure of the program, and helps to determine which paths should be examined and in which form. It therefore increases the reliability and correctness of the maintenance work. The tool also can identify unreliable and unmaintainable design structures of the code. Hence, the tool can implicitly provide quality to the
@I. The information produced by the reverse engineering tool can also be used to calculate complexity memcs which has a great impact in evaluating software quality. Another very important factor in software quality and productivity is reusable components. The reusability of the components of a software system can be measured by using reverse engineering tool. The reusable components of an existing system could be selected and tested easily from the information produced by reverse engineering tool. The reuse of code reduces the development work and increases productivity and quality of a software
I 22
-
Reverse
Engineering Tool
I
/Assienine ” ”
I weightsto the
A’ I
Optional
1-H
Score1
I
U
P E 9 y.l---,-A--k Mandatory RE Attributes
I
1-
Figure 1. MECCA -preprocess. system. The program test paths could be quickly detected by the reverse engineering tool. So the test cases become easier to develop for the test personnel.
Whenever mandatory attributes are not present in a specific tool, the tool should not pass the evaluation threshold, and further evaluation will be worthless.
User functions
4. Strategies for the assessment of reverse engineering tools. In order to obtain objective measures to compare a set of software engineering tools, we need a standard set of attributes or characteristics to compare, and a procedure for the final evaluation. We have structured the attributes into user functions, interface, inpur/output, verification strategy, and metrics collection and analysis. This structuring scheme should be valid for all types of software engineering tools. For all software development environments, it is possible to categorize functionality as: Functions which are “need-to-have”or mandatory, and functions which are “nice-to-have” or optional. The list of these attributes will always be tool specific. Each type of software engineering tool will have its own requirements. The attributes should be structured and possibly also given a grade of importance. The attributes which are mandatory fall in one category, and the attributes which are optional or “nice-to-have’’ fall in the second category.
The reverse engineering tool should provide a few functions which are absolutely necessary and mandatory in order to understand the structure of an existing system. It includes module chart, procedure call graph, control flow graph, structure charts, identification of global and local data structures and global data flow diagram. Reverse engineering tools must be capable of recognizing these design artifacts out of existing source code. The optional functions include production of natural language based specifications and requirements of the program, language independent program processing capabilities, natural language based user interface, dynamic program slicing capability etc.
Interface We categorize the interface in two different types: Interface to other tools or toolsets, and man-machineinterface. Especially for reverse engineering tools, interfacing capability with other tools is important. The output produced from the tool will need to be exported to other tools used in the development process, and the input
to the RE tool will normally be source code representation, in a machine readable format. Tools integration will be an important criteria for the assessment. Possible altematives: full data integration, integration through well defined input/output formats etc. It is desirable that a reverse engineering tool have very flexible and friendly interface. The modem window interface should be used in interaction with the user. And users must have enough room to navigate the source code as much as they want. It is also recommended that user
1 IIReverse Engineering Tool I
-t
should have enough opportunity to contribute to the reverse engineering process. It is noted from several papers, [lo]. [lll, that it is sometimes not possible to capture the design information from the source code alone, it can only be possible with the gathering of additional background knowledge as well as the development history of the system. The original source code does not contain sufficient information to be automatically reverse engineered to recapture the underlying design decision. The maintainer can also
~
38%
Mandatory Functions
90%
-Functionality
-
I
1
Man Machine Interaction Ability
Interface I
I
to OtherTools Accessability to the Repository
I
I 30%1ReportGeneration I I Capability
?El---65%
Me trics
-
Metrics Collection
3 5Metrics 9 Analysis - 7 1 Strategy
" " 7 1 Checking
E [ Constraining
I
Mechanisms
Figure 2. MECCA-model for the assessment of RE Tools.
I24
identify the missing gap between the design components of a system. Maintainer must fill this gap with hidher own experience and reasoning capabilities. So it is obvious that for reverse engineering tools, user intervention and active interaction is highly required. The interface requirements will normally be categorized as optional.
Input/output requirements Reverse engineering tools should have flexible and comprehensive report generation capabilities. Due to the non-standardized mode of working in the maintenance phase, the design and generation of ad hoc reports should be possible. The access to a common repository is also important, especially when talking about tools integration and software reuse. The inputt to the reverse engineering tool will primarily be source code, but also any available documents. An important evaluation criteria will be the scope of programming languages supported by the tool. The categorization into mandatory and optional will be application dependent.
Verification strategy Verification strategy is used in the tool in order to be more useful, a tool should contain mechanisms to check consistency, completeness etc. Based on a work by Stinson [14], we have proposed a set of constraining mechanisms to be included in the ESA SDE [13]. Constraint systems apply rules which check that objects adhere to predefined standards such as method or performance requirements. There are two kinds of constraints: ondemand and preemptive. On-demand constraint systems detect violations of standards upon request. For example, a user may ask a graphical editor to check an object for consistency with associated objects. Preemptive constraints prevents violations from occurring, they narrow the space of possible development actions to those which are known to be valid, thereby enforcing restrictions upon the user. A graphics editor which enforces data flow diagram construction rules is applying preemptive constraints. In our opinion, RE tools should be of the on demand type. The assessment criteria would be the level of constraining mechanisms. And in relation to Quality assurance - the “goodness” of the verification.
Metrics Software meuics collection and analysis is supposed to be more and more important in the development of
quality software. It is therefore important that software development tools contain metrics collection facilities.
Quality requirements to the tools
The enforced use of tools in the software development is often due to critical requirements to the software system being developed. This might be reliability requirements. safety requirements, availability requirements etc. The important question is: what are the additional requirements to the assessment of tools being used for development of critical software. Is it necessary to verify these tools with the same procedures as the software itself? For software being reverse engineered it is probably not advisable to apply this type of software to critical applications at the current level of maturity. 5. The assessment process We propose to use an approach similar to the MECCA (Multi-Element Component Comparison and Analysis) method. The MECCA method has been used as a system evaluation technique. It uses percentage weighting and a hierarchy of attributes [9]. The arrangement of the attributes identified in evaluation must be a hierarchical structure. A percentage weight is assigned to each component attribute throughout the structure. The MECCA-model identifies every component attribute, and determines the relative importance of each attribute or component. In addition to the weighted attributes, consideration has to be taken to the “need-to-have” functions. We have identified that “need-to-have” attributes are vital in reverse engineering tool for quality assessment. These are inevitable, if these not present, the tool will not pass into the MECCA evaluation. We assume that the functionality of a reverse engineering tool is the central component in MECCA evaluation. Therefore it is important to assess whether the mandatory functions described in the earlier section are present in the tool. We need a MECCA preprocess to consider whether the tool is qualified for MECCA-model, or not. If the weighted “need-to-have” attributes are satisfactory, the tool would be qualified for a MECCAassessment. Figure 1 illustrates this view of MECCApreprocess. The MECCA-evaluation will give numbers as results which are the direct scores for the current tool, and comparable in ranking the tools.
References:
A MECCA model for the reverse engineering tools
[ l ] Quality Management and Quality Assurance Standards, I S 0 9000-3, Part 3: Guidelines for the application of I S 0 9001 to the development, supply and maintenance of software. [2] T. C. Jones, “Measuring Programming Quality and Productivity”, IBM Syst. Joumal, Vol. 17, no 1,1978. [3] T. S. Chow, “ Part 2: Software Quality: Definition. Measurements,andApplications”1985IEEE Standards [4] McCall. J., Richards, P.; Walters, G., ” Factors in Software Quality”, three volume, “TIS AD-A049-014. ADA049-015. AD-A049-055, November 1977. [5] J. P. Cavano. J.A. McCall, “ A Framework for the Measurement of Software Quality’’ The proceedings of the ACM Software Quality Assurance Workshop, Nov. 1978. pages. 133-139. [6] Elliot J. Chikofskv and Burt L. Rubenstein. “CASE Reliability Engineering for Information Systems” IEEE Software, March 1988. page 11-16. Elliot J. Chikofsky and James H. Cross II, “ Reverse Engineering and Design Recovery: A Taxonomy” IEEE Software, January 1990. page. 13. Keith D. Gillis and David G. Wright, ‘‘ Improving Software Maintenance Using System-Level Reverse Engineering”, IEEE Roc. Conf. on Software Maintenance, 1990 page 84. T. Gilb: Software Metrics, Cambridge - MA, Winthrop 1977. [ 101 J.M. Boyle and M.N. Muralidharan, “Program Reusability through Program Transformation” IEEE Trans. on Software Engg. Vol. SE-10, No.5, Sept. 1984 p. 574-588. [ 111 Harry M. Sneed, “Software Renewal:A Case Study”, IEEE on Software, Vol. 1, no. 3. July 1984, p. 56-63. [ 121 Philip Gilbert: Software Design and Development. Science Research Associates Inc. 1983. [ 131 T. Skramstad, T. Henriksen: Constraining mechanisms in the ESSDE. ESSDE/Working Documentl/B22, Issue 1, revision 0, Oct. 1991 [ 141 W. Stinson, “Views of Software Development Environments: Automation of Engineering and Engineering of Automation”. ACM SIGSOFT, Software Engineering Notes, vol. 14 no 6. [ 151 Md. Khaled Khan, “Reverse Engineering and Program Understanding in Software Maintenance: An Optimal Approach.”cand. scient. thesis, Dept .of Informatics, University of Trondheim, 1992.
Figure 2 cited above is a proposed MECCA-model for reverse engineering tool evaluation. It should be noted that cost, reliability, and similar relative factors are not taken in consideration in this model. It is recommended that any other component attributes could be added to this model if the concerning personnel feel so. However, the natural question is how the measurements of the attributes would be determined. As Philip Gilbert [12] suggests, the weighting facts could be gathered from the manuals, handbooks. sales offers, references to performance tests and from similar documents of the tool. A numeric measure is assigned for each lowest level attributes based on the facts collected from the tool related documents. He also suggests that the numeric measure ranges from 0 (barely acceptable) to 5 (average) to 10 (super temfic). Once the lowest level attributes have been weighted, the preliminary evaluation is complete. Then the weights can be summed up. For instance, if Man Machine Interface Capability in figure 2 is given a weight of 8, and Interface to Other Tools is 5, then the Interface component will be (8 * 0.6) + (5 * 0.4) = 4.8 + 2.0 =6.8. After assigning weights to other components, the final measure of the feverse engineering tool will be calculated. And in our example, the contribution of Interface component in MECCA-model will be 6.8 * 0.25 = 1.7. The process of assigning scores to the attributes will depend on the information and facts gathered about the specific tool. So some importance should be given in gathering information as much as possible about the tool.
6. Conclusion We have presented an approach for the assessment of reverse engineering tools based on an extension of the Mecca-model. The extension proposes to categorize the attributes in “need-to-have”, or mandatory attributes and “nice-to-have” or optional attributes. If the mandatory attributes are not present, or the final score is not satisfactory, the product will not be assessed. Our proposed model is not absolute or limited, it might be redefined according to the new components or attributes identified by the concerning personnel.
126
