Mapping of Datrix™ software metrics set to ISO

Mapping of Datrix™ software metrics set to ISO 9126 Maintainability subcharacteristics Bruno Laguë Information Technology Procurement Bell Canada 2265 boul. Roland-Therrien Longueuil, Quebec, Canada J4N 1C5 Phone: (514) 448-5543 Fax: (514) 647-3163 [email protected]

Alain April Customer Services Assistance Solutions Bell Canada 700 DeLaGauchetière West Montréal, Québec, Canada H3B 4L1 Phone: (514) 396-8682 Fax: (514) 621-2713 [email protected] Abstract

The mapping of draft standards work in the area of product quality measurement to industry practice has lead to a discussion on the importance of the previous knowledge of the design and support environment is necessary for correct interpretation of static source code metrics. We have found no intersection between the proposed internal metrics of the ISO9126 draft standard and the one extracted with our tool (Datrix). This has lead us to present a mapping and rationale for each Datrix Domain for the four ISO 9126 maintainabilty sub-characteristics. Keywords: Software Engineering Standards, Software Metrics, ISO 9126, Internal Metrics Bell Canada has been performing supplier assessments and supplier management since 1993 in its Quality

1. Introduction Bell Canada telecommunication network operations depends on more than 50 million lines of code. This number increases yearly as the network becomes more digital and the operational support systems are introduced to mechanize the administration. The median size of these software systems lies between 200 to 400 Kloc (thousands of lines of code), with the largest having over 20 million lines of code. Most of this software is real-time, with a majority of the systems of a hybrid nature (combination of real-time, database and graphical). In this environment, some software control safety/critical systems and must be reliable (a few minutes downtime over 50 years) in order for Bell Canada to offer the level of service required by its customers and the telecommunications regulator. Competitiveness in this market is imposing quick turn around on delivery of new services. These new services are software intensive and concentrate mostly on enhancing the existing base of software to deliver new features.

Engineering Laboratory in Montréal, Canada. Acquiring and comparing products which include software has always been a difficult activity. Obtaining rapid, and in depth, insights of a supplier and his product and the software content is key to good acquisitions practices. In our industry products that fail in operations and in the hands of customers means lost market share. To address these issues Bell Canada in conjunction with University research has developed frameworks and tools for the software product evaluation and assessment [1,5]. The objective of an assessment is to get the maximum information for decision making within a limited number of days. This effort constraint has imposed mechanization of some key steps of the software acquisition process. As presented by Boloix [5] “the metrics interpreting is difficult because there are no mechanisms for synthesizing the information.”

1

An important part of the supplier assessment process is the Software Product Assessment Process (SPAP). A key component of this assessment process is the software source code assessment. Getting access to the source code of the supplier product and analyzing it gives an objective understanding of the actual state of the product under investigation. It also gives insight into the legacy of techniques and technologies imbedded in the product by looking at the structure and internal documentation of the software portion of the product. We have observed from years of supplier evaluation that documentation and intermediary products like functional descriptions, designs, test plans and other intermediary development products are seldom present or kept current. The most mature organizations keep their testing data and environment up to date and available to the maintenance team. Even if time was available, the task of doing formal review of all the intermediary products would still be a major challenge for the assessor. Practice led to the development of a support tool (Datrix) that focuses on a static source code evaluation that is mostly concerned with the maintainability attributes. In the early 90’s the assessment attributes were focused on the model proposed by McCall [6] and mostly on efficiency and reliability. Since 1991, the influence of the standards work on ISO/IEC/SC7/WG3 [7] and the six software quality characteristics has moved the focus of the Bell Canada work towards the maintainability characteristic. This paper will focus on the mapping of the current practices of our team versus the proposed metrics included in the ISO JTC1/SC7/WG6 - Internal metrics (DIS). More precisely we will discuss the maintainability issues from an internal metrics point of view. This paper is composed of the following sections: Introduction to ISO9126 maintainability internal metrics, maintainability sub-characteristics assessment using Datrix, mapping of Datrix metrics to proposed ISO9126 maintainability internal metrics and conclusions for future work.

The focus of this paper is on maintainability (a set of attributes of the software that influences the effort needed to make specified modifications). Views of maintainability are different when approaching the subject from an external or internal view. External maintainability is viewed as the effort needed to make specified modifications to software. Internal maintainability is a set of attributes of the software that influence that effort. The sub-characteristics of the maintainability aspects of software are analyzability, changeability, stability and testability. One can view the software maintenance work as a four step process. The first step is analyzing existing software for diagnostics, deficiencies, failures but mostly identification of the parts to be modified. The second and third steps are modifying the software while identifying the potential impacts of the modification. In the last step, the maintainer must test the modification to ensure correctness according to specifications. The Bell Canada Quality Engineering proposed to study the intersection between the internal metrics used in our product evaluation product called Datrix and the that proposed by the ISO 9126- internal Metrics current draft standard. Internal metrics currently proposed in the (draft) that address the maintainability issues are: Analyzability: implementation ratio of diagnostic function, Activity recording ratio; Changeability: Parameterisation ratio, Recording ratio of module change information; Stability: Affected data ratio; and Testability: Implementation ratio of test function, Checkpoint readiness. When reviewing these draft internal metrics we found no metric associated to source code analysis. The Bell Canada product evaluation metrics and interpretation associated with four sub-characteristics of the ISO9126 standard are described in the next section. 3. Maintainability sub-characteristics assessment 3.1 Information extracted with Datrix

2. ISO 9126 and the maintainability internal metrics

The metrics extracted from the source code allows one to build an objective view of the software. However, in order to correctly understand the results, it is critical to have a good knowledge of the design environment. The tools available to the designers have a strong impact on the characteristics of the code, and can change the range of acceptable values for some metrics. As a result, when Bell performs a software

The six characteristics proposed by the ISO9126 quality model are functionality, reliability, usability, efficiency, maintainability and portability. These characteristics are then decomposed into subcharacteristics and metrics. This level of detail is currently being reviewed in ISO IEC/JTC1/SC7/WG6.

2

source code assessment, some designers are interviewed, at their desk, in order to see and understand their design environment. Datrix provides source code metrics that describe properties of three types of components: the routines, the classes, and the files. Datrix measures 53 characteristics of routines, which cover 5 domains (Table 1). The Programming Rules is the first domain, mostly evaluating whether the structured programming rule (each block has one point of entry and one point of exit) are applied. The Coupling domain includes metrics that determine to what extent a given routine is coupled to the rest of the system, regarding the number of global identifiers it uses (global variables references, and call to other routines). Dimension and Complexity forms the third domain. Various measurements are made on the size of

the routines, as well as the complexity of the statements and of the control flow graph (CFG). The last 3 metrics in this domain also provide information on the size and complexity of the routine, but in the dimension of the data instead of the control flow. Metrics that influence the number of test cases required for unit testing, as well as the effort required to build each test case are classified in the Testability domain. The fifth and last domain is made up of all metrics related to the internal documentation. Datrix takes various aspects of internal documentation into account. For example, by considering the location of the logical statements, it is possible to check to what extent sequential statements, conditional statements and variable declaration statements are documented.

Table 1 - Routine Metrics by Domain

3

METRIC NAME Programming Rules

RtnCovStaNbr

METRIC DESCRIPTION Number of static cast

RtnCtrBrcNbr

Number of breaches of structure

RtnNdsEntNbr

Number of entry nodes

RtnNdsExtNbr

Number of exit nodes

RtnNdsPndNbr

Number of pending nodes

Coupling

Number of functions called

RtnCalNbr

RtnCalUnqNbr

Number of distinct functions called

RtnVarGlbUsdNbr

Number of references to global variables

Dimension/Complexity

RtnArcNbr

Number of arcs

RtnCndSpnAvg

Average span of conditional arcs

RtnCndSpnMax

Maximum span of conditional arcs

RtnLocNbr

Number of lines of code in the routine body

RtnLopNbr

Number of loop constructs

RtnNodNdsNbr

Number of nodes

RtnRtnCplLvl

Routine complexity level

RtnStmCtlNbr

Number of control statements

RtnStmDecNbr

Number of declarative statements

RtnStmExeNbr

Number of executable statements

RtnStmNbr

Number of statements

RtnVarSpnMax

Maximum variable span

RtnVarUsdNbr

Number of references to variables (local and global)

RtnVarUsdUnqNbr

Number of references to distinct variables (local and global)

Testability

RtnCndCplAvg

Average conditional arc complexity

RtnCndCplMax

Maximum conditional arc complexity

RtnCndNbr

Number of conditional arcs

RtnCtrNstAvg

Average control structure nesting

RtnCtrNstMax

Maximum control structure nesting

RtnPrmNbr

Number of parameters

RtnPthIndNbr

Number of independent paths

RtnRtnCycCpl

McCabe’s cyclomatic number

RtnStmCplAvg

Average complexity of statements

4

Documentation RtnArcComAvg

Ratio of commented arc

RtnComdDecVol

Volume of comments in declaration

RtnComLogNbr

Number of logical comments

RtnComStrAvg

Comments volume average

RtnComStrVol

Comments volume in structure

RtnCtrComAvg

Ratio of commented control structure

RtnVarLenAvg

Average length of variable name

For object oriented systems, Datrix can characterize class methods with the same set of metrics as explained above for routines in procedural systems. Regarding the

characterization of the classes themselves, 17 metrics are currently available, and are classified in four domains(Table2).

Table 2 - Class Metrics by Domains METRIC NAME Attributes

ClaAttHid

METRIC DESCRIPTION Ratio of hidden (private) attributes.

ClaAttNbr

Total number of attributes

ClaAttPriNbr

Number of private attributes

ClaAttProNbr

Number of protected attributes

ClaAttPubNbr

Number of public attributes

ClaAttStaNbr

Number of class (static) attributes

ClaClaNstNbr

Number of nested class

Methods

ClaCstrNbr

Number of constructors

ClaMetNbr

Total number of methods

ClaMetPriNbr

Number of private methods

ClaMetProNbr

Number of protected methods

ClaMetPubNbr

Number of public methods

ClaMetStaNbr

Number of class (static) methods

Inheritance coupling ClaInhDirNbr

Number of direct parent classes

ClaInhIndNbr

Total number of parent classes

ClaInhLev

Inheritance level

Documentation

ClaNamLen

Length of the classe name

The first domain is Attribute, and includes characteristics of the attributes of the class, like simple counts on the number of private, protected and public attributes. The Methods domain provides similar information about the nature of the methods found in each class. Metrics extracted from the inheritance tree are classified in the Inheritance coupling domain. These metrics quickly show whether or not the object

oriented concept of inheritance is used, and how well. Finally, the Documentation domain is limited, for now, to the length of the class name (in number of characters). The two metrics characterizing the files are classified in a single domain: Coupling (Table 3). Other metrics are computed for each, but are not listed in the table as

5

they are simple aggregation, at the file level, of routines

and class metrics.

Table 3 - File Metrics by Domain METRIC NAME Coupling

METRIC DESCRIPTION Number of files included directly (#include statements)

FilIncDir

FilIncUnqNbr

Total number of distinct files included, directly and transitively.

3.2 Mapping Datrix Metrics to ISO-9126 Maintainability Sub-Characteristics The set of Datrix cover many of the ISO 9126 Maintainability sub-characteristics, although other measurements would also be required to achieve a full evaluation of these sub-characteristics. In the following, we explain how datrix metrics are used to assess the maintainability of a software system, and we expand on what other measurements would be required to improve the results. As mentioned previously, it is important to keep in mind that the maintainability of a software system depends not only on the source code characteristics, but also on the development environment. If the designer can use tools to reproduce the environment where the problem was found (configuration information,

sequence of events, etc…), as well as tools to trace the execution of the code for example, it can be expected that the average time required to find the cause of problems and fix them would be reduced. However, the maintainability of a system is still highly dependent on the source code characteristics. In systems made up of independent components with well defined roles, linked through strict interfaces, a designer knows where to start to solve a problem since determining which component is responsible for an undesired behavior is unambiguous. Table 4 shows which Datrix metrics domains are used to assess each of the four ISO-9126 maintainability sub-characteristics.

Table 4 - Datrix metrics domain mapping to ISO 9126 maintainability sub-characteristics

ISO 9126 MAINTAINABILITY SUB-CHARACTERISTICS DATRIX DOMAIN

ANALYZABILITY CHANGEABILITY

Routines Prog. Rules

x

Routines Coupling

x

Routines Dim/Complexity

x

x

STABILITY

TESTABILITY

x

x

x

Routines Testability Routines Documentation

x

Class Attributes

x

x

Class Methods

x

x

x

x

x

x

x

x

Class Inherit. Coupling

x

Class Documentation

x

File Coupling

x

classes and files. High level of coupling, particularly between routines and global variables, indicate that the components are not independent. Even though

The first sub-characteristic is the analyzability of the code. Datrix provides coupling metrics for routines,

6

industrial experiments correlating size and complexity metrics to

The last sub-characteristic is testability. The Testability domain contains a set of metrics that can be directly used to assess this sub-characteristic. Though these metrics can reveal a lot about the effort required to test a given system, the testability metrics set only characterize the unit testing effort. Coupling metrics, again, are related to the effort required for performing integration testing, but no specific metrics are available yet for this aspect of testing.

the time it takes for a designer to understand a given function are not common, it is reasonable to assume that having to work with large and complex routines increases this time. Datrix metrics in the size and complexity domain thus also contributes information to form an opinion on the system analyzability. Similarly, the application of programming practices, by reducing the variability in the code, can only help the designer to understand the different parts of the code in the system. Hence, Datrix metrics in the programming rules domain can be used to assess this maintainability subcharacteristic. Internal code documentation will also help the designer, at a detailed level, particularly to validate assumptions about what the code is actually doing as he or she reads it. The conclusions that can be drawn from these metrics about the analyzability of the source code are obviously limited. Additional insight could be gained with the ability to extract design patterns. Systems which use a high degree of design patterns should be easier to analyze, since problems have fewer possible causes. The changeability of the code is the second subcharacteristic. Once the cause of a problem is found, the effort required to change the code in order to fix it is also related to the extent to which a given behavior is localized in a given component. Most of the time, the changes should be localized in a few, tightly related functions and classes. Thus, again, coupling metrics provide useful insight in the assessment of the source code changeability. Another aspect that should be considered when changing the code, is whether or not other parts of the system would require the same change. Again, the automatic identification of design patterns in the code would prove useful in this regard. Initial work in this direction is referenced [3]. The next sub-characteristic is the stability of the code. The risk of introducing undesirable impacts when making changes to the source code also tightly depends on the coupling between components. Datrix coupling metrics help to assess this sub-characteristic in general. The number of public attributes and methods of classes is a measurement of the potential communication channels between classes. Metrics are required to assess the actual usage of these channels.

4. Conclusions and future work In general, it can be said that Datrix provides a lot of insight about the code being assessed, which gives Bell Canada a much better understanding of the ability of a system to evolve and be maintained. Datrix metrics introduce complementary information to the internal metrics proposed for assessing the maintainability characteristics in the draft version of the ISO 9126 standard. It is our opinion that such information is required to obtain an accurate and objective view of the actual status of the source code of a software system. The set of Datrix metrics is not complete in order to fully assess the four ISO9126 maintainability subcharacteristics, and more information must be collected. It must be noted that coupling metrics provides significant insight in the assessment of all four of the maintainability sub-characteristics. Datrix provides many metrics in this aspect, but additional ones are still required. Relationships such as aggregation and association also introduce coupling, but are not yet taken into account in Datrix. The set of metrics proposed in [10] are promising to be used to build an accurate coupling picture for object oriented systems. Coupling metrics are also required in order to assess to what extent a system is made of components, each of them being as self-contained and independent as possible. The ability to identify design patterns would help to refine the definition of a component and its interfaces. Only once that definition is clear will it be possible to automate the extraction of patterns in the code, and extract the most meaningful measurements to support the maintainability assessment. Although this functionality is not yet available in Datrix, a significant first step in that direction is expected from the cloning analysis as described in [2,3]. [1] Mayrand J., Coallier F., “System Acquisition Based on Software Product Assessment”, 18th International Conference on Software Engineering, Berlin, March 2529, 1996.

5. References

7

[2] Mayrand J., Leblanc C., Merlo E., “Experiment on the Automatic Detection of Function Clones in a Software System Using Metrics”, International Conference on

[3]

[4]

[5]

[6]

[7]

Software Maintenance, Monterey, California, November 4-8, 96. Mayrand J., Laguë B., Hudepohl J., “Evaluating the Benefits of Clone Detection in the Software Maintenance Activities in Large Scale Systems”, Workshop on Empirical Software Studies, Monterey, California, November 8, 96. ISO 9126, “Information technology - Software product evaluation - Quality characteristics and guidelines for their use”, 1991. Datrix is a trademark of Bell Canada. G.Boloix, P.Robillard, A Software System Evaluation Framework, Computer, Magazine of the IEEE Computer Society, December 1995, pp. 17-26. McCall, J.A.; Richards, P.K.; Walters, G.F. Concepts and Definitions of Software Quality Factors in Software Quality, Vol. 1, Springfield, Va: NTIS, November 1977. Azuma, M. Japanese Contribution on Quality Metrics, ISO/IEC JTC1/SC7/WG3, International Organization for Standardization, May 29, 1991.

[8] Bevan, N. Measuring usability as quality of use. Journal of Software Quality, 4, p115-130, 1995. [9] Mayrand Jean, Guay François, Merlo Ettore, “Inheritance Graph Assessment Using Metrics”, International Software Metrics Symposium, Berlin, 25-29 march, 1996 [10] Hitz J, Montazeri B., “Measuring Coupling in Object-Oriented Systems”, Object Currents, April 1996.

Datrix is a trademark of Bell Canada.

8