The Java Metrics Reporter – An Extensible Tool for OO Software Analysis Jaspar Cahill, James M. Hogan, Richard Thomas Centre for Information Technology Innovation Faculty of Information Technology Queensland University of Technology GPO Box 2434, Brisbane, QLD, 4001, AUSTRALIA {
[email protected], j.hogan,
[email protected]} Abstract It has been argued for many years that software engineering lacks the repeatability and well-defined monitoring characteristic of the traditional engineering disciplines. Over time, numerous authors have addressed this issue by proposing a range of software metrics — although it is generally agreed that no one measure is sufficient to capture software quality, and a well chosen suite of metrics must be employed. While substantial progress has been made, adoption of metrics has been limited in the software development community, and metrics have long suffered from a lack of comprehensibility. Further, critics have argued that many metrics have been introduced in isolation, with little regard for their relationship to existing measures, and without appropriate validation against a sufficient body of source code. This work introduces the Java Metrics Reporter, a new tool which addresses a number of these concerns in the domain of object oriented languages. The tool provides integrated tutorial support and benchmarking of user code against professional code bases. Moreover, the architecture allows ready adaptation to other languages, and extension to other metrics through a straightforward plug-in approach. The paper provides detailed consideration of the architecture and the metrics selected, together with a number of examples of its use. Finally, we outline plans for the further development of the tool, together with its release to the professional and research communities.
1. Introduction It has been argued for many years that software engineering lacks the repeatability and well-defined process monitoring characteristic of the traditional engineering
disciplines. In the absence of a readily definable measure of overall software complexity, many authors have suggested a range of complexity measures, each based on some data collection from the source code. Such metrics vary in their usefulness as indicators of quality, but it is clear that no single metric provides the breadth of coverage necessary to be a stand-alone measure and a well-chosen suite of metrics must be employed. Many of the metrics suggested have significant merit – particularly those described for Object Oriented Development by a series of authors since the mid 1990s. (See for example Chidamber and Kemerer [4], and Abreu et al. [2].) However, their uptake by the software development community has been remarkably limited. While there are a number of factors in play, in the past, metrics have been plagued by a lack of comprehensibility, and their associated tools characterised in the main by limited attention to the needs of the professional user. At a deeper level, researchers have attracted criticism for proposing single metrics with little insight into their relationship to existing metrics, and for collecting suites of existing metrics without providing any unifying, coherent framework for their usage [4]. Moreover, a number of metrics proposed in the literature have not been validated against a sufficiently large body of professional source code, exacerbating the difficulty in interpreting the figures generated from a novel system. The Java Metrics Reporter (JMR) is a new tool which attempts to address a number of these concerns within the domain of modern object oriented languages. Central to the tool design is the separation of language processing and metric collection, with the initial pass over the source generating a system model independent of the supplied source code. In this fashion, the tool support extensions to additional object oriented languages, through the substitution of a parsing module specific to the target language – although there are inevitably 1
some issues inherent in the mapping of the metric definition to the source language. Subsequently, the model is used as the input to the metrics collection suite, itself constructed to allow ready extension through the installation of new metrics collection plug-ins. The tool not only provides the raw data collected from the model, but also a mechanism to browse this data at different levels of detail — from the system level down to the individual method. Our approach addresses the issue of comprehensibility through integrated tutorial descriptions of each metric, and benchmarking of the users code against norms determined from professional code bases. In particular, the system allows the user to select these norms according to the type of application being undertaken, and to be alerted to potential deficiencies when the figures produced lie well outside the usual ranges for such an application. This paper is organised as follows. Section 2 examines the tool architecture in some detail, focusing in particular upon the separation of model construction and metrics calculation – with extensibility of each module considered in turn. Metric selection for the initial release and their organisation within the tool are considered in section 3, along with discussion of candidates for future release. Section 4 addresses usability and provides some screenshots of the tool display. We conclude in section 5 with some discussion of ongoing tool development and of plans for its release to the software engineering community.
2. Architecture The architecture of Java Metrics Reporter is broadly divided into the language model (here the java model) component – which builds a structured representation of the selected source code – and a separate metrics calculation component - which operates directly upon the model without reference to the source. Extensibility of the system was a critical consideration in the design of the architecture, and this is reflected in both of the major components. Modifications to the system to allow alternative source languages are facilitated in a similar fashion to a number of existing tools – through careful isolation of source language specific features, allowing the substitution of a parser appropriate to the target language. Inclusion of additional metrics is more convenient, requiring only the development of a suitable metric plug-in , which will normally rely upon metrics and information gathering plug-ins already in use in the system. The system architecture and its underlying components are considered in the following sections.
2.1 The Java Model The Java Model is essentially a tree-based independent representation of the Java source. Nodes in the tree reflect elements in the source such as classes and methods, and the hierarchy reflects nesting of scopes. In order for the model to be useful, references, such as type references, are resolved to the associated elements, thus forming a network of references between element nodes, assembled around the basic tree structure. The elements which form the building blocks of the model are considered first, allowing model construction to be viewed as the incremental addition of elements as the source code is parsed. Resolution of references is accomplished through a series of traversals over the model tree, and these are considered in the subsequent paragraphs. Java Model Elements: There are a number of different types of element, as reflected in the element class hierarchy shown in figure 1. At the top of the hierarchy lies JElement, an abstract class from which all elements are derived and which has attributes name, parentScope, and modifiers. The element which forms the basis of the tree structure is JScope, as it contains a collection of child elements, and allows nesting of scopes. JSystem contains all the java elements in the parsed source, and JEnvironment contains primitive types and the java.lang package, incorporating the root class java.lang.Object. Building the Java Model: The Java Model is constructed as a result of messages sent from the source Parser to a Builder object, each detailing declarations encountered
Figure 1. Class hierarchy of Java elements. 2
in the source code. In response, the Builder creates elements and incorporates them into the model tree. The range of messages which may be received by the Builder is defined through a BuilderInterface, implemented by the Builder, and this includes a startElement and endElement method for each scoped element type. The basic algorithm used by the Builder is as follows: 1. On start scope: Create new element and add to model. Make new element the current scope. 2. Process elements in scope 3. On end scope: Make current scope the parent of the current scope
The Parser itself is generated by the ANTLR parser generator [3] from a grammar with embedded semantic actions. Accessing the Model using a TreeWalker: The model can be traversed using a Document Object Model (DOM) TreeWalker. This approach is facilitated by JNode which implements a subset of the DOM Node interface - from which all elements are derived - and JScope - which maintains a linked list of siblings from its children. Different logical views of the tree can be created by using filtering in conjunction with a TreeWalker traversal. Resolving Type References: Once the basic structure of the tree is established, type resolution may proceed. This involves matching type names to the referent elements - for example a JVariable’s type to a JClass somewhere in the model. Prior to resolution, all types are given the placeholder representation of ClassReference, which contains the name of the class to which it refers. Resolution uses a ResolveTypesTraversal, that sends a resolveTypes message to each element as shown in figure 2, and each element sends a resolve message to each of its associated ClassReference objects. In turn, this message leads to a lookup of the class name to locate the matching element, leading to successful resolution or the use of an ExternalClass generic match. Java Model ResolveTypesTraversal resolveTypes(Lookup) lookup Updates context during traversal
ClassReference name element Resolved to a JClass
lookup(ElementDescription)
Lookup context
Figure 2. Resolving types with model traversal.
Name Lookup: Resolving types involves name lookup - matching of a name to an element in the model. Lookup is implemented with a Lookup class, and the search is guided by an assigned LookupContext, which contains the currentScope and imported classes and packages for that scope. A lookup is requested by sending Lookup an ElementDescription, which contains the name of the element, its type, and if a method, number of parameters. Each element implements a lookup method, and in most cases the message is passed on to JScope ,which tries to find a match with one of its child elements. However, there are exceptions to this rule: in the case of JClass, for example, if a matching element is not found in its scope, the lookup request is passed to its superclass, and on failure to the interfaces it implements. The lookup algorithm implemented within Lookup is complicated by the occurrence of names that contain multiple identifiers, and by the appearance of imported classes and packages. In the context of type resolution, a lookup is passed to each element, with its context set by the ResolveTypesTraversal as it proceeds from node to node. Resolving References in Expressions: References which occur within the body of a source scope, such as a field reference obj.attribute or method reference obj.method(), are recognised, resolved to the element they reference, and stored in the model in the associated JScope. Such references frequently occur in Java expressions, where most of the work of a program is done. Given the range of Java expressions and the complex grammar for parsing them, identifying and resolving references within expressions is not straightforward. A simple solution to this problem is to limit consideration to references of the form a.b and a.method(), which may be handled by the standard lookup routine. In the latter case, the number of parameters is stored with the reference. Such references can be collected during the build parse, and resolved along with type references. The problem is that this approach performs poorly when faced with reference chaining and method overloading. Consider the method invocation: func1(obj.func2().func3())
The method reference obj.func2() is recognized and resolved properly, but the additional reference cannot be handled: • func3() is not recognized as a reference because to do this we require the return type of func2() • func1() may not be resolved correctly if it is overloaded, as to do this, its argument type is required (the return type of func3()). 3
To overcome these problems, the implemented system must work out the expression type. As an expression is parsed, type is tracked, identifiers are resolved, methods calls are correctly resolved using argument types, method return types are taken into account, and types are combined and promoted as described in the Java Language Specification. This enables references in complex expressions to be recognized and accurately resolved. The tracking of expression type, and in the processing recognizing and storing references, is done in a second parse of the source code. A Parser-Builder configuration is again employed, with the builder, here a ReferenceBuilder, processing startElement and endElement messages to set up lookup context, and to provide special routines to support the parser in tracking type. The ReferenceBuilder has two lookup objects, a Lookup and a MethodLookup. Tracking Expression Type: Expression type is tracked with the use of a ParseInfo object, which is created at the start of an expression and holds the state of the expression at any given time during the parse. Choosing the Most Specific Method: As part of tracking expression type and reference resolution, method lookup takes place as each invocation is encountered. Method lookup is performed by a MethodLookup object, based on method name and argument types. This task is complicated by two frequently occurring cases: • the case of formal parameter types that do not exactly match the argument types; and • the case of method overloading, where there may be more than one applicable method. MethodLookup deals with this problem by finding all applicable methods - those whose parameters are compatible with call arguments - and then selecting the most specific alternative, i.e. the one whose formal parameter types most closely match those of the actual argument types.
ily replaced with that for a different language, generated from a grammar using ANTLR or some alternative parser generator tool such as JavaCC. The BuilderInterface which defines the messages that can be received from the Parser during model building may be set up to suit a particular language, along with a corresponding Builder which implements this interface. In short a Builder may be added easily for each language where necessary, and possibly extended from other more generic Builder classes. And finally in this step, generic model elements could easily be extended, or new elements added, to handle additional language features and constructs. For example, a generic Class element could possess a single superclass, and a Java Class element could extend this model with the addition of an attribute for implemented interfaces. In constrast, a C++ Class element might extend a generic Class that has multiple superclasses. Issues of type and reference resolution could probably be handled through the use of language specific subclasses inheriting from generic base classes embodying much of the existing behaviour. In the former case, we require the implementation of language–specific lookup algorithms, and in the latter the embedding of semantic actions in the expression section of the language grammar. Possible enhancements of this support for extensibility to other languages are considered in section 5.
2.3 The Metrics System The Metrics System is a component of the tool which computes metrics directly from the Java Model, thus separating the modeling function from the metrics calculation. As the component incorporates a plug-in architecture, it allows the convenient addition of new metrics in a fashion apparently not found in other tools of this kind. The Metrics System is composed of a model extension component which is plugin-based, and the metrics plug-ins themselves. These are considered in the following paragraphs.
2.2 Extensions to other Languages The Java Model component of the system was developed specifically for Java, based on an existing cross-referencing program from a student project. However, as noted earlier in this section, the system has been developed with the idea of extensions to other languages firmly in mind, and the tool as it stands provides a good foundation for such an application. The basic process used in producing a model for Java – that of building the model structure from elements, the resolution of type references, and finally the resolution of expression references, would readily transfer to other languages within the OO paradigm, with relatively minor porting issues due to their inherent similarity. Looking at this in more detail, the java parser is eas-
Plugin Extension of the Java Model. Much of the information needed for the computation of particular metrics is not directly available from the Java Model. The model is intended to provide a ‘pure’ and uncluttered representation of a Java System, not to contain derived information for various applications such as the computation of metrics. While a plug-in component for a certain metric could derive the information that it needs, such an approach may lead to unnecessary duplication of code and calculation if other plug-ins have a similar need for the same derived information. It would seem desirable to have the information stored in a shared repository of some kind, in order to ease the task of writing additional metric plugins, and to provide a common set of data and behavior to assist in exploration and analysis 4
of the metrics themselves. The system implemented achieves this through an object oriented extension of the Java Model which supports software plug-ins, allowing the model to be extended as new metrics are added. In this way, an object such as a JClass may be sent a message to access the set of methods that it inherits without affecting the underlying model, eliminating the need to look up some external database where the data and behaviour are removed from the object with which they are associated. Ideally, additional data and behaviour to support metrics computation would be added to the model by subclassing model elements – but this approach does not provide for extension through plug-ins. To achieve this, each element in the Java Model contains a Hashtable of modules that can be accessed by name. Each module has a reference to the element to which it belongs so that element can be accessed. Element extension modules are created and assigned to each element. To hide the module implementation - so that data and behaviour in the module appear seamlessly as part of the element - an AccessorWrapper can be provided which wraps the element and provides access to a particular module. It effectively provides a temporary interface to access extended information about the element. This is illustrated in figure 3. Model extension plug-ins compute certain extended information for an element, such as inherited methods for a JClass, put it in a module, and assign it to the element. Each model extension plug-in has its own module type, and module accessor wrapper. Each has a run(JModel) method where it is given the opportunity to compute extended information Metrics System Model Ext. Plugins Extended Model Model Ext. Plugin Basic info. Model Ext. Plugin Java Model Info. derived from model
Derived info. Metric Plugins Metric Plugin Metric Plugin
Figure 3. The MetricsComponent architecture and its relationship to the Java Model.
System Root Classes
Class
Method
Variable
Basic Sets
Inheritance Tree
Ancestors & Descendants
Method Inheritance
Overriding
Overriding
Referenced Classes
Referenced Classes
Overriding
Figure 4. Model extension plugin dependencies. for the model, and a list of dependencies - other extension plug-ins that must be run prior to the invocation of the plugin’s run method. Current model extension plugins are listed on the following page and the dependencies between them are shown in figure 4. It is not necessary to have a separate module for each extension plug-in. The approach adopted has the effect of providing an interface for accessing data specific to the plugin, rather than accessing it from a generic structure and doing type casts. This also organizes the data and associates it with the plug-in which created it. Metric Plugins. A Metric plugin computes an individual metric for elements in the model. In common with the extension plug-ins, it has a run(JModel) method in which it is given the opportunity to perform its task. Every element has a Metrics module that contains a Hashtable of metric values accessed by metric code. A computed metric value is stored in this module, with the element for which the metric was computed. Metric plug-ins utilise the extended information generated by model extension plugins - which are always run before metric plugins - to ease the task of computing metrics. Below is a simple example of code that might be used to compute Number of Ancestors using extended model information: processClass(JClass) { // access basic sets info. for class ADWrapper w = new ADWrapper(class); // compute metric int metricVal = w.getAncestors().size();
5
Element Type System
Plugin Name
Description
Root Classes
Class
Basic Sets
Class
Inheritance Tree
Computes root classes in system. Computes commonly used sets for a class, such as declared methods, and non-private declared methods. Computes superclasses and subclasses for each class to form an inheritance tree. An interface may have multiple superclasses. Computes the set of ancestors and the set of descendants for each class.. Computes the set of inherited methods and the set of inherited attributes for each class. Takes into account overriding of methods, and access modifiers. Computes the set of methods in the class that override inherited methods, and the same for attributes. Computes the reference information for each class. Includes lists for attributeTypes. Computes whether or not a method is overriding. Computes reference information for each method
Class
Ancestors and Descendants
Class
Method Inheritance
Class
Overriding
Class
Referenced Classes
Method
Overriding
Method
Referenced Classes
// assign metric to class element element.getModule(“Metrics”).put(“NOA”,me tricVal); }
Generic Plugin System. The model extension plugins and metric plugins just described are based on a generic plugin system, based in turn upon the plugin system used in the open source JEdit software. The system consists of three components: • the plugin itself • the plug-in loader which makes classes and other resources in the plug-in available to the program; • the plug-in library which loads all plug-ins in a specified directory. Each plug-in takes the form of a JAR archive, a convenient method of packaging its constituent files. Plug-ins contain a core plug-in class which is the main interface with the application, and has behaviour predefined (by extending a common base class known to the application) so that the application may communicate with it. It may also contain an option pane
for displaying options for the plug-in, a properties file where default options are stored, and other files, such as HTML files which may be used for tutorial support. The suite of metrics implemented as the first set of plugins, and the rationale for their selection, is considered in the following section.
3. Metric Selection and Support As noted in the introduction, numerous software metrics have been proposed in the literature since the foundation of the field in the 1970s. While an extensive fraction of the relevant literature was examined in making the initial selection of metrics for JMR, and in identifying candidates for future inclusion, our focus is at present on coherent suites of metrics designed for the object oriented environment such as those of Chidamber and Kemerer [4], and the MOOD suite of Abreu et al. [1]. These issues are examined in the following sections.
3.1 Present Selection and Rationale The present selection is comprised of fifteen metrics organised into the categories labeled Basic, Complexity, Inheritance, and Polymorphism. Choice of individual metrics was based upon the following criteria: • the metrics should cover a range of different metric categories • the metrics should be well known and accepted as being useful in measuring software quality • the metrics may be implemented readily • there should be a preference for design-level metrics as they may result in the greatest improvement in software quality. At this point it has not been the intention to select an ideal suite of metrics. Rather, we aim to implement a range of measures and in time derive small subsets of metrics that are seen to be useful measures of certain aspects of software quality. The metrics in the initial selection are now considered in turn, organised according to category.
3.2 Basic Metrics Basic metrics are those which primarily have to do with size, and which are obtained by doing a simple count. There are four metrics in this category. Source Lines of Code (SLOC) The commonly accepted definition of a source line is a line that is not blank and is not a comment. In JMR this is based instead upon a count of the number of statements, as this is seen as a more accurate way of measuring source size. Comment Lines of Code (CLOC) A complementary measure to SLOC, it is seen as useful in determining how 6
well the source has been commented which affects reusability, and maintainability. Number of Instance Variables (NIV) [7] is a size measure of the number of instance variables in a class. It is easily computed, easily understood, is used by other metrics, and is a fundamental measurement of class size. A large number of instance variables may indicate that a class is doing more than it should, and that it may need to be split into smaller classes. Number of Local Methods (NLM) [6] is a size measure of the number of methods declared in a class that are accessible outside the class (in other words, the size of the class interface). As for NIV, it is easily computed and commonly used. It can be used to assist in determining comprehensibility and maintainability of the class – the more methods, the more difficult it is to comprehend and maintain.
3.3 Complexity Metrics Complexity metrics are those which measure structural complexity with the flow on effect of their relationship to the cognitive complexity of the system. There are three metrics in this category. Cyclomatic Complexity (CC) [8] measures the number of distinct linearly independent paths through a block of code, or in other words, the amount of decision logic. While there are a number of ways of calculating this quantity, the most convenient is to use the number of decision nodes plus one, where a decision node is a branch statement in the code. CC is a widely used metric, which has an obvious direct relation to structural complexity, is used by other metrics to measure method complexity, and can be used to determine the comprehensibility, maintainability, and reliability of code. If complexity of code is high, it is difficult to understand and to maintain, and is more likely to be error prone. A common rule of thumb is that the cyclomatic complexity of a method should not exceed ten. Weighted Methods per Class (WMC) [4] measures the sum of complexities of all methods of a class. (CC may be used as the measure of method complexity.) In principle, the metric may serve a dual purpose – at a design level a method weight of one may be used in the absence of source code (essentially a method count), and at code level with an appropriate complexity metric. Usually it is limited to the code level, and method count is treated as a simpler metric. WMC is part of the well known and ground-breaking object oriented metrics suite by Chidamber and Kemerer. It is easy to implement (assuming a complexity measure is available) and can be used to determine understandability, maintainability and reusability of a class. In addition, it may provide insight into the suitability of the class for reuse and some measure of the impact of the class on the complexity of its subclasses.
Average Method Complexity (AMC) [5] is the average complexity of the methods of the class, based again on some complexity measure such as CC. Evidently, AMC is closely related to WMC but avoids undue inflation of the latter measure due to an excessive number of methods.
3.4 Inheritance Metrics Inheritance metrics are concerned with both the availability of methods and fields from parent classes, and in the degree to which these resources are exploited in the class structure. Inheritance is generally considered an important part of good object oriented design, as • it reduces complexity; • it avoids recoding of behavior; and • it supports polymorphism, through which specific child classes may benefit from the functionality of their ancestors. The four members of this category are described in turn below. Number of Children (NOC) [4] is a measure of the number of immediate subclasses of a class. It can be used to indicate: • improper use of inheritance – through an excessive number of subclasses • class complexity • the amount of reuse • the influence of the class on the overall design. It has been suggested that a suitable range for this metric is one to four. Depth of Inheritance (DIT) [4] measures the length of the ancestor path from a class to its root (the Object class in Java). It can be useful in determining the degree of specialisation of a class (the greater the depth the higher the specialisation) and the difficulty there may be in predicting the class’ behavior and understanding the class design. While there are drawbacks in having deeper inheritance trees, they have to be weighed against the benefits of inheritance such as reuse and abstraction. Method Inheritance Factor (MIF) [1] measures the amount of method inheritance in a system. It is defined as the proportion of the inherited methods of a class relative to all available methods of that class, averaged over all classes in the system. Available methods in a class are those declared plus those inherited, where inherited methods are defined as those available in ancestors that are not overridden. (See figure 5.) This metric is useful in providing a direct measure of inheritance in a system, and from this, the level of reuse in the system. Research by the author of the metric has pointed to there being a loss in the benefit of inheritance when MIF exceeds 70 to 80%. Attribute Inheritance Factor (AIF) [1] measures the degree of attribute inheritance in a system. It is defined in 7
Available methods Methods declared Class
Superclass
Methods inherited & not overridden
Overridden methods
New methods
Available methods
Figure 5. Method inheritance terms for MIF. the same fashion as MIF, except that it applies to attributes rather than methods.
3.5 Polymorphism Metrics This class of metrics measures the usage of polymorphism, where an operation may take on a particular form according to the nature of the object in the inheritance tree. The three metrics in this category are described in turn below. Polymorphism Factor (PF) [1] measures the amount of polymorphism in a system. It is defined as the proportion of the number of polymorphic situations of a class (a situation being a binding between a message sent to a class and a method form that it can take, there being as many situations as there are overriding methods), relative to the maximum potential number of polymorphic situations, averaged over all classes in the system. The maximum potential number of polymorphic situations occurs when every method in a class is overridden by every one of its descendents. This metric is useful in indicating complexity, comprehensibility and maintainability. A system with a high PF value is likely to be less complex, and consequently more understandable and maintainable. However, an overuse of polymorphism can increase complexity as it can make control flow harder to follow. It is suggested that in order to benefit from polymorphism, the PF should not exceed 10%. Number of Methods Overridden (NMO) [7] measures the number of methods in a class that override inherited methods. It is useful in that a large value can indicate that a class may not be a correct subclass by specialisation. A subclass is supposed to extend functionality rather than produce excessive overrides. It is suggested that NMO should not exceed three. Number of Added Methods (NMA) [7] measures the number of methods added by a class (i.e. added to those available through inheritance). This metric is useful in combination with DIT. The number of added methods should decrease with inheritance depth. If not, and a class has a large number of added methods, it may indicate that the class differs significantly from its superclass, and therefore may not be correctly subclassed or need to be split up.
Usage of these metrics, and the support for interpretation provided within the tool are considered in the following section.
4. Usability and Interpretation Perhaps the most important aspect of the JMR tool is its attention to usability, through the integration of tutorial and interpretation support for each metric. In this section, we examine each of these facilities in turn.
4.1 Browsing and Tutorial Support The core display of the JMR tool is the metrics spreadsheet window, available metrics being organised into the categories discussed earlier, and individual metrics denoted by a short abbreviation at the top of each column. In the most common configurations, this spreadsheet is coupled with a graphical representation of the Java Model, a display which is linked strongly to the metrics spreadsheet. As discussed earlier, the java model component of the tool provides a tree-based representation of the source code selected for analysis (in JMR, code packages and classes are selected using an intuitive GUI). In addition to forming the input to subsequent metrics collection, this representation – and the associated metrics extension architecture - allows the resulting data to be displayed at numerous levels of resolution. Within the JMR, this facility is controlled through mouse clicks upon nodes within the graphical tree representation, with the spreadsheet display of metrics data adjusting transparently to the appropriate selection. In this way, the tool may be used in the diagnosis of software at the package, class and method level, with the programmer able to identify the relationship between the different descriptions. This tree structure is further exploited through the availability of an integrated source browser, with the appropriate class or method code available through selection within the GUI. Tutorial information is provided to the user at three levels. At the simplest level, metric codes such MCX and NMO at the head of each column, emerge in unabbreviated form within tool-tips, appearing as the mouse is moved over them. At a summary level, these same column headers may be clicked to display a short description of the metric within the tutorial window. This frame also contains a tree of metrics, grouped by category, for browsing the short descriptions. Selection of a metric code in the tree brings up the associated description. A screenshot showing the features discussed so far in this section appears here as figure 6. At a detailed level, lengthy tutorial information is provided through the JMR Help window. This information is accessible through the main menu, or from the metric summary frame by clicking on a “More...’ hyperlink that is appended to the 8
Figure 6. JMR screenshot. end of each description. These three levels of metrics help information cater to the different levels of experience of users in assisting them to understand the metrics and use them properly. More significantly, it is our belief that they will remove one of the key barriers to the adoption of metrics tools within the wider software development community.
4.2 Interpretation The JMR also goes some way toward addressing what we believe is the second of the key factors in the poor uptake of metrics tools within the wider SE community – that of the comprehensibility of the information produced – and in the process, supporting further evaluation and validation of metrics as they are implemented. Central to both objectives is the context in which the metrics are understood, and it is critical that the software engineer be able to compare these figures with those obtained
by developers on similar classes of project. Moreover, the developer must be alerted when their code lies well outside the usual limits experienced by professional developers on such tasks. These ideas have been considered earlier by Abreu et. al. [2] in their application of the MOOD metrics, and they proposed that the developer should be advised if the resulting figure was outside some confidence interval based on an empirical distribution. These authors provided addition guidance through the analogy of the low-pass, band-pass and high-pass filters from Electrical Engineering, the shapes characterising the profile of particular metrics. (These are being incorporated into the JMR metric summary information.) We adopt a similar approach to the confidence interval, warning developers through highlights in the spreadsheet whenever the figure lies more than two standard deviations from the mean (yellow) or three standard deviations from the mean (red). Naturally, this approach will not apply to all metrics, but it provides a useful indicator of potential trouble. 9
However, the JMR provides an additional level of flexibility which we believe will be warmly received by developers – the option of configuring the tool so that these distributions are based upon particular subsections of the code used in the benchmarking process. Thus, users may select to have these indicators calculated from the entire code base, or from a particular application domain such as GUI programmes. This feature will be especially useful in large organisations, which may draw upon their own code base for such comparisons, and as a tool for managing software quality within their teams.
[5] Etzkorn, L., C. Davis and W. Li. “A Practical Look at the Lack of Cohesion in Methods Metric.” JOOP, Sept. 1998, pp. 27-34. [6] Li, W. and J. Talburt. “Empirically Analyzing ObjectOriented Software Evolution.” JOOP, Sept. 1998, pp. 1519. [7] Lorenz, M. and J. Kidd. Object-Oriented Software Metrics. Prentice-Hall, Englewood Cliffs, NJ, 1994. [8] McCabe, T. “A Complexity Measure.” IEEE Transactions on Software Engineering, December 1976.
5. Summary and Further Work In this paper, we have described a substantial new tool for the analysis of OO software, outlined its architecture and initial suite of metrics, and discussed its usability and integrated tutorial and interpretation support. In future development of this tool, our focus will be on three main areas: • Increasing the sophistication of the interpretation guidance provided to professional and student users; • Incorporating additional metrics, particularly those known to complement those already implemented; • Improving the extensibility of the tool at the language model level – perhaps incorporating a more generic OO modeling system. Most importantly, however, it is our intention to release the JMR to the wider software engineering community in late 2002, and use the resulting feedback to develop the tool so that it becomes a more natural part of the toolkit of many professional developers.
Acknowledgements This work was supported by QUT as part of a Teaching Learning Grants Scheme Large Project Grant.
References [1] Abreu, F. and R. Carapuça. “Object-Oriented Software Engineering: Measuring and Controlling the Development Process.” Proceedings of the 4th International Conference on Software Quality, ASQC, McLean, VA, USA, October 1994. [2] Abreu, F., R. Esteres and M. Goulão. “The Design of Eiffel Programs: Quantitative Evaluation Using the MOOD Metrics.” Proceedings of TOOLS USA’96 (Technology of Object Oriented Languages and Systems), Santa Barbara, California, USA, August 96. [3] ANTLR (Another Tool for Language Recognition). http: //www.antlr.org/ [accessed: July 3, 2002]. [4] Chidamber, S. and C. Kemerer. “A metrics suite for object oriented design.” IEEE Transactions on Software Engineering, Volume 20, Number 6, pp. 476-493. June, 1994.
10
