Building Reverse Engineering Tools with Software Components: Towards a Dedicated Development Process for Academia Holger M. Kienle University of Victoria Victoria, Canada
[email protected]
Abstract
Traditional approaches to tool building have resulted in tools that have a high degree of custom code and exhibit little reuse. This approach offers the most flexibility, but can be costly and can result in highly idiosyncratic tools that are difficult to use. To compensate for the drawbacks of building tools from scratch, researchers have started to reuse existing functionality, leading towards an approach that leverages software components—such as commercial off-theshelf components and products—as building blocks. This approach can be characterized as component-based toolbuilding (CBTD), as opposed to traditional tool-building The use of components fundamentally changes the development of tools and has unique benefits and drawbacks. However, this often is not realized by tool builders. Furthermore, CBTD is currently pursued in an ad hoc manner, because of a lack of known experiences and guidelines. As a result, the building of tools resembles a craft rather than professional engineering [13]. My dissertation is based on the hypothesis that the current state of CBTD in academia can be significantly advanced by introducing a more disciplined, predictable approach to tool building in the form of a dedicated development process. In fact, each development effort should follow a process regardless of the development effort’s domain. Presumably, tool builders in academia are following an implicit, undocumented process when they build their tools. However, this approach is too unstructured and hinders, for instance, communication among (new) team members, project assessment, and preservation of tool-building knowledge. To tackle these problems, the dissertation proposes a process framework based on work products (WPs). These WPs are created and evolved during the cause of the tool-building project and help to communicate assumptions (e.g., what are the tool’s requirements?), conduct assessments (e.g., does the tool meet the expectations of users?), and preserve knowledge (e.g., what is the architecture of the tool?). In the rest of the paper I first describe the two major contributions of my dissertation: Section 2 identifies the
My dissertation explores a new approach to construct tools in the domain of reverse engineering. The approach leverages already available (off-the-shelf) components and products, assembling and customizing them programmatically. This approach can be characterized as componentbased tool-building (CBTD). While many academic tools follow CBTD—presumable with the goal to save time and effort—, this approach is still pursued in an ad hoc manner. In my dissertation I strive to advance the current state of CBTD towards a discipline that is more predictable and formal. This is achieved with two major research contributions. The first contribution states the necessary requirements for a tool-building process in academia in the domain of reverse engineering. These process requirements have been identified with a literature survey of the current tool-building approaches that exist in academia as well as the author’s first-hand experience. The second contribution is a dedicated process framework for CBTD that meets the process requirements identified by the first contribution. The process framework is a first step towards formalizing CBTD with the goal to help researchers to build tools in a more predictable and effective manner.
1. Introduction Typical research in the reverse engineering domain proposes techniques to improve the comprehension and maintenance of software systems. Many researchers choose to implement a (prototype) tool to demonstrate the feasibility of their proposed technique, and to conduct user studies that show the tool’s effectiveness. My dissertation [8] explores the current state of tool building in the reverse engineering domain with the aim to improve upon the current practice. This topic is worthwhile to address because even though tool building is a popular technique to validate research, it is neither simple nor cheap to accomplish.
978-1-4244-2614-0/08/$25.00 © 2008 IEEE
416
ICSM 2008
process requirements for tool building in academia (i.e., feedback-based, iterative, prototype-based, lightweight, and component-based), and Section 3 describes the derived process framework (based on WPs) that meets these requirements. Section 4 then provides an evaluation of the process framework based on the NIMSAD method evaluation framework. Section 5 closes the paper with conclusions.
fill certain tool requirements. For instance, a prototype can demonstrate the usefulness of a novel algorithm without meeting the scalability requirements of a production tool. There are many examples of research prototypes but few researchers describe the benefits and drawbacks of this approach (e.g., Yang [16]). Wong proposes an iterative strategy where “each iteration builds an evolutionary prototype that is intended to be a solid foundation for the next iteration” [15]. lightweight: A lightweight process strives to minimize (intermediate) artifacts, recognizing that these artifacts are not the goal of a process, but a means to an end. Agile methods are typically more lightweight than plan-driven approaches [1]. An example of a lightweight development approach is Extreme Researching from Ericsson Labs [2]. It is an adaptation of Extreme Programming and used for applied research and development in a distributed workenvironment with changing human resources and rapid prototyping. component-based: CBTD proposes to leverage software components such as off-the-shelf products, IDEs, domain-specific tools, frameworks, and libraries. CBTD promises similar advantages to the ones reported for component-based development (CBD) in general; for instance, immediacy of acquisition, lower development costs, higher productivity, and greater ease of modernization. Many researchers leverage components when they build tools, but there are few examples of researchers who discuss their approach and experiences of CBTD. Reiss reports on his experiences with using FrameMaker to implement the Desert software engineering environment [12]. Coppit and Sullivan discuss how they leveraged Microsoft Word and Visio to implement the Galileo fault tree analysis tool [3]. To summarize, a strong process that addresses the building of reverse engineering tools in academia should accommodate the above five requirements.
2. Process Requirements It is my believe that for a development process to be effective, it has to meet the domain’s unique requirements and constraints. In other words, I advocate a strong method, which is domain-specific and addresses a specific type of problem as opposed to a weak one that is domainindependent and can be applied to diverse types of problems [6]. In order to arrive at a strong development process, it is necessary to first understand the problem domain that the method should address. This evaluation of the problem domain is carried out in the dissertation with a literature review. The review identifies experiences of researchers that have published about their tool-building approach. Based on this review and my own experiences I have been able to distill five requirements that a suitable process should have. I have gained experience in the building of reverse engineering tools with the evolution and maintenance of the Rigi (University of Victoria) and Bauhaus (University of Stuttgart) tools as well as the construction of several component-based reverse engineering and visualization tools (implemented on top of Visio, GoLive, Lotus Notes, and others) [8]. In the following, I briefly describe the identified requirements and important related work. The complete survey with the full set of references is described in the dissertation. feedback-based: Initial and ongoing involvement of the tool’s users is important to elicit the tool’s requirements and to improve on it in terms of features and usability. Examples of feedback-based development of reverse engineering tools are reported by Storey et al. based on user studies with students [14], and by Lethbridge et al. based on observation of professionals developers [11]. iterative: Requirements in tool-building can change frequently (e.g., because of user-feedback and modified hypothesis). Iterative development is most suitable to cope with requirements changes [10]. As an outcome of the SHriMP tool, Storey proposes an iterative approach for designing and implementing reverse engineering tools consisting of “several iterative phases of design, development and evaluation” [14]. Similar to Storey, Wong argues for an approach where the tool’s “evolution follows a spiral model” [15]. prototype-based: Prototypes are well suited to mitigate risks and changing requirements in applied research [2]. A proof-of-concept prototype does not need to (initially) ful-
3. Process Framework The process framework consists of eight WPs (cf. Figure 1), whereas a WP is defined as “any planned, concrete result of the development process” [7]. The framework focuses on WPs because often there is agreement on what should be produced, but not on how it should be produced. Thus, the WPs provide necessary guidance for tool builders without unnecessarily constraining them. Importantly, even though a process makes the building of tools more formal, my process framework is lightweight and flexible enough so that it does not place undue burden on researchers and does not stifle their creativity. I do not propose a full process because of the diversity of tool building projects in academia. Instead, the WPs in the framework are kept to a minimum with the expectation that researchers will add WPs as the need arises in the cause of
417
The Host Product Customization WP explores the component’s customization support and thus mitigates the risk of basing the tool development on an unsuitable component. Lastly, it is important to make the architecture explicit because it places constraints on the tool’s design and implementation. In CBTD, the Tool Architecture WP identifies the tool’s leveraged components and exposes the interaction mechanisms between them (e.g., sharing of data and transfer of control). WPs also support each other in reinforcing desirable goals for tool development (see dashed arrows in Figure 1). For example, the Functional Requirements WP provides the necessary information for the construction of a UI Prototype. Nonfunctional Requirements can have an influence on the Tool Architecture (e.g., scalability and interoperability). Certain requirements for the user interface exposed by the UI Prototype may preclude the use of a component otherwise suitable. Technical Prototypes play an important role to assure that the envisioned customizations of the host products are indeed feasible and that certain Nonfunctional Requirements (e.g., scalability) can be met. These dependencies show that if a certain WP is omitted there will be undesirable implications for the WPs that it supports. The goal of the process framework is to make tool building more predictable and effective. In this context it is important to note that the WPs that are produced in the cause of a tool building project are a vehicle to preserve valuable knowledge across projects and people. For instance, WPs can be used by new team members to rapidly understand the current state of a tool project, and by other researchers to jump-start their own projects.
Figure 1. Work products of the process framework and how they support CBTD their tool project. As a result, the framework is lightweight and enables a quick ramp-up time. In the follow, I touch on selected WPs and discuss how they help to support CBTD. The full description is available in my dissertation. The Intended Development Process is a meta-WP that instantiates a suitable CBTD process that meets the requirements identified in Section 2. Importantly, this WP makes the process explicit and provides rationalizations for the WPs and other process elements. Furthermore, it allows to change the process during execution, enabling tool developers to “find what works for them, and alter the process to fit” [5]. The other WPs (see right side of Figure 1) support specific aspects of the development cycle, covering elicitation of the tool’s requirements; identification, selection, and customization of suitable host components to build the tool; prototyping of the tool to identify risks early on and to gain experience with the selected components and new technologies; and a description of the tool’s (conceptual) architecture. The framework describes each WP concisely (in about 600–900 words) using a template (similar to [7]) that provides an introductory outline; the WP’s purpose and process constraints; techniques to realize the WP; advice and guidance; and references to related work. In the following, I outline how the WPs support CBTD (see solid arrows in Figure 1). Leveraging components for tool building has various risks, among them missing functionality of components and developers’ lack of experience with a component’s technologies. Prototype-based development is an important tool to mitigate these risks. Similarly, iterative tool development can more easily accommodate changes due to modified functional or nonfunctional requirements, and due to components or technologies that have to be exchanged while development is already underway. The Candidate Host Products WP supports CBTD by explicitly requiring a rationalization for the selection of the leveraged components. A recurring problem in CBTD is that a component that seems a promising candidate at first later turns out to be missing crucial support for customization (e.g., due to limitation of its API or scripting support).
4. Evaluation of Process Framework To better understand the properties and scope of the proposed process framework, I characterize it using the NIMSAD method evaluation framework [4]. This allows researchers who are interested in using my process framework to compare it with other available methods and processes. NIMSAD has been used, for instance, to evaluate methods for component-based software development [4]. A subset of the categories and elements of the framework are given in Table 1; the full set is described in my dissertation. The evaluation framework allows to characterize the process framework as follows: process goal: The goal of my process framework is to enable the effective construction of component-based tools that satisfy the requirements of tool users. process outputs: The output of the process are a set of WPs. In terms of the process’ goal, the resulting tool is the most important work product. stakeholders’ motivation: Both tool user and tool developer strive for a tool that satisfies the requirements of the tool user. The tool developer is interested in a process that
418
Category context
Element process goal process outputs
user
stakeholders’ motivation process guidance
contents
process structure artifacts
validation
method maturity output quality
Question What is the specific goal of the process or method? What are the results of the process? What are the stakeholders’ benefits of using the process? How does the method guide the user? What are the steps to accomplish the process’ goal? What are the artifacts created and managed by the process? Has the process been evaluated in practical case studies? How does the process evaluate the output it produces?
a more disciplined approach that resembles professional engineering—without stifling the creative nature that is present in tool building research. Other approaches should be pursued as well: more documented experiences and case studies are needed; and more disciplined approaches for tool construction have to be proposed, tried out, and evaluated. For example, I have distilled my tool building experiences into lessons learned (also described in my dissertation) [9]. I hope that other researchers will more actively discuss and publish their experiences as well. A platform for this is provided, for example, by the International Workshop on Advanced Software Development Tools and Techniques (WASDeTT).
References
Table 1. Characterization framework (subset)
[1] B. Boehm and R. Turner. Observations on balancing discipline and agility. Agile Development Conference (ADC’03), pages 32–39, 2003. [2] O. Chirouze, D. Cleary, and G. G. Mitchell. A software methodology for applied research: eXtreme Researching. Software—Practice and Experience, 35(15):1441–1454, Dec. 2005. [3] D. Coppit and K. J. Sullivan. Multiple mass-market applications as components. 22nd ACM/IEEE International Conference on Software Engineering (ICSE’00), pages 273–282, June 2000. [4] M. Forsell, V. Halttunen, and J. Ahonen. Evaluation of componentbased software development methodologies. Fenno-Ugric Symposium on Software Technology (FUSST’99), pages 53–63, Aug. 1999. [5] M. Fowler. The new methodology. MartinFowler.com, Apr. 2003. http://www.martinfowler.com/articles/ newMethodology.html. [6] R. L. Glass and I. Vessey. Focusing on the application domain: Everyone agrees it’s vital, but who’s doing anything about it? 31rd IEEE Hawaii International Conference on System Sciences (HICSS’98), pages 187–196, Jan. 1998. [7] IBM Object-Oriented Technology Center. Developing ObjectOriented Software: An Experience-Based Approach. Prentice Hall, 1997. [8] H. M. Kienle. Building Reverse Engineering Tools with Software Components. PhD thesis, Department of Computer Science, University of Victoria, Nov. 2006. https://dspace.library. uvic.ca:8443/dspace/handle/1828/115. [9] H. M. Kienle. Building reverse engineering tools with software components: Ten lessons learned. 14th IEEE Working Conference on Reverse Engineering (WCRE 2007), pages 289–292, Oct. 2007. [10] P. Kruchten. The Rational Unified Process: an introduction. Object Technology Series. Addison-Wesley, 1999. [11] T. C. Lethbridge and F. Herrera. Assessing the usefulness of the TKSee software exploration tool. In H. Erdogmus and O. Tanir, editors, Advances in Software Engineering: Topics in Comprehension, Evolution, and Evaluation, chapter 11, pages 73–93. Springer-Verlag, Dec. 2001. [12] S. P. Reiss. The Desert environment. ACM Transactions on Software Engineering and Methodology, 8(4):297–342, Oct. 1999. [13] M. Shaw. Prospects for an engineering discipline of software. IEEE Software, 7(6):15–24, Nov. 1990. [14] M.-A. D. Storey. A Cognitive Framework for Describing and Evaluating Software Exploration Tools. PhD thesis, Simon Fraser University, Dec. 1998. [15] K. Wong. The Reverse Engineering Notebook. PhD thesis, Department of Computer Science, University of Victoria, 1999. [16] H. Yang. The supporting environment for a reverse engineering system—the maintainer’s assistant. Conference on Software Maintenance (CMS’91), pages 13–22, Oct. 1991.
allows to build tools in an effective, repeatable, and predictable way. process guidance: The framework provides a skeleton that needs to be instantiated. There are—deliberately—few constraints on how to accomplish this. Instead, guidance can be obtained from suitable process frameworks (e.g., RUP [10]). process structure: The framework does not define concrete sequences of activities. It does not prescribe when a certain WP needs to be created or modified. However, there are implicit dependencies between WPs (e.g., creating a Tool Architecture before any tool requirements have been obtained does not make much sense). artifacts: The artifacts of the framework are the defined WPs (and others that a process instantiation may add) as well as the tool itself. method maturity: So far there is no case study of an instantiation of the framework. However, there are several CBTD case studies (described in my dissertation) and existing processes that indicate the usefulness of individual WPs. output quality: The framework does not prescribe or require an evaluation of the produced WPs. Generally, few processes address evaluation; RUP is an example of a process that gives evaluation criteria to assess the project after each of its four phases [10].
5. Conclusions My dissertation strives to advance CBTD beyond the craft stage by providing a dedicated process framework. The process framework is a strong method in the sense that it specifically addresses the requirements of the construction of reverse engineering tools in an academic environment. The process framework is one approach to advance the current state-of-the-art in tool building towards
419
