Evolution Is Essential for Software Tool Development - CiteSeerX

National Research Council Canada

Conseil national de recherches Canada

Institute for Information Technology

Institut de technologie de l’information

Evolution Is Essential for Software Tool Development M. Wein, S. A. MacKay, D. A. Stewart, C.-A. Gauthier and W. M. Gentleman Software Engineering

July, 1995

NRC No. 38378

This report also appears in the “Proceedings of the 1995 International Workshop on Computer-Aided Software Engineering (CASE-95),” Toronto, Ontario, July 9-14, 1995.

Copyright 1995 by National Research Council of Canada

Copyright 1995 par Conseil national de recherches du Canada

Permission is granted to quote short excerpts and to reproduce figures and tables from this report, provided that the source of such material is fully acknowledged.

Il est permis de citer de courts extraits et de reproduire des figures ou tableaux du présent rapport, à condition d’en identifier clairement la source.

Additional copies are available free of charge from:

Des exemplaires supplémentaires peuvent être obtenus gratuitement à l’addresse suivante:

Publication Office Institute for Information Technology National Research Council of Canada Ottawa, Ontario, Canada K1A 0R6

Bureau des publications Institut de technologie de l’information Conseil national de recherches du Canada Ottawa (Ontario) Canada K1A 0R6

ii

Evolution Is Essential for Software Tool Development M. Wein, S. A. MacKay, D. A. Stewart, C.-A. Gauthier and W. M. Gentleman Institute for Information Technology National Research Council of Canada Ottawa, ON K1A 0R6 email: @iit.nrc.ca Abstract We examine the problems encountered in evolving software development tools, in particular, configuration management tools, drawing on our experience in producing the DaSC configuration management toolset. Building good software development tools is a time-consuming, iterative process where the tool designers must take into account the evolving nature of the software development process embodied in the tools, the evolving nature of the environment in which the tools exist, and the evolving understanding that the tool designers have of the users of the tools. We believe that our experience with the tools that we have developed is applicable to other kinds of tools used in industrial settings.

Key Words:

Software development, software tools, configuration management, optimistic concurrency control.

1. Introduction In the Software Engineering Laboratory at the National Research Council of Canada, we have been developing tools for configuration management for eight years. These tools were built to support a software development process based on DaSC, a layered configuration management model that we developed. The DaSC model and its associated process have been used on several development platforms in the development of Harmony, a multiprocessor realtime operating system that supports multiple targets and processor variants. What may seem surprising is that in the eight years that we have been working on the tools, we have found it worthwhile to re-implement them several times, each time with new functionality. We found that it was impossible to finalize the functionality of the tools because of the evolutionary nature of the software development process. As we used tools, our understanding of the process we NRC number

38378

were using changed. More than that, the tools themselves influence the process, thus creating a feedback loop. We also changed the tools because our understanding of how users of the tools perceive and organize the information the tools present to them changed, making it necessary to present users with different information or with the same information in different formats. Tools also had to be reimplemented in response to changing technologies and changing markets. When we started, we used VMS extensively and wrote many tools under that operating system. We recently abandoned support for VMS. Our preferred development platform is the Apple Macintosh. Many tools that we wrote under VMS have been replaced by third party tools. This paper discusses the difficulties in implementing software development tools, and how we either avoided or overcame some of them in the implementation of tools to support our configuration management system and its associated process. The discussion presented here applies to a wide range of software development tools. The theme of this paper is that good software development processes and their associated tools must be developed iteratively, and take years to mature.

2. A Summary of DaSC Configuration Management To place the discussion in context, a brief summary of our configuration management system (DaSC) is necessary. A complete discussion of DaSC can be found in [1]. The primary objective of modern configuration management tools is to allow teams of programmers to work on a project, sharing the various components that comprise the project, yet maintaining the integrity of the repository. The principal control mechanism in tools like SCCS [2] and RCS [3] and their successors has been the check-out/check-in lock, which allows access to a configuration item, typically a file, by only one person at a time. The most common mechanism to handle concurrent modi-

1

fications is by creating new branches. Branching can become extremely complex in a large software project as new branches are added to support families of components with similar yet unique pieces, to permit bugs to be fixed in previously released software, and to permit bug fixes and new development on the same configuration item. Also, the check-out/check-in model suffers from the “haste-makes-waste” approach, that is, developers are strongly encouraged to make changes quickly and return the configuration item to the repository so that it is available for someone else to check it out. Unfortunately, this emphasis often leads to errors due to inadequate thought and inadequate testing. An alternative control mechanism, optimistic concurrency control, is most commonly implemented as copymodify-merge. A developer is given an unlocked copy of a configuration item and the time to modify it properly. When the modification is complete and fully tested, the item is returned to the repository. Because no one else is blocked from working with the same item, other modifications may be made simultaneously, so the procedure for returning the configuration item must manage potential clashes (usually with human supervision). The Database and Selectors Cel (DaSC) approach to software configuration management [1], builds on the copy-modify-merge technique. DaSC manages multiversion software through a software database, and handles concurrent, distributed evolution through a multilayered approach (analogous to drawing on cels, or sheets of celluloid, as used by film animators).

Derived .obj, .exe, .lib tools

Working layers, W1 and W2 Master

Figure 1 : A set of layers used by a developer. Tools operate on items in Master, W1 and W2 and deposit the results in the Derived layer. In DaSC, the software components that comprise an application or a system are constructed as a set of layers. The layers constitute an ordered set, where the order roughly corresponds to the temporal evolution of the software under configuration control. (Time stamps do not work well in an intermittently connected world.) Figure 1 shows an example of a near-minimum set with four layers. The Master layer at the bottom contains the baseline

reference. The Working layers, containing items being modified, are stacked above; for illustration purposes, we show two groupings of changes on layers W1 and W2. The top layer, Derived, is a throw-away layer where development tools, such as compilers and linkers, deposit their results. Manual operations are never used to add or modify items on the Derived layer and all contents may be regenerated as needed. For the sake of generality, we initially decided to implement DaSC with a minimum of custom software and to use the underlying development environment as much as possible. In our current implementation, each layer is stored in a parallel tree in the file system. The Master layer contains a complete and a self-consistent release of the system and all the necessary files are stored there. A release in this context is a baseline of all the components under configuration control. The release will have passed some defined quality assurance measures and is ready for a “customer.” Working layers and the Derived layer contain sparse trees, in that the only nodes and branches present are those that are necessary to store the modified files. To the developer, the program being worked on is the composite of all the layers (other than Derived), where the most recent copy of an item to be used is the one from the highest layer in which the item exists. DaSC currently uses the file system tree as the code database. Each node in the tree differentiates a specific variant. Variants are different implementations of a component that remain valid at a given instant. They are created to handle environmental differences (for example, different execution processor or for a specific peripheral chip). DaSC uses the inclusion capability of the C preprocessor in a somewhat unconventional way, to define a version, which is a specific collection of components, both common code and required variants. The desired version of a module is specified by a selector file, which consists of a series of #include statements. Each #include statement contains a full pathname that points to the specific variant of a C function or a “.h” header file that must be included in that version of a module. The structure is illustrated by a simple example in Figure 2 for a program, called tool, with three versions: an MPW C version targeted at Macintosh platforms, a Microsoft C version targeted at MS-DOS platforms, and a GNU gcc version targeted at generic UNIX platforms. The selector files are shown on the left and the source code files are on the right. The selector file tool.c for the UNIX gcc version, at the bottom of Figure 2, specifies two common functions and two variant functions (in the shaded directories). For the sake of simplicity, the single selector file in this example defines not just a module but the entire program. In a realistic situation, the organization is considerably more complex, but DaSC scales up. The Harmony

2

tool

selectors

source database

inc

src

macmpw

dosms

unixgnu

tool.c makefile version.doc

tool.c makefile version.doc

tool.c makefile version.doc

tool.h main.c comm.c abstract.doc

mac

dos

unix

mac.h getfile.c macfile.c variant.doc

dos.h getfile.c dosfile.c variant.doc

unix.h getfile.c unixfile.c variant.doc

/* * Headers */ #include "MASTER/tool/src/tool.h" #include "MASTER/tool/src/unix/unix.h" /* * Common code */ #include "MASTER/tool/src/main.c" #include "MASTER/tool/src/comm.c" /* * Unix specific code */ #include "MASTER/tool/src/unix/getfile.c" #include "MASTER/tool/src/unix/unixfile.c" Figure 2: DaSC Code Structure. Selectors are on the left and the source files on the right. realtime Operating System [4] has dozens of variants; some of them include: the development platform (as illustrated in Figure 2), the specific compiler, the target processor type, the model of the processor board, the interprocessor interrupt integrated circuit (I/C), the serial port I/C, plus many others related to communication protocol suites and the file system. There are two consequences of the organization in DaSC: all dependencies on the development system and pathname notation are localized on the selector side of the code database, and there are no #include statements in files

on the source side of the tree. Each file contains one function, represents a unique variant and hence is free of #ifdef statements. Revisions, being the steps a configuration item goes through over time, are created in new layers. A new version of a program, a system or a sub-system is composed of a selection of common code and variants from any valid revision layer. Figure 3 illustrates revision layers superimposed on the original one from Figure 2. Usually a new layer is created to facilitate a single logical change to the software. It is a logically complete entity—applying the

3

/* * Headers */ #include "MASTER/tool/src/tool.h" #include "MASTER/tool/src/unix/unix.h" /* * Common code */ #include "Work-change/tool/src/main.c" #include "MASTER/tool/src/comm.c" /* * Unix specific code */ #include "MASTER/tool/src/unix/getfile.c" #include "Work-add-del/tool/src/unix/posixfile.c"

tool

inc

unixgnu tool tool tool.c makefile version.doc inc inc

src

Derived

unixgnu

unix tool tool posixfile.c

tool.c

unixfile.c

Work-add-del

src src

main.c tool tool

src

inc inc

Work-change

macmpw

tool.c makefile version.doc

dosms

tool.c makefile version.doc

unixgnu

tool.h main.c comm.c abstract.doc

mac

mac.h getfile.c macfile.c variant.doc

tool.c makefile version.doc

dos

dos.h getfile.c dosfile.c variant.doc

unix

unix.h getfile.c unixfile.c variant.doc

MASTER

Figure 3: Revision Layers applied to the example of Figure 2

4

new layer to the layer (or layers) below it will result in a stable and complete rendition of the software with that change applied to it. In Figure 3, the Work-change layer introduces a new revision of main.c while the Work-adddel layer replaces unixfile.c by posixfile.c in the directory unix . The process described by the example in Figure 3 illustrates that the layered approach to software evolution is ideally suited to team software development, in that different Working layers are being developed independently by different individuals. Cels (layers in the sense used in animation) provide a representation of change without violating the integrity of the original. Evolution is supported by adding subsequent layers to a master repository. We have built two major tools to support the management of DaSC layers as well as some auxiliary ones. The first major tool is derive, which generates selectors on the Derived layer to extract the most recent set of functions and declarations that make up a program (shown as the top layer in Figure 3). For example, the selector file tool.c in the Derived layer in Figure 3 was produced using the derive tool. For each item in the set, derive performs an ordered search through the Master and relevant Working layers, top to bottom, to locate the copy from the “highest” applicable layer. The second major tool, consolidate, resolves conflicts among a set of layers and consolidates or merges the candidate layers into a new composite layer. We also use an extensive set of editor macros (scripts) to provide the glue between the tools and the editor.

3. Evolutionary Nature of the Process Having introduced DaSC, we now return to the discussion of the problems of implementing software development tools.

3.1 Understanding the process The first step towards implementing successful software development tools is to understand the software development process that the tools should support. This is an obvious observation but is, in fact, a complex task. The task is complicated by the fact that usage of the tools inevitably leads to changes in the process itself. Because of this feedback loop, there can never be a final version of the toolset. The early implementations of the DaSC tools assumed that each developer had only three layers: a readonly copy of the baseline Master repository, a Working layer containing those items being revised, and the Derived layer. The original model of development was that, once all the changes were tested, the modified Working

layers from the various developers would be merged back into the Master layer to create an updated Master. Experience with the early version of the tools showed that the initial process used with the DaSC model was inadequate. We found that it was difficult for a single developer to start a new activity before the previous one was consolidated with the Master. Also, for temporary experimentation and for quick bug fixes, it is convenient to introduce additional layers. Such temporary layers are also useful for sketching out a feature that might be implemented sometime, but is not part of the current product. Additionally, when a developer wants to share a layer with a colleague, it is imperative that the shared layer be frozen and a new active layer started. Therefore, developers can, and typically do, have several intermediate layers that have not been incorporated into a new revision of the Master [5]. The change in the process led to changes in the tools. We changed the consolidate tool to allow us to combine the various active intermediate layers into an interim composite layer that is super-imposed on the previous baseline, both to test for possible conflicts and to simplify the view presented to the developer. The individual layers are archived, but for the day-to-day use the interim composite layer is used. A significant lesson that we learned using DaSC is that concurrent development means that new activities may start before completion of earlier ones and consequently the tools must support this overlap flexibly. Clearly, an operational model must support many groupings of layers or sub-layers. In the present implementation, each developer tends to work with typically six to eight layers, limited mainly by the visualization tools and by the cognitive load on the developer, as will be discussed later.

3.2 Extending the process As the first generation of a tool begins to be used and accepted by the intended user community, users see the domain of applicability expand and they demand increased power from the tool. For example, our consolidate tool was originally designed to create a new Master layer from a set of Working layers. However, developers soon wanted to use the tool to create intermediate layers (consolidate several layers of changes into one, but not update the Master). Also, for marketing reasons, a software release may need to be shipped to some customers with some restricted functionality. It was observed that a change to consolidate would permit this to be performed easily with a mask layer. The tool was modified (over a couple of iterations) to provide these features. Today, consolidation to intermediate layers is the most common way in which the consolidate tool is used.

5

One can generalize that tools grow to solve new and unforeseen problems. As the understanding grows, so do the needs and expectations. A significant part of the evolution of tools is driven by this growth in the domain of applicability.

4. Evolutionary Nature of Markets and Technologies

3.3 Scaling the process

The body of program source text is often one of the most important capital assets in a corporation. A toolset may require that the source code be converted into a proprietary format. For example, in many configuration management systems, source code is stored as compressed deltas. With time, the toolset may change not only when the process changes, but also when new tools become available or when the tool supplier no longer supports the product. It is therefore critical to be able to recover the source from its proprietary format. A key question when evaluating a toolset is whether it will be possible in the future to migrate to an alternative set of tools without the assistance of the suppliers of the original tools, and at what cost. We have avoided the risks of proprietary formats by keeping the source code in a conventional file system rather than in proprietary databases. We maintain source code in uncompressed form as a set of text files in the ASCII (ISO–646) character set with careful attention being paid to the naming conventions and to limits on treedepth to minimize clashes for the various development systems that we have been using. For example: • File names must be distinct in the first eight characters, as required by MS-DOS (the rest are ignored, but they are not ignored in OS/2).

Scaling the process and its associated tools to the size of the project is critical to its success, but this issue is not adequately covered in the literature. Industrial developers are most familiar with the problems of scaling but are less likely to publish than academics. A successful tool must be usable over a wide range of project sizes and especially must be effective for large scale problems where developers can number in the tens and the body of source code can exceed one million lines. Scale presents several problems for the tool developer. Clearly, the tool must be usable for large projects, but it must also be usable on small projects. For configuration management tools, scale has an impact on the tool structure. The core function of the configuration management tool is to maintain the integrity of the program components with clear definitions of releases, interim releases and working sets. Auxiliary functions are record keeping tasks: the maintenance of change logs, of documentation, of who changed the file and why. These tasks may be done semi-manually for small projects, but as problems get larger, more of the procedures must be automated and the approval process formalized. Where the record keeping might be done manually for a small project, it must be done by automated scripts and procedures for larger projects. In order for a CM tool to scale effectively, it must have facilities for automating, as much as possible, the record keeping process, yet permit doing it manually for smaller undertakings. Automated record keeping may impose larger overheads than manual records keeping. This overhead is more acceptable in large projects than in small projects because of the dire consequences of not keeping records in large projects. We implement change logs as human-readable text files with a structured format so that not only could automated tools process these change logs, but also the logs became more readable by humans. The copy and paste facilities in modern window systems and scriptable editors make this approach practical, so that the format of the templates is preserved. Manual record keeping succeeds because multi-window editors permit and encourage updating the log at the same time as the revisions are made in step with the revision of the code, not as a separate step. This approach has blended the transition from manual, to semi-automated, towards fully automated record keeping. Above all, record keeping is done at all scales.

4.1 Adapting to a changing tools market

• The depth of the path in the file tree is limited to eight nodes, as imposed by VMS, which we have used in the past. • The length of the total pathname is most severely limited under MS-DOS to 64 characters. With the observed migration of development for embedded systems to using Microsoft Windows, the DOS file limitations are more widely recognized as a serious inconvenience that will hopefully disappear in 1996. It is relatively easy to resolve name clashes for individual files, but when thousands of files are involved, in highly structured file trees, questions as to naming conventions, tree depth, etc. must be resolved a priori. Our disciplined use of the file system facilitates porting to other development systems and using new tools as they evolve. Yet, porting even just plain text files is plagued by problems that are so trivial as to be embarrassing to the industry. For example, with the release of the Harmony Operating System on CD-ROM that was prepared in the portable ISO 9660 format for the three major development

6

platforms, we had to deal with three line-break conventions: one with CR’s for the Macintosh, one with LF’s for UNIX and one with both for MS-DOS. Fortunately, it is easy to move a body of code kept in ASCII text files between these three platforms. One approach is to archive the files in a format available on all three platforms. The archiving program converts the platform-specific linebreaks into the archive-specific one when archiving the source, and again does the proper platform-specific conversion when extracting the source.

4.2 Adapting tools to a changing market With time, the preferred development platforms change. For example, VMS used to be a common development platform. Today, VMS is almost never used as a platform for new development projects. If a toolset is to have a long and useful life, it must not only be portable to different platforms, that is, to different hardware architectures and operating systems, but it should also exist on different platforms. Many software products are offered on a variety of platforms. It is highly desirable to use the same tools to develop all variants of a given application. Different strategies for such cross-platform development are used by various companies: (a) several independent teams carry out development for each of the platforms; (b) the code is organized as a core engine plus personality modules for each of the window systems; (c) portability abstraction tools, like XVT [6] or Galaxy [7], are used to support one body of code; or (d) a trickle down process of finishing for a lead platform and then reworking for the other platforms is used. Apart from the question of the merit of the various approaches, a configuration management toolset must support several of them and offer some flexibility in how the tools may be used. In all the approaches other than (a), there is a need for convenient access to the source from any of the development platforms. Furthermore, the access must be bi-directional so that modified source can be returned seamlessly into the primary repository. Even if one platform is the principal and preferred one, some development usually takes place on the other platforms, so cross-platform migration of source text must be supported. In the limit, one could take the approach that the non-primary platforms be treated as runtime targets with no development taking place there, as is the practice with targets in embedded systems. However, such a centralist view does not appear desirable, nor is it likely to resolve all the “look and feel” issues because the developer would never have experience on the target platform. We have been very successful at accessing our source files from a variety of platforms by carefully crafting the source tree to facilitate this. We have been less successful

at migrating tools across different platforms. We have had a high degree of success with migrating command line driven UNIX-like programs because almost all platforms offer the basic UNIX I/O calls and all supply the ANSI-C library calls. However, migrating tools with rich graphical user interfaces is much more difficult. We have implemented some tools using the XVT portability toolkit. However, XVT and other similar portability toolkits offer only a limited set of abstractions. They either do not offer or are late in delivering the functionality that users have come to expect on the various platforms, such as drag-anddrop copying or context sensitive help. There can be fundamental differences in the way that such functionality is delivered on different platforms, making it difficult, or even impossible, to provide common abstractions across all platforms that implement the native look-and-feel of each platform. For example, Apple’s proposed approach to port OpenDoc [8] components, which must comply with the Apple Open Scripting Architecture [9] and with IBM’s SOM implementation [10] of the OMG CORBA ORB [11], is to implement the OpenDoc architecture in its entirety on non-Macintosh platforms, ignoring equivalent native functionality.

4.3 Adapting to changing technology Changes in computing technology cause changes in the way that software developers work. The current generation of personal desktop workstations are as powerful as the mainframes of a decade ago. Interconnected by highspeed networks, these workstations caused development to become increasingly concurrent. Managing concurrent development requires new software development processes and requires new tools. Also, changes in interapplication communication technologies and in user interface technologies have increased the expectations of developers, who now demand richer user interfaces and the seamless exchange of data between applications. Tools should help to improve the development process and the developed product. As technology changes and the development process changes, tools must evolve to remain effective. Unfortunately, it may be almost impossible to evolve a toolset in the right direction if that toolset implements a bad software development process or if the toolset shows a fundamental lack of understanding of the way in which developers work. Configuration management tools based on the classic check-out/check-in approach are difficult to use with a concurrent development process. The problems are compounded by the common practice of organizing C program source into files that contain a (sometimes large) number of C functions and of tracking the configuration at the file level. What functions are packaged into any particular file

7

is determined by the judgment of the original developer. What needs to be checked out during a revision cycle is determined by the nature of the revision: the few functions to be revised may be spread across several files that contain many other functions as well. The two views rarely agree, resulting in having to check out many extra functions in order to freeze a small set of scattered but related functions for revision. Parts that are checked out become unavailable for write access. The impact of this unavailability must be minimized by the project manager. The problem becomes more acute as the project approaches a release. The check-out/change/test/check-in cycles must accelerate to accommodate all the quasi-concurrent changes, often leading to “haste-makes-waste” problems. Keeping the project under control puts a strangle hold on the entire development team. We specifically chose to keep multiple and distributed copies of the Master layer and to use optimistic concurrency control in DaSC to reduce the problems caused by locking mechanisms [12]. Our current approach to configuration control in DaSC is to keep each C function (or a set of related definitions) as a separate file and to group a collection of the items that comprise a “logical change” into one layer. Thus the programmer works, not on one file, not on one function, but on one layer. What is in that layer is grouped by what makes sense at the time of the revision cycle, not at the time when the set of functions was created, so it holds the minimum data necessary to represent the logical change. This structure reduces considerably the difficulty of managing the revision/release process. The structuring does however increase the number of files a programmer must work with at any one time. This increase, in turn, creates a need for a more sophisticated programmer’s editor—one that can manipulate many open files simultaneously and that includes such features as multifile search and replace, window tiling, etc. Such editors used to be rare; now they are available on all platforms. Because of new trends in the representation of text in rich, yet widely readable formats [13, 14, 15], the time is ripe for us to re-examine the choices made a decade ago. There have been several suggestions that a richer representation of source code enhances readability [16, 17, 18]. New technologies are now being put into place that will allow the use of font styles and graphics to enhance the readability and understanding of programs. We also need to consider how to adapt our tools to deal with compund documents and component techologies mentioned in section 4.2.

5. Evolutionary Nature of Models of Users We have argued that software development tools must adapt to ever-changing software development processes, technologies and markets. These truths are widely recognized. What is less recognized is that the tools should also be changed as the developers of the tools gain a better understanding of the users of the tools.

5.1 Adapting tools to users’ cognitive abilities The size and complexity of software systems are growing rapidly. Recent victims of “creeping featurism," such as Microsoft Word 6.0, attest to this. Telecommunication systems that contain over a million lines of code are not uncommon. Unfortunately, the ability of programmers to store information and understand a system remains constant. Consequently, tools must organize and present information to users in increasing levels of abstraction if the users are to understand the systems they are to maintain. Too often, software development tools present information in small bits full of low-level details, typically the source text, and leave it to the user to formulate a mental model of the overall system. We have made some efforts in DaSC to provide visualization tools that can help users develop mental models of the software systems stored in the repository. However, we found that DaSC also provides, by its very nature, a visual representation of the evolution of the repository, something that is equally important. When a programmer needs to understand what has changed and how, he or she needs to look at many configuration items. Observing what has changed in any one function in isolation is not sufficient, and hence, grouping revisions into conceptual layers has permitted viewing the entire set. Of course, there are instances when an alternative “historical view” of the evolution of changes to a single function is useful. In DaSC either of the two views is available to the developer. We have learned that the tools must support easy navigation through many layers. Furthermore, there must be an easy way to open all the changed files in an editor such that there is a fluid transition between the view of the layer and the text in the editor. Given that the programmer may wish to select a different editor from the one we prefer, we are investigating various scripting systems, which have been gaining popularity, for controlling applications, such as Tcl [19], Perl [20], AppleScript [21], Frontier [22], and Visual Basic [23]. With such scripting tools we are able to integrate disparate commercial tools into a composite window-oriented environment.

8

Our effort to develop visualization tools for managing programs composed of a large number of functions and having many valid versions (and hence inter-dependencies) is gaining unexpected allies—developers of browsers for object class libraries. While our work predates the growth in popularity of object-oriented programming, it shares a common thread: programs are becoming progressively more complex, and tools are needed for gaining a larger scale view, in fact, multiple views, of the aggregate, not merely for seeing what happens inside one function. The significant idea in browsers is that the programmer can browse the source code and must have tools for navigating and seeing a larger scale view.

5.2 Adapting tools to users’ cultures Tools should also fit into the cultural context of the users. Three different meanings of cultural contexts can be identified: (1) “social” culture, which is the usual one and which relates to languages, beliefs and other social aspects; (2) “corporate” culture, which relates to the way in which humans work within an organization; and (3) “engineering” culture, which relates to the common practices and the well known and accepted methods of software development. It is finally becoming standard practice to pay attention to the social culture in which applications will be used. Recent internationalization standards, such as the XPG4 Common Application Environment (CAE) standard [24, 25], have begun to address the issues and to propose solutions. These are far from complete, but at least represent a step in the right direction, even if only because they are forcing developers to think about the issues. For software development tools, fitting into a corporate culture mostly implies that the toolset must be adaptable to the software development processes already in place or, at least, not perturb those processes too much, otherwise the tools will more than likely become “shelfware.” The need to adapt tools to the process was discussed at length in Section 3. Tools should also fit into the engineering culture. There exist a number of well-known paradigms for software development and various methods that cover the various stages of software development. Programmers are used to working with one or a few of these and are more likely to use a particular toolset if it supports a paradigm or method that they know and that suits the problem domain. To be successful, a tool must not introduce radical departures from the familiar structure. One abstraction that the programmer uses to work with a large body of code, such as an operating system, is the source file tree, which can be partitioned into coherent areas, such as the kernel,

the drivers, the shell, etc. As long as the programmer sees the code as a tree, the configuration management system must preserve this view, and it must offer procedures for manipulating the files and use conventional ways to refer to the nodes in the tree. The DaSC configuration management system introduced a strong cognitive model, that of a set of “transparent” layers, where each layer contains a group of functions belonging to a “logical change” [26]. The actions of addition of a function, deletion of a function or replacement by an updated function have strong visual metaphors. Progressive minor releases that are not yet merged into a new Master layer form an ordered set of layers, top to bottom. We have found that the model works well, particularly in tracking concurrent development.

6. Conclusion This paper discussed the difficulties in building software development tools. We argued that tools must evolve to match the evolving software development process. We stated that the process and its associated tools change as tool developers gain a better understanding of the development process, as the process and the tools are used on projects of widely different sizes, and as the tools are applied in unforeseen ways. Tools also change with changes in the development platform market, with changes in the third party tools market, and with changes in technologies. Finally, tools should change as the developers of tools refine their understanding of the users of the tools and of how the latter use the tools to capture design and historical information for the repository. We have been developing tools to support the DaSC configuration management model for eight years and we are still not satisfied with the toolset that we have. Part of the problem is that the process associated with the DaSC model keeps evolving as we try successive generations of tools and understand more about the way we really work rather than how we thought we worked. Specifically, only as we gained experience using tools that made it easier to work with layers did we realize the great potential benefits offered by using many layers. An a priori requirements analysis would never have revealed this. The problems we encountered are not unique. The long evolutionary process can also be observed in the way word processors have advanced over the last 15 years: current ones incorporate the collective experience of many suppliers over many releases. Unfortunately, we do not forsee any improvements in the problems of tool evolution in the near future. Recent advances in cross-platform graphical user interface toolkits and builders and in document centric architectures promise

9

solutions. However, they remain hampered by problems of technology. The portability toolkits offer limited abstractions that are either late appearing or under-powered. The document centric architectures fail to deal with the fundamental differences in the way that various features are implemented on each platform.

7. Acknowledgements Several summer assistants contributed to the development of various versions of the DaSC tools: Kevin Wood, Farhana Sheikh, Alain Hajjar and Kathryn Noddin. Marc Gauthier has evaluated these tools as part of the development of the Harmony Operating System.

8. References 1.

Gentleman, W.M., MacKay, S., Stewart, D., and Wein, M., “Commercial realtime software needs different configuration management,” Proceedings of 2nd International Workshop on Software Configuration Management (SCM), Princeton, NJ. October 24–27 1989. Published in Software Eng. Notes, Vol. 17, No. 7, 1989, pp. 152–1 6 1 . 2 . Marc J. Rochkind, “The Source Code Control System,” IEEE Transactions on Software Engineering, Vol. SE-1, No. 4, December 1975, pp. 364–370. 3 . Walter F. Tichy, “RCS—A System for Version Control,” Software—Practice and Experience, Vol. 15, No. 7, July 1985, pp. 637–654. 4. W.M. Gentleman, S.A. MacKay, D.A. Stewart, and M. Wein, Using the Harmony Operating System, Release 3.0, NRC/ERA-377, National Research Council of Canada, Ottawa, ON, February 1989. 5. K.E. Noddin, Derive and Consolidate in the DaSC Configuration Management Model, National Research Council of Canada, Institute for Information Technology Technical Report. In preparation. 6. XVT Software Inc., XVT Toolkit, Release 4.0, Boulder CO. 7. Visix Software Inc., Galaxy, Version 1.2, 11440 Commerce Park Drive, Reston, VA 22091 8 . Apple Computer, Inc., OpenDoc Programmer’s Guide for the Macintosh, To be published by Addison-Wesley Publishing, Reading, MA. 9 . Apple Computer, Inc., Inside Mac: Interapplication Communication, Addison-Wesley Publishing, Reading, MA, 1993. 10. International Business Machines Corporation, SOMobjects Developer Toolkit, Version 2.0, International Business Machines Corporation, June 1993. 11. Open Management Group, The Common Object Request Broker: Architecture and Specification, Revision 1.1, QED Publishing, Wellesley, MA, 1992.

12. Stephen A. MacKay, “The State of the Art in Concurrent, Distributed Configuration Management,” Proceedings o f the 5th International Workshop on Software Configuration Management (SCM5), Seattle, WA, April 24–25 1995. 13. R.M. Baecker and A. Marcus, Human Factors in Typography for More Readable Programs, ACM Press, Addison Wesley, 1990. 14. W. M. Gentleman and M. Wein, “Mass Market Computers for Software Development,” Proceedings of the 1991 Symposium on Personal Computers and Small Computers, Toronto, ON, June 12–14 1991. ACM Press. 15. Paul W. Abrahams, “Typographical Extensions for Programming Languages: Breaking out of the ASCII Straightjacket,” ACM SIGPLAN Notices, Vol. 28, No. 2 , February 1993, pp. 61-68. 16. FrameMaker Technology Corp., FrameMaker Reference, San Jose CA, 1991. 17. International Standards Organization, Standard Generalized Markup Language (SGML), ISO-8879:1986. 18. CERN, HTML+ Markup Format Document, http://info.cern.ch/hypertext/WWW/MarkUp/HTMLPlus /htmlplus_1.html. 19. J. K. Ousterhout, Tcl and the Tk Toolkit, Addison Wesley Publishing, Reading, MA, 1994. 20. L. Wall and R.L. Schwartz, Programming Perl, O’Reilly & Assoc. Inc. Sebastobol, CA, 1991. 21. Apple Computer, Inc., AppleScript Language Guide, Cupertino CA. 1993. 22. UserLand Software Inc., UserLand Frontier, UserTalk Reference Guide, Palo Alto CA, 1992. 23. Microsoft Corp., Visual Basic, Visual Basic User Manual, Redmond, WA. 1994. 24. X/Open Company Ltd., U.K., X/Open CAE Specification, Issue 4. July 1992, (3 volume set). 25. X/Open Company Ltd., U.K., Internationalization Guide, Version 2, July 1993. 26. M. Wein, Wm. Cowan, and W.M. Gentleman, “Visual Support for Version Control,” Proceedings of the 1992 ACM/SIGAPP Symposium on Applied Computing (SAC ’92), Kansas City KS, March 1–3 1992, pp. 1217–1223.

9. Trademarks VMS is a trademark of Digital Equipment Corporation. Apple, Macintosh, MPW, OpenDoc, AppleScript are trademarks of Apple Computer, Inc. MS-DOS, Visual Basic, and Windows are trademarks of Microsoft Corporation. UNIX is a trademark of UNIX Systems Laboratories, Inc. OS/2 and SOM are trademarks of International Business Machines Corporation. XVT is a trademark of XVT Software Inc. Galaxy Application Environment is a trademark of Visix Software Inc. CORBA is a trademark of the Object Management Group, Inc. (OMG). UserLand Frontier is a trademark of UserLand Software Inc.

10