Dynamic Traceability Links Supported by a System Architecture Description Published in Proc. Int’l Conference of Software Maintenance, Bari, Italy, Oct.1-3,1997 pp. 180-187. Principal author can be reached at
[email protected] Eirik Tryggeseth and Øystein Nytrø Department of Computer and Information Science Norwegian University of Science and Technology N-7034 Trondheim, Norway Abstract To reduce the effort spent on system comprehension during software maintenance, easy access to different type of information describing the system features is necessary. This is used by the maintainer to build a mental model of the software. Links relating different types of information are often implicit, and called traceability links. Extracting all traceability links of possible interest from the system documentation would be an extremely complex operation. Storing and updating a database of such information manually would be an expensive task. We propose that the identification of information about a feature is done dynamically. The maintainer is provided with a powerful query mechanism for identifying a starting point for collecting information. Further information is identified dynamically by automatically expanding several predefined types of traceability links. This avoids the problems of the traditional database approach. To make the query mechanism scalable, an architectural description of the system is used to limit the size of information which must be inspected by a query. Keywords: Maintenance, documentation, system understanding, traceability, maintenance efficiency.
1 Introduction The complexity of software systems is steadily increasing. The functions they support increase both in number and in complexity, resulting in larger systems to maintain. Investigations report that while the average size of application systems was around 50kLoC in 1980 (Lientz and Swanson [8]), current systems are about 4-6 times larger (Krogstie [4], Nosek and Palvia [10]). Traditional software maintenance processes have often resulted in a degradation of the structure, documentation, and source code of the maintained software system, as described by Parnas [12]. Additional problems of maintainer turnover and recruitment of experienced maintainers have made long term maintenance a problem, e.g. [4] and
[10]. The system becomes difficult to understand, particularly for new maintainers, and hence maintenance costs are increasing. To fight increasing maintenance problems, systems may be re-engineered, but this adds additional costs. Thus, software producers must focus on how to efficiently evolve their systems to be able to stay competitive. Again, literature reports that one of the largest problems in the maintenance task is related to understanding how the system is organized and how it works. Fjelstad and Hamlen [3] presents very interesting results on the distribution of effort in the processing of modification requests: 25% of the effort is related to actually implementing the change, verification accounts for 28%, while understanding the software accounts for 47%. Devanbu et al. [2] reports about a study performed by L. Modica at AT&T which shows that 30%-60% of maintenance cost is due to what Modica calls the ‘discovery task’, similar to the understanding task. Von Mayrhauser and Vans [16] describe a study of how maintainers assemble their system knowledge suggest that they build up an incremental model of causal associations among the system components. The systematic and as-needed strategies for the understanding task described by Littman et al. [9], show two complementary approaches to understanding a small program. The as-needed strategy must be used for large systems, but lacking support to locate only the information needed, hinders effective understanding. Another major problem is lacking or obsolete documentation to support this understanding task. Why is this lacking? We suggest an answer with at least two parts: 1. Documentation has traditionally been a part of the delivery which the developers have signed off for maintenance. The documentation has not been available online, and no good way of relating information in the documentation to particular parts of the code has existed. Hence, documentation has been difficult to maintain. 2. Maintenance managers may have been short-sighted.
How can they defend spending extra effort on updating documentation when customers are pushing the release which was due two months ago? In [15] we presented results from an experiment, which demonstrated that documentation significantly improves maintainers’ ability to understand a system and make efficient changes to it. A reference group without documentation available formed a basis for the statistical analyses. In an experiment debriefing, subjects with documentation available asked for more time or a smaller system to improve their experiment scores. The reference group expected higher score if their C++ expertise had been higher. Both groups had a similar C++ expertise on a test prior to the experiment. This suggest that support for identifying the needed information in the different documents will increase the performance of the maintenance. The rest of this paper is organized as follows: Section 2 presents our approach for dynamically identifying traceability links in a software system. Section 3 gives a short overview of related work, while Section 4 draws some conclusions.
2 Selected Approach 2.1 Motivating discussion Parnas [11] reports about a visit to a major U.S. airport. When being guided around the airport, he asked which specific situations that would trigger a red alarm lamp on the air traffic controllers’ desk. The supervisor told him that all such situations were described in the system documentation. The problem was that the amount of available documentation covered a considerable amount of shelf space. The supervisor said: “Well − you could check the documentation, but it will take you some weeks”. Further investigations showed that changes were not integrated into the existing documentation, rather added as separate reports. The requirements specifying which situations that triggered the alarm were hence divided in a number of small reports − Very inefficient for retrieving the information answering Parnas’ question. Still, several motives to organizing documentation as described can be identified: • The maintainer focus on describing the actual changes made to the system with regard to a particular modification request. This will speed up the process of documenting the changes, and make the effort visible to the client asking for the change. • If maintainers modify the same system components simultaneously, access conflicts may occur when modifying the same documents. If changes are described separately, such situations are avoided. • If the original documentation is only available in
printed form, or unavailable to the maintainer, the only solution may be to produce separate documents. This is better than producing no documentation at all. However, we find this form of system documentation to be very ineffective for system understanding. If only few changes are anticipated during the system life-time, the approach will cause few problems. When thousands of changes are made, the problems may be severe. Our experiment and the above example lead us to the following proposal: During software maintenance, system documentation should be updated rather than added and at any time describes the current state of the software application. When this is the case, the software system is in internal equilibrium. We propose to support system understanding by locating information by utilizing dynamic traceability links. This requires the system to be in internal equilibrium with on-line documentation. Furthermore: • Configuration management is used for all items produced in the development and maintenance processes. • The architectural structure of the software system is made explicit enabling the maintainer obtain a topdown understanding of the system organization. • The maintainer is provided with support to locate particular pieces of information which are needed to understand a specific part of the system. • Dynamic traceability links allow the maintainer to navigate among different types of system components. The approach is described in the rest of the paper.
2.2 Representing traceability links We identify three different approaches to relate components in a software system: 1. Store representation of relationships in a database following a predefined schema. 2. Manually insert explicit relationships in the system documentation and source code as hypertext-like links. 3. Dynamically generate relationships based on a query of certain system features, defined by the maintainer. A thorough discussion of these approaches is given in [14]. Below follows a short rationale, while a summary is given in Table 1. For system understanding, it is important to show different kind of connections among components. If documentation was static, interesting connections could be manually inserted when identified. Software documents change, and a dynamic way of extracting interesting relations is preferable, since this is more cost effective. Relations which describe static connections among source code components can be automatically updated after a change. The relations are defined in [14], but briefly explained later in this paper. To avoid unnecessary
Table 1. Comparison of relation capturing approaches Issue
Predefined
Manually inserted
Dynamic generation
Initial cost
Low
High
Low
Maintenance cost
Low
High and error-prone
Low
Correctness
All specified are captured, but not all that we may need
Rels. may be omitted
All captured, but depending on query detail, also possibly too many?
Storage required
Part of model
Extra storage req.
Extra or no storage
Flexibility
Limited
High
High
parsing, the relationships are extracted and stored in a versioned database at check-in when a source code file is changed. If we select a manual insertion approach, we should verify that inserted relationships are correct when changes are made. If this is not ensured we may have false or dangling relationships. The verification must be done manually, which make the manual insertion technique expensive. We propose that relationships among software document components should be generated dynamically at the user’s request. There is no need for a persistent representation, as this inhibits maintainability and flexibility. Using dynamic generation is subject both to errors of omission and false relations. Such errors do not preclude the usefulness of the approach. What is important is that the number of valuable links found is significantly higher than what we want to maintain manually, and that the noise ratio is low. We show later how to minimize false relations, using a particular naming convention. Relations connecting source code members to appropriate documents are also generated dynamically. The architectural system description is used to constrain the number of files needed to inspect for computing all relevant relationships. This ensures scalability for large systems.
2.3 Describing the system architecture A language for modelling system families and their evolution, the Proteus Configuration Language (PCL), is used to model the system’s logical composition. We model both the structure of the application and the documentation, and the relations among these two structures. This is conceptually shown in Figure 1. The left part of the figure gives a general picture, while the right part visualizes the coarse-grained PCL example given Table 2. We do not attempt to describe the PCL here, but guide the reader to [13] for further details. Tools have been implemented in the Proteus project to manipulate and visualize the PCL models, and regenerate arbitrary system configurations.
Table 2. Coarse-grained PCL example family Calculator attributes terminal : string := “vt100” default; end parts appl => CalcProg; doc => CalcDoc; end end ... family CalcProg parts parser =>Analyser; ui => if terminal = “vt100” then AsciiUI elseif terminal = “win95” then Win95UI endif; math => Arithmetic; end end ... family Win95UI relationships D1 : is_documented_by => (Win95Design); end physical def => “win95.h” attributes repository := “include”; end; impl => “win95.c” attributes repository := “impl”; end; end end
The description of the system structure can be done to an arbitrary level of detail. The PCL is an excellent tool for gaining understanding of a system in a top-down manner. Hence, the model should not be too detailed. The details are handled by the complementary dynamic traceability links, which supports system understanding in a bottom-up manner. Experience shows that he structures should be detailed down to a level where the physical objects of a logical component are naturally related. A rule-of-thumb is to make the description of the logical system mimic the directory structure of the application. Each leaf logical component of the PCL description will then have the leaf directory files as its physical objects. For particularly important subsystems, the inherent structure of the components in the files can also be modelled. The system architect decides the level of detail of each description. An organization may determine a strategy for how their systems should be described using PCL.
Application structure
Calculator
Document structure CalcProg Analyser
AsciiUI
Arithmetic
Several relation categories are of interest in a system comprehension process: • Architectural relations (A-relation): These relations show the architectural composition of the logical system structure. The relations are described in PCL, e.g. parts sections of Calculator and CalcProg entities, as well as the is_documented_by relation in Table 2. Arelations are depicted in Figure 1. • Instance relations (I-relation): I-relations link a logical component in PCL to unique physical components on the disk. These physical components can be text file documents, diagrams, or source code files. These relations are primarily maintained manually, but the PCL provides a mechanism for automating the maintenance. An example is the physical part of the Win95UI entity in Table 2. • Document element relations (DE-relation): A relationship of this type relates a document element in one document to one or several document elements in other documents or in source code. E.g. requirement traceability links are DE-relations. Relationships of this type are generated dynamically. • Document type relations (DT-relation): A relationship of this type relates a document element to other document elements in documents of the same type. Source code files are one particular document type. E.g. call relations in a C program are DT-relations. The DT-relations in proper documents are generated dynamically, while DT-relations among elements in the source code are stored in a database as explained in Section 2.2. When the maintainer has received a modification request, he uses the different relations to locate information necessary to understand the current state of the system and how to comply with the modification request. Table 3 lists the different types of relations that we identified to be of interest to the maintainer, and why the relation was per-
CalcDoc Req’s
Design
Win95UI
Figure 1. Software structures
2.4 Relations identified
PCL Model
Repository (physical objects)
ceived interesting. Three basic requirements must be true for our approach to be successful: • The system must be in internal equilibrium. • The requirements must be uniquely numbered, following a standard numbering scheme. No particular numbering scheme must be used, but an example is: R-, so that R-961029-12 is the twelfth requirement registered on October 29th. All must follow the same format, as this alleviates identification of requirements and references to them. • All names of identifiers in the program must follow a predefined naming format, using natural names for all identifiers, using the same format in source code and documents. In addition to making the source code easier to read, as shown by Laitinen [5], this will make it more manageable to relate information in different sources. A short explanation of the extraction policy for the DE and DT relations is described in the subsections below. A. Document element relations. RD and RT relationships. The start of each requirement is uniquely recognizable due to the numbering scheme. In the design document, the description of the design of a component, at some level in the document’s logical sectioning structure, refers which requirement(s) the component support by listing the requirements’ numbers. If no requirements are listed, the design information contained in the section is defined to support the requirements listed in the containing section. This approach is also used in the test case description. Identification of RD and RT relationships requires that the document format can be searched for the appropriate information. We insert a hypertext mark at both ends of the relationships. The maintainer can select the interesting relationship and navigate to the appropriate sections in the documents. We have demonstrated how this can be done for LaTeX with emacs, and for FrameMaker [14].
Table 3. Relations identified Type
Name
A
Target
Intentional description of relation
Hierarchical PCL entity composition
PCL entity
Describe hierarchical system structure. PCL supports alternatives in the decomposition, to describe variant system structures, see Table 2.
A
Requires
PCL entity
PCL entity
A PCL entity is dependent on other PCL entities.
A
Documents
PCL entity
PCL entity
A PCL entity contains documentation.
I
Instance
PCL entity
File
A PCL entity maps to file(s).
DE
RD
Requirement Section in design doc.
The maintainer may need to know which parts of the system design that supports a requirement. The maintainer may need to know which requirements that put restrictions on a system component described in the design.
DE
RT
Source
Requirement Test case
The maintainer wants to know whether a test has been made to ensure that a requirement has been correctly supported by the system. Given a test case, the maintainer wants access to the requirement which controls the test.
DE
RUD
Requirement
The maintainer needs to know how a requirement is implemented and looks like from a user’s point of view as described in the user manual. Given a feature described in the user documentation, the maintainer Paragraph in the user doc. needs to know what requirements that has lead to this feature.
DE
DUD
Design element
The maintainer is interested to know how the function described in the design is presented for the user as described in the user manual. The maintainer would like to know where in the design the compoParagraph in the user doc. nent that implements a particular feature is located.
DE
DSC
Design element
The maintainer reads the design document and would like to know where the component that he reads about is located in the source code. When working with the source code, the maintainer wants to inspect Source code the design of the component that he is working on. element
DE
SCT
Source code element Test case
Document element
Document element
When a maintainer is inspecting a component in a source code file, he wants to know whether a test case is made for testing this component. When a maintainer is reading a test case in the test report, he wants access to the source code files which contain the functionality which will be tested by the test case. The maintainer is also interested in locating the correct test drivers and stubs for the test that is described in the test case.
DT
Document
Used to generate dynamic cross references among document elements in the same type of documents.
DT
Source code Source code Source code This is the standard collection of relations among source code elements, element element like calls, uses, includes, etc.
RUD and DUD relationships. When the maintainer wants to know how the functionality specified in a requirement is implemented application, he specifies a query using the key terms used for specifying the requirement. These terms are then searched for in the user manual. The matches approximately describe functionality specified in the requirement. To find out which requirements that constrain some functionality in the application, terms used in the user documentation to specify that functionality are selected. These terms are used in a search in the requirements specification. The corresponding matches are requirements which have impact on the functionality.
A corresponding procedure is used to identify DUD relationships. Note that the matches returned by the searches are not exact. The RD relationships described before were exact matches, as they were explicitly defined in the documentation. The relationships generated using the search mechanism are only approximate to the correct relationships. The correctness of the extracted relationships are dependent on the choice of terms used in the query. The maintainer needs to inspect the relationships to investigate how well the query matched his intentions.
DSC relationships. These relationships involves linking the information in the documentation directly to source code elements. We are able to relate these different forms of information by requiring that a predefined naming format is used for naming identifiers in the system. Since programming languages have scope rules, the same name may be used for several components; e.g. the same name may be used for member functions in different classes. To solve this problem, the description of a member function in the design must uniquely identify which class the member function belongs to. We had the opportunity to test one strategy in a project course at our department, where the students were asked to develop a system using C++. A similar solution may be used for other modular languages. In the design document, the description of the different entities had to follow this particular scheme: • The name of the components documented in a (sub)section must be part of the (sub)section name, following this rule: :. • The legal type names which can be used in (sub)section names at all levels are: Class: the same as a C++ class. Module: a set of closely cooperating classes. GlobalFunction: global function in C++. GlobalData: variable defined outside a class in C++. • Type names that can only be used in subsections are: StaticFunction in Module chapter (local static funct). MemberFunction in Class chapter (class member funct) MemberData in Class chapter (data member in class). StaticData in Module chapter (local static variable). The student groups did not have any problems in following this scheme. They found it a bit strange at first, but when they were told that the reason behind it was that a tool sometime in the future would automatically extract information from their design document, they agreed to that the scheme was OK to use. We use the information in the PCL description (the hierarchical, documents, and instance) relations to minimize the number of files to search to establish the needed DSC relationships.
defined for the most important relationships. A C++ parser (CPPP from Brown University) generates an abstract syntax tree, from which relationships are identified and exported to a database. It is an expensive operation to produce the full database of DT-relations for a considerably sized software system. Due to this we do not extract the DT-relations for source code elements dynamically. We update the database incrementally when changes are made to the source code. To avoid this expensive operation when an old configuration is checked out of the repository, a full save of the database is stored together with any named configuration, i.e. release provided to a customer.
2.5 Query mechanism and supporting thesaurus The query mechanism. To locate information to support the maintainer in the understanding task, we propose a query mechanism. The queries are used to locate a starting point for further information retrieval, or to dynamically generate relationships (RUD, DUD, and Document). We propose a query mechanism with three aspects: 1. Search for exact phrases. 2. Search for a set of particular terms in a phrase of text of a defined window length. 3. Same as #2, but allow expansion of terms used based on synonym relations in the thesaurus. For queries with terms, there is an implicit AND between the terms, term ordering is not important, and white space is not counted. All queries are context insensitive. Examples of queries are given in Table 4. Table 4. Examples of queries Type 1
(“calculate nutrition intake”)
2
(production manager policy),20
3
(+production +organization),50
Combination SCT relationships. These are identified and extracted in much the same manner as RD relationships, as we specify that a test case should reference what source code element which is tested. The maintainer can now construct queries to obtain the needed relationships. The interested reader should consult [14] for details. B. Document type relations. Document type relations are cross references dynamically generated using queries over a particular document type, or extracted from source code based on a data model for the particular language. For cross references in source code, a data model is
Query
Using OR
(“software production” organization),5 (project (manager OR leader)), 2
Note that an plus sign (‘+’) prior to a term indicates that term expansion is used on that term. The user is allowed to compose queries incorporating all three aspects, as shown in the fourth line of Table 4. An exact phrase cannot be expanded, and the window size for an exact phrase is 1. Finally, we add the possibility of combining queries using the OR operator. The return value of a query is a reference to the section header for the section yielding the match. For example, the return value to the query (“return value” +query),30 would be “7.12 The query mechanism”.
assemble
0.2
structure
0.1
0.3 0.2 0.4 arrangement organization Figure 2. Small thesaurus example The thesaurus. The system thesaurus acts like an index to important terms used in the system. We propose to store terms which describe the key properties (i.e. nouns) of the system, or some key behaviour (i.e. verbs) of the system. The system thesaurus should be initiated at the beginning of the project and evolve with the system through development and maintenance. A particular system property or behaviour can be referred to differently by different maintainers, or in different contexts. The thesaurus therefore cannot be merely a list of terms used in the system. Synonym relations must exist among terms to express that the terms are similar. The synonym relation is bidirectional and there is an associated weight on the relation. The weight is on the relation is used for expressing how distant two terms are. The weight value is from 0 to 1, where 0 means identical and 1 means very distant. The user choose this weight subjectively from experience. For example, term W can be expressed similarly using term X, Y or Z. The weights on the synonym relationships from W to X, Y, and Z would determine which of X, Y, and Z that were closest to W. Figure 2 gives an example of a small thesaurus with synonym relations and weights. It is better to insert too many than too few terms into the thesaurus. In this way, the query mechanism described earlier can be better exploited.
2.6 An example of DSC generation In Figure 3, a maintainer has searched for the algorithm for calculating loan interest rates, e.g. (interest rate +calculate),10. The search result was the gray-colour section in file PersFinances.doc. This file is a physical object of a PCL entity in the documentation, the LoanDesign, which documents the LoanModule in the application part of the system PCL description. The number of source files
to investigate to locate the other end of the DSC relationship are the source files described as PCL physical objects in the structure below LoanModule. These are shown in the lower right part of Figure 3. The identification of the complete relationship is shown by the arrow pointing into loan_account.C. This illustrates how the architectural description support up-scaling the approach.
3 Related work The DOCKET project, described by Layzell et al. [6]/ [7], proposes to utilize multiple knowledge sources to build a static model of how important system concepts are related. The system information at different abstraction levels are integrated into a global system model which describes how components in the different knowledge sources relate to each other. This integration process is not fully automated and requires human interaction. The text processing mechanism locates concepts and events in the text. These are located by pattern matching using certain heuristics, such as “important concepts are emphasized in the text” and “events are often introduced by if or when”. The DOCKET approach is lacking under its attempt to be general. No restrictions are made to the organization of the system documentation. While this allows the DOCKET approach to be used directly on a number of existing systems, the amount of useful links among different type of information is restricted. In the LaSSIE system [2], Devanbu et al. propose to hard-code a knowledge base with information about the source code. Users can query the knowledge base. A retrieval algorithm is then used to provide answers to questions formed in natural language, such as “what global variables are accessed by a function that flashes a display lamp at an attendant’s console?”. As knowledge is explicitly represented in the knowledge base, the acquisition and maintenance of the knowledge base becomes tedious and expensive. While this approach makes LaSSIE capable of answering very detailed questions about the source code, there is no information available to answer questions like “why does the lamp flash”.
PersFinances.doc
PCL description
... 6.0 Module:Loan ... 6.3 Class:LoanAccount ... 6.3.2MemberFunction:CalculateInterest
doc
loan_account.C
app
Figure 3. Example of information location, a DSC relationship
Both the DOCKET and the LaSSIE approaches will experience the problems of keeping knowledge base information synchronized with the rest of the software system. Boldyreff et al. [1] describe the approach taken to application program understanding in the AMES project. The AMES approach links related elements in the documentation by inserting links between two document elements when they contain similar words, phrases, or a colocation of words. They also relate requirements documents to design documents using this approach, and finally design to source code. The words, phrases and colocations leading to a link is decided after an analysis of the vocabulary of the documentation. The document elements and source code elements are listed so that the user can point-and-click navigate among them. All of these approaches maintain a static knowledge base of the information available for the maintainer to use during the understanding task. This means that a lot of processing must be done to create the knowledge base. None of them describe how they update the knowledge base during system evolution, and how much effort that is required for this. Neither are any of them taking into consideration the potential problems of versioning the knowledge base, when for example maintenance is done concurrently on several parallel variants of the system.
4 Conclusion To efficiently perform the understanding task in the software maintenance process, it is important to be able to locate information when needed, and to navigate among related pieces of information. Persistent representation of relationships which allow the maintainer to locate and access information is expensive to build and evolve, as well as restrictive in use. We proposed that such traceability relationships should be generated dynamically when needed. We presented a set of relations which we identified as necessary to aid the maintainer in the system understanding task. Most of these relations are extracted dynamically during the system understanding task. We gave a brief description of how this extraction was done. A analyser for PCL is implemented together with an accompanying tool set providing graphical manipulation of the PCL model. An extractor for C++ has been implemented, and the DT relations for C++ is inserted into a database which can be browsed using a graphical browser. Test extractors have been implemented to be able to demonstrate the feasibility of the different extraction policies explained. Further work includes working towards an integrated prototype which incorporates the full approach presented in this paper, and to experimentally verify the usefulness of the approach.
Acknowledgements. The PCL team members in the Proteus project are acknowledged for making PCL a success. Suggestions from the anonymous referees were appreciated for improving the state of the final paper. References. 1. C. Boldyreff,E.L. Burd, R.M. Hather, R.E. Mortimer, M. Munro, and E.J. Younger: The AMES Approach to Application Understanding: A Case Study. In (Submitted to) International Conference on Software Maintenance 1995. 2. P. Devanbu, R.J. Brachman, P.G. Selfridge, and B.W. Ballard. LaSSIE: A Knowledge-Based Software Information System. Communications of the ACM, 34(5):34–49, 1991. 3. R.K. Fjelstad and W.T. Hamlen. Application Program Maintenance Study - Report to Our Respondents. Technical report, IBM Corporation, DP Marketing Group. Also in Proceedings of GUIDE 48, The Guide Corporation, Philadelphia, 1979. 4. J. Krogstie. Survey Investigation: Development and Maintenance of Information Systems: Version 1. Technical report, Norwegian Institute of Technology, Department of Computer Science and Telematics, Trondheim, 1994. 5. K. Laitinen. Natural Naming in Software Development and Maintenance. PhD thesis, Univ. Oulo, Finland, October 1995. 6. P.J. Layzell, R. Champion and M.J. Freeman. DOCKET: Program Comprehension In-the-Large. In 2nd Workshop on Program Comprehension, July 1993, Capri, Italy, pp. 140–148. 7. P.J. Layzell, M.J. Freeman and P. Benedusi. Improving Reverse-engineering through the Use of Multiple Knowledge Sources. Journal of Software Maintenance: Research and Practice, 7:279–299, 1995. 8. B.P. Lientz and E.B. Swanson. Software Maintenance Management. Addison-Wesley, Reading MA, 1980. 9. D.C. Littman, J. Pinto, S. Letovsky and E. Soloway. Mental Models and Software Maintenance. In Journal of Systems and Software, 7:341–354, 1987. 10.T. Nosek and P. Palvia. Software Maintenance Management: Changes in the Last Decade. In Journal of Software Maintenance: Research and Practice, Vol. 2, pp 157–174, 1990. 11.D. L. Parnas. Studying Design Practice - Empirical Findings and Future Challenges. One-day seminar at the Dept. Of Informatics, University of Trondheim, October 29, 1993. 12.D.L. Parnas. Software Aging. In Proc. 16th Int’l Conference on Software Engineering, Sorrento, Italy, pp. 279–287, 1994. 13.E. Tryggeseth, B. Gulla and R. Conradi. Modelling Systems with Variability using the PROTEUS Configuration Language. In Proc. of 5th Int’l Workshop on Software Configuration Management, Lecture Notes in Computer Science, no. 1005, Springer Verlag, pp. 216–240, 1995. 14.Tryggeseth, E. Support for Understanding in Software Maintenance. PhD thesis, Department of Computer and Information Science, Norwegian University of Science and Technology, 339 pages, March 1997. 15.E. Tryggeseth. Report from an Experiment: Impact of Documentation on Maintenance. In Journal of Empirical Software Engineering, vol. 2(2):199-205, Kluwer Publishing, 1997. Also in First Int’l Workshop on Empirical Studies of Software Maintenance, Monterey, CA, Nov. 8, 1996. 16.A. Von Mayrhauser and A.M. Vans. Comprehension Processes During Large Scale Maintenance. In Proc. 16th Int’l Conf. on Software Engineering, Sorrento, Italy, pp. 39–48, 1994.
