Testing Adaptive Software During Class Evolution

Testing Adaptive Software During Class Evolution Linda Seiter Keszenheimer and Karl J. Lieberherr Northeastern University, College of Computer Science Cullinane Hall, Boston MA 02115 Phone: 617-373-5204 Fax: 617-373-5121 fseiter,[email protected] February 27, 1995 Abstract

Class evolution has signi cant impact on program maintenance and testing. Changes in class structure require maintenance of existing class methods, which must then be retested. Existing test cases can not simply be rerun, since the test cases may no longer be consistent with the class structure. The bene t of reuse is reduced if generic software requires testing to start from scratch each time a component is reused. Adaptive software components are designed to be exible to changes in class structure, allowing reuse with many class designs. This paper presents an incremental approach to testing adaptive software during class evolution. Paper Category: Research Topic Area: Object Testing and Metrics, Templates and Generic Programming

1 Introduction Object-oriented software engineering is being increasingly used for system development, primarily due to advantages gained from the production of reusable components. While reuse is an assumed bene t, it is vital that a component be well tested before being reused. A well tested component will require some amount of testing in its new environment due to potential speci cation-based errors [7, 26], however the testing eort must be minimized or the bene t of reuse is reduced. The development of adaptive component-based software is becoming an increasingly popular means of software engineering [27, 28]. This growing methodology must address the issue of adequate testing during component reuse. Class evolution has signi cant impact on the maintenance and testing of an application. While class evolution has been well researched, the focus has been on maintaining the structural consistency of objects in a database[1, 5]. Adaptive Software as de ned by Lieberherr, Palsberg and Xiao [20, 27] refers to a model of software development in which program components are designed to be exible to changes in class structure. Behavior is implemented based on a set of high-level constraints. The class constraints de ne a language of compatible customizers , which are speci c class designs. An adaptive component can be used by an in nite number of customizers, thus allowing algorithmic reuse. The adaptive software model has been implemented in Demeter System/C++, a CASE tool for developing object-oriented applications. 1

Each time an adaptive component is customized by a speci c class structure, the question of how to test the customization arises. An error may exist when the adaptive component is used by one customizer that does not exist with another customizer, due to control ow dierences caused by class structure. This paper presents a model for incrementally testing adaptive software, focusing on the process of testing a sequence of evolving class structures. A process for regression testing during class evolution is described based on program-based or white-box testing, in which test cases are developed to test the actual implementation of a program rather than its intended functionality. This can be applied to testing other generic software components during class evolution, such as template classes and patterns. Speci cation-based testing is based on testing the intended functionality regardless of the actual implementation, and is not addressed in this paper. This paper is organized as follows. Section 2 provides an introduction to adaptive software, describing a model for class structure and behavior. Section 3 describes a model for testing adaptive software. Section 4 describes the process of regression testing during class evolution. Section 5 discusses related work in objectoriented testing. Section 6 concludes with future research.

2 Adaptive Software Object-oriented programs are developed based on a speci c design of class structure. A programmer de nes the structure of a class by specifying its attributes and relations to other classes. Behavior is implemented by attaching functions or methods directly to classes. It is often necessary to have several classes collaborate on a task. This is implemented through message passing, where a message is sent along a speci c relation between two classes. The class design becomes hard-wired into the methods, using relations between classes to implement collaboration. When the class structure must change, the maintainer must modify existing methods to refer to the new structure.

2.1 Describing Class Structure While there are many representations for describing class graphs, the material presented in this paper is based on the Demeter data model [12]. A class graph is a nite, directed, labeled graph that describes a speci c class design. Vertices represent concrete and abstract classes, while edges represent subclass and reference relations. Concrete classes are instantiable while abstract classes are not. A subclass edge is used to represent a generalization/specialization relation between an abstract superclass and a subclass. A construction edge is used to indicate a reference relation, where the source class of the edge has a reference to the target class, with the edge label indicating the reference name. For purposes of code generation, edges are assumed to be ordered as well. AppExp rator rand body

LitExp

Number

Exp

ProcExp

val

val formal

Ident

id

Variable

var

AssignExp

Figure 1: Example class structure for Scheme program expressions. Figure 1 shows an example class graph containing a design for simple Scheme program expressions. An expression is either a literal, variable, assignment, procedure, or application. Class Exp is an abstract class, 2

drawn as , for describing generic Scheme expressions. Class Exp has subclass edges, which are drawn as =), to the concrete classes LitExp , Variable , AssignExp , ProcExp and AppExp . Concrete classes are drawn as . Construction edges describe reference relations between classes and are labeled. For example, a literal expression has a value that is a number, which is represented with the edge labeled val between LitExp and Number . The construction edge is drawn as LitExp val ! Number. A speci c ordering of the edges is assumed. A class graph is at if every abstract node has only subclass edges emanating from it. The class graph in gure 1 is at since Exp does not have outgoing construction edges for its subclasses to inherit. Given a class graph, it is straightforward to generate an equivalent at one using object-preserving class transformations [3]. During attening, construction edges are pushed down the inheritance hierarchy to concrete classes.

2.2 Describing Class Behavior void ProcExp::findFreeVars(VariableList* boundVars) {

virtual void Exp::findFreeVars(VariableList * boundVars) { // virtual member function }

boundVars−>push(formal); body−>findFreeVars(boundVars);

void AssignExp::findFreeVars(VariableList* boundVars) { val−>findFreeVars(boundVars); var−>findFreeVars(boundVars); } void AppExp::findFreeVars(VariableList* boundVars) { rator−>findFreeVars(boundVars); rand−>findFreeVars(boundVars); }

boundVars−>pop(); }

void Variable::findFreeVars(VariableList* boundVars) {

if (!boundVars−>contains(this)) this−>g_print(); }

Figure 2: Example C++ Program. In an object-oriented program, functionality is traditionally implemented by attaching behavior to speci c classes, in the form of methods or functions. As an example, assume it is desirable to print out the free variables within a Scheme program. This requires recursively traversing the expressions contained in a program, printing each variable encountered during the traversal that is not bound within a procedure scope. If a procedural expression is encountered during traversal, its formal parameter is pushed onto a list of bound variables. Variables encountered in the procedure that are not contained in the bound variable list are free and therefore printed. The formal parameter variable is popped o the bound variable list when traversal of the procedural expression is complete. Figure 2 shows example C++ code to implement this task, based on the class structure of gure 1. AppExp rator rand body

Exp

ProcExp

val

Traversal graph for

formal

Variable

var

AssignExp

[Exp, Variable]

: source vertex (where traversal starts) : target vertex (where traversal ends)

Figure 3: Traversal Graph The program consists of several methods or member functions, all having the same signature. The functionality implemented by the methods consists primarily of traversal-oriented behavior. Figure 3 shows the subgraph of the class graph used for traversal, which contains all paths from Exp to Variable . Aside from the 3

traversal-oriented behavior, there is a small amount of processing to push and pop bound variables from a list, and check if a variable is contained in the list. This is shown in gure 2 in bold type. The speci c structure of the class design is clearly hard-coded into the class methods, since edge labels depicting relation names are used for message passing. Changes to the structure of the class graph will require maintenance of the C++ program. 1 operation void findFreeVars(VariableList* boundVars ) 2 traverse 3 [Exp; Variable] 4 wrapper ProcExp 5 pre x 6 { boundVars->push(formal); } 7 sux 8 { boundVars->pop(); } 9 wrapper Variable 10 pre x 11 { if (!boundVars->contains(this)) this ->g_print();

}

Figure 4: Adaptive program Adaptive Software alleviates many of the maintenance problems that arise during class evolution [9, 11, 27] by supporting a model for describing class behavior while placing minimal assumptions on class structure. An adaptive program consists of a set of Propagation Patterns [20, 27], where each propagation pattern de nes the implementation of a speci c task. A propagation pattern may consist of a signature , traversal speci cation , transportation speci cations and code wrapper speci cations . A traversal speci cation succinctly describes a set of paths to be traversed during execution of the task. The set of paths is described in a traversal graph . As it is sometimes necessary for one object to send information to another during collaboration, a transportation speci cation describes the transportation of information between objects which must occur during collaboration. Code wrapper speci cations describe task-speci c behavior aside from traversal and transportation. The adaptive program to implement the same behavior is shown in gure 4. It consists of a single propagation pattern . The signature describes the name of the operation, ndFreeVars , its return type, void and formal argument, V ariableList  boundV ars. The traversal speci cation [Exp; V ariable] describes the traversal graph, which is a set of paths from the Exp class to the Variable class. Applying the traversal speci cation to the Scheme class graph produces the traversal graph in gure 3. While this traversal speci cation simply consists of a source and target, more complex speci cations can be created using union and concatenation operators. The propagation pattern also de nes several wrapper speci cations de ning behavior to be performed in addition to traversal. The pre x wrapper for class ProcExp is executed each time a ProcExp object is traversed, before any outgoing edges for the ProcExp object are traversed. The sux wrapper for ProcExp is executed after outgoing edges are traversed. Finally, the pre x wrapper for Variable is executed each time a Variable class instance is traversed. The generated C++ code for the propagation pattern is similar to that previously shown in gure 2. This propagation pattern can be customized by any class graph having at least one path from the Exp class to the Variable class. There are an in nite number of class structures that meet this constraint. For each class in the traversal graph, a method is generated with the signature speci ed in the propagation pattern. Class graphs are attened during code generation, therefore pre x and sux wrapper speci cations attached to abstract classes will be pushed down to their concrete subclasses, providing incremental inheritance of behavior rather than overriding inheritance. While overriding inheritance is possible, this paper does not present it. Abstract classes along the traversal graph will have an empty virtual method generated, since their pre x and sux code wrappers have been pushed down to concrete subclasses. This handles the potential issue where a class such as LitExp , which is not in the traversal graph, is sent a message at run-time. For each concrete class C in the traversal graph, a method is generated to provide the following execution behavior:

4

1. The method will rst execute the pre x wrappers de ned for class C's superclasses, in order from the top of the hierarchy down. In the case of multiple inheritance, the ordering of direct superclasses is used to determine which inheritance path to follow rst. 2. The pre x wrapper speci ed for class C will be executed. 3. Each outgoing construction edge contained in the traversal graph for class C will be traversed in order. 4. The sux wrapper for class C is executed. 5. The sux wrappers de ned for the superclasses of C will be executed in the reverse order of the inherited pre x wrappers.

3 Testing Adaptive Software Many errors in procedural programs may occur in object-oriented programs as well, such as control errors and data anomalies. However, object-oriented programs introduce certain issues not found in procedural programs, and testing methodologies must be adapted to account for them. Inheritance and encapsulation introduce complexity into the testing process [21]. Dynamic binding introduces complexity into control ow and data ow algorithms. The hierarchical nature of object structure requires modi cations to traditional data ow algorithms [25]. The process of testing object-oriented software diers signi cantly from procedural software [7, 21, 24]. Object-oriented unit testing is based on testing a single class, which involves unit testing each method of the class as well as integration testing the interacting methods of the class. Object-oriented integration testing involves testing the method interactions between several classes. The process of testing adaptive software diers from that of object-oriented software. Unit testing involves testing a single propagation pattern. This requires unit testing the individual methods for each class contained in the traversal graph, as well as integration testing the interactions and dependencies between methods of dierent classes. Integration testing refers to testing the dependencies between multiple propagation patterns, and will not be covered in this paper. The goal of unit testing is to discover errors in the propagation pattern based on its customization with a class graph. While some amount of static analysis is possible before a propagation pattern is customized, many errors can only be found after customization. While exhaustive testing is typically not feasible, the level of con dence in the correctness of a customized propagation pattern can be boosted by developing a suite of test cases that cover many of the paths in the code. Adequate program-based testing of a propagation pattern will be measured using certain coverage criteria.

3.1 Procedural Coverage Criteria Coverage criteria for traditional procedural programs include statement coverage , decision coverage , condition coverage , decision/condition coverage , and multiple condition coverage [18]. Adequate testing based on

coverage criteria implies having enough test cases in a test suite so that every program statement is executed at least once (statement coverage), every predicate has a true and a false outcome at least once (decision coverage), every sub-expression within a predicate expression has a true and a false outcome at least once (condition coverage), and various combinations of statement, decision and condition coverage. While coverage criteria are useful in showing how well the control ow and dependence in a program is tested, many errors are based on data anomalies. Data ow testing involves exercising paths between the statements that de ne a value for a variable and those that access the value of the variable. A variable de ne occurs when a variable is declared or assigned a value. A variable use can either be a p-use indicating use in a predicate, or a c-use indicating use in a calculation. There are several variations of data ow testing based on coverage criteria: all-c-uses , all-p-uses , all-uses and all-du-uses [22].

5

3.2 Adaptive Software Coverage Criteria When a propagation pattern is customized with a speci c class graph, a Traversal Dependence Graph is computed to show inter-method control ow and control dependence [10]. This graph is an object-oriented version of the procedural program dependence graph [6]. It is produced by attaching transportation and code wrapper speci cations to the traversal graph, and then applying graph attening as described in the algorithm for the generation of methods along the traversal graph. Test case adequacy is based on covering paths in a traversal dependence graph. While a traversal dependence graph can describe data ow and data dependence, this paper discusses only control ow and dependence. Additionally, the control and dependence of individual statements contained in a single code wrapper are grouped into one node. The traversal dependence graph for the Scheme expression customizer is shown in gure 5. Control dependence is depicted by the existence of an edge between nodes, while control ow is depicted in the ordering of edges which is shown by edge numbering. Polymorphic control ow based on dynamic binding is depicted through the use of abstract edges (=)). AppExp 1

rator

rand

2 body

Exp

LitExp

3

ProcExp

1

prefix

4

suffix

val

2

prefix

1

Variable

var

2 formal

AssignExp

1

Figure 5: Traversal Dependence Graph. Unit testing a customized propagation pattern requires the development and execution of a suite of test cases fT1 ; T2; ::; Tng that adequately cover paths in the traversal dependence graph. Each test case contains information relevant for documenting an individual test. Berard [2] provides a class design for test cases that includes attributes for object id, test case id, description of purpose, and a list of test case steps including a list of states for the object being tested, list of messages to be sent to the objects, list of exceptions, and other information relevant to a test. Each test case Ti at a minimum should consist of an input object state, expected output object state, expected output, actual output, and traversal history. The traversal history describes the sequence of messages and code wrappers executed for a given object. Figure 6 shows some partial test cases for testing the Scheme expression traversal dependence graph. Object state is described in an object graph. Each node represents an object, with the object id (oid) and class given as the node label. Edges in the graph represent relations between objects. The traversal history portion of a test case describes the paths in the traversal dependence graph covered by the test. Coverage criteria will use this traversal history to determine test case adequacy. 3.2.1 Traversal dependence graph Coverage

Since adaptive programs are structured dierently than procedural programs, coverage criteria must be modi ed to test the dierence. The possible behavior that could result due to dynamic binding and polymorphism must be tested. There are several coverage criteria to be applied to testing adaptive software, in addition to the standard procedural coverage criteria.

 Minimal Customizer Coverage : The test set instantiates every concrete class in the traversal dependence

graph at least once, and the traversals of the test objects, mapped back to the traversal dependence graph, cover each construction and subclass edge at least once. If there exist abstract classes in the traversal dependence graph, the test set must cover a combination of polymorphic substitutions for 6

Test Case #

Traversal History

Input, State id

i1

Variable

i2

prefix

#1 Variable

#2

id

var

i5

AssignExp

1

Ident

AssignExp

id

i1

i2

val 1

Ident

Variable val

var

id

i3

prefix 2

3

Variable

i4

Variable

Variable

prefix

Ident

4

Figure 6: Example test cases.

id

i1

Ident

Variable

val

i3

i2

Number

LitExp

val

i3

val

i4

Number

LitExp

i5

formal

i2

Variable

Number rator

i2

Variable

Ident

i4

Number

LitExp

AppExp

id

i1

val

i3

rand i7

ProcExp

id

i1

i4

LitExp

i6

var

AssignExp

val

i3

body

i4

Ident

id

i1

i2

Variable

Ident

Minimal Customizer Coverage i3

val

val

i5

i8 id

i1

i3

val

i6 formal

var

AssignExp

i1

Variable

i1

Variable

Number

i1

i11

i2

i1

Variable

val

i1

i4

Number

rand val

id i2

Ident

i3

i13

var

val i4

Number

LitExp

O7

AppExp rator

AssignExp

i1

Variable

Variable

id i2

Ident

id i2

Ident

i1

Variable

id i2

Ident

Level−1 Recursion + Polymorphic Coverage for AssignExp

Figure 7: Example Test Case Object Structure

7

i4

Number

var

AssignExp var

AssignExp

Ident

val

LitExp

i5

val

id

Variable

LitExp

ProcExp

Variable

i3

val

var

Ident

body

i10

AssignExp

i2

Variable

i9

val

var

AssignExp

si12

val

i4

Number

LitExp

id i2

Ident

id i2

Ident

each abstract class. For example, the AppExp class has two construction edges, AppExp rator ! Exp and rand AppExp ! Exp. The Exp class has ve concrete subclasses: LitExp , Variable , AssignExp , ProcExp and AppExp , therefore there are ve possible polymorphic substitutions for each of the edge targets. While LitExp is not in the traversal graph, it is contained in the traversal dependence graph as having no traversal or code wrapper behavior, as it must be tested since it is a valid polymorphic substitution of the Exp class. Total polymorphic coverage of the two construction edges would require 5  5 = 25 combinations. Minimal customizer coverage would select a set of the possible combinations. While not every possible combination of polymorphic substitution for the parts of a class is tested, a representative group is covered such that every concrete class is instantiated in some test case, and every edge is covered in a traversal history. Orthogonal latin squares can be used to avoid ad-hoc test case development [14, 15]. An example set of test objects for minimal customizer coverage of the Scheme expression class graph is shown in gure 7. Every concrete class in the traversal dependence graph is covered, as is every subclass and construction edge. This is a very weak type of coverage, as many errors will only be discovered given certain combinations of polymorphic substitutions.

 Total Polymorphic Coverage : The test set covers all possible polymorphic substitutions that can occur

when an edge having an abstract class as its target is traversed. For instance, testing the traversal of the construction edge AssignExp val ! Exp must cover the possible concrete class instances that are polymorphic substitutions of the Exp class. A test set for the construction edge AssignExp val ! Exp is shown in the lower half of gure 7.

 Level-K Recursion Coverage: Given a recursive class structure, the test set covers K levels of recursion

in object structure. Some errors may only be discovered given a certain level of recursion. The Scheme expression class structure is recursive, since subclasses of Exp have reference relations with Exp as target. Exhaustive testing of all possible paths through this class structure is not possible, due to its recursive nature. Level-K recursion coverage refers to the level of recursion of objects within test sets. The constant K is determined by the propagation pattern and customizer. Figure 7 shows level-1 coverage combined with polymorphic coverage of test cases for the AssignExp class.

 Exhaustive Traversal Coverage : The test set covers all possible paths through the traversal dependence

graph. Testing of all possible run-time paths through the Scheme expression class structure is not possible due to its recursive design.

The tester must apply coverage criteria to testing the possible combinations of polymorphic substitutions of the sending, receiving, parameter, and return objects for each edge traversed in the graph[10, 15, 19].

4 Testing Adaptive Software During Class Evolution The evolution of class structure is an unavoidable part of the object-oriented software development process. Evolution may occur due to an improved understanding of the application domain, due to changes in the application domain, or due to class optimization. Changes to a class graph may introduce errors in a customized propagation pattern by causing dierent traversal behavior. Regression testing is conducted to determine if a structural modi cation introduces an error. There are several considerations to be taken into account when regression testing adaptive software: 1) It is important to reduce the amount of eort required to perform regression testing, therefore it will be useful to rerun only those tests which cover code aected by the changes in class structure. 2) Existing test cases may require maintenance of object structure in order to be consistent with the modi ed class graph. 3) Additional tests may be required to cover new paths introduced by the change. 4) Some tests may no longer be valid if they cover portions of the class graph that have been removed. Class evolution occurs when a change is made to the structure of the class graph, for example the addition, deletion, or modi cation of a class or edge. This paper concentrates on two categories of class modi cation, 8

namely object-preserving class transformations [3] and object-extending class transformations [13]. Objectpreserving transformations involve primitive operations that modify the structure of the class graph, yet maintain the same set of objects that can be instantiated from the graph. An example would be class attening, where the attributes of a superclass are pushed down to its subclasses. Object-extending class transformations are those which extend the set of objects or extend the structure of existing objects that can be instantiated by the class graph. For example, addition of a new class to the class graph extends the set of objects the graph can instantiate. If unit testing has been performed on propagation pattern P customized by class graph G, using test suite T, then given a set of primitive modi cations to G that produces class graph G , the process for regression testing will be based on the following steps: 1. Compute the traversal dependence graph for class graph G . 0

0

2. Identify a set of tests T  T to be rerun based on dierences in the traversal dependence graphs. Test cases are selected whose traversal histories cover a part of the traversal dependence graph that has been changed. 3. Flag invalid tests in T. 0

4. If necessary, update object structure in each test case ti 2 T to be consistent with the modi ed class graph G . 5. Run the updated tests in T . 0

0

0

6. Add additional tests to T based on coverage of new paths in the traversal dependence graph. 7. Create T , the new test suite for the propagation pattern P given the current customizer G . In the following sections, the impact of primitive class transformations on the regression testing process is analyzed. Previous studies have identi ed the maintenance eort required to maintain object-oriented programs during class evolution versus that of maintaining propagation patterns [4, 9, 11]. 0

00

0

4.1 Object-Preserving Class Transformations The object-preserving class transformations proposed by Bergstein [3] are based on maintaining the structural consistency of objects in a database while optimizing the class design. Two class graphs are objectequivalent if they de ne the same set of objects. The object-preserving transformations did not take into account the presence of behavior for classes, nor were they based on class graphs in which edges are ordered. The primitives assume all superclasses are abstract and use the term alternation class rather than abstract class . All object-preserving class transformations can be composed of ve primitive operations: 1. Addition of useless alternation : An alternation (abstract) class is considered \useless" if it has no incoming edges and no outgoing construction edges. This primitive describes the addition of an abstract class v with subclass edges to a set of existing classes fv1; v2; ::; vng. Since v has no incoming edges, it is not a part of any class nor does it inherit parts from any class. Also, it does not provide any construction edges for its subclasses to inherit. 2. Deletion of useless alternation : This is the inverse of the previous primitive, where an abstract class with no incoming edges and no outgoing construction edges is deleted. 3. Abstraction of common parts : If all of the immediate subclasses of an abstract class v have an outgoing construction edge with the same label l and target class w, then the construction edges can be removed and a single construction edge v !l w added in their place. This primitive describes the abstraction of a common reference relation up the inheritance hierarchy. 9

4. Distribution of common parts : This is the inverse of the previous primitive, where an inherited part is distributed down the inheritance hierarchy. This primitive describes graph attening, where construction edges are pushed down to concrete classes. 5. Part replacement : If two abstract classes v and v have the same set of concrete classes as subclasses, then any construction edge w !l v can be replaced by w !l v . Intuitively, if two abstract classes v and v have the same set of instantiable subclasses, then the set of de nable objects does not change when a construction edge has v or v as its target class. 0

0

0

0

Examples of the primitives are shown in gure 8, depicting a sequence of incremental transformations being applied to a class graph to optimize its structure. Object-preserving class transformations will not require maintenance to object structure in test cases, since the class graphs are object-equivalent. Drawing shape

shape

shape

shape

Figure

shape

Drawing

Drawing

Drawing

Drawing

Figure

Figure

Figure Circle

Square width

Circle

Square width

Point

Circle

Square

Circle

Point

Circle

Square

Point

Point

Square

width

Point Number

width

width

width

Number

Number

width

2DShape Shape

2DShape Shape 2. Addition of Useless Alternation (applied twice)

1. Original Class Graph

3. Abstraction of Common Parts

Number

Number

2DShape Shape

2DShape Shape

5. Deletion of Useless Alternation

4. Part Replacement

Figure 8: Object-Preserving Class Transformations

4.2 Object-Extending Class Transformations The object-extending class transformations proposed by Lieberherr, Hursch and Xiao [13] are based on extending the structure of objects in a database. Object-extension is required for a database when existing objects need additional structure or when completely new object structure must be added to the database. A class graph G is an extension of another graph G when (1) the set of concrete classes for G is a superset of the set for G, (2) the inherited and direct construction edges for each class in G are a superset of those for the corresponding class in G, and (3) the target class of each construction edge in class G is a superclass or the same class as that of the target of the edge in G. The object-extending class transformations can be composed of three primitive operations: 0

0

0

0

1. Class addition : New classes and edges are added to the existing graph, however, existing classes may not obtain new outgoing construction or subclass edges, nor new incoming subclass edges. This primitive does not change the structure of existing classes. 2. Part addition : A construction edge is added between two existing classes. This modi es the structure of the source class to have an additional part, or reference relation. 10

3. Part generalization : A construction edge v !l w is replaced with the edge v !l w , where w is a superclass of w. This primitive transformation generalizes the domain of the edge, expanding the possible set of objects that can be instantiated as the target of the edge. 0

0

Examples of the primitives are shown in gure 9, depicting a sequence of primitive transformations being applied to a class graph. Each primitive has an inverse, namely (1) class deletion, (2) part deletion, and (3) part specialization. Object-extending class transformations may require maintenance to the object structure contained in existing test cases before tests can be rerun. Drawing

Drawing

Drawing

shape Drawing

shape Point

Square

Square Point

Point

shape Square

widthSquare

Point

Number

Point

Shape

Shape

1. Original Class Graph

shape

shape

Drawing

2. Class Addition (object−extending)

3. Addition of Useless Alternation (object−preserving)

4. Part Generalization (object−extending)

Shape

5. Part Addition (object−extending)

Figure 9: Object-Extending Class Transformations.

4.3 Regression Testing During Class Evolution The process of regression testing a customized propagation pattern during class evolution will be presented using the example transformations shown in gure 10. The example shows the addition of the LexExp class into the class graph, which allows Let expressions to be supplied in Scheme programs. A Let expression allows a global variable with an initial value to be bound in the body of its sub-expression. The transformation shows a sequence of object-preserving and object-extending primitives being applied to the class graph, with incremental changes to the graph shown in bold. The transformations require modi cation to the traversal speci cation, shown as step 6 in gure 10. The traversal dependence graph created after each step is shown in gure 11. Adding the LetExp class will introduce an error into the program, since it erroneously prints variables bound within the scope of a Let expression as being free. The LetExp will introduce new paths in the traversal dependence graph, which must be covered. The following example describes only the process of program-based testing. 1. The rst graph in gure 10 shows the initial class graph. Its corresponding traversal dependence graph is shown in gure 11. A suite of test cases will be developed to cover the traversal dependence graph. 2. Class Addition : The transformation in step 2 of gure 10 shows the addition of the LetExp class, with three construction edges LetExp var ! V ariable,LetExp val ! Exp and LetExp body ! Exp. This primitive will never modify the traversal dependence graph, since the structure of existing classes is not modi ed. New paths from Exp to Variable are not created. Regression testing is not necessary. 3. Addition of Useless Alternation : The next transformation shows the addition of the Exp2 class and its two subclass edges Expr2 =) LetExp and Expr2 =) Exp . This transformation is required as an intermediate step in reorganizing the class structure to include LetExp as a subclass of Exp . This primitive will also never modify the traversal dependence graph, since it can not add new paths between existing classes. Regression testing is not necessary. 11

AppExp

val

AppExp

rator

rand

Number

Exp2

AppExp

rator

Number

LitExp body

Exp

rator

rand

val LitExp

body

val

Exp

LetExp

rand

LitExp

body

ProcExp

ProcExp

body

Number

ProcExp body val

val formal

Variable

Ident

id

var

LetExp

val

val

var

AssignExp

formal

Variable

Ident

1. Original Class Graph

var

id

Exp val

var

AssignExp

formal

Variable

Ident

2. Class Addition

id

var

AssignExp

3. Addition of Useless Alternation

Exp2 rator

rator

Exp2

AppExp

AppExp

rand

[Exp,Variable]

rand

Number LitExp

Number LitExp

val

val

body

val body

body

Exp

val body

LetExp

LetExp

Exp

ProcExp

Exp3

formal

var Variable

Ident

var

id

AssignExp val

Ident

formal

var

id

val

AssignExp

Variable

4. Part Generalization

[Exp3,Variable]

ProcExp

var

5. Addition of Useless Alternation

Exp2

6. Traversal Specification Modification

AppExp

AppExp

rator rand

LitExp

Number

rator

Number

val

body

body

LetExp

LetExp

Exp Exp3

val var

body

ProcExp

body

rand

val LitExp

ProcExp

Exp3

val

val

formal

var

Ident

val

Ident

id

Variable

var

formal

Variable id

var

AssignExp

AssignExp

8. Deletion of Useless Alternation

7. Part Replacement

Figure 10: Example Sequence of Primitive Class Transformations.

12

1

rator

rand

AppExp

rand

Exp

ProcExp

prefix

4

suffix

1

prefix

var

4

Variable

1

prefix

1

var

2

body val

2

body 3

Exp

LetExp 1 var

ProcExp

1

Variable

prefix

var

AssignExp

1

body val body 3

Exp

ProcExp

LitExp

var

AssignExp

1

1

ProcExp

1

2

1

1 rand

AppExp

body

LetExp 3 body

4

prefix suffix

val

2

AssignExp Variable

prefix

1

ProcExp

Exp3

var

1

3

2 val

1

2 formal

var

1

2 formal

AssignExp

1

8. After Deletion of Useless Alternation (no change)

7. After Part Replacement

Figure 11: Example Sequence of Traversal Dependency Graph Transformations.

13

suffix

val 2

2

prefix suffix

4

6. After Traversal Specification Modification [Exp3,Variable]

LitExp

val

var

var 1

prefix

3

prefix

AssignExp

Variable

val 2

rator

4

Variable

Exp3

var

AppExp

Exp3

var

prefix

1

1

Exp formal 2

2

2 val

1

body 3

ProcExp

5. After Addition of Useless Alternation (no change)

body

LetExp 3 body

LitExp 2

1

suffix

4

AppExp

LetExp

prefix

1

2

formal 2

4. After Part Generalization

1

1

rand

3

prefix

rand

1

2

1 var

val 2

rator

formal

AssignExp

3. After Addition of Useless Alternation (no change)

LitExp

Variable

Variable

var

rator

formal 2

1

1

AppExp

suffix

4

suffix

2

2

formal

2

LetExp

prefix

1

prefix

Exp2 rand

3

prefix

4

1

AppExp

LitExp

1

val

AssignExp

rator 1

ProcExp

suffix

Exp2

rand

3

prefix

2. After Class Addition (no change)

Exp2 rator

Exp

LitExp

2

2

formal

1. Original Traversal Dependence Graph

body val

1

val

AssignExp

3

body

ProcExp

2

2

AppExp 2

3

Exp

LitExp

1

rator

rand body

1

val

Variable

AppExp 2

3

body

LitExp

1

rator

2

4. Part Generalization : This transformation shows the rerouting of all construction edges having Exp as target to point instead to its superclass, Exp2 . The traversal dependence graph is modi ed, since Exp2 and LetExp are now contained in paths from Exp to Variable . Although the traversal dependence graph appears to be dierent, this primitive does not modify the behavior of existing test cases contained in the test suite, therefore they do not need to be rerun. Nor does it require modi cation to the object graphs contained in the existing test cases. It also does not cause any existing test cases to become invalid. However, new test cases must be added to the test suite to cover the new paths from Exp to Variable through the LetExp class. 5. Addition of Useless Alternation : The next transformation shows the addition of the Exp3 class and its six subclass edges Expr3 =) LetExp , Expr3 =) Variable ,Expr3 =) AssignExp ,Expr3 =) ProcExp , Expr3 =) AppExp and Expr3 =) LitExp . Since the traversal dependence graph is never changed by this primitive, regression testing is not necessary. 6. At this point, the traversal speci cation is modi ed to have Exp3 as the source rather than Exp . This allows LetExp objects to be included as top-level objects in test cases. This transformation would require any code wrapper speci cations attached to Exp to be moved to Exp3 . Updating the traversal speci cation modi es the traversal dependence graph, in that the edge Exp3 =) LetExp indicates that traversal can now start with a Let expression. Test cases must be added to cover this new path. While the traversal dependence graph appears very dierent from the previous, existing test cases will exhibit the same traversal behavior, therefore they do not need to be rerun. 7. Part Replacement : This object-preserving transformation reroutes the construction edges to have Exp3 as target rather than Exp2 , since they both have the same set of concrete subclasses. While part replacement may change the structure of the traversal dependence graph, it is only changing the dynamic binding links. The targets of the dynamic binding edges are the same in both graphs, namely the same set of concrete subclasses. Therefore the valid polymorphic substitutions for a method call remain the same. The traversal dependence graphs may have dierent structure, however the run-time behavior is equivalent. Regression testing is not necessary in this example. Part replacement may sometimes require maintenance of the propagation pattern before testing can occur. Maintenance is required when part replacement causes rerouting of edges with target class w to a dierent target class w , where code wrappers are attached to w or its abstract subclasses. The code wrappers must be moved to class w . As this may introduce changes in the ordering of wrapper speci cations, the traversal dependence graph must be generated and compared to the original. Dierences in ordering of the code wrappers will require retesting of aected test cases. The test cases do not require maintenance of object structure since this is an object-preserving transformation. 0

0

8. Deletion of Useless Alternation : The next transformation shown in gure 10 shows the deletion of the Exp2 and Exp abstract classes. They are considered useless since they did not have incoming construction edges and provided no outgoing construction edges for subclasses to inherit. Since the traversal speci cation was updated in step 6 to [Exp3,Variable], the classes being deleted are not in the traversal graph, therefore the traversal dependence graph is not modi ed. Regression testing is not necessary in this example. While not the case in this example, this primitive may require maintenance of the propagation pattern, since the abstract class being deleted may be used in the traversal speci cation or wrapper speci cations. If this is the case, the speci cations must rst be maintained before testing can occur. If multiple inheritance is not present in the traversal graph, then it is possible to maintain the traversal speci cation and code wrappers in a manner that produces an equivalent traversal dependence graph. If there is multiple inheritance, then deleting the abstract class may cause a change in ordering of inherited wrapper speci cations, thus changing the structure of the traversal dependence graph. Aected test cases must be identi ed and rerun, new test cases created to cover new paths, and invalid test cases agged as such. 14

Abstraction of common parts and distribution of common parts in principle should not change the behavior of the propagation pattern since attening (distribution of common parts) is performed automatically for the traversal dependence graph. However, it is possible for manual application of the primitives by the programmer to produce a dierent edge ordering than that produced by the attening algorithm. This may change the traversal dependence graph, and therefore require identi cation of aected test cases which must be rerun. The test cases can be used as is, since these are object-preserving transformations. The part addition, class deletion, part deletion and part specialization primitives may change the traversal dependence graph. The changes can add, delete and modify paths in the traversal dependence graph. Existing test cases aected by the changes will require object structure maintenance before being rerun, since these are object-extending or restricting transformations. Test cases covering deleted paths must be agged as invalid. New test cases must be produced to cover new paths.

5 Related Work There has been an increase in recent literature on the topic of testing object-oriented software [8, 17, 16], however the issue of regression testing during class evolution is not discussed, nor is the issue of reuse when testing generic components. A process for incrementally testing object-oriented programs during class derivation is presented in [7]. The technique analyzes the manner in which a derived class inherits from a base class, and reuses test cases of features that are not modi ed by the base class. Inherited features that are overriden can not reuse existing test cases, nor can new features. Rothermel and Harrold present a process for regression testing changes in statements in procedural and object-oriented software [23], based on analyzing changes in the program dependency graph. Their approach does not discuss regression testing changes to class structure and the impact on existing behavior, and does not discuss maintenance of object structure in test cases. McDaniel and McGregor [15] describe an approach to testing coverage using orthogonal arrays [14] to avoid exhaustive testing of polymorphic substitutions. Regression testing and class evolution are not discussed.

6 Concluding Remarks The concept of reuse must extend beyond program implementation and into the testing process. Class evolution has signi cant impact on maintenance and testing. Adaptive software anticipates the need for structure to evolve, and supports a programming model for de ning behavior with minimal assumptions on structure. Individual components are developed based on generic class constraints, and potentially reused by many programs with varying class structure. Future directions of research in testing adaptive software will include formalization of customizer equivalence classes, where an error aects all customizers within a class in the same manner. This can greatly reduce the amount of testing required during customization, by allowing adequate program-based testing with one customizer to imply adequacy of its equivalence class. Other research includes the process of testing customizers that are language-equivalent rather than object-equivalent, which may have drastically dierent structure, yet may reuse test cases when based on textual descriptions rather than object graphs. While this paper has presented the process of testing adaptive software, the ideas and issues presented are relevant to many areas of object-oriented software engineering. The issue of regression testing during class evolution will exist in every object-oriented application. The eort required to adequately retest a component when it is reused is an important metric in determining its true reusability.

15

References [1] Gilles Barbedette. Schema modi cations in the lispo2 persistent object-oriented language. In European Conference on Object-Oriented Programming. Springer-Verlag, 1991. [2] Edward Berard. Essays on Object Oriented Software Engineering. Prentice-Hall, Englewood Clis, NJ, 1993. [3] Paul Bergstein. Object-preserving class transformations. In Object-Oriented Programming Systems, Languages and Applications Conference, in Special Issue of SIGPLAN Notices, pages 299{313, Phoenix, Arizona, 1991. ACM Press. SIGPLAN Notices, Vol. 26, 11 (November). [4] Paul L. Bergstein and Walter L. Hursch. Maintaining behavioral consistency during schema evolution. In S. Nishio and A. Yonezawa, editors, International Symposium on Object Technologies for Advanced Software, pages 176{193, Kanazawa, Japan, November 1993. JSSST, Springer Verlag, Lecture Notes in Computer Science. [5] Eduardo Casais. Managing class evolution in object-oriented systems. In Dennis Tsichritzis, editor, Object Management, pages 133{195. Centre Universitaire D'Informatique, Geneve, 1990. [6] J. Ferrante, K.J. Ottenstein, and J.D. Warren. The program dependence graph and its use in optimization. ACM Transactions on Programming Languages and Systems, 9(3):319{349, 1987. [7] Mary Jean Harrold, John D. McGregor, and Kevin J. Fitzpatrick. Incremental testing of object-oriented class structures. In Proceedings of the 14th International Conference on Software Engineering, pages 68{80. IEEE Computer Society, 1992. [8] Paul C. Jorgensen and Carl Erickson. Object-oriented intergration testing. Communications of the ACM, 37(9):30{38, September 1994. [9] Linda M. Keszenheimer. Specifying and adapting object behavior during system evolution. In Proceedings of the 8th International Conference on Software Maintenance, pages 254{261. IEEE Computer Society, 1993. [10] Linda M. Keszenheimer. A uni ed graph representation for testing and maintaining object-oriented programs. Technical Report NU-CCS-94-19, Northeastern University, August 1994. [11] Linda M. Keszenheimer. Utilizing behavioral abstractions to facilitate maintenance during class evolution. In Proceedings of the 6th Conference on Advanced Information Systems Engineering, pages 325{338, Utrecht, Netherlands, 1994. Springer Verlag, Lecture Notes in Computer Science. [12] Karl J. Lieberherr. Adaptive Object-Oriented Software: The Demeter Method with Propagation Patterns. PWS Publishing Company, Boston, 1996. [13] Karl J. Lieberherr, Walter L. Hursch, and Cun Xiao. Object-extending class transformations. Formal Aspects of Computing, 6:391{416, July 1994. [14] Robert Mandl. Orthogonal latin squares: An application of experimental design to compiler testing. Communications of the ACM, 28(10):1054{1058, October 1985. [15] Robert McDaniel and John D. McGregor. Testing the polymorphic interactions between classes. Technical Report 94-103, Clemson University, 1994. [16] John D. McGregor and Timothy D. Korson. Integrating object-oriented testing and development processes. Communications of the ACM, 37(9):59{77, September 1994. 16

[17] Gail C. Murphy, Paul Townsend, and Pok Sze Wong. Experiences with cluster and class testing. Communications of the ACM, 37(9):39{47, September 1994. [18] G.J. Myers. The Art of Software Testing. John Wiley, New York, NY, 1979. [19] Jan Overbeck. Integration testing for object-oriented software. Technical report, Vienna Technical University, 1993. Ph.D. thesis. [20] Jens Palsberg, Cun Xiao, and Karl Lieberherr. Ecient implementation of adaptive software. ACM Transactions on Programming Languages and Systems, 17(2):264{292, March 1995. Also presented at the OOPSLA 1994 poster session. [21] Dewayne E. Perry and Gail E. Kaiser. Adequate testing and object-oriented programming. Journal of Object Oriented Programming, 2(5):13{19, 1990. [22] S. Rapps and E. J. Weyuker. Selecting software test data using data ow information. IEEE Transactions of Software Engineering, 11:367{375, 1985. [23] Gregg Rothermel and Mary Jean Harrold. Selecting regression tests for object-oriented software. Technical Report 94-111, Clemson University, 1994. [24] M.D. Smith and D.J. Robson. A framework for testing object-oriented programs. Journal of Object Oriented Programming, 5(3):45{53, June 1992. [25] Satish Subramanian, Wei-Tek Tsai, and Shekhar H. Kirani. Hierarchical data ow analysis for o-o programs. Journal of Object-Oriented Programming, 7(2):36{46, May 1994. [26] Elaine J. Weyuker. The evaluation of program-based software test data adequacy. Communications of the ACM, 31(6):668{675, June 1988. [27] Cun Xiao. Adaptive Software: Automatic Navigation Through Partially Speci ed Data Structures. PhD thesis, Northeastern University, 1994. [28] Daniel M. Yellin and Robert E. Strom. Interfaces, protocols, and the semi-automatic construction of software adaptors. In Object-Oriented Programming Systems, Languages and Applications Conference, in Special Issue of SIGPLAN Notices, pages 176{190, Portland, Oregon, 1994. ACM Press.

17