Getting More Out Of Your Classes: Building Families Of Programs in OOP Yoav Intrator, Hafedh Mili† Computer Technology Associates Inc. 6116 Executive Boulevard, Suite 800 Rockville, MD 20852 e-mail:
[email protected] De´partement de Mathe´matiques et d’Informatique Universite´ du Que´bec a` Montre´al Case Postale 8888 (A) Montre´al (Quebec) H3C 3P8, CANADA
Abstract Object-oriented programming is a powerful packaging technique for reusable software components [5], but in and of itself, is not sufficient to produce highly reusable and good quality software components; additional development methods and design guidelines are needed. Developing a set of closely related applications as a program family is one such development method. In the context of OOP, a program consists of a set of related classes. OOP languages provide different constructs to support abstraction, parametrization, and (semi-) automatic program derivation; all key techniques for implementing program families. This paper evaluates and compares inheritance, abstract classes, generic classes, and metaprogramming with metaclasses, as key techniques for implementing program families. Using meta-programming in metaclasses to define program families, as we suggest in this paper, is the most expressive form but also has some limitations.
This work was performed while Yoav Intrator was with Software Productivity Consortium, Virginia. Mili’s work is funded through grants from NSERC and Quebec’s ministry of higher education (MESS) through the SYNERGIE program.
1.
Introduction
Software reuse is possibly the most valuable contribution of object-orientation to the engineering of software. However, in the same way that procedural abstraction alone does not guarantee (good) functionally modular designs, object packaging alone does not yield the desired reusability and extensibility of software modules; additional guidelines and methods are needed. In this paper, we explore the mechanisms offered by OO programming languages to support the development of the so-called program families. It has become customary in the software reuse literature to consider the reusable artifacts at a higher level of abstraction than simple (and single) code components (see e.g. [7, 15, 37]). This involves: 1) going up the software lifecycle, such as reusing designs of components, 2) considering larger components, e.g., subsystems and frameworks [13], and 3) considering collections or classes of ‘‘functionally equivalent’’ or ‘‘non-essentially differing’’ components, independently of the stage of the lifecycle; when such components refer to code, we talk of program families. Roughly speaking, a program family is described by: 1) a defining abstraction representing the similarities (common aspects) among the members of the family, and 2) a set of parameters whose bindings that uniquely identify individual members of the family. Within the context of OOP, a program may be considered as a set of inter-related/interacting classes. Objectoriented programming introduces a number of concepts and language-level constructs that support sharing— of code and behavior— and parameterization. In this paper, we study three particular constructs: 1) abstract classes, 2) generic classes, and 3) meta-classes and meta-programming. These constructs involve a different blend of expressivity, flexibility, and safety. They all rely more or less heavily on class inheritance, dynamic binding and polymorphism, and basic knowledge of these constructs is assumed (see e.g. [33]). Five commercially available languages are used to illustrate the various constructs: Smalltalk-80, CLOS, Eiffel, C++, and Classic-Ada. We show how the metaclass programming construct can provides the means for creating the other constructs if they do not exist in the language. In section 2, we define program families in more detail, and propose a number of "quality" and "safety" criteria for implementing them. Section 3 provides a tutorial-like discussion of inheritance, as it provides the basis for the remaining mechanisms. Abstract classes, generic-classes, and meta-classes are briefly discussed within the context of program families in section 4. In section 5, we illustrate the power of metaprogramming with metaclasses on a simple
Introduction
Page 2
Smalltalk example. We conclude in section 6.
2.
Program Families
"We consider a set of programs to be a program family if they have so much in common that it pays to study their common aspects before looking at the aspects that differentiate them... We are motivated by the assumption that if a designer/programmer pays conscious attention to the family rather than a sequence of individual programs, the overall cost of development and maintenance of the programs will be reduced" (parnas 1979). Abstraction in programming in general, and program families in particular, have been around for some time [31]. IBM’s OS/360 can be ‘‘SYSGEN’’ed for its entire 360 and 370 family of computers using macro-processing. The code for operating system routines contained a number of macro references ("calls") whose parameters— and resulting expansion— depended on the specific "member" of the family. Conditional compilation in languages such as C is another case where different high-level source code is generated from the same "family description", depending on environmental parameters. However, these and other lexically-based techniques are limited in two major ways: 1) the members of the family have to be known— enumerated— in advance, and 2) the code to be generated for each element has to be coded in advance (e.g., as alternative macro expansions). This makes the unplanned— as it often is— "instantiation" of new, unanticipated, family members significantly costlier than the simple specification of a parameter, and costlier that the simple addition of that member’s specific code. Krueger stresses abstraction in software reuse, and argues that reuse is most successful when dealing with abstract components rather than with individual concrete components [15]. His definition of a reusable component— called component abstraction— resembles that of a program family. A component abstraction may be described using an abstraction specification, and abstraction realizations[15]. The abstraction specification embodies both the shared properties among the elements— called the fixed part of the abstraction— and the ones that distinguish between them — called the variable part or parameters of the abstraction [15]. The abstraction realizations represent the different instances of the abstraction— however it is convenient to represent them [15]. Much of the AI work on automatic programming is based on instantiating or specializing "reusable" program templates [1] (sometimes called frames[2] or cliche´s [28]), which may also be considered as program families of sorts. There are a number of factors to consider when designing a program family. By and large, program families
Intrator & Mili
Page 3
have the potential of: 1)
Saving on development and maintenance for the common aspects of the family members.
2)
Saving on the development and maintenance of the family members, above and beyond the savings resulting from the implementation of the common aspects.
Savings on the common parts are maximized if the common aspects are properly — not necessarily physically— separated from the variable parts of the family members. Savings on the variable parts are the by-product of reducing the conceptual complexity of individual members. For instance, provided a conceptually "clean" separation of the common and variable parts, the user of a program family need not see the implementation details of the common parts and only has to deal with the variable parts. And with some ingenuity, the "implementation" of the variable parts can be reduced to specifying parameter values. Program families have to be evolutive in the sense of allowing for the specification/derivation of new unanticipated family members without too much changes to the family description— obviously, provided they are members of the same family, i.e., they share the same common parts. The contrary would be symptomatic of a conceptual gap between the expression/description of the family and its intent1. We may refer to this property as completeness. Conversely, they have to be safe in the sense that they should not allow for the— inadvertent— creation of programs that are not within the family. We refer to this property as soundness. Program families also have to be evolutive in the sense that new families can be derived from older ones, either through specialization— e.g. by optimizing the implementation of a subfamily— or through generalization— by parameterizing a fixed aspect of the current family members. Ideally, this sharing between families extends beyond creation and into maintenance, i.e., maintenance of the common aspects between a family and its derivatives is centralized; in essence building a family of program families! Addressing the issues raised above involves a mix of methodological design guidelines and packaging techniques for designing and implementing program families. In packaging terms, the above issues may be translated into the following requirements, which we group under three major categories: Ergonomy: __________________ 1. For example, the intent could be to build the family of operating systems for the IBM 360/370 architecture, which includes several configurations for each kind of peripheral devices. The actual description may— implicitly— assume the basic configuration for one of the options (e.g., a single printer).
Program Families
1)
Page 4
A clear separation of the fixed part from the variable part of the family, from the point of view of both the developer of the family (designer) and the user (the "instantiator" of the members).
Safety: 2)
‘‘Constants should be’’: Enforcing the constancy of the fixed part of the family during the "legal" derivation of the members.
3)
Specify the parameters/variable parts intensionally (e.g., in terms of obligations) and support the proof of conformance of particular values, rather than enumerate the different values and matching instantiation values against them.
Evolvability: 4)
Incremental parameterization/resolution: the specialization or generalization of an existing family should involve local and conservative2 changes to the existing family,
5)
Shared maintenance of common aspects to a family and its derivatives.
Object-orientation supports a number of constructs for (data) abstraction, sharing, and parameterizing implementations, and appears to be the "dream" packaging technology for implementing program families. In the remainder of the paper, we review the various OO techniques in light of the issues raised above, with a particular attention towards safety issues.
3.
Inheritance: A Tutorial
Inheritance is perhaps the most commonly misunderstood and least agreed upon concept in object-orientation, for several reasons including object-orientation’s rich and diverse ancestry [36], and its simultaneous use of all those ‘‘inherited’’ meanings in analysis, language design, and programming. For the purposes of this paper, we present three interpretations of inheritance, with a particular attention to its uses as an implementation technology for program families: 1) generalization a` la knowledge representation, 2) subtyping, and 3) subclassing as a code sharing mechanism. Issues of formality and safety are central to our discussion. __________________ 2. conservative in the sense of not requiring to undo previous work (see e.g. [19]).
Intrator & Mili
Page 5
3.1. Inheritance in knowledge representation In knowledge representation, the term ‘‘inheritance’’ is used to refer to both generalization relationships between concepts/classes, and to the default inference mechanism that ensues from such relationships [23]. Broadly speaking, concepts have intensions— what they ‘‘mean’’— and extensions— what they refer to. For example, the concept ‘‘The morning star’’ means (intension) "the star that shines in the morning", and refers to (extension) the planet Venus3. In logical terms, we can think of the intension of a concept C as a one-place predicate PC(.) that returns true for instances of the concept, and its extension as the set EC of such instances. In this context, generalization is equivalent to logical implication between concept intensions and implies set inclusion between their extensions. Let C1 and C2 be two concepts, PC and PC their corresponding intensions, and EC and EC their corresponding exten1
2
1
2
sions. We have: (C1 is more general than C2) ≡ ((\⁄— X) PC (X) → PC (X) ) 2
1
→ ( EC ⊆ EC ) 2
1
Inheritance as an inference process has received considerable attention in the knowledge representation community (see e.g. [35]). In its simplest form, inheritance can be expressed by the following informal rule: Let C1 and C2 be two concepts such that C1 is more general than C2, then, whatever is true for C 1 is also true for C2. We can’t make the description of inheritance— as an inference mechanism— more explicit without adhering to one particular definition/school of thought. In one interpretation, let F(.) be a one-place predicate such that PC (.) → F(.). 1
We may say that F(.) is a property of C1. Inheritance in this case consists of using the implication PC (.) → PC (.) to 2
1
infer that PC (.) → F(.), or that F is also a property of C 2. All would be peachy keen if penguins could fly4, but they 2
can’t and knowledge representation languages must handle this problem one way or another. In this case, the implication PBirds(.) → Flies(.) is obviously not true, since some birds— namely, penguins!— do not fly. This led researchers to distinguish between essential or definitional properties, which are always inherited without exception, and nondefinitional (or normative, or prototypical) properties which may be cancelled or overridden by subconcepts [30]. __________________ 3. Incidentally, ‘‘The evening star’’ means (intension) "the star that shines in the evening", and happens to also refer to (extension) to the planet Venus... 4. Intuitively, Penguins are Birds (PPenguins(.) → PBirds(.)), Birds fly (PBird(.) → Flies(.)), and yet, Penguins don’t fly (i.e. ——— X s.t. PPenguins(X) Λ ¬ Flies(X)).
Inheritance: A Tutorial
Page 6
Others have questioned the logical interpretation of generalization relationships themselves within natural taxonomies, and proposed what amounts to non-monotonic/non-transitive generalization relationships (see e.g. [29] and [24]). Some knowledge representation languages such as the KL-ONE family (see e.g.[3]) do not allow the representation of "cancellable properties" and support such monotonic inferences as hierarchical classification, while others are based on non-monotonic logics and support a number of defeasible inferences (see e.g. CYC [16]). The underlying differences, from a knowledge representation point of view, are both philosophical/epistemological (essence of knowledge) and pragmatic (representing imperfectly coded knowledge). In KL-ONE, the intension of a concept is described by the (logical) conjunction of a number of "non-cancellable", inheritable properties. For example: PC (.) ≡ F1(.) Λ F2(.) Λ F3(.) 1
A subconcept C2 may either extend the set of applicable properties, as in: PC (.) ≡ F1(.) Λ F2(.) Λ F3(.) Λ F4(.) = PC (.) Λ F4(.) 2
1
or specialize one (or more) of the properties of C 1, as in: PC (.) ≡ F1(.) Λ F2(.) Λ F′3(.), where F′3(.) → F3(.) 2
or both. Classification in KL-ONE compares the properties that make up concepts’ descriptions [18].
3.2. Inheritance as subtyping Both the monotonic (classical logic) and non-monotonic (default logics) interpretations of generalization relationships and inheritance— as an inference process— have a role to play in object-oriented analysis. Application domain models may involve natural categories which need the expressivity and flexibility of non-monotonic logics [9]. However, viewed as model-based specifications of target applications, analysis-level object models need to be executable, as least in principle (see e.g. [20]), and the non-monotonicity of default logics would stand in the way of correctness proofs, verification and validation. In programming language terms, the end product of object-oriented analysis can be seen as a (specification of a) hierarchy of abstract data types based on the subtyping relationship [9, 25]. Subtyping rules may be defined either axiomatically, i.e. in terms of relations between type descriptors 5, or ‘‘behaviorally’’, in terms of object substitutability: T is a subtype of T’ if values of T are acceptable wherever values of T’ are expected (see e.g. [32]). __________________ 5. In the same way that generalization relationships between concepts are defined in KL-ONE.
Intrator & Mili
Page 7
Much theoretical research in OO deals with translating "object substitutability" into an equivalent consistent set of logical relationships (rules) between type descriptors that type systems can— effectively, or ‘‘decidably’’— check. Two
such
relations
include
extension,
as
in
extending
the
data
record
to
, or specialization, as in specializing an operation into a more ‘‘restrictive’’, yet ‘‘conformant’’ one. We can consider operations on a type T as mappings T×I → T×O, where I is the set of inputs values, and O is the set of output values. Let f1: D1 → R1 and f2: D2 → R2 be two mappings. f2 is said to conform to f1 iff: (A1) D2 ⊇ D1, i. e. f2 is defined everywhere f1 is defined, and (A2) (\⁄— x ∈ D1) f2(x) ⊆ f1(x), i.e., wherever f1 is defined, f2 assigns a subset of the values assigned/mapped by f1. Typed OO programming languages such as C++ [34] and Eiffel [21] use a variant approximation of conformance. Let f1: T×I1 → T×O1 and f2: T′×I2 → T′×O2 be two operations/methods where T’ is a subtype of T. f2 is said to conform to f1 iff: (a1)
I2 ⊇ I1, and
(a2)
O2 ⊆ O1.
The combination of (a1) and (a2) is called contra-variance. The reader may notice that such conditions do not ensure behavioral conformance, in the sense that what f2 is not guaranteed to be ‘‘semantically consistent’’ with f1. For example, if f1: CARDINAL → REAL returns the positive square root of positive integers, one can think of many functions f 2: INTEGER → REAL that have nothing to do with computing the square root. In order to enforce true behavioral conformance, we need to be able to check condition A2 above intensionally, i.e. without having to compute the values of f2 on the domain of f1. We need to formally specify the behavior of methods in a language that supports some sort of conformance proof. A number of such researchoriented specification languages exist (see e.g. [12]). However, they haven’t made their way into production environments, and in addition to the computational intractability of proof systems, they tend to be inaccessible and impractical for the average developer. The Eiffel language, through its use of assertions— class invariants and method pre- and post-conditions— comes closest to supporting formal specifications that are ‘‘user-friendly’’, precise, and that support ‘‘conformance validation’’; all this, within a commercially available environment. In Eiffel, assertions are themselves regular Eiffel
Inheritance: A Tutorial
Page 8
(boolean) expressions. Further, behavioral conformance between an inherited method and its redefinition is implicitly assured by the way the compiler treats pre-conditions and post-conditions6, provided that the pre-conditions and postconditions tell the whole story about what a method requires (pre-conditions) and what it promises (post-conditions)! This depends on how much intellectual (and programming) effort is consented by developers for the specification of pre-conditions and post-conditions.
3.3. Inheritance as code-sharing Object-oriented programming languages, be they typed (e.g. C++ and Eiffel) or untyped (e.g. Smalltalk and CLOS) support subclassing, i.e., the possibility to create a class as a subclass of another. After much confusion about the semantics of subclassing, and the relation between classes, types, subclassing, and subtyping, it is now widely agreed that: •
classes implement types
•
subclassing and subtyping are two different concepts
•
subclassing is a code-sharing mechanism.
In particular, it is possible to have two classes that are in a subclass relationship that implement two non-hierarchically related types, and vice-versa (see e.g. [4]). Whether that is desirable or not is another matter. A number of researchers argued that class hierarchies that are not organized along subtyping relationships are hard to understand [6], unsafe [4, 32], unpredictable [14], and may even involve sub-optimal code reuse [25]. Typed OO programming languages such as Eiffel and C++ attempt to ensure that classes that are in a subclass relationship implement types that are similarly hierarchically related. As a code-sharing mechanism, inheritance is not much more powerful than import mechanisms such as C (and C++)’s "#include" macro, or Ada’s "with" clauses. Meyer argued that class inheritance is a developer (server) mechanism, and not a user (client) mechanism, and need not even be shown to clients7 [22]. The C++ language enables developers to select the visibility of subclass relationships themselves, with the understanding that you can’t use— and __________________ 6. The actual (versus locally-defined) pre-conditions of a method consist of the logical disjunction of all the pre-conditions collected along the inheritance path, and the actual post-conditions consist of the logical conjunction of the post-conditions collected along the inheritance path to the root [21]. 7. In fact, the Eiffel environment includes a "hierarchy flattener" that delivers an inheritance-free, equivalent implementation of a class.
Intrator & Mili
Page 9
hence depend on— what you can’t see. However, unlike C’s "#include" or Ada’s "with" clauses, the "imported" code interacts with client code in ways that are hard to predict [14], in part thanks to dynamic binding and polymorphism. Hence the importance of ensuring that the dynamic type of an object be an "acceptable substitute" for its static type, i.e., ensuring that subclassing follows subtyping relationships. A number of researchers and practitioners have suggested delegation as a safer and cleaner alternative to subclassing for code-sharing between classes (see e.g. [32]).
4.
Implementing Program Families
4.1. Abstract Classes "[An abstract class is] A class that specifies protocol, but is not able to fully implement it; by convention, instances are not created of this kind of class" [11].
Using the usual separation between specification and implementation in data abstraction, an abstract class consists of a class specification, and an incomplete implementation. Fully abstract classes have no implementation. Fully concrete classes have all of their specified methods implemented. Because they are not fully implemented, abstract classes cannot— or should not— be instantiated, and they are created to be subclassed/inherited. In actual applications, abstract classes appear as "internal" nodes of class hierarchies, where all the leaf nodes are fully implemented. In reusable libraries and frameworks, abstract classes may appear as leaf nodes of class hierarchies, but it is understood that they need to be subclassed and fully implemented before they can be instantiated. Abstract classes may appear in class hierarchies for two reasons. First, from a "mechanistic"/technical point of view, abstract classes may be used to support polymorphism and dynamic binding in typed— and otherwise early-bound— languages such as C++ [34] and Eiffel [21]). Second, an abstract class may be created to factor out the common aspects among a number of classes— of which it is made a superclass. As such, it centralizes the development, documentation, and maintenance of those common aspects. Finally, even fully abstract (i.e. no implementation) classes can be quite useful: 1) they define the obligations of their subclasses (see below), 2) providing a conceptually ‘‘clean’’ organization of the class hierarchy, which can be its own reward, in addition to facilitating future evolution of the class hierarchy [22]. Abstract classes and subclassing may be used to define program families and generate family members. The abstract class itself embodies all the common aspects of the members of the family. These may consist either of
Implementing Program Families
Page 10
method specifications or of method specifications and implementations. Family members are created as subclasses of the abstract class. Subclassing allows a class to differ from its superclass in one of three ways: 1) implementing a method that was specified, but not implemented, in the superclass— e.g. virtual functions in C++ or deferred methods in Eiffel—, 2) re-implementing/overriding a method implemented in (or simply, available to) the superclass, and 3) adding new methods. Accordingly, the variable part of family members can be either the implementation of an already specified— and possibly implemented— method, or the addition of new methods. As a method of implementing program families, abstract classes combined with subclassing may be characterized as follows: Ergonomy: this method does separate the fixed part of the family from the variable part both from the viewpoint of the family designer, and from the viewpoint of the user. Safety: we identified two safety-related characteristics in § 2: •
Enforcing the constancy of the fixed part: in this case, this amounts to preventing subclasses from overriding selected aspects in the abstract class. OOP languages usually do not allow programmers to redefine data fields (instance variables). As for methods, the best they can do is to ensure that the redefinition is conformant with the original version (see § 3.2 and 3.3). This depends on: i) what is intended to be maintained constant, i.e. signatures and/or behavioral specifications, versus implementation, and ii) the language used. For example, no language prevents the redefinition of implementations— conformant or not. Further, only C++ and Eiffel enforce signature conformance, and only Eiffel enforces some measure of behavioral conformance (see § 3.2).
•
Specifying the variable parts intensionally: First, this method does not limit the methods that can be added to a class, and thus, may fail to specify the variable parts altogether. When the variable part is the implementation of an already defined (whether it is implemented or not) method, the specification of that method— in effect, the common part— either in terms of signature or behavior, defines implicitly the range of possible "values" that methods can take, and this technique may be seen as quite effective.
Evolvability: •
Incremental parameterization/resolution of an existing family: subclassing, which is used to related family members to the description of the family, can also be used to relate an existing family to its subfamilies and superfamilies. In this way, superfamilies can be generated by: i) removing an instance variable, ii) removing a method implementation (but keeping its specification), or iii) removing the method altogether (specification
Intrator & Mili
Page 11
included). The opposite operations may be used to generate subfamilies. •
Shared maintenance of common aspects between a family and its derivative: it is guaranteed because a family and its derivative will be related by inheritance.
4.2. Generic classes A generic class is a class whose specification— and implementation, if one is available— contains a type parameter. Genericity is useful in typed languages to support general data structures. For example, instead of defining a linked list structure to contain integer values (e.g. IntLinkedList), one to contain strings (StringLinkedList), and so on, we could define a generic LinkedList structure that can contain values of some type T, to be specified when needed. Using Eiffel-like syntax, we could define a generic linked list structure as follows: class LinkedList[T] export successor, predecessor, empty, full,... feature ... successor(x: T): T is ... do ... end ... end -- class LinkedList and use the one we need simply by supplying a value for the type parameter at variable declaration time, as in: ... myIntLinkedList : LinkedList[INTEGER]; myStringLinkedList : LinkedList[STRING]; i,j : INTEGER; s,t : STRING; ... do ... j := myIntLinkedList.successor(i); t := myStringLinkedList.successor(s); ... The generic class LinkedList can be ‘‘instantiated’’ for any type T that supports all the operations on elements of the list that are Genericity is supported in Eiffel, C++, and Ada. Naturally, it is not supported in untyped languages such as Smalltalk and CLOS because all classes are ‘‘type-generic’’, by default. In developing LinkedList, a developer may have assumed that some operations are supported by the type T, such as comparison operators, arithmetic operators, etc. This restricts the set of legal types to those that support these
Implementing Program Families
Page 12
operations. The specification of genericity comes in two flavors: i) constrained genericity, where the requirements of the type parameter are explicitly— and declaratively— stated, and ii) unconstrained genericity, where such requirements are only implicit in the generic code [21]. Ada (and Classic Ada) supports constrained genericity. The specification of the generic package LinkedList would look something like this: generic type T is private; with function "="(a,b: T) return BOOLEAN is ; with function "
