is little support for the development of corresponding software systems. Thus, one ... Local search based heuristics use as common "representation" primarily the ...
Building Reusable Software Components for Heuristic Search Andreas Fink, Technische Universitat Braunschweig Stefan Vofi, Technische Universitat Braunschweig David L. Woodruff, University of California at Davis Summary: Today, advanced heuristic approaches for discrete optimization are usually implemented from scratch. We discuss some crucial problems that have so far prevented us from having a "heuristics stockroom" with ready-to-use software components available. Then, we analyze the domain to be covered by reusable software and introduce and discuss a design for a framework for heuristic search. Finally, we summarize our experiences and discuss future research topics.
1. Introduction In practice there is a great variety of discrete optimization problems. Since most of them are NPhard, the modeling of the problem is an approximate one, and the data are generally imprecise, heuristics are the primary way to tackle these problems. The use of heuristics generally meets the needs of decision makers to efficiently generate "satisfactory" solutions. However, though in practice there is a huge number of problems to be solved, the following statement of Wolsey [24] may still be regarded as valid: "This profession [Operations Research] has presently more algorithms than applications [... ]". Compare, e.g., the number ofresearch papers in the field of meta-heuristics with the number of implementations of such heuristics in real, large-scale application systems. In a statistical review of flowshop scheduling research, Reisman [20] criticizes the lack of relevance to practice for the overall majority of research in this field. He emphasizes "that flowshop scheduling research is in dire need of a paradigm shift to enhance its probability of ever becoming a tool for the practice of OR/MS. [... ] The literature should also seek out and recognize work that is satisfied with less elegant but implementable, better yet implemented, solutions." That is, in practice we need easyto-use application systems that incorporate the results of basic research in the corresponding fields. Following Nievergelt [18], this may be subsumed by "No systems, no impact!" Therefore, we also have to deal with the issue of efficiently building such implementations to bridge the gap into practice, instead of just investigating and solving simplified problems by ad-hoc-approaches. Why have we not seen extensive applications of advanced heuristic search methods in practice? Our ultimate goal is decision support, so we have to observe the characteristics of real-world problems. Problems in practice are diverse and evolve over time, so robust and flexible heuristic strategies are required. The common incorporation of advanced heuristics in application systems requires to find a way to reuse much of such software and to redo as little as possible each time. However, there is little support for the development of corresponding software systems. Thus, one should strive to develop efficient mechanisms to aid in building software that implements robust heuristics to solve real-world problems. This especially concerns the development of reusable and adaptable software components for heuristic search. As an example, meta-heuristics like tabu search or adaptive memory P. Kall et al. (eds.), Operations Research Proceedings 1998 © Springer-Verlag Berlin Heidelberg 1999
211
programming [10] have been successfully applied to a great variety of problems. In order to apply such methods for a new type of problem, the corresponding algorithms have to be "materialized" as executable implementations-a non-trivial task if this is done every time starting from scratch, especially if carried out by someone who is not deeply involved in these topics. The crucial problem is the transition from high level descriptions of algorithms (like those in scientific papers) to specific systems. One way to achieve this step would be the definition and use of a modeling language as, e.g., for mathematical optimization (e.g., AMPL). Such modeling systems provide an excellant interface between models and solvers, but only for a limited range of model types (e.g., linear) due, in part, to limitations imposed by the solvers. Furthermore, while modeling systems for heuristic search is an active research topic, it is still open whether such an approach may be generally successful [17]. One might reflect the corresponding situation for the mathematical programming approach, which is based on the declarative specification of problems as well-defined mathematical programs. This facilitates the use of highly advanced methods that are to a large extent independent from specific types of problems. Another example is constraint logic programming with logical constraints as a common representation. Such well-defined representation schemes allow independent development of problem solving methods that are applicable for different types of problems. With respect to software development, this enables the development and use of software modules that work without adaptation for a wide range of applications. Local search based heuristics use as common "representation" primarily the concept of solution spaces and neighborhood structures. With this, different heuristics can be described on a high level like usually used in scientific papers. Whereas mathematical programming software defines the requirement to specify problems in an explicit low-level formalism (like text files in a specific format as, e.g., MPS-files), the neighborhood search paradigm is a more abstract concept. Here, the search space is described implicitly-the search space is "virtually available" by traversing the solution space locally, step-by-step. Unfortunately, the way solutions and neighborhoods are defined for various types of problems is generally highly problem dependent.
2. Software reuse and frameworks In this paper, we focus on building application-specific heuristic search systems by means of the reuse and adaptation of sufficiently universal and generic software components for heuristic optimization. Ideally, software might be built-analogously to the production of physical goods-by selecting, adapting, and combining appropriate components from a huge set of existing components. This goal has been discussed for a long time [14, 15]. Reusability may be defined as the ability of software elements to serve for the construction of many different applications [16]. Corresponding reusable software elements may be termed components. The need for reusability comes from the observation that software systems often follow similar patterns; it should be possible to exploit these similarities and avoid reinventing solutions to problems that have been encountered before. By capturing such a pattern, a reusable software component will be applicable to many different developments. Due to the unreasonable assumption that we may generally use these components "as is", it is crucial to define appropriate mechanisms for adaptation. Thus, our goal should be to incorporate the knowledge of experts in different fields into corresponding software components, which may easily
212
be reused and adapted without the need to understand their inner workings. The application of modern software development techniques has been neglected for a long time in the field of algorithm research. Accordingly, the software developed mostly lacks certain properties required to be reused, e.g., robustness, adaptability, and ease of use. Object-oriented software technology provides a way to build reusable software, but does not guarantee reusability per se. Especially the reuse of algorithms may be regarded as "Still a Challenge to Object-Oriented Programming" [23]. One approach primarily devoted to the goal of achieving a higher degree of reuse is the framework approach [3, 4, 13]. Taking into account that for the development of application systems for given domains quite similar software is needed, it is reasonable to implement such common aspects by a generic design and embedded reusable software components. Here, one assumes that reuse on a large scale cannot only be based on individual components, but there has to be to a certain extent a reuse of design. Thus, the components have to be embedded in a corresponding architecture, which defines the collaboration between the components. Furthermore, there must be explicit mechanisms for adaptations of these components to the specific application. A framework may be defined as a set of classes that embody an abstract design for solutions to a family of related problems (here, heuristics for discrete optimization problems). A framework provides us with abstract applications in a particular domain, that may be tailored for individual applications. A framework defines in some way a definition of a reference application architecture ("skeleton"). That is, frameworks provide not only reusable software elements but also some type of reuse of architecture and design patterns, which may simplify software development considerably. Thus, frameworks represent implementation-oriented generic models for specific domains. The flexibility of a framework is dependent on the specific possibilities for adaptation. As certain aspects of the framework application cannot be anticipated, these aspects have to be kept flexible, which implies a deliberate incompleteness of respective generic software components. These semifinished components provide "holes" to be "filled" when building specific applications. Pree [19] terms these variable aspects hot spots and calls a framework well-designed if it provides adequate hot spots for adaptations. Here, one may differentiate between white-box and black-box reuse or frameworks. Frameworks often start as white-box (inheritance-based) frameworks which are primarily customized and reused through subclassing. When the variable concepts have stabilized or been realized, it may be possible and appropriate to evolve to black-box frameworks, which are mostly adapted through static parameterization and composition [21]. There are several potential advantages of this adaptation by parameterization: run-time efficiency, decoupling of components, and ease-ofuse without the need to understand internal details of components. One potential disadvantage of parameterizable components is the requirement of a solid understanding of the stable and flexible concepts of the domain [3]. In reality, of course, a framework generally does not embody a pure design concerning the properties described above, but uses different concepts and techniques in a pragmatic way.
213
3. Design of a framework for heuristic search While there are some well-known approaches for reusable software in the field of exact optimization, 1 there is, as far as we know, no ready-to-use and well-documented component library in the field of local search based heuristics that meets the goals set above. However, we would like to point to the class library for heuristic optimization of [25], which includes both local search based methods and genetic algorithms, the framework for local search heuristics of [1], and the class library described in
[9]. For reasons of space, an appropriate evaluation of the different approaches is not possible here. 3.1 Domain analysis Building a non-trivial software system requires a solid understanding of the respective domain. Thus, it is essential to perform an appropriate domain analysis, which results in a model of the domain to be covered by reusable software with the constant and variable aspects of the domain. The domain to be captured here may be described in the following simplified, tight model for local search based heuristics. This model might appear rather trivial, but it is important to formalize the domain to be covered by reusable software in some way, to achieve a common understanding of the domain, which allows the software developers to start from reasonable assumptions. Here, we apply a "back-to-thebasics" approach and try to include the basic ingredients of the different concepts and neglect some subtleties, especially if these are difficult to describe within the space limitation here. Further, here we restrict the discussion to concepts related to the meta-heuristics simulated annealing and tabu search. 2 There are different types of discrete optimizations problems, P, each with different problem instances, pEP. There may be different ways to obtain these problem instances; e.g., reading from a file, reading
from a data base, using a problem generator, by coupling with a software system that provides the data, etc. For every problem instance p there may exist one or more solution spaces, Sp, with solutions, S
ESp. If unambiguous we may neglect the index p. As a "solution" for a given type of problem may
be an incomplete solution in the sense that not all decision variables are set, we may use more than one solution space. For example, a solution for the Steiner problem in graphs may be constituted by the actual Steiner tree or by a selected subset of the Steiner nodes that may be treated as required nodes. Furthermore, we mayor may not include infeasible solutions in solution spaces S. Infeasibility generally affects the evaluation function,
1:S
~
r.
A fully ordered set
r
corresponds to a simple
preference function of a decision maker. The objective is to find a solution
S
E
S with minimal
evaluation I(s), or an evaluation as low as possible. (Of course, maximization problems may easily be transformed into minimization problems.) For each solution space, S, there are one or more neighborhood structures, N, that define for each solution s E S a linear ordered set of neighbors N(s) = {nl(s), ... , nIN(s)l(s)}. To each neighbor
n(s) E N(s) corresponds a move p,(s, n(s)) that captures the transitional information from s to n(s). 'Consider, e.g., mathematical programming software like CPLEX (http://www.cplex.com). ABACUS, a framework that incorporates cutting plane and column generation methods for combinatorial optimization problems (cf. [22]), and class-libraries for constraint logic programming such as ILOG Solver (http://www.ilog.com/). 2With regard to these heuristic methods, we refer to the descriptions given in [12, 10, 5].
214
To make use of tabu search, one must store information about the search history (trajectory), which requires the notion of move (solution) attributes. A move J-L is an aggregation of
IJ-L-I negative move attributes J-Ll,···, J-Lj;".-I'
J-Li E 111
and
IJ-L+I positive move attributes
J-Lt,···, J-L~+I' J-Lt E 111,
where 111 is the set of attributes that imply, and are implied by, the neighborhood. The negative (positive) move attributes correspond to those properties of the current solution that are "destroyed" ("created") when the move under consideration is performed. With this, tabu criteria may be defined in a flexible way. 3.2 Framework design In the following we briefly describe a framework for heuristic search (first discussed in [8] under the acronym HOTFRAME, Heuristic OpTimization FRAMEwork). Intended users of the framework are both software developers that aim to develop optimization software to be used in practice and researchers in the various fields applying meta-heuristics. Before describing the framework in more detail, a discussion of the general concept is given. We use abstraction as the primary mechanism to build adaptable components. That is, common behavior is factored out and grouped in generic classes, applying static type variation that enables one to keep respective concerns flexible. A generic component should be expressed in a way that assumes as little as possible about the specific type on which it is acting. The specific application, i.e. adaptation, of generic classes by defining (fixing) the type parameters is called instantiation. For example, a local search component may be parameterized by the neighborhood used. Specific neigborhood types, which may be used to instantiate the component as a specific application, must meet a number of requirements defined be the generic component. This may be called conceptual inheritance: all type parameters of the component represent specializations of such an abstract type with respective syntactical and semantical requirements. In C++, generic classes are defined by using the template mechanism (sometimes termed as static polymorphism), which enables one to leave certain types and values unspecified until the code is actually instantiated and used (compiled). With this, in connection with traditional object-oriented mechanisms such as encapsulation, etc., adaptation of concerns such as the representation of data, the implementation of algorithms, and the tabu criterion used is possible. Naturally, this means that the basic data types on the low level have to be parameterized. Denoting such a type parameter by adding it enclosed by "< ... >" to the respective classes may, e.g., result in a template class such as X..P that represents problem instances for a specific problem type X, with the type T defining the data type of the cost coefficients of the problem (e.g., integer or floating point) and the type Range defining the maximum number of objects. However, more important is the parameterization of the heuristic classes by problem dependent types. For steepest descent, this leads to two important type parameters: the solution type and the neighborhood type. For example, the instantiation of a generic steepest descent component for a sequencing problem with a shift neighborhood may be defined as
215
SteepestDescent< Sequencing< T, Range >, ShiftNeigborhood< T, Range> > .
As a more advanced example, consider using an object-oriented data base to persistently store the search trajectory of a reactive tabu search component, with simulated annealing used as escape mechanism. Then, we may instantiate a respective component as TabuSearch< Sequencing< T, Range >, ShiftNeigborhood< T, Range >, ReactiveTabuCriterion< Sequencing< T, Range >, ShiftNeigborhood< T, Range >, EdgeOrientedHoveAttributes< T, Range >, ExactSolutionlnfo< T, Range >, PersistentTrajectory< T, Range> >, SimulatedAnnealing, ShiftNeigborhood< T, Range> > > .
Summarizing, the approach followed here is to extensively use static polymorphism in this way. Thus, the framework is primarily based on generalization by genericity (by heavy use of templates), and uses inheritance only in a secondary way. Potential advantages of this design approach are run-time efficiency, a decoupling of components, less "intrusiveness", and possibly a better way to reuse more granular components, what might lead to appropriate flexibility, while partially achieving the ease of use of black-box reuse. For example, the architectural design described below provides appropriate flexibility on the problem side to enable reasonable implementation variations. Inheritance is essentially used to factor out common behavior for similar problems. For example, the similarities between different sequencing problems and corresponding neigborhood structures can be represented in corresponding class hierarchies. With that, the internal representation of a sequence or different neighborhoods (shift, swap, inversion, etc.) are coded only once, whereas specific functionality such as the evaluation of a solution is implemented in corresponding subclasses. The architectural design covers especially the separation into components, and the definition of a predefined component collaboration including the syntatical interface and the semantical requirements to be met. Naturally, our principal problem is, how to separate the problem side from heuristics. If it is possible to define a universal interface between these concepts, the number of implementations needed is reduced to an additive (instead of a multiplicative) combination of the number of problems and the number of heuristics. According to the domain model discussed above, the abstract problemspecific types involved from a specification perspective (with X representing the problem type) would be a problem class X..P and one or more solution classes X _8 for all local search based heuristics, and eventually some additional classes for advanced heuristics. For example, various tabu search heuristics may require for each solution class X..8 one or more corresponding neighborhood classes
X ..8fl, for each neighborhood class X..8fl one or more classes X ..8.A, which encapsulate the transitional information of a move J1, as corresponding move attributes, and possibly for each solution
216
class X_S corresponding classes X...sJ, which encapsulate information about solutions (e.g., exact or hash codes). These types are visualized by the simplified class diagram in Figure 1, which includes the relevant associations and the most important functionality to be provided by the corresponding types (as specialized methods).
f-I
P P( Istream)
1
1
S
*
S_I(S) S_I(S_N)
multiplicity in depeI1de!Ice injectivity ----- ofS_I (yes-exact: I, O
