Extending Object-Oriented Programming to Address Hard Optimization Problems Yves Caseau, Glenn Silverstein, and Peter Koppstein Bellcore 445 South Street Morristown, NJ 07960 Email:
[email protected],
[email protected], and
[email protected] phone: (201) 829-4471 FAX: (201) 829-5981
Abstract Many problems in the telecommunication business and elsewhere seem to resist an object-oriented approach. Although objects and classes are easily identified and requirements can be mapped into methods, the object-oriented paradigm does not shed much light on how to solve problems with combinatorial computational complexity. On the other hand, these problems often occur in large projects that require reuse of data and methodologies for which the objectoriented programming paradigm can help significantly. In this paper we present two successful applications of a new language, called LAURE, which extends the object-oriented programming with high-level constructs such as rules and constraints that apply to objects and interoperate with other object-oriented tools. The applications discussed are examples of mixed optimization problems which combine individual optimization problems for which a number of sophisticated techniques have been developed, but whose combination seems to elude many of these approaches. Previous attempts at applying an object-oriented methodology to implement these applications have failed due to the complexity of the optimization problems they contain. With LAURE, we have obtained very impressive results in solving these problems and have developed a methodology for integrating rulebased and constraint-based techniques into object-oriented software.
1.
Introduction
Mixed optimization problems are quite common in the telecommunications industry and their solution generally requires the implementation of very complex algorithms for constraint satisfaction. In particular, they often contain a combination of individual optimization problems for which a number of algorithms
and approaches have been developed, but whose combination seems to elude any straightforward adaption of the individual approaches. Two such problems are work-force management and staff scheduling with constraints. "Mixed optimization problems" are problems that contain a combination of two or more individual optimization problems. For instance, a we have examined a work force management application, which includes a traveling salesman problem (TSP) component and a job scheduling component. In trying to adapt a TSP algorithm that utilizes a set of Operations Research (OR) heuristics to reduce the search space of potential solutions, we found that the time constraints of the scheduling aspect of the problem destroy the equivalence between permutations of tasks, which is critical for the OR approach. Similarly, constraint satisfaction techniques often work well for scheduling problems, but the added goal of minimizing travel costs requires a global analysis which is difficult to achieve in a purely constraint-based scheduling approach. On the one hand, constraint satisfaction technology is quite useful in addressing these problems. The problem specifications are often easily described as a set of constraints, the specifications themselves can be dynamic and may change at the last minute, and the declarative nature of constraints makes them ideal to handle this. However, problem solving is only one aspect of the overall problem and must be fit in the context of project as a whole. Here we need to consider issues such as managing the complexity of a large software project and sharing data between software modules for which object-oriented technology is ideally suited. Hence, it would appear that a combination of object-oriented and constraint-based technology would be appropriate. Unfortunately, object-oriented technology and constraint satisfaction technologies are somewhat at odds, as efficient constraints satisfaction requires access to low-level data structures which violates the principle of encapsulation. Still, there is a strong need for an integration of constraint-based and object-oriented technology, since the nature of the data significantly influences the constraint satisfaction performed. In this paper, we describe solutions for two mixed optimization problems: w o r k - f o r c e m a n a g e m e n t and staff scheduling with constraints, which were developed using LAURE, a language that attempts to overcome these obstacles to merge the object-oriented programming, rulebased, and constraint satisfaction paradigms. We will begin by discussing some relevant aspects of LAURE and then turn to the two case studies, followed by a discussion of why we chose LAURE instead of some of the available alternatives.
2. LAURE
A description of the LAURE language can be found in [Ca91b]. Here we focus on some of the aspects that distinguish LAURE from other languages. 2.1 Rapid prototyping LAURE provides a number of features that are useful for rapid prototyping including a development environment, a flexible knowledge representation, automatic deduction and constraint satisfaction, and support for dynamic as well as static typing. The programming environment of LAURE includes an interpreter, debugger, and inspector for viewing the entities of the LAURE language. LAURE allows the mixing of compiled and interpreted code which is especially important for managing a trade-off between flexibility and efficiency when dealing with large data sets. LAURE's flexible knowledge representation is homogeneous in that everything in LAURE can be described as an entity including relations and sets. Providing sets as first-class citizens of the language in LAURE greatly facilitates the manipulation of sets for collecting and querying objects. Defining relations as entities makes it possible to create new types of relations that inherit from existing ones and to associate properties with new and existing relations. LAURE provides a logical language consisting of rules and constraints that can be used to define problems in logical terms and supplies a built-in deductive engine and constraint solver to allow the user to quickly experiment with the rules and constraints. Finally, LAURE supports both dynamic and static typing which can be used together. Hence, the user can prototype using mostly dynamic typing and gradually move to precise static typing to gain the expected efficiency without having to change languages. 2.2 Multi-paradigm support for rules and constraints Rules and constraints describe what a solution is rather than how to solve the problem. LAURE provides rules and constraints in a manner that is fully integrated with its object-oriented system. In particular they can be attached to objects and determine the values of logical relations of objects (attributes). Rules in LAURE can be used to derive new information from old information by specifying conditions under which the new information can be derived. Constraints, on the other hand, focus on determining when new information can be derived as they specify conditions which must be true of any new or existing information. In the problems we have addressed, the ability to define rules and constraints on objects greatly simplifies the problem. For instance, in the staff scheduling problem, the specifications of the problem contain of a number of constraints on the activities a given person can perform on a given day. If we represent people and days as objects then we can define constraints on them, such as the fact that person cannot
have more than five total vacation days, and these constraints will apply to the specific instances of people and days as well as their subclasses (e.g., managers and workers). 2.2.1 Rules LAURE supports production rules, called axioms, which are guarded actions that are executed only if their conditions become true. In this sense, axioms are data-driven, executing their actions when information is added. For instance, we could use an axiom to raise an alarm in response to a particular state in the database (e.g., a high reading on a temperature gauge). LAURE also supports rules which derive new information from old information on demand. Unlike axioms, which are active, these rules are passive in that they are used to satisfy goals that are supplied by the user or derived by LAURE. For instance, if we use a rule to represent the transitive closure of a relation, then it will be used only in response to queries or goals involving that relation and will compute only the portions of the transitive closure needed to satisfy specific goals or queries. 2.2.2 Constraints Constraints in LAURE are used to enforce conditions on logical relations between objects. In LAURE there are three types of constraints: integrity constraints, definition constraints, and negative constraints. Integrity constraints are used to ensure that certain conditions are true of one or more objects logical data base. The condition checked can be arbitrarily complex. D e f i n i t i o n c o n s t r a i n t s maintain logical assertions in the database. They are somewhat more powerful than integrity constraints in the sense that they can be used by the constraint solver to choose values for the logical relations contained in their conditions. The conditions of definition constraints can be logical relations or mathematical formula. Negative constraints are definition constraints with negative conclusions, i.e., they forbid certain conditions to be true in the database. They too can be used to help choose a value for a relation, only they act as filters reducing the set of choices available rather than defining what choices are available. 2.3 Advanced problem solving The constraint solver in LAURE maintains a set of possible values for its goals (i.e., logical relations) and uses the constraints to narrow the choices through constraint satisfaction. Constraint satisfaction consists of a sequence of legal choices (i.e., choices which satisfy the constraints) and backtracking when no legal choices can be made (i.e., retracting the existing choices and making new choices). The choices themselves are made by using a strategy which can be supplied by the user (the first-fail principle is the default).
Hard optimization problems like the ones we have addressed, have combinatorial search spaces that become unmanageable very quickly and require sophisticated constraint satisfaction techniques to manage their search space. The approach we have taken in problem solving in this context is to help out the constraint solver with domain specific knowledge in the form of axioms, rules, and methods. LAURE's unique contribution is that it provides the lowlevel support needed to do this. As the constraint solver makes choices, rules and axioms can be triggered to derive new information that can further constrain the subsequent choices that the solver needs to make or initiate backtracking. Methods can also be triggered by axioms to perform computations that provide additional information for the constraint solver (including pruning search paths by generating contradictions). In this sense, axioms form the "glue" for methods that help solve the problem. 2.4 Efficiency One of the main goals in developing LAURE was to provide a language in which one can experiment with naive as well as complex "intelligent" solutions, but still provide efficiency (through compilation) that is comparable to equivalent solutions implemented as handwritten C or C++ programs. LAURE provides a compiler that produces C code, which our experience has shown, is comparable to the equivalent handwritten C programs in efficiency. The LAURE compiler can also be used to produce C++ code of comparable efficiency 1 . A key component of this is an advanced compiler for the logical rules and constraints which produces very efficient C or C++ code. This compiler is described in [Ca91a].
3. Two Case studies In this section we describe our experience with two applications of LAURE involving a task-technician assignment problem and a staff scheduling problem with constraints. As noted earlier, these problems are representative of a class of problems called "mixed optimization problems" that are essentially a mix of competing optimization problems for which none of the "simple" approaches work effectively alone. These types of applications are ideal for LAURE, although they are by no means the only types of applications for which LAURE is appropriate. 3.1 A task-technician assignment problem 1 Note that since LAURE provides an interface to the underlying C or C++ language, there are
a number of reasons why it is desirable to provide compilation to C++ including the ability to access C++ libraries and interact with C++ programs.
The task-technician assignment problem we addressed is the following: given a set of technicians, a set of prioritized work requests, a set of time constraints that define a time window in which the tasks need to be performed, a set of skills required for the tasks, and skills associated with each technician; find a set of schedules for each of the technicians which satisfies the time constraints, minimizes the overall travel time for the technicians, and maximizes the overall load of tasks. An important step in solving this problem is to solve it for a single technician. This is known as the time-constrained traveling salesman problem (TCTSP) [Sa86]. We have developed a solution for the TCTSP component in LAURE as part of a solution to the overall problem. Figure 1 displays a representation of the TCTSP problem where the nodes are the tasks with their associated time windows and the edges are the travel costs between the nodes. The goal of the problem is to create a Hamiltonian circuit which covers as maximizes the load of tasks and minimizes the travel cost. task 2 10:00 - 13:00
task 6 10:00 - 12:00
9.0
4.5
2.5 1.0
task 4 6:00 - 11:00
7.2 3.0 task 1 10:00 - 11:00
task 7 7:00 - 10:00 11.6
2.2
2.7
task 5 13:00 - 21:00 task 3 10:00 - 17:00
Figure 1: The Time-Constrained Traveling Salesman Problem
The TCTSP is a combination of two optimization problems, the job scheduling and the traveling salesman (TSP) problems. A naive approach for solving the TCTSP is to attempt to adapt solutions for one of two subproblems. To this end we explored three classical approaches: dynamic programming, operations research (OR) heuristics, and constraint logic programming. The first two of these approaches address the TSP and can be implemented easily with C or C++. The last approach is ideal for job scheduling and can be implemented with one of the more efficient constraint solvers such as PECOS [PA91] or CHIP [VH89], [VH90].
Our solution in LAURE was to use a combined strategy that included parts of all three approaches and added some additional heuristics for increased efficiency and robustness. The program we produced was relatively short (2000 total lines, about 500 of which were for the TCTSP portion of the problem), efficient (same worst case performance as the dynamic programming approach and a better average case performance), and easily modifiable (i.e., new rules and constraints can be added, as the problem specifications change). The important result is not the individual techniques that we applied, but the fact that we were able to apply them together in a cooperative fashion to produce a straightforward program that is relatively easy to understand and modify. Below we summarize the results of our attempts at adapting the three individual approaches and discuss the approach we used in LAURE. A more detailed description all four attempts to solve the TCTSP can be found in [CK92b] 3.1.1 Approach 1: Dynamic Programming The dynamic programming approach is well suited for producing optimal solutions to the TSP problems. It can be adapted to solve the TCTSP problem by using in a matrix to memoize the shortest distance to get to a given node (from the start node) given a set of nodes to explore. The algorithm is relatively straightforward and robust with a worst case complexity of O(n2 n ). Despite the fact that the number of tasks is very large (our database consists of 20,000 tasks), the number of tasks assigned to a technician is much more manageable (10 to 15 on the average), which makes achieving an optimal solution feasible in this domain. The dynamic programming solution was able to solve a graph of size 10 in 500 ms on a workstation. Unfortunately, its average complexity was about the same as the worst case complexity, as it was unable to recognize and take advantage of the "simple cases" and cases for which there was no solution. In addition, the size of the matrix produced was prohibitive for looking at larger problems or to be reused on other problems, which is important in dealing with the larger task technician problem. 3.1.2 Approach 2: Operations Research Heuristics Heuristic approaches are useful for producing good approximate solutions to the TSP [LK73]. OR Heuristics such as trying to schedule the largest jobs first and building a circuit by choosing the furthest city tend to work well and can produce algorithms that are efficient in many cases. However, for the particular TCTSP problem we addressed, we have found that although the combinatorics of the problem require vast amounts of computational resources to find an optimal solution, if it is possible to find an optimal solution, it often pays to do so. The OR heuristics can be used to find good
solutions, but optimal solution cannot be guaranteed by these approaches. 3.1.3 Approach 3: Constraint Logic Programming The constraint approach has at least two distinct advantages. First, the overall problem is represented very naturally as a set of constraints, with the time windows represented as time constraints and general constraints to represent the mutual exclusion of tasks, the notion of performing a task in its time window, etc. This leads to a very concise and clean program. The second advantage is the declarative nature of constraints makes it relatively easy to change the definition of the problem to accommodate new constraints and add existing ones. However, the main disadvantage is that search space is prohibitively large and a global analysis is needed for efficiency. This is beyond the capabilities of the existing constraint solvers that we are aware of or at least requires a lot of low level programming of the constraint solver. 3.1.4 Using LAURE on the TCTSP The approach used in LAURE was a combination of three strategies: a resource allocation strategy, a TSP strategy, and a constraintbased scheduling strategy, and a set of heuristics to provide increased efficiency and robustness. The resource allocation strategy used a double time-window to create an empty schedule consisting of n consecutive boxes and then used the time-window constraints to reduce the tasks that can fit in each box. This approach is useful for providing a more global analysis to detecting bottlenecks, where too many tasks must be executed due to the time constraints. The TSP strategy attempted to optimize travel time by building a Hamiltonian circuit (avoiding loops) based on a next task relationship and performed pruning on the total travel time. Finally, the scheduling strategy defined an ordering on pairs of tasks based on the time constraints and a greedy heuristic which tries to order the bigger pairs first. The strategies were combined by using the TSP strategy as a basis, building a Hamiltonian circuit of tasks, and using the rules from the scheduling strategy to provide information on the task ordering and the rules from the resource allocation strategy to compute estimates on minimal travel times as the possible assignments (of the boxes) change. Each of these strategies were developed independently LAURE and tried in various combinations. It is interesting to note that it took only a few hours to combine the three strategies used to solve the problem. The overall result was that on the TCTSP problem, the LAURE solution retained the robustness of the dynamic programming approach producing results in under 500 ms in the worst case, while improving on the average case complexity (about 300 ms in the average) by solving simple problems efficiently and recognizing when there was no solution. In addition, LAURE improved on the
quality of the results it produced when compared against a benchmark heuristic system by increasing the load by up to 3% and providing a savings in travel costs up to 8% for the data sets we examined. This is quite a significant improvement due to the magnitude of the work force scheduled and the labor rates (see [CK92b]). 3.2 Staff scheduling For our second application, we worked with a group at NYNEX telephone Co. on a staff scheduling problem that called for creating a timetable for a set of people and a set of days given some global constraints on the type and number of activities that can be associated with a given person or a given day, as well as, constraints on activities that can be performed on consecutive days. The timetable that we needed to produce consisted of a matrix of activities where the rows corresponded to the individual people being scheduled, the columns represented the days to schedule over, and the cells of the matrix corresponded to the activities performed by a particular person on a particular day (e.g., taking vacation, working on a morning, and working at night). Figure 2 displays a sample timetable along with examples of a constraint and an activity. The timetable Monday
Rules & constraints "a morning shift should not follow a night shift" Kirk
night
Tuesday
Wednesday
night
night
vacation
night
day
vacation
"no more than two people should take vacation on a given day"
...
Spock
day
McCoy
morning
Activities morning day night vacation
... Figure 2: A sample time table for the staff scheduling problem
The scheduling task was not automated before our initial effort, so it was important to have a development environment that supported exploratory programming. At first, we were tempted to solve the problem with a purely constraint-based approach. The problem
definition was easily translated into a set of constraints and the constraints themselves were dynamic and could change even at the last minute, the constraint-based approach seemed very appropriate. However, the combinatorial nature of the problem space made an approach based solely on constraints infeasible even for small to moderate sized problems. Again, our solution combined multiple paradigms and strategies in an attempt to maintain the flexibility of a constraint-based approach and achieve efficiency by pruning the search space of the constraint solver. Through the addition of heuristics such as measuring the entropy of the cells in the matrix 2 and providing intelligent cut-off algorithms (e.g., monitoring the number of activities that can be associated with a person or a day and maintaining a min and a max), we were able to achieve a solution that provided a four magnitude increase in performance of the constraint-based approach. Again the three main factors that made this possible were the ability to experiment efficiently with combinations of methods (the combinatorics of this problem necessitate C-like efficiency even in the exploratory mode), the availability of the multiple paradigms (constraints, deduction, and object-orientation), and the ability to piece the technologies together easily to produce clean and concise code. Below we present some LAURE code for a similar application which demonstrates the cooperation of rules, constraints, and methods in the staff scheduling application. A more detailed description of the application and results will appear in a forthcoming paper. The resources for the scheduling problem are days (the column of the matrix) and people (the rows of the matrix). Scheduling consists of filling in the cells of a matrix timetable with the activities of the corresponding people on the corresponding day. Person and Day are defined below along with the activities and Person-Day, which represents the cells of the matrix. A Monitor class is also defined which records the minimum and maximum number of times a particular activity can be associated with a resource as well as the current number. This is useful for some pruning heuristics including the one described below. [define Monitor class with slot(Min -> integer, default 0), slot(Max -> integer, default MAX)] [define Current attribute domain Counter, -> integer] [define Day class superclass {Resource}, with slot(next_day -> Day)] [define Person class superclass {Resource}] [define Activity class with list_slot(monitors -> Monitor)] [define morning-assignment Activity] ... 2 Entropy is an estimate of how much a choice will reduce the size of the search space.
[define Person-Day class with slot(person -> Person), slot(day -> Day), slot(next_person_day -> Person-Day), slot(activity -> Activity)]
In addition to global constraints on the resources, the problem definition contains a number of exclusion rules which are constraints on activities that occur on consecutive days. Incompatible activities are activities that should not occur on consecutive days. Below we define a general incompatibility constraint to handle incompatible activities. [define incompatible multi_attribute domain Activity, -> Activity] [define incompatibility constraint for_all d:Person-Day, a:Activity, if [or [exists d2:Person-Day, d2 = next_person_day(d), incompatible(a,activity(d2))], [exists d3:Person-Day, d = next_person_day(d3), incompatible(activity(d3),a)]], check no(activity(d,a))] ;; cannot have a morning assignment after a night or day assignment (incompatible(morning-assignment) is {night-assignment,day-assignment}) ;; cannot have a day assignment after a night assignment (incompatible(day-assignment) is {night-assignment})
Often a constraint will be used to ensure some condition or relationship between attributes whose value is maintained by rules, axioms, and/or methods. This is one of the ways in which the various programming paradigms in LAURE can cooperate. An example of this is shown below with the check_min_matrix constraint. The constraint check_min_matrix makes sure that the overall minimum associated with an activity for the matrix (min_matrix) does not exceed the overall maximum (max_matrix). The axioms min_compute is used maintain the minimum value for a monitor associated with a resource using the method update_min_matrix to do most of the work. [define check_min_matrix constraint for_all a:Activity, check min_matrix(a) > max_matrix(a)] [define min_compute axiom mode reversible, for_all a:Activity, m:Monitor, if activity(m) = a, Current(m) > Min(m), then update_min_matrix(a)] [define update_min_matrix(a:Activity) method -> integer => [let sum as 0,
[for c in monitors(a), sum
