Ta ) Tc; Ta 2 ; dead end. Sa = c : (2) ) Sb 2 fa; c=; d0g, Sd 2 fa; b; c=; d0g. (4) ) Sc 2 fa=; b; d0g. (7) ) Ta + 3. Tc. Sb = a : (2) ) Sd 2 fa=; b; d0g. (4) ) Sc 2 fb=; d0g.
An Exact Constraint Logic Programming Algorithm for the Traveling Salesman Problem with Time Windows ? Gilles Pesant1 , Michel Gendreau1;2, Jean-Yves Potvin1;2, Jean-Marc Rousseau1;2;3 Centre de recherche sur les transports, Universite de Montreal, C.P. 6128, succursale Centre-ville, Montreal, Canada, H3C 3J7 2 Departement d'informatique et de recherche operationnelle, Universite de Montreal, C.P. 6128, succursale Centre-ville, Montreal, Canada, H3C 3J7 3 GIRO inc., 75, rue de Port-Royal est, bureau #500, Montreal, Canada, H3L 3T1 1
Abstract
This paper presents a constraint logic programming model for the traveling salesman problem with time windows which yields an exact branchand-bound optimization algorithm without any restrictive assumption on the time windows. Unlike dynamic programming approaches whose performance relies heavily on the degree of discretization applied to the data, our algorithm does not su er from such space-complexity issues. The data-driven mechanism at its core more fully exploits pruning rules developed in operations research by using them not only a priori but also dynamically during the search. Computational results are reported and comparisons are made with both exact and heuristic algorithms. On Solomon's well-known test bed, our algorithm is instrumental in achieving new best solutions for some of the problems in set RC2 and strengthens the presumption of optimality for the best known solutions to the problems in set C2.
Introduction In the last few years, constraint programming (cp) has brought a novel way of tackling combinatorial optimization problems. Its most successful incursion to date is probably in the area of scheduling (Baptiste et al., 1995) (Caseau and ? to appear in
Transportation Science
Laburthe, 1994). A good share of their success comes from the incorporation of edge- nding (Carlier and Pinson, 1989), an operations research (or) technique, into a cp setting. Indeed, (Baptiste et al., 1995) argue that such a combination can be quite fruitful. In that spirit, we propose a constraint logic programming (clp) algorithm for the traveling salesman problem (tsp) with time windows (tsptw) which incorporates arc elimination and time window reduction rules from the literature but, instead of restricting their use to the preprocessing phase of the algorithm, uses them throughout the search in a more powerful way. The resulting branchand-bound algorithm does not require any restrictive assumption about the time windows nor any discretization of the problem data. The attractiveness of this type of programming language is the built-in mechanism to dynamically handle the rules (constraints), which greatly simpli es the programming task by making implicit the various interactions of constraints during program execution. The diversity of constraints that can be expressed and uniformly handled in that framework also serves the addition of side constraints often present in real-life problems. The tsptw consists of nding a minimum cost tour of a set of cities where each city is visited exactly once. To be feasible, the tour must start and end at a unique depot within a certain time window and each city must be visited within its own time window. Early arrival is allowed but implies a waiting time until the beginning of the window. The cost usually corresponds to the total travel distance or to the total schedule time (travel time + waiting time + service time). (Savelsbergh, 1985) showed that simply deciding whether there exists a feasible solution to an instance of the tsptw is NP-complete. Nevertheless, the full problem has important applications in bank and postal deliveries, school-bus routing and scheduling, disjunctive scheduling with sequence-dependent processing times, automated manufacturing environments and as a subproblem of the vehicle routing problem with time windows (vrptw). To our knowledge, there has been little work on the tsptw from a constraint programming perspective | we believe the recent birth of the eld to be mostly responsible for that oversight. (Caseau and Koppstein, 1992) encountered a variety of small tsptws (about 10 cities) within a large task assignment problem and so designed a robust exact branch-and-bound algorithmto handle thousands of these little problems with good average performance. To achieve robustness, the authors combined tsp, scheduling and resource allocation models to cope with instances where the routing or scheduling elements may dominate. Constraint programming o ers a clean way to implement this combination since the three models are kept separate and interact through shared variables. (Puget, 1992) reduced a locomotive scheduling problem to a tsp with a time window
avor given by the departure time of the trains. A simple greedy heuristic builds an initial solution which is then repeatedly improved within a descent method. Therefore, the nal solution is not necessarily optimal. Much more work has been done in operations research to develop both exact 2
and approximate algorithms for this problem. Branch-and-bound algorithms developed in (Christo des et al., 1981) and (Baker, 1983) have been reported to solve instances with up to 50 vertices but require either \moderately tight" time windows or little overlap between them. (Langevin et al., 1993) presented a branch-and-bound algorithm based on a two-commodity ow formulation and reported solving instances with up to 40 vertices. (Dumas et al., 1995) devised a dynamic programming algorithm which minimizes the total travel distance. They reported solving instances with up to 200 vertices and fairly wide time windows, though the need to work with integers asks for a truncation of the data. Their success comes from the use of rules that eliminate states and transitions from consideration during the execution of the algorithm. (Mingozzi et al., 1993) report better computational results than the previous authors on similar test problems by using a two-phase dynamic programming strategy. They introduce bounding functions in both phases in order to eliminate states which cannot lead to an optimal solution. First an iterative procedure based on \through circuits" yields either an optimal solution or a lower bound in which case a second iterative procedure is applied which ends with a possibly sub-optimal solution since some states are heuristically discarded. Turning to approximate algorithms, (Savelsbergh, 1985) originally proposed arc interchange heuristics to achieve good solutions for the minimization of the total travel distance and later extended his results for total schedule time (Savelsberg, 1992). (Gendreau et al., 1995) adapted the generalized insertion heuristic GENIUS (Gendreau et al., 1992) for the tsptw. They reported frequent optimal or near-optimal solutions on problem sets taken from (Langevin et al., 1993), (Dumas et al., 1995) and (Potvin et al., 1993). The rest of the paper is organized as follows. The next section presents clp, explains how it deals with constraints over nite domains and highlights some important di erences with other approaches. Section 2 formulates our constraint programming model for the tsptw while the following section presents redundant constraints which improve its e ectiveness. Throughout these two sections, we aim to present the model in a manner that lends itself well to a comparison with mathematical programming models. Section 4 describes the branch-and-bound algorithm. Computational results and comparisons with other algorithms are reported in section 5.
1 Constraint Logic Programming The purpose of this section is to give an overview of clp and in particular of some issues pertaining to constraints over nite domains. A more thorough exposition can be found for example in (Ja ar and Maher, 1994). This section also discusses other popular combinatorial optimization approaches in relation with constraint programming. Constraint logic programming arose from the generalization of the uni ca3
tion algorithm, which is at the core of logic programming, to a constraint solver (Ja ar and Lassez, 1986) (Ja ar and Lassez, 1987). This brought constraints to the forefront within a general purpose programming language (in fact, a whole family of programming languages). They not only became a primitive construct of the language but could also appear in both the input and output. The resulting programming languages are declarative and thus allow a high level of abstraction | the constraints of the problem are stated in a natural way and its solution is in turn formulated with constraints. Behind the scenes, algorithms have been developed to decide whether a feasible solution exists, to simplify the constraints, to deduce information from them (for example, from x + y 1 and x ? y 5 deduce that x 3), and so on. Di erent types of constraints (e.g. arithmetic constraints over the real numbers, equality constraints over trees, constraints over nite domains) usually call for di erent, specialized algorithms. The declarative nature of constraints made their integration into logic programming, a declarative programming paradigm, the natural choice. Nevertheless, they have since appeared in other paradigms and areas of computer science (Pujet, 1994) (Saraswat, 1989) (Kanellakis et al., 1995). The common ground is constraint programming, where the computation is driven by constraints, thus giving them an active role. For dicult problems involving search, this translates to guiding us as much as possible when faced with alternatives. Tackling combinatorial problems generally involves manipulating variables which can take a nite number of values. In constraint programming over nite domain variables, a domain is associated to every variable: each value in that domain represents a possible value for the variable. A constraint on two or more variables restricts the possible combinations of values for these variables. Several constraint programming languages have been designed to o er constraints over nite domains (Dincbas et al., 1988) (Pujet, 1994) (ECLiPSe, 1995) (Diaz and Codognet, 1993) (van Hentenryck et al., 1993). Since it is well known that deciding about the satis ability of a collection of such constraints is NP-complete, the constraint solver implements a compromise: it performs a polynomial amount of work but remains incomplete in the sense that unsatis ability may go unnoticed (though every unsatis ability reported is genuine, that is, no feasible solution is lost). Such an algorithm helps control the combinatorial explosion in dicult problems involving search and as such is often used within a trial-and-error framework. It was rst introduced in the early seventies for an arti cial intelligence application (Waltz, 1972) and has since given rise to the eld of constraint satisfaction problems (csp) which has focused on search and propagation techniques to solve general combinatorial problems (see for example (Tsang, 1993)). Picturing the constraints arranged in a network whose vertices are the variables and whose (hyper)edges are the constraints provides insight into the algorithm. A vertex will be labeled with the values in the domain of the corresponding variable. An edge will be incident to those vertices representing the 4
a
_ > 7, 11, 20
5, 19
X
Y
=/
1, 3, 5, 6 Z
b
_ > 7, 11, 20
5, 19
X
Y
=/
1, 3, 5, 6 Z
c
_ > 7, 11, 20
5, 19
X
Y
=/
1, 3, 5, 6 Z
d
_ > 7, 11, 20
5, 19
X
Y
=/
1, 3, 5, 6 Z
Figure 1: An example of constraint propagation variables appearing in the associated constraint. Looking locally at a particular constraint, the algorithm attempts to remove from the domain of each variable involved in that constraint values which cannot be part of any solution. This reduction of a variable's domain triggers the examination of all constraints involving this variable, which in turn may reduce other domains. This recursive process stops when either no new domain reduction has taken place or a domain becomes empty, in which case we learn that no solution exists. The overall behavior is called constraint propagation. Consider the example in gure 1. Let X 2 f7; 11; 20g; Y 2 f5; 19g; Z 2 f1; 3; 5; 6g; X Y; Y 6= Z; : : : be some of the constraints in a problem: the corresponding portion of the constraint network appears in (a). Suppose the value 11 has been selected for variable X , thus discarding the other two possible values. Since X 's domain has been modi ed, the relevant constraints (i.e. the incident edges) will be examined again to verify whether some of the values of other variables have become inconsistent (b). In particular, the constraint X Y causes the value 19 to be removed from the domain of Y since 11 6 19. As Y 's domain has been reduced, propagation to neighboring constraints is triggered (c). In particular, constraint Y 6= Z has the e ect of removing value 5 from Z 's domain since it has now become the only possible value for Y , which causes more propagation and so on (d). 5
The propagation algorithm sketched above can be implemented using a list of pointers to relevant constraints for each variable, a list of pointers to involved variables for each constraint, and a queue of active variables. A variable is put at the end of the queue when its domain is modi ed. The algorithm repeatedly removes a variable from that queue and processes all the relevant constraints. A constraint is processed by removing inconsistent values from the domain of each variable involved in it. Variables whose domain has been modi ed are then added to the queue, if they are not already there. An interesting property of the propagation algorithm is its con uence: regardless of the order in which variables and constraints are processed, in the end the domains have been reduced in the same way. Another property is that constraints can be dynamically added during the search for a solution. It may not appear obvious that the propagation algorithm ever terminates. Notice that propagation only shrinks domains: values are removed but never added. It follows that constraint propagation must terminate (in polynomial time) simply because there are nitely many (and polynomially many, in the number of variables and the size of the domains) values to remove. The e ectiveness and cost of such an algorithm can be adapted by deciding when a particular constraint is worth triggering. Three types of propagation events are generally recognized: the value event, the range event and the domain event. A value event on a variable occurs when that variable only has a single value left in its domain and so has been assigned that value. A range event on a variable occurs when either the minimum or maximum value in its domain has changed. A domain event on a variable occurs anytime its domain is modi ed. Take for example a disequality constraint such as X 6= Y on variables X; Y 2 f1; 2; 3g. It is sucient to trigger it by value events on either X or Y : once X (resp. Y ) has been assigned a value, we can remove that value from Y 's (resp. X 's) domain. Triggering that constraint on a domain event, such as the removal of value 1 from X 's domain, would be useless: there are still two possible values for X and nothing can be removed from Y 's domain. For a constraint such as X Y , range events are appropriate. Since constraint propagation is incomplete the solution process still requires search and its potentially exponential cost, especially if we are not simply looking for a solution but an optimal one. Propagation will continue to be useful in guiding that search.
1.1 Relationship with other approaches
The aim of this section is to highlight the complementary nature of constraint programming with respect to some of the other combinatorial optimization techniques. Because constraint propagation has most often been used within a branchand-bound scheme, the rst parallel that comes to mind for cp is with classical branch-and-bound algorithms. Though both start with a formal model, the na6
ture of the models will most likely greatly di er in preparation for the distinct solution strategies to be applied. Classical branch-and-bound algorithms most often relax the integrality constraints or possibly some other constraints so that the remaining model has a structure suitable for its resolution using well-known ecient algorithms; the solution strategy for cp does not relax constraints as it is far less sensitive to model structure and in particular works with the original integer variables. The use of relaxations in mathematical programming possibly yields infeasible solutions: to regain feasibility, solving relaxations becomes a building block within a problem decomposition approach such as branchand-bound and branch-and-cut. The incompleteness of constraint propagation possibly yields indeterminate solutions (when several possible values are left in the domain of a variable): to obtain a solution, constraint propagation almost invariably becomes a building block within a branch-and-bound search. In the second approach, at each node of the search tree a relaxation is solved which provides a lower bound on the optimal cost of our minimization subproblem; in the rst one, at each node the result of constraint propagation on a nite domain variable representing the cost of the solution provides a lower bound as well: the smallest value left in its domain. Because dynamic programming also works directly with discrete variables, it may seem akin to cp at rst glance. Dynamic programming methods essentially solve a recursion on a state space graph. Their main drawback is the quickly unmanageable size of the state space. Understandably, a lot of e ort has been put into reducing that size through state space relaxation and state elimination techniques. Constraint programming, with its usual depth- rst tree search approach, does not share such space-complexity concerns (like running out of computer memory) but su ers from potentially exponential time-complexity like all implicit enumeration methods. In contrast, dynamic programming resembles a breadth- rst graph search. There is no notion of a recursion in cp | its execution is rather driven by constraints. Dynamic programming does not have a formal model for the constraints of the problem whereas cp does in the same way as mathematical programming formulations. Our portrayal of cp has been that of an exact solution method but nothing prevents it from being combined with heuristic methods (see for example (Pesant and Gendreau, 1996)). This appears to be a promising area of research.
2 A constraint programming model We rst describe the constraint programming model for the tsp part of the problem. Let V = f2; : : :; ng represent the cities to visit and duplicate the unique depot into an origin-depot and a destination-depot, identi ed as 1 and n + 1, respectively. A tour thus becomes a Hamiltonian path starting at 1 and ending at n + 1. For convenience, de ne V o = V [ f1g, V d = V [ fn + 1g and V o;d = V [f1; n +1g. For each pair of cities i and j , let cij denote the travel cost 7
from i to j . Central to the model are variables Si ; i = 1; : : :; n associated with each city (and the origin-depot) and which represent its immediate successor in the tour. Their domain will therefore be an integer in the range 2; : : :; n + 1. A valid tour assigns a distinct successor to each city and avoids sub-tours. As values are given to the Si 's, corresponding partial (directed) paths are formed. Let i and i respectively denote the beginning and end of the current partial path through i; initially, i = i = i. A formal model for the TSP would be: Minimize
Xc 2
i Vo
subject to
i;Si
(1)
Si 6= Sj ; 8i; j 2 V o ; i 6= j (2) Si 6= i; 8i 2 V o (3) o (4) Si = j ) Sj 6= i ; (provided j 6= n + 1) 8i 2 V o Si 2 f2; : : :; n + 1g; 8i 2 V (5) Objective function (1) attempts to minimize the total cost of the tour: ci;Si refers to the travel cost from i to its immediate successor Si . Constraints (2) and (5) ensure that every city is visited exactly once (n variables assigned distinct values among n) while (3)-(4) are the sub-tour elimination constraints. Note the correspondence between value j in the domain of variable Si and 0-1 variable xij in a conventional network ow formulation of the problem. This model involves n variables and O(n2 ) constraints. Though it signi cantly departs from a usual mathematical model, particularly in its generous use of disequality constraints (6=), such a formulation is perfectly adapted to a clp setting. Constraint programming uses these constraints in an active way, removing inconsistent values from the domains of the variables when sucient information becomes available. The disequalities spell out the various ways in which values are eliminated. The conditional constraint at (4) is also dealt with very naturally in this computational paradigm since it gives the condition whose ful llment triggers the elimination of a value. We now explain how each constraint is used. (5) simply initializes the domains. (3) also acts at the very beginning by trivially removing i from the domain of each Si . (2) waits until Si or Sj becomes xed to some value k, at which time it eliminates k from the domain of the other variable (an example of value event propagation). Finally we come to (4): as Si is xed to j , the partial path ending at i becomes merged with the one starting at j . Since sub-tours are forbidden, we may then reason that the end of this new path, j , cannot be followed by the beginning of it, i (see gure 2). As a special case, if j = n +1, i.e. the path has reached the destination-depot, nothing more needs to be done. To the tsp we add a scheduling component in the form of a time window [ai; bi] associated with each city. As usual, if we arrive at a city before ai we are allowed to wait but we cannot arrive after the deadline bi . Let tij denote the 8
βi
i
j
εj
Figure 2: Dynamically forbidding a sub-tour travel time between cities i and j . We introduce variables Ti ; i = 1; : : :; n + 1 representing the time at which service begins at city i. Without loss of generality, let us x T1 at 0. A model for the tsptw adds the following constraints to (1)(5):
ai T i b i ; 8i 2 V d (6) o Si = j ) Ti + tij Tj ; 8i 2 V (7) (6) restricts service times to their respective time windows while (7) ensures the feasibility of the schedule by imposing a temporal constraint on each arc of the tour. The former establishes the initial domains of the Ti 's whereas the latter works during the search each time an arc is selected to possibly trim the appropriate domains. For example, given Ti 2 f0 : : : 10g, Tj 2 f3 : : : 11g and tij = 5, including arc (i; j ) in the tour will trim the upper limit of Ti 's domain down to 6 and the lower limit of Tj 's domain to 5. This may in turn propagate changes to the service time variables of neighboring cities on the tour. Note that model (1)-(7) and what will be added to it in section 3 remains general: data ai, bi, cij and tij are not required to be integers nor are the last two assumed to be symmetric or obeying the triangle inequality. The actual implementation will make such assumptions based on the test problems taken from the literature for our experimental work, as will be described in section 5.
2.1 A small example
In order to illustrate the inner workings of such a model, we will at this point apply it to a small example. Leaving aside the objective function, let us simply enumerate the feasible solutions. Consider three cities a; b; c plus depot d with the following travel time matrix and time windows:
9
travel time matrix a b c d a 0 1 3 2 b 1 0 1 2 c 3 1 0 1 d 2 2 1 0
time windows ai bi a 0 4 b 1 2 c 5 8 d 0 8
For clarity, the cities will be identi ed with letters in the model rather than integers. To distinguish between the origin-depot and the destination-depot, we will respectively use d and d0. We therefore have successor variables Sa , Sb , Sc , Sd and variables for the time of beginning of service Ta , Tb , Tc , Td , Td . (5) and (6) initially set the domain of Si to fa; b; c; d0g and the domain of Ti to the time window of city i (with Td = 0). The action of (3) eliminates some trivial values to yield: Sa 2 fb; c; d0g, Sb 2 fa; c; d0g, Sc 2 fa; b; d0g, Sd 2 fa; b; c; d0g. In order to observe the dynamic behavior of the remaining constraints, we will explore this small search space in a systematic way by assigning values to the Si 's ( gure 3 illustrates this exploration): 0
Starting with Sa = b, (2) removes value b from the domain of the other Si 's: Sc 2 fa; =b; d0g, Sd 2 fa; =b; c; d0g. (4) prevents the sub-tour ha; b; ai through the update Sb 2 fa=; c; d0g. Finally, (7) imposes the new constraint Ta + 1 Tb which trims the domain of Ta to f0 : : : 1g but leaves that of
Tb unchanged. { Going further with Sb = c, (2) produces Sd 2 fa; =c; d0g. (4) prevents the sub-tour ha; b; c; ai through the update Sc 2 fa=; d0g. (7) imposes the new constraint Tb +1 Tc which does not modify either domain. Now, Sc is automatically set to d0, which implies Sd = a (from (2)). Skipping a few inconsequential actions, we come to (7)'s reaction to Sd = a: the new constraint Td + 2 Ta annihilates Ta 's domain (f0 : : : 1g) since the l.h.s. of the inequality equals 2. We have come to a dead end. { Alternatively trying Sb = d0, (2) produces Sc 2 fa; d=0g and Sd 2 fa; c; d=0g. (4) does not act since d = d0 (i.e. n + 1). (7) imposes the new constraint Tb +2 Td which trims the domain of Td0 to f3 : : : 8g. Now, Sc is automatically set to a which, through (7), eventually introduces Tc + 3 Ta , which is violated by every combination of values in the domains of Tc and Ta . A new dead end that forces us to reconsider our initial choice of value for Sa since we have exhausted Sb 's domain. Going on to Sa = c, (2) produces Sb 2 fa; =c; d0g, Sd 2 fa; b; =c; d0g. (4) prevents the sub-tour ha; c; ai through the update Sc 2 fa=; b; d0g. Finally, 0
0
10
Sb = c :
(2) ) Sd 2 fa; =c; d g (4) ) Sc 2 fa=; d g (7) ) Tb + 1 Tc Sc = d ^ (2) ) Sd 2 fa; d=g Sd = a ^ (7) ) Td + 2 Ta ) Ta 2 ; dead end Sb = d : (2) ) Sc 2 fa; d=g, Sd 2 fa; c; d=g (7) ) Tb + 2 Td ) Td 2 f3 : : : 8g Sc = a ^ (7) ) Tc + 3 Ta ) Tc ; Ta 2 ; dead end 0
0
Sa = b : (2) ) Sc 2 fa; =b; d g, Sd 2 fa; =b; c; d g (4) ) Sb 2 fa=; c; d g (7) ) Ta + 1 Tb ) Ta 2 f0 : : : 1g 0
0
0
0
0
0
0
0
0
0
Sb = a : (2) ) Sd 2 fa=; b; d g (4) ) Sc 2 f=b; d g (7) ) Tb + 1 Ta ) Ta 2 f2 : : : 4g Sc = d ^ (2) ) Sd 2 fb; d=g Sc = d ^ (7) ) Tc + 1 Td Sd = b ^ (7) ) Td + 2 Tb : : : ) Ta 2 f3 : : : 4g, Tb 2 f2 : : : 2g, Tc 2 f6 : : : 7g, Td 2 f7 : : : 8g feasible solution: hd; b; a; c; d i Sb = d : (2) ) Sc 2 fb; d=g, Sd 2 fa; b; d=g (7) ) Tb + 2 Td ) Td 2 f3 : : : 8g Sc = b ^ (7) ) Tc + 1 Tb ) Tc ; Tb 2 ; dead end 0
0
0
0
0
Sa = c :
(2) ) Sb 2 fa; =c; d g, Sd 2 fa; b; =c; d g (4) ) Sc 2 fa=; b; d g (7) ) Ta + 3 Tc 0
0
0
0
0
0
0
0
0
0
Sb = a : (2) ) Sc 2 fa=; bg, Sd 2 fa=; b; cg Sc = b ^ (7) ) Tc + 1 Tb ) Tc ; Tb 2 ; dead end Sb = c : (2) ) Sd 2 fa; b; =cg (4) ) Sc 2 fa; =bg (7) ) Tb + 1 Tc Sc = a ^ (7) ) Tc + 3 Ta ) Tc ; Ta 2 ; dead end
Sa = d : (2) ) Sb 2 fa; c; d=g, Sc 2 fa; b; d=g, 0
0
Sd
0
2 fa; b; c; d=g 0
(7) ) Ta + 2 Td ) Td 0
0
0
2 f2 : : : 8g
Figure 3: Exploring the search space 11
(7) imposes the new constraint Ta + 3 Tc which leaves the domains unchanged. { Going further with Sb = a, (2) produces Sd 2 fa=; b; d0g. (4) prevents the sub-tour hb; a; c; bi through the update Sc 2 f=b; d0g. (7) imposes the new constraint Tb +1 Ta which trims Ta to f2 : : : 4g. Note that this modi cation should cause the re-examination of Ta +3 Tc but nothing new would come out of it. Now, Sc has indirectly been set to d0, which from (2) provokes Sd = b. Repeated action of (7) introduces Tc + 1 Td and Td + 2 Tb whose combined e ect, together with the two others of the same type, eventually yields Ta 2 f3 : : : 4g, Tb 2 f2 : : : 2g, Tc 2 f6 : : : 7g, Td 2 f7 : : : 8g. We have found a feasible solution: hd; b; a; c; d0i. { Pursuing with Sb = d0, (2) produces Sc 2 fb; d=0g, Sd 2 fa; b; d=0g. (4) does not act. (7) imposes the new constraint Tb + 2 Td0 which trims Td to f3 : : : 8g. Sc is automatically set to b which eventually introduces Tc + 1 Tb (through (7)) which cannot be satis ed. Finally trying Sa = d0, (2) produces Sb 2 fa; c; d=0g, Sc 2 fa; b; d=0g, Sd 2 fa; b; c; d=0g. (4) does not act. (7) imposes the new constraint Ta + 2 Td0 which trims Td to f2 : : : 8g. { Choosing Sb = a, (2) produces Sc 2 fa=; bg, Sd 2 fa=; b; cg. We see that Sc becomes xed and that eventually the introduction of Tc + 1 Tb stops us short. { The last alternative is Sb = c. From (2) we get Sd 2 fa; b; =cg; from (4), Sc 2 fa; =bg. (7) imposes Tb + 1 Tc which yields no update. The value of Sc can now only be a and a dead end is reached with Tc + 3 T a . As we have just experienced, following every step of the constraint propagation is a tedious task. It will not be attempted later on the bigger model or larger examples, though now and then we will come back to this simple instance to illustrate further points. Throughout this example, we applied the various steps in a particular order but one should realize that the nal outcome does not depend on that order and that it only a ects the eciency of the propagation. 0
0
0
0
3 Adding redundant constraints Though (1)-(7) constitute a valid model for the tsptw and (2)-(7) exactly describe the set of feasible solutions, additional constraints, though redundant, may improve the logical reasoning and drastically reduce the solution space. In fact, the reader may have realized that the sub-tour elimination constraints 12
12 5 Ti ε [0,100]
12 5
Tj ε [0,100]
5 Tk ε [0,100]
Ti ε [0,90]
(a)
5 Tj ε [5,95]
Tk ε [10,100]
(b)
Figure 4: A sub-tour undetected by temporal constraints (4) are no longer needed once we have (7). On the other hand, keeping (4) is advantageous. For example, consider cities i; j; k with time windows and travel times as illustrated in gure 4(a). Choosing arcs (i; j ) and (j; k) causes (7) to trim the windows as in gure 4(b) but does not immediately disallow arc (k; i). Only when we branch on that arc will the illegal sub-tour be detected. Our inspiration for redundant constraints comes from the operations research literature but carries them further. They will not only be used before but, more importantly, during the search. The next two sections respectively describe dynamic arc elimination and dynamic time window reduction.
3.1 Arc elimination
Firstly, we may better exploit the temporal constraints. An arc (i; j ) is admissible only if ai + tij bj : We shall incorporate this into the model in a slightly di erent form freeing it from simply an a priori static reduction of the number of arcs. Conditional constraint
Ti + tij > Tj ) Si 6= j; 8i; j 2 V o;d ; i 6= j (8) is used to discard inadmissible arcs both a priori and throughout the execution of an algorithm as the domains of the Tk 's are trimmed and their smallest and largest value consequently change. (Note that although (8) is logically equivalent to (7) (it is its contrapositive), their operational meanings di er in that the latter is triggered when a Si variable becomes bound whereas this new one is triggered upon satisfaction of a condition on Ti variables. Therefore, each conditional constraint plays a di erent role.) Let Tkmin and Tkmax denote the smallest and largest value in the current domain of Tk , respectively. It suces to consider Timin and Tjmax when checking the antecedent in (8). Certainly, if Timin + tij > Tjmax holds then Ti + tij > Tj is veri ed. (Note that this would not be powerful enough to detect the potential sub-tour in gure 4 and so could not replace (4) either.) 13
Rules are introduced in (Langevin et al., 1993) to eliminate additional arcs from consideration. They stem from the simple observation that every city must appear either before or after a given arc in the tour:
8 admissible arc (i; j ); 8k 6= i; j : (Tk + t?ki Ti ) _ (Tj + t?jk Tk ); where t?ab stands for the shortest time path from a to b. It proves more evocative (and powerful) in our model to express them in the following way:
Ai \ Bj 6= ; ) Si 6= j;
8 i 2 V o ; j 2 V d ; i 6= j
(9)
where sets
Bi = fk 2 f1; : : :; n +1g j Timin + t?ik > Tkmax g Ai = fk 2 f1; : : :; n +1g j Tkmin + t?ki > Timax g contain cities which must be visited respectively before and after city i, because of time window restrictions. Again they come into play whenever the bounds on the domains of the service time variables are modi ed. We will state separately two special cases: B1 = ;; A1 = f2; : : :; n +1g; Bn+1 = f1; : : :; ng; An+1 = ;: On the small example of the previous section, a priori usage of (8) would already remove a; b from the domain of Sc while (9) would disallow d0 for Sa ; Sb; Sd and c for Sd , thus greatly reducing the search e ort.
3.2 Time window reduction
(Desrochers et al., 1992), while dealing with the more general vrptw, describe rules for time window reduction. Here the idea is to consider the set of possible successors and predecessors of each city in order to tighten the upper and lower bounds of its service time variable. Like the previous rules, they implement a global constraint encompassing a collection of cities rather than merely looking locally at a single arc, or pair of cities. What we gain from this more global view is increased pruning capabilities. It will prove useful for the formulationof the constraints corresponding to the rules in (Desrochers et al., 1992), and later on in section 4 as well, to introduce predecessor variables Pj ; j = 2; : : :; n + 1 which represent the symmetrical counterpart of the Si 's. Let ffX gg denote the domain of variable X . We link the Pj 's to the model as follows:
j 2 ffSigg , i 2 ffPj gg Pi 6= Pj ;
8 i 2 V o; j 2 V d 8i; j 2 V d ; i 6= j 14
(10) (11)
Observe that the counterpart of (3)-(5) are not required since (10) subsumes them. On the other hand, we add (11) so that we can infer Si = j from Pj = i, a powerfully binding consequence. Here is an instance of it, using the example of section 2.1. The domain of each Si is as it appeared before we started assigning values. Applying (10) yields Pa 2 fb; c; dg, Pb 2 fa; c; dg, Pc 2 fa; b; dg, Pd 2 fa; b; c; dg. Fixing Pb to a, we would like to infer that Sa = b. We label each reasoning step with the appropriate line in the model: 0
Pb = a ) c; d 62 ffPbgg (10) ) b 62 ffSc gg; ffSd gg and without (11) we can go no further. But with it:
) a; c; d0 62 ffSa gg ) Sa = b; Pb = a (11) ) a 62 ffPa gg; ffPcgg; ffPd gg (10) the desired result. We are now ready to state the time window reduction constraints: Ti k2f min (T + t ) 8i 2 V d (12) fPigg k ki Ti k2f max (T ? t ) 8i 2 V o (13) fS gg k ik 0
i
(12) says that service at i cannot possibly start before the earliest arrival from one of its potential predecessors and (13) says something similar about the latest departure from i to its potential successors. (Though four rules are given in (Desrochers et al., 1992), the other two, while reducing the size of the time windows, have little impact on constraint propagation.) There are several ways in which these constraints may become violated in the course of an algorithm. Consider (12): 1. If Timax decreases below the value of the r.h.s.. 2. If Tkmin changes (i.e. increases), where k0 was the index minimizing the r.h.s. and therefore yielding the bound on Ti . The new value Tkmin + tk i (or alternatively Tkmin + tk i if k00 becomes the new minimizing index) may then exceed Timax . 3. If k0 no longer belongs to the domain of Pi and so a tighter bound will be derived from some other k. As reported in (Desrosiers et al., 1995), the time windows can be further reduced in the case of the tsptw. Instead of looking at direct successors and predecessors, we turn to all cities which must be visited before and after a given one. The time at which we visit city i must necessarily follow the maximum of 0
0
00
00
15
0
Tk + t?ki over all k among Bi and similar reasoning about Ai leads to: Ti max (Tk + t?ki) 8i 2 V d k 2Bi Ti kmin (T ? t? ) 8i 2 V o 2Ai k ik
(14) (15)
The nal model, (1)-(15), involves 3n + 1 variables and O(n2 ) constraints.
4 A branch-and-bound algorithm Though the constraints of the model contribute to signi cantly reduce the number of arcs to consider, search of course still plays a major role. The natural way to go about it in clp is via a branch-and-bound algorithm which actually di ers in a number of ways from its operations research namesake | we will therefore describe the clp version in the following section. Since it is a generic scheme, we developed various strategies whose combination attempts to increase the robustness of the algorithm. Robustness has been a driving concern as the success of such an approach is heavily dependent on the non-deterministic choices to be made. One may be very lucky and quickly nd the optimal solution to one problem | we strive to be \lucky" most of the time.
4.1 Branch-and-bound a la clp
In a clp setting, we work directly with the model and no relaxation is sought. The discrete variables remain so and logical reasoning steps are applied to them through constraint propagation. To progressively increase the amount of propagation and thus reduce the domains further, decision variables are successively xed to some value in their domain while maintaining lower and upper bounds on the objective value. The lower bound is updated at each node of the search tree and its nature will be discussed in section 4.4 | the upper bound corresponds to the value of the best feasible solution found so far (an initial upper bound may also be derived from a known solution). The choice of which variable to set to what value may be unlucky and force us to try another option. This non-deterministic process, simulated by backtracking, yields a depth- rst tree search in which pruning will take place if some variable has no value left in its domain or if the bounds indicate that we cannot hope to improve on the best solution found so far.
4.2 Variable and value selection heuristics
In this section we address the order in which variables are xed and their values tried. Variable ordering is by far the most important since it can drastically alter the shape of the search tree. Value ordering does not a ect the topology of the tree but rather which parts of the tree are rst explored and ultimately 16
how soon a very good or even optimal solution will be found. (One could argue that pruning through bounds is sensitive to the order in which parts of the tree are explored and so value ordering a ects topology in that way.) Much recent research is being devoted to these concerns in the related discipline of csp (section 1). Dynamic variable ordering is generally recognized as a key factor of success: rather than xing an ordering a priori, the most promising variable is chosen according to the information available at that moment. Within this scheme, the rst-fail principle (Haralick and Elliott, 1980) seems to emerge as a good strategy. It states that one should provoke dead-ends as early as possible to reduce the search e ort. This usually translates in favoring the most constrained variable | the one with the smallest domain and involved in the greatest number of constraints. This primarily domain size heuristic also has the advantage of avoiding bushy nodes which run the risk of scattering the e ort in a multitude of small portions of the search space right from the start. We follow these guidelines but specialize them for the particular problem at hand. We developed the sparse heuristic, a modi cation of the previous one which tries to build a sparse search tree. Small domains are still favored but in case of a tie, rather than selecting the one which participates in the greatest number of constraints overall, we adopt a more nearsighted attitude and choose the one that is likely to induce the most important domain reduction among its peers (i.e. of same small domain size). More formally, for the successor variables: 1. Let s be the size of the smallest domain among the Si 's. Collect V = fSi ; i = 1; : : :; n : jffSi ggj = sg. 2. If s = 1 then arbitrarily choose a variable among V . 3. Otherwise: S (a) For each element e in v2V ffvgg, compute e# , the number of occurrences of e in the domains of variables in V . P (b) Choose the variable which maximizes f (v) = e2ffvgg e# . The choice of f was in uenced by the nature of the model: Si and Sj (i 6= j ) cannot take the same value (by (2)) and thus xing one of them to a popular value forces its removal from the domain, already small, of several others. Since every value of the selected variable will be tried, taking the sum of the occurrences seemed a better measure than, say, the maximum. Both the Si 's and Pj 's are actually used as decision variables. The previous variable selection heuristic is independently applied to the successor and predecessor variables, and we branch on the best of the two winners. (We would not be justi ed to apply it to the two types of variables together as the mutual exclusion holds solely within each type.) This adds robustness to the overall algorithm since there may be times when the predecessor variables are more 17
3 1
2
5 4
Figure 5: A topology favoring the choice of P2 constrained than the successor variables and help keep the search tree sparse. For example, gure 5 illustrates possible arcs for a Hamiltonian path through ve cities. Here P2 has a single possible value whereas each of S1 ; S2 ; S3; S4 has two. The advantage of a symmetrical model has also been argued in (Caseau and Koppstein, 1992). While we look for \bad news" when selecting a branching variable ( rst-fail), we want \good news" when selecting a branching value. A sensible heuristic tries values in increasing order of cij in the hope of discovering cheap (i.e. good) solutions early to improve the upper bound on the objective value.
4.3 Dichotomic exploration of large domains
When the previous heuristic nevertheless leads to a variable with a large domain, we select a group of branches at a time. (A parallel can be drawn with branching on either side of a fractional value for an integer variable in traditional branchand-bound using linear programming relaxations.) If the size of the domain lies beyond a certain threshold, the domain is split in the middle and each half examined in turn, rather than selecting values individually. We do this in the hope that each cut propagates a sucient amount of information to eliminate or at least shrink the corresponding subtree; the overhead is small and in the worst-case we end up branching on individual values.
4.4 Lower bounds on the total cost
In keeping with our goal of robustness, we maintain two lower bounds on the objective value within our branch-and-bound strategy. They correspond to the sum of the edge costs of the closest neighbor (cn) and minimum spanning tree (mst) subgraphs, both of which underestimate the minimum Hamiltonian path. The cn bound is derived directly from the objective function (1). Let Ccn
18
be a nite domain variable tied to the model by the constraint
Ccn =
Xc 2
i Vo
i;Si
making it the sum of the costs of the arcs selected for the tour. As long as some Si remains indeterminate because its domain still contains several values, so will Ccn but its smallest possible value provides a lower bound on the total cost. Whenever the domain of some Si is reduced, the propagation through the constraint above implicitly updates the domain of Ccn accordingly. To illustrate this, consider again the example of section 2.1, identifying cost with travel time (we will ignore time windows here). For each city, a value must be chosen in its corresponding row of the matrix. Initially, Ccn has for domain f4; 5; 6; 7; 8; 9; 10g which represents the possible outcomes of the summation involving one element from each of f2; 2; 1g, f1; 3; 2g, f1; 1; 2g and f3; 1; 1g. Suppose solution hd; a; b; c; di with cost 5 has already been found and that we are now investigating partial solution hd; b; : : :i. At this stage, the remaining free successor variables Sa , Sb and Sc have respective domains fc; dg, fa; cg and fa; dg, thus propagating changes to the domain of Ccn and reducing it to f6; 7; 8; 9g (the elements of the sum are henceforth taken from f2g, f3; 2g, f1; 1g and f3; 1g). The derived lower bound, 6, exceeds the current best cost, 5, and so the partial solution can be abandoned. One easily realizes that the smallest value for Ccn comes from the sum of the smallest value in each of the Si 's domain whence its relationship with closest neighbors. The complementary mst bound also implicitly tightens as the domains of the Si 's shrink (reducing the number of arcs in the graph) and in particular as the variables become xed (imposing a skeleton for the mst). To avoid redundant computations, an incremental version of Kruskal's algorithm (Aho et al., 1983) was implemented. Instead of starting from scratch at each node of the search tree, we complete the skeleton from the parent node. Note that the construction is triggered by domain modi cations and requires no explicit follow-up, as in the case of the other bound.
5 Experimental results
Our branch-and-bound algorithm was implemented in ECLiPSe, a Prolog-based programming language equipped with a solver for constraints over nite-domain variables (ECLiPSe, 1995), and run on a Sun SS1000 (8xSuprSP, SPECfp92 123.1). A limit of one day of computation was allowed for each problem instance. Test problems were taken from the literature, namely the tsptw instances in (Dumas et al., 1995) and single-vehicle decompositions of Solomon's vrptw instances in (Solomon, 1987). In both cases, the travel cost and travel time between cities i and j (at location (xi ; yi) and (xj ; yj ) resp.) correspond to the 19
n
jA0 j optimal number of CPU time w (this paper/(Dumas et al., 1995)) value backtracks (seconds)
20 20 40 60 80 100 40 20 40 60 20
33.0/34.2 64.6/72.2 115.4/115.2 143.0/143.8 208.2/207.0 108.8/121.8 207.0/230.2 195.2/226.0
361.2 316.0 309.8 311.0 275.2 486.6 461.0 581.6
4.4 52.6 300.0 2741.4 1572.4 221.0 30345.6 100932.8
Table 1: Computational results on some of the version 1 TSPTW problems of Dumas et al. Euclidean distance separating them (i.e. cij = tij = t?ij = [(xi ? xj )2 + (yi ? yj )2 ] 21 ). The threshold value was set to 5 for the dichotomic exploration of large domains (section 4.3). The rst set of problems consists of groups of ve instances for various combinations of number of cities (n) and maximum width of time windows (w) (see (Dumas et al., 1995) for details). The diculty of the problems naturally increases with n, as well as with w for a xed n. As mentioned earlier, the dynamic programming algorithm described in (Dumas et al., 1995) requires integer data and to accommodate this they do not keep decimal places for the distances. Table 1 reports our computational results on some of their problems with such truncated distances (we will call these version 1 problems). Each entry represents an average over ve instances. As their algorithm is also optimal, it comes as no surprise that the optimal values in column four coincide with their values. In the notation of (Dumas et al., 1995), jA0 j represents the size of the reduced network after preprocessing, using much of the same rules that inspired our redundant constraints. In our case, it is computed as the sum of the domain sizes over the Si 's after the initial propagation of the constraints of the model but before the branch-and-bound search is actually started. The third column of the table gives our jA0 j followed by theirs. Though they are close, those values do not correspond exactly: the di erence of mechanics between constraint propagation and the iterative application of each rule in some xed order may be responsible for the discrepancy. Our values are generally lower for small w. The fth and sixth columns respectively give the number of backtracking steps performed in the search tree and the computation time. On the n=20 instances, our computation times are already two to three orders of magnitude larger than those reported in (Dumas et al., 1995) on the same instances and in (Mingozzi et al., 1993) on similar problems, both dynamic programming algorithms relying on a discretization of the data. Such a gap lies beyond what can be attributed to the di erent computers and programming languages used. 20
1.6 3.7 20.6 212.2 111.7 19.2 2614.2 10759.2
50
0
18.71 [9,21]
26.95 [24,42]
22.95 [3,23] 9.22 [2,13]
21.95 [9,21]
25.11 [24,42]
17.70 [3,23] 9.22 [2,13]
50
Figure 6: Two di erent optimal tours (left), depending on the precision of the data. Enlargement of the two distinct segments (right) gives the time window and time of service (in bold) for each city. The segment in dashed line violates time window [9; 21]. However, there is a de nite danger in truncating the original data as one then solves a di erent problem whose optimal solution may not be optimal or even feasible in the original setting. Table 3 reports additional computational results on the same set of problems, this time truncating the Euclidean distances after four decimal places (version 2). The optimal values are consistently higher than the ones in the previous table, thus con rming the presence of optimal solutions for version 1 problems which become infeasible when more accurate distances are used, as in version 2. For example, gure 6 illustrates our version 2 optimal tour (in solid line) for one instance of the n=20;w=20 set and what is di erent (in dashed line) in the version 1 optimal tour: in this case, two consecutive cities have been interchanged to decrease the overall cost but this actually violates the time windows. Table 2 gives the solution cost of each using Euclidean distances truncated after four decimal places in both cases. Worse, the third column of the table reveals that some of the instances (in fact, about half of the w=20 instances) have no feasible solution. One of the n=80;w=20 and three of the n=60;w=40 instances could not be solved within the time limit. Accordingly, we may only safely say that at least two of the 21
optimal tour version 1 version 2
cost
389.13 390.97
Table 2: Solution cost of optimal tours using same precision of data
n
20
feasible w instances
20 40 60 80 100 40 20 40 60 60 20 40 80 20 100 20
4 5 5 5 5 3 5 5 3 2 to 5 2 or 3 1
jA0j
25.5 56.4 109.6 140.2 205.4 100.0 196.6 370.2 159.3 380.0 288.7 336.0
optimal number of CPU time value backtracks (seconds) 363.0 328.2 317.7 329.2 281.5 497.2 477.6 432.2 588.2 y593.3 y713.3 775.2
1.5 41.4 217.4 1515.0 1337.6 59.0 12015.8 227712.0 1755.7 y86636.0 y6399.0 441216.0
1.8 3.5 14.4 125.9 102.2 10.3 1111.1 31386.3 134.5 y12189.7 y511.9 18003.5
Table 3: Computational results on some of the version 2 TSPTW problems of Dumas et al. (y: over instances which were solved to completion)
n=60;w=40 instances are feasible and that at least two but at most three of the n=80;w=20 instances are feasible.
The number of states generated in dynamic programming algorithms is exponentially related to the granularity of the data (time windows and travel times), even though some state elimination techniques may be applied. One can only speculate as to what the computation times would become if an appropriate precision in the data were used for these problems. In contrast, our algorithm does not su er from an increase of precision in the manipulated data | oddly enough, it even seems to be bene cial, aside from the obvious gain in correctness. Indeed, better performance (number of backtracks or cpu time) is observed in table 3. The higher optimal value and lower jA0 j may be an indication that the original problem is more constrained. The vrptw instances in sets RC2 and C2 of (Solomon, 1987) were used to generate our second set of problems. The original problem instances feature 100 cities and require several vehicles to service them while obeying the side constraints. The C2 set features cities arranged in clusters whereas the RC2 set contains a mixture of clustered and uniformly distributed cities. A partition of the cities where each group is visited by a single vehicle yields a 22
instance c201 c202 c203 c204 c205 c206 c207 c208
jA0j
38.3 516.3 862.3 1076.7 227.7 365.7 448.7 489.7
solution number of CPU time value backtracks (seconds) 591.56 591.56 591.17 ??? 588.88 588.49 588.29 588.32
0 603 12125 | 228 769 15088 1654
5.3 127.8 3705.3 1 day 16.4 47.0 1131.3 126.2
Table 4: Computational results for the C2 set number of smaller tsptw problems which will provide our test instances. The partition used for C2 simply assigns a di erent vehicle to each cluster; for RC2 we borrowed from the results of (Taillard et al., 1995) (for problems RC201, RC202, RC205, RC206 and RC207) and (Rochat and Taillard, 1995) (for problems RC203, RC204 and RC208) the partition yielding the best known solution. The number of cities in each subset varies from 16 to 49. (Solomon, 1987) also describes other problem sets but they did not meet our needs: type 1 problems (C1, R1, RC1) have too few cities per route to be interesting to us whereas the R2 set often has too many to suit our time limit. These vrptw instances, as well as their tsptw subproblems, are known to be quite dicult. To our knowledge, heuristic approaches have so far been the only e ective tool to consistently obtain good solutions. Tables 4 and 5 summarize the application of our optimal algorithm to the tsptw instances, again keeping four decimal places. We used initial upper bounds on the optimal value corresponding to best known solutions for these instances. In table 4, each entry represents an average over the clusters found in these problems (except for the solution value, which is naturally the sum). It is remarkable that none of the subproblems in c201 required any backtracking, thus highlighting their very constrained nature. The solution values found coincide with what has been achieved by several independent sources in the literature. Nonetheless, this has the merit to con rm that these solutions are optimal under the assumption that each cluster must be visited by a di erent vehicle. The computing times are quite reasonable, with the exception of c204 on which we could not terminate within our time limit. In table 5, an instance 'rc20x.i' corresponds to the group of cities in route i of the best known solution to problem rc20x. The fourth column lists the upper bounds used, obtained by rounding upward at the rst decimal place the solution value of each route. The other column labels are the same as those found in previous tables. Optimal values preceded by y improve on the best known values. When this is the case, it leads to an improvement over the best 23
100000 C2 RC2
CPU time
10000
1000
100
10
1 0
5 10 15 20 25 30 35 40 average out-degree after initial propagation
45
Figure 7: How computing time is a ected by the constraints known solution for the corresponding vrptw instance, as reported in table 6 (the actual solutions can be found in the appendix). Four out of the eight problems were thus better solved. ((Thangiah et al., 1994) report a value of 1294 for instance rc201, which seems improbable.) The values in table 5 that are not improvements bear witness to the quality of the routing component of the heuristics from which these values were obtained in the rst place. The next ve graphs analyze di erent aspects of our algorithm. Note that in each of them the ordinate scale is logarithmic. There is a strong relationship between the performance of our algorithm and the proportion of cities with time windows. Those proportions for the RC2 problems are 25% (rc204), 50% (rc203), 75% (rc202) and 100% (rc201, rc205, rc206, rc207 and rc208, the last four with increasing average width of time windows). As our computational paradigm is driven by constraints, the more of them the better. jA0 j re ects to some extent how much an instance is constrained. If we plot the computing time against the ratio of the latter quantity to the number of cities (yielding the average out-degree) as in gure 7, that relationship is hinted (the points on the ceiling of that graph require at least that much time). One novelty of our branch-and-bound algorithm has been the sparse heuristic for variable selection (in section 4.2). Though it only diverges from the usual heuristic when a tie between variables of equal domain size needs to be broken, this di erence alone can bring considerable savings. Figure 8 compares the two 24
instance n rc201.0 rc201.1 rc201.2 rc201.3 rc202.0 rc202.1 rc202.2 rc202.3 rc203.0 rc203.1 rc203.2 rc204.0 rc204.1 rc204.2 rc205.0 rc205.1 rc205.2 rc205.3 rc206.0 rc206.1 rc206.2 rc207.0 rc207.1 rc207.2 rc208.0 rc208.1 rc208.2
25 28 28 19 25 22 27 26 35 37 28 32 28 40 26 22 28 24 35 33 32 37 33 30 44 27 29
upper jA0j bound
191 238 231 128 446 301 398 438 1084 1201 561 937 740 1472 387 232 384 306 697 696 680 1076 867 770 1980 756 870
optimal number of CPU time value backtracks (seconds)
378.7 378.62 376.6 y374.70 428.0 y427.65 232.6 232.54 246.3 246.22 206.6 206.53 345.1 y341.77 369.8 y367.85 384.8 ??? 357.3 ??? 337.6 ??? 221.5 ??? 206.4 ??? 379.0 ??? 251.7 251.65 271.3 271.22 442.5 y 436.64 362.9 y361.24 495.8 ??? 334.8 ??? 335.4 335.37 439.6 y 436.69 396.4 396.36 246.5 246.41 381.1 ??? 239.1 ??? 214.0 ???
607 4034 4879 77 32215 19526 5155 65393 | | | | | | 4776 1329 | 90982 | | 301599 | | 62905 | | |
51.1 311.0 261.8 6.5 4163.0 1803.9 696.2 11107.7 1 day 1 day 1 day 1 day 1 day 1 day 652.4 128.1 1 day 7026.2 1 day 1 day 67543.1 1 day 1 day 6790.6 1 day 1 day 1 day
Table 5: Computational results for subproblems of the RC2 set (y: improvement on best known value)
instance rc201 rc202 rc205 rc207
solution previous best value (Taillard et al., 1995) 1413.52 1162.37 1320.75 1079.45
1413.79 1164.25 1328.21 1082.32
Table 6: New best solutions for the RC2 problems 25
1e+06
number of backtracks
ffs ffc
100000
10000 rc202.0 rc202.1 rc202.3 rc205.3 rc206.2 rc207.2
Figure 8: Gain of using the sparse heuristic ( s) instead of the conventional variable selection heuristic ( c) strategies on the most dicult RC2 subproblems that could be solved within a day of computation. The computation of lower bounds on the cost of partial solutions helps to prune the search tree as long as a good upper bound on the total cost is known. This will be the case once a good solution has been found but it may happen late in the search. Initial upper bounds like the ones appearing in table 5 have been crucial in improving the solution value of some of the more dicult instances. This is already clear on the simpler instances, as illustrated in gure 9. The numerous redundant constraints introduced in section 3 also contributed to a signi cant reduction of the search e ort. Figures 10 and 11 show the e ects of gradually removing some of these constraints from the model. Though a corresponding increase in the number of backtracks can be observed, there often seems to be situations where the computation time is lower without the full model. This does not come as a surprise since we work harder (i.e. more propagation takes place) at every node of the search tree when the full model is used. On the easiest instances, such as rc201.0-3 in gure 11, that extra e ort may not seem worth it but on the harder instances, such as rc202.3, where eciency becomes crucial, the e ort does pay o in terms of both performance measures. As the size of the search tree grows, it is generally better to spend more time at a node in order to prune potentially large subtrees. A more careful management of these constraints may also lessen the propagation overhead. 26
100000
number of backtracks
with upper bounds without upper bounds 10000
1000
100
10 rc201.0
rc201.1
rc201.2
rc201.3
Figure 9: The e ect of good initial upper bounds on the optimal value 1e+06 full model without (15),(16) without (13)-(16) without (9),(13)-(16)
number of backtracks
100000
10000
1000
100
10 rc202.3 rc201.0 rc201.1 rc201.2 rc201.3
Figure 10: The e ect of redundant constraints on the number of backtracks 27
100000 full model without (15),(16) without (13)-(16) without (9),(13)-(16)
CPU time
10000
1000
100
10
1 rc202.3 rc201.0 rc201.1 rc201.2 rc201.3
Figure 11: The e ect of redundant constraints on the computation time
Conclusion We demonstrated how constraint programming can be used to solve dicult combinatorial problems like the tsptw. One important lesson has been to borrow and transpose operations research expertise into our constraint model. The resulting exact algorithm showed advantages over the fundamental shortcoming of other optimal methods based on dynamic programming since the degree of discretization applied to the data has no major impact on its performance. Another important advantage of constraint programming is that it is far less sensitive to model structure. Consequently the addition of ad hoc constraints, which are so pervasive in real-life problems, rarely demands a reconsideration of the algorithm, as is often the case with traditional mathematical programming algorithms. In addition, our model may be easily generalized in several ways: allowing multiple time windows is straightforward since it corresponds to removing particular ranges of values inside the domain of our service time variables; adding precedence constraints is equally easy since requiring that i be visited before j is modeled by constraint Ti < Tj . Our algorithm obtained better results than heuristics on several dicult problems in the literature. It provided new best solutions for Solomon's vrptw problems in set RC2 and strengthened the presumption of optimality for most of the best known solutions in set C2. The computation times can be somewhat large but it must be noted that this implementation was coded in a Prolog-like 28
language as opposed to a fast imperative language such as C. As we have seen, the algorithm can also be used as a post-optimization procedure for a vrptw heuristic in order to optimize each individual route. We are currently seeking improvements to our algorithm's behavior on problems where the time windows have little impact, such as c204 and rc204. We are also investigating the combination of our constraint model with local search methods.
Acknowledgments We wish to thank three anonymous referees and Helmut Simonis for their constructive criticism which improved the quality of this paper. Financial support for this research was provided by the Natural Sciences and Engineering Research Council of Canada (NSERC).
References A. V. Aho, J. E. Hopcroft, and J. D. Ullman, Data Structures and Algorithms, Addison-Wesley, 1983. E. Baker, \An Exact Algorithm for the Time Constrained Traveling Salesman Problem," Operations Research 31, 938{945 (1983). P. Baptiste, C. Le Pape, and W. Nuijten, \Incorporating Ecient Operations Research Algorithms in Contraint-Based Scheduling," In Proceedings of the First International Joint Workshop on Arti cial Intelligence and Operations Research, Timberline Lodge, Oregon, 1995.
J. Carlier and E. Pinson, \An Algorithm for Solving the Job Shop Problem," Management Science 35(2), (1989). Y. Caseau and P. Koppstein, \A Rule-Based Approach to a Time-Constrained Traveling Salesman Problem," In Symposium on Arti cial Intelligence and Mathematics, Annals of Mathematics and Arti cial Intelligence, 1992. Y. Caseau and F. Laburthe, \Improved CLP Scheduling with Task Intervals," In P. van Hentenryck, editor, Proceedings of the 11th International Conference on Logic Programming, MIT Press, 1994. N. Christo des, A. Mingozzi, and P. Toth, \State Space Relaxation Procedures for the Computation of Bounds to Routing Problems," Networks 11, 145{164 (1981). M. Desrochers, J. Desrosiers, and M. Solomon, \A New Optimisation Algorithm for the Vehicle Routing Problem with Time Windows," Operations Research 40, 342{354 (1992). 29
J. Desrosiers, Y. Dumas, M.M. Solomon, and F. Soumis, \Time Constrained Routing and Scheduling," In M.O. Ball, T.L. Magnanti, C.L. Monma, and Nemhauser G.L., editors, Network Routing, volume 8 of Handbooks in Operations Research and Management Science, pages 35{139, North-Holland, Amsterdam, 1995. D. Diaz and P. Codognet, \A Minimal Extension of the WAM for clp(FD)," In Proceedings of the 10th International Conference on Logic Programming, pages 774{790, 1993. M. Dincbas et al., \The Constraint Logic Programming Language CHIP," In Proceedings of the International Conference on Fifth Generation Computer Systems, pages 693{702, Tokyo, 1988, Ohmsha Publishers. Y. Dumas, J. Desrosiers, E . Gelinas, and M.M. Solomon, \An Optimal Algorithm for the Traveling Salesman Problem with Time Windows," Operations Research 43(2), 367{371 (1995). European Computer-Industry Research Center, Munich, Germany, ECLiPSe 3.5.1 User Manual, 1995. M. Gendreau, A. Hertz, and G. Laporte, \New Insertion and Postoptimization Procedures for the Traveling Salesman Problem," Operations Research 40, 1086{1094 (1992). M. Gendreau, A. Hertz, G. Laporte, and M. Stan, \A Generalized Insertion Heuristic for the Traveling Salesman Problem with Time Windows," Publication CRT-95-07, Centre de recherche sur les transports, Universite de Montreal, Montreal, 1995. R.M. Haralick and G.L. Elliott, \Increasing Tree Search Eciency for Constraint Satisfaction Problems," Arti cial Intelligence 14, 263{313 (1980). J. Ja ar and J-L. Lassez, \Constraint Logic Programming," Technical Report 86/73, Dept. of Computer Science, Monash University, June 1986, (An abstract appears in the Proceedings of the 14th ACM Symposium on Principles of Programming Languages, Munich (January 1987), 111-119.). J. Ja ar and J-L. Lassez, \From Uni cation to Constraints," In K. Furukawa, H. Tanaka, and T. Fujisaki, editors, Proceedings of the 6th International Conference on Logic Programming, Tokyo, June 1987. J. Ja ar and M. J. Maher, \Constraint Logic Programming: A Survey," Journal of Logic Programming 19/20, 503{581 (1994). P. C. Kanellakis, G. M. Kuper, and P. Z. Revesz, \Constraint Query Languages," Journal of Computer and System Sciences 51(1), 26{52 (1995). 30
A. Langevin, M. Desrochers, J. Desrosiers, and F. Soumis, \A Two-Commodity Flow Formulation for the Traveling Salesman and Makespan Problems with Time Windows," Networks 23, 631{640 (1993). A. Mingozzi, L. Bianco, and S. Ricciardelli, \Dynamic Programming Strategies for the Traveling Salesman Problem with Time Window and Precedence Constraints," Technical report, Department of Mathematics, University of Bologna, 1993. G. Pesant and M. Gendreau, \A View of Local Search in Constraint Programming," In Principles and Practice of Constraint Programming | CP96: Proceedings of the Second International Conference, volume 1118 of Lecture Notes in Computer Science, pages 353{366, Springer-Verlag, Berlin, 1996. J.-Y. Potvin, T. Kervahut, B.L. Garcia, and J.-M. Rousseau, \A Tabu Search Heuristic for the Vehicle Routing Problem with Time Windows," Publication CRT-855, Centre de recherche sur les transports, Universite de Montreal, Montreal, 1993. J.-F. Puget, \Object-Oriented Constraint Programming for Transportation Problems," In Proceedings of Advanced Software Technology in Air Transport (ASTAIR), 1992. J.-F. Pujet, \A C++ Implementation of CLP," Technical Report 94-01, ILOG S.A., Gentilly, France, 1994. Y. Rochat and E . Taillard, \Probabilistic Diversi cation and Intensi cation in Local Search for Vehicle Routing," Journal of Heuristics 1, 147{167 (1995). V.A. Saraswat, \Concurrent Constraint Programming Languages," PhD thesis, Carnegie-Mellon University, 1989. M.W.P. Savelsberg, \The Vehicle Routing Problem with Time Windows: Minimizing Route Duration," ORSA Journal on Computing 4, 146{154 (1992). M.W.P. Savelsbergh, \Local Search in Routing Problems with Time Windows," Annals of Operations Research 4, 285{305 (1985). M.M. Solomon, \Algorithms for the Vehicle Routing and Scheduling Problem with Time Window Constraints," Operations Research 35, 254{265 (1987). E . Taillard, P. Badeau, M. Gendreau, F. Guertin, and J.-Y. Potvin, \A New Neighborhood Structure for the Vehicle Routing Problem with Time Windows," Publication CRT-95-66, Centre de recherche sur les transports, Universite de Montreal, Montreal, 1995.
31
S.R. Thangiah, I.H. Osman, and T. Sun, \Hybrid Genetic Algorithms, Simulated Annealing and Tabu Search Methods for Vehicle Routing Problems with Time Windows," Technical Report UKC/OR94/4, Institute of Mathematics and Statistics, University of Kent, Canterbury, UK, 1994. E. Tsang, Foundations of Constraint Satisfaction, Academic Press, 1993. P. van Hentenryck, V. Saraswat, and Y. Deville, \Design, Implementations and Evaluation of the Constraint Language cc(FD)," Technical Report CS-93-02, Brown University, 1993. D.L. Waltz, \Generating Semantic Descriptions from Drawings of Scenes with Shadows," Technical Report AI-TR-271, MIT, 1972.
32
Appendix In the following, each route starts and ends at depot 1 and is shown between angular brackets.
Improved solution for rc201: h1; 6; 46; 3; 99; 70; 83; 13; 12; 16; 17; 76; 88; 87; 58; 100; 54; 10; 11; 98; 75; 14; 18; 61; 101; 71; 1i h1; 66; 15; 48; 60; 53; 84; 65; 20; 24; 22; 19; 77; 86; 85; 52; 50; 23; 21; 67; 57; 97; 55; 38; 44; 36; 94; 92; 81; 1i h1; 9; 96; 64; 34; 29; 28; 30; 32; 31; 63; 68; 72; 39; 41; 42; 82; 91; 95; 51; 35; 33; 27; 90; 49; 25; 26; 78; 59; 1i h1; 73; 37; 40; 43; 45; 62; 89; 74; 79; 80; 8; 7; 9; 47; 4; 5; 2; 56; 69; 1i Improved solution for rc202: h1; 92; 93; 96; 86; 64; 34; 29; 27; 28; 30; 32; 31; 63; 68; 72; 73; 42; 41; 36; 44; 55; 97; 94; 95; 81; 1i h1; 66; 83; 13; 15; 48; 17; 16; 12; 89; 99; 54; 74; 79; 80; 8; 9; 7; 47; 5; 3; 56; 69; 1i h1; 46; 6; 4; 2; 43; 40; 37; 38; 39; 45; 62; 82; 91; 100; 58; 87; 88; 10; 11; 98; 60; 75; 14; 18; 61; 101; 71; 1i h1; 70; 65; 20; 24; 49; 19; 77; 52; 85; 50; 23; 21; 84; 67; 57; 51; 35; 33; 90; 25; 26; 78; 76; 59; 53; 1i Improved solution for rc205: h1; 70; 99; 12; 16; 17; 48; 15; 13; 89; 79; 74; 80; 8; 7; 9; 47; 6; 4; 5; 2; 44; 36; 38; 94; 97; 81; 1i h1; 66; 84; 65; 20; 24; 22; 19; 58; 87; 53; 100; 10; 88; 60; 76; 98; 11; 14; 18; 61; 101; 71; 1i h1; 93; 96; 34; 29; 28; 30; 32; 31; 64; 77; 86; 68; 85; 23; 50; 52; 51; 35; 33; 27; 90; 21; 25; 75; 78; 59; 26; 49; 1i h1; 3; 46; 43; 40; 37; 73; 72; 63; 95; 62; 45; 41; 39; 42; 82; 91; 54; 83; 67; 57; 92; 55; 69; 56; 1i Improved solution for rc207: h1; 66; 84; 65; 96; 68; 32; 30; 29; 31; 64; 77; 52; 20; 22; 19; 24; 76; 60; 88; 75; 87; 58; 23; 21; 50; 78; 59; 98; 14; 11; 18; 61; 2; 71; 101; 56; 69; 1i h1; 89; 3; 46; 6; 43; 45; 41; 39; 73; 72; 94; 91; 62; 82; 95; 93; 57; 85; 86; 63; 51; 35; 28; 27; 33; 34; 90; 49; 26; 25; 67; 92; 81; 1i h1; 70; 99; 83; 100; 53; 10; 12; 16; 17; 48; 15; 13; 54; 79; 74; 80; 8; 7; 9; 47; 5; 4; 44; 37; 36; 38; 40; 42; 55; 97; 1i
33
