Available online at www.sciencedirect.com
Computers & Operations Research 32 (2005) 465 – 490
www.elsevier.com/locate/dsw
Embedded local search approaches for routing optimization P.I. Cowlinga;∗ , R. Keuthenb a
MOSAIC Research Centre, Dept. of Computing, Univ. of Bradford, Bradford BD7 1DP, UK b University of Nottingham, School of Computer Science & IT, Nottingham NG8 1BB, UK
Abstract This paper presents a new class of heuristics which embed an exact algorithm within the framework of a local search heuristic. This approach was inspired by related heuristics which we developed for a practical problem arising in electronics manufacture. The basic idea of this heuristic is to break the original problem into small subproblems having similar properties to the original problem. These subproblems are then solved using time intensive heuristic approaches or exact algorithms and the solution is re-embedded into the original problem. The electronics manufacturing problem where we originally used the embedded local search approach, contains the Travelling Salesman Problem (TSP) as a major subproblem. In this paper we further develop our embedded search heuristic, HyperOpt, and investigate its performance for the TSP in comparison to other local search based approaches. We introduce an interesting hybrid of HyperOpt and 3-opt for asymmetric TSPs which proves more e4cient than HyperOpt or 3-opt alone. Since pure local search seldom yields solutions of high quality we also investigate the performance of the approaches in an iterated local search framework. We examine iterated approaches of Large-Step Markov Chain and Variable Neighbourhood Search type and investigate their performance when used in combination with HyperOpt. We report extensive computational results to investigate the performance of our heuristic approaches for asymmetric and Euclidean Travelling Salesman Problems. While for the symmetric TSP our approaches yield solutions of comparable quality to 2-opt heuristic, the hybrid methods proposed for asymmetric problems seem capable of compensating for the time intensive embedded heuristic by :nding tours of better average quality than iterated 3-opt in many less iterations and providing the best heuristic solutions known, for some instance classes. ? 2003 Elsevier Ltd. All rights reserved. Keywords: Travelling salesman; Local search; Iterated local search; Variable neighbourhood search
∗
Corresponding author. E-mail addresses:
[email protected] (P.I. Cowling),
[email protected] (R. Keuthen).
0305-0548/$ - see front matter ? 2003 Elsevier Ltd. All rights reserved. doi:10.1016/S0305-0548(03)00248-X
466
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
1. Introduction This paper describes local search heuristic algorithms for the symmetric and asymmetric Travelling Salesman Problem (TSP). The heuristics in this paper were inspired by heuristics which we designed for a complex scheduling problem arising in printed circuit board assembly. In order to provide appropriate background, we now describe problems arising in printed circuit board assembly. Surface Mount Technology has enabled the electronics manufacturing industry to mount a large number of components on a single printed circuit board. With these advances the reduction of assembly time has become a very important issue in order to cut down costs and increase production throughput. Several diCerent models and solution techniques for this problem have been proposed with diCering techniques depending on the component placement machine in use [1–5]. The problem we have been especially interested in considers a multi-headed component placement machine as shown in Fig. 1. A machine like this typically consists of one or two feeder magazines, where components are stored for pick up, and at least two component placement heads mounted on an arm [3,4,6,7]. This arm moves between board and feeder magazine to perform the pick-and-place operations. The advantage of a machine of this type is that a number of components can be picked up simultaneously from the feeder magazine and then be placed successively at the appropriate locations on the circuit board without returning to the feeder magazine. However, the multiple heads introduce multiple movement times between each pair of placement locations depending on whether component pick ups are necessary between two successive placements. In order to tackle this complex problem we developed a new model in which we consider it as a minimum weight Hypertour problem [3,4,8]. In this Hypertour problem each hyperedge represents a sequence where the placement arm starts with all heads empty at a placement location, moves to the feeder magazine to pick up one or more components and then moves back to the circuit board to place these components, :nishing with all heads empty at the last placement location.
arm nozzle
head
circuit board table feeder slots ATC tape reel feeder Fig. 1. Multi-headed component placement machine.
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
467
Our heuristic approach considers neighbourhood moves on hyperedges, which are seen as small subproblems of the sequencing problem, instead of single placement locations. The subproblems are then solved to optimality using exact algorithms and the optimal solution embedded within the original problem. This makes this approach a hybrid where an exact algorithm is used as part of a local search heuristic. Hence we retain the time complexity of the outer local search heuristic, to within a constant factor which depends on the size of the small subproblems. Empirical evidence shows that we also retain much of the solution quality of the time intensive exact approach. The printed circuit board assembly problem has, at its heart, the TSP. In this paper we use the TSP as a test-bed for a detailed investigation of embedded local search heuristics. Hundreds of publications and books [9,10] have been devoted to the TSP and various exact and heuristic solution approaches have been proposed. The TSP represents a permutation problem where one is given a set of cities {c1 ; c2 ; : : : ; cn } and distances between each pair of cities d(ci ; cj ). We wish to minimize N −1
d(c (i) ; c (i+1) ) + d(c (N ) ; c (1) );
i=1
over all permutations . This problem is well known to be strongly NP-hard [11], making fast heuristic approaches to determine good local optimal solutions very desirable for large problems occurring in practice which usually have complicating side constraints. The TSPs apparent simplicity on the one hand and the di4culty of :nding optimal solutions on the other has established it as a test bed for new heuristics and exact algorithms. However, the problem is not only of a theoretical value as there exist many industrial applications of the TSP and its variants [10]. These applications range from problems in transport and logistics [9] to Jow shop scheduling [12,13], steel hot strip mill scheduling [14], the screening of cytological samples [15] and, in the :eld of genetics, multiple genome rearrangement and breakpoint phylogeny [16]. Applications in electronics such as the layout driven chaining of scan Jip-Jops [17] and the ordering of cascade FIR :lter structures [18] have been proposed. Further applications arise for sequencing in the X-ray crystallography [19] and for numerically controlled punch press [20] or drilling operations [21]. Various local search algorithms have been suggested for the TSP. Many of the most successful local search engines, such as the approaches described in [22–25], for the TSP are based on an strategy proposed by Lin and Kernighan [26]. However, algorithms of a much simpler nature, such as 2 and 3-opt [27], are still very popular, especially for solving real-life sequencing problems which frequently introduce side constraints to the TSP. Here, we present a new class of heuristic which embeds an exact solution algorithm for solving small subproblems within a local search heuristic. We investigate the performance of our class of heuristics and their hybrids, described in the next section, in comparison to other local search heuristics on asymmetric and Euclidean TSPs and show, for the asymmetric TSP, how the hybridisation of simple search neighbourhoods can lead to a powerful heuristic approach. The paper is structured as follows. In the next section we introduce and motivate our choice of local search neighbourhood and discuss the search strategy we chose for the TSP. The third section gives an overview of iterated local search strategies proposed in the literature and how we applied them to allow us to search a larger part of the solution space. Computational results for our methods
468
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
applied to symmetric and asymmetric TSP are presented in Section 4. The last section discusses the conclusions we have drawn from our experiments and gives a brief insight into possible extensions and directions for further research. 2. Embedded search In the following we will refer to our embedded local search heuristic as HyperOpt, drawing an analogy with the heuristic of the same name which we have applied to printed circuit board assembly [3,4]. This local search heuristic belongs to the family of local improvement algorithms where simple modi:cations are repeatedly applied to a feasible solution for as long as improvements can be found. 2.1. Basic HyperOpt Essentially, the HyperOpt approach breaks edges between two segments of adjacent vertices in the current tour, and then uses e4cient enumeration to reconnect the tour optimally (see Fig. 2). This process is then iterated until a local optimum is found. More formally, starting with an initial tour, :rst, two non-intersecting hyperedges of the same length are de:ned by the cities p0 and q0 and their h successors (Fig. 2a). The edges within each of these hyperedges are then removed from the tour while the starting and ending cities of each hyperedge still stay attached to the tour (Fig. 2b). This leaves the inner cities of both hyperedges unconnected to the rest of the tour. These inner cities are then reconnected with the tour optimally. Figs. 2c and d illustrate two diCerent ways of reconnecting the tour. Obtaining the optimal reconnection involves :nding an optimal tour of the small TSP induced by the cities p0 ; p1 ; : : : ; ph and q0 ; q1 ; : : : ; qh where the two edges between ph ; q0 and qh ; p0 have been :xed. Essentially, this hyperedge deletion/restoration process is equivalent to restricted variant k-opt where k = 2h. However, since the k edges to be deleted have to be successive tour edges in order to qualify as a hyperedge and further only two hyperedges are chosen to perform a hypermove, we retain the same order of time complexity as 2-opt, for constant h. In the printed circuit board assembly problem reconnection involves exchanging placements between two small pick-and-place sequences, where multiple components are picked up in each sequence. The optimal reconnection process was included to ensure optimality of these pick-and-place subsequences. Computationally this is feasible since the number of heads the placement machine is equipped with
q0
q0
q0
q0
p0
p0
p0
p0
(a)
(b)
(c)
(d)
Fig. 2. Schema of HyperOpt moves.
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
469
is limited, typically lying between 2 and 4. In the following we denote a HyperOpt move where each hyperedge consists of a starting city and its h successors as a h-HyperOpt move. To illustrate the basic schema of HyperOpt two possible outcomes of a 4-HyperOpt move are shown in Fig. 2c and d. Even for small h the evaluation of a HyperOpt move is computationally expensive, e.g. for a 4-HyperOpt move there 10080 possible ways to restore the tour. In order to cut down computing time the optimal reconnection is in this paper performed by dynamic programming [28,29] for h ¿ 3. We use a fast dynamic programming implementation where a static dynamic programming tree is held in memory, and only the numbers at each node need to be instantiated to :nd an optimal solution. Thus our heuristic approach is a hybrid where an optimal enumerative approach (dynamic programming) is embedded in a local search heuristic. We thus hope to obtain some of the bene:ts of using an exact algorithm, within a local search framework of low time complexity (for small h). Once two hyperedges have been chosen, optimally reconnecting the tour requires (2h − 2)2 · 22h−2 iterations using dynamic programming, in contrast to 2 · (2h − 1)! for complete enumeration. The computational results we gained for this basic HyperOpt approach applied to the symmetric TSP will be discussed later in Section 4. 2.2. Hybridisation with 3-opt An obvious drawback of the basic HyperOpt approach as described above, which we encountered for the component placement sequencing problem, is that it does not allow the reordering of successive pick-and-place sequences or tour segments. However, especially for asymmetric problems, such as our component placement sequencing problem, the more successful local search approaches for the ATSP, like 3-opt and the Kanellakis–Papadimitriou Lin–Kernighan-type local search procedure [13], are based on reordering tour segments. Our basic HyperOpt approach is only able to reorder tour segments of length limited by h. Therefore, we hybridise our HyperOpt approach with the simplest segment reordering algorithm for the ATSP, 3-opt, as we originally suggested in [30]. If, for a chosen hyperedge starting at p0 , no improving hypermove was found the tour is scanned for an improving 3-opt move involving the edge connecting p0 and its successor p1 in the current tour. The idea behind this approach is to pro:t from the complementary characteristics of both search neighbourhoods, HyperOpt and 3-opt. While HyperOpt is unable to reorder tour segments involving more than h − 1 successive cities, it is able to perform deletion/reinsertion operations out of the reach of the 3-opt neighbourhood. A similar strategy has been suggested for symmetric problems in [31] to hybridise 2-opt with 2:5-opt. While basic HyperOpt as illustrated in Fig. 2 is easily applicable to symmetric instances of the TSP, it cannot be applied e4ciently to asymmetric instances. The basic HyperOpt approaches allows the reversal of tour segments, as illustrated in Fig. 2d. For asymmetric problems the evaluation of a tour segment reversal is considered computationally too expensive, being of order O(n), and therefore is usually avoided. For our HyperOpt local searches we do not allow tour segments to be reversed. However, for nearly symmetric ATSP instances the reversal of tour segments may lead to improvements that are very unlikely to be found by local searches which ignore segment reversals. In the next section we will propose a strategy to also introduce the 2-opt neighbourhood as part of an iterated local search method in order to bene:t from the bene:ts of tour segments reversals for the
470
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
ATSP at low cost. Empirical evidence that considering tour reversals may be an advantage for some asymmetric instances is given in Section 4. 3. Iterated local search The drawback of any local search approach is that we cannot avoid sometimes getting trapped in a local optimum of poor quality. We can avoid this by “kicking” a locally optimal tour and restarting the local search on the “kicked” tour, in order to :nd a new, and hopefully better, local optimum. Here a “kick” is a random rearrangement of the locally optimal tour, which is both far away enough so that falling back to the current local optimum is unlikely, and close enough so that the better features of the current locally optimum tour are not all lost. The approaches we are going to consider here are the iterated or chained multi start local search. The basic schema of this approaches is illustrated in Fig. 3. Here we are going to concentrate on the very popular Large-Step Markov Chains (LSMC) approach by Martin et al. [32–34], iterated Local Search by Johnson [22], and a variant of the Variable Neighbourhood Search (VNS) approach proposed by Hansen and MladenoviOc [35–37]. Basically, these approaches follow a strategy where, once a feasible tour has been created, simple tour modi:cations and local search are repeatedly applied to the current best tour found, often referred to as champion tour. The simple restart strategy enables the local search engine to reach a new local optima quickly since the local search engine needs to change only those parts of the tour that have been subject to modi:cation. 3.1. Large-Step Markov Chains The LSMC approach is a highly popular and successful multi-start technique for the TSP. Initially, Martin, Otto and Felten’s LSMC approach [32–34] incorporated Simulated Annealing [38], accepting local optima according to the annealing schedule. Here in step 2b of Fig. 3 a tour longer than the
Fig. 3. Iterated local search schema.
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
471
Fig. 4. Double bridge kick.
current champion tour may also be accepted following a probabilistic acceptance criteria. Because of this randomized acceptance and it’s similarities to Simulated Annealing, the authors referred to this process as a LSMC. However, Martin, Otto and Felten discovered in their work that ignoring this feature and only accepting improved local optima does not lead to any loss in tour quality. Therefore, this strategy is often dropped, i.e. only local optima are accepted that are better than any tours found earlier. This strategy was also followed by Johnson for his iterated Lin–Kernighan approach [22] which represents a simpli:ed variant of Martin, Otto and Felten’s approach and was inspired by early results of theirs. This non-probabilistic strategy is adopted in many successful implementations of their approach, such as the iterated Lin–Kernighan solvers by Johnson [22,27], Applegate, Bixby, Cook, ChvOatal and Rohe [23,39], and the iterated 3-opt and iterated Kanellakis–Papadimitriou [13] approaches Cirasella et al. studied in [40] for asymmetric TSPs. Our heuristics will accept only improved local optima. Ideally, the tour modi:cations, which are referred to as kicks in [32–34], should achieve two goals. Firstly, the kick should not be too strong and modify large parts of the tour, in order to keep the computation time to :nd a new local optimum low. Second, the modi:cation should lead to a new tour out of reach of the local search neighbourhood in use, as otherwise we may fall back to a local optimum formerly visited. The tour modi:cation strategy we are going to consider in this paper is the double bridge kick, as used by Otto, Martin and Felten in [32–34] which was also by Johnson for his iterated local search approach [22]. The principle of a double bridge kick is illustrated in Fig. 4. This kick was suggested by Johnson [22] as well as Martin, Otto and Felten following the observation of Lin and Kernighan [26] that this type of move is “non-sequential” and therefore very unlikely to be found by their local search approach, since a large number of consecutive local search moves is required to reverse a single double bridge kick. Due to the non-sequential nature, Lin and Kerninghan [26] initially suggested the incorporation of 4-opt moves in their local search engine. However, :nding the best 4-opt move usually requires too much computational eCort. The question remains of how the edges involved in the double bridge kick should be chosen. For symmetric TSP instances various strategies have been suggested. Martin, Otto and Felten suggested to choose the edges at random but only those of relatively small cost in the current champion. An investigation of diCerent techniques for Euclidean problems can be found in a recent paper by Applegate, Cook and Rohe [39]. Here, because of its general applicability to symmetric as well as asymmetric problems we opted for selecting the edges at random as suggested by Johnson in [22].
472
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
Fig. 5. VNS schema.
3.2. Variable Neighbourhood Search Although the Variable Neighbourhood approach suggested by Hansen and MladenoviOc [35–37] works in a very similar fashion it abandons the idea of a unique tour modi:cation. In their work VNS, as well as a local search approach called Variable Neighbourhood Descent Search, is not only applied the TSP but to various other problems including the p-median problem and the continuous location–allocation problem. The basic idea behind this VNS strategy is to strengthen the kicks, which they refer to as shakes, after failing to :nd an improved local optimum in order to escape large basins in the solution space. Hansen and MladenoviOc studied this approach in [35,36] for the symmetric TSP using randomly chosen expanding k-opt shakes and 2-opt for local search. They start with k = kmin and apply shakes using a randomly chosen member of the k-opt neighbourhood. If local search does not yield a better local optimum than the best found so far and k ¡ kmax the shaking parameter k is increased by one, otherwise if a new best tour is found k is reset to kmin , as illustrated in Fig. 5. In our work we use this approach although we have only chosen k-opt moves in our implementation which do not include reversals of tour segments, as shown in Fig. 6, in order to ensure that the shakes are applicable to symmetric as well as asymmetric TSPs. For their approaches, Hansen and MladenoviOc chose parameters kmin and kmax as 2 and N (the number of cities) and the local search engine in use is a fast variant of 2-opt using candidate lists to restrict the search. The computational results they report only compare this approach to simple local search 2-opt without any speed ups and the results gained by VNS are reported as improvements on this when given similar CPU times. For or implementation we set kmin = 4 as shakes may otherwise be easily repaired by the local searches and restrict kmax to a value much less than the number of cities N in order to keep computing times low. We compare this approach to a much wider range of
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
473
Fig. 6. VNS k-opt kick.
other heuristics in Section 4. An investigation on the how the value kmax inJuences solution quality and CPU times of this approach is also presented in Section 4. Other techniques following a strategy similar to VNS and LSMC have been investigated in the literature as well. In an early paper Baum [41,42] suggested an iterated descent approach similar to the one we described above using random 2-vertex-exchange kicks to overcome local optima. A very comprehensive study of LSMC strategies with k-opt kicks, using 2- and 3-opt as well as Lin– Kernighan [26] as local search engines can be found in Hong, Kahng and Moon [43]. In contrast to VNS the size of k remains :xed in their approach. In [43] the authors further investigate the Simulated Annealing side of Martin, Otto and Felten’s initial work and propose a population based iterated local search strategy. A diCerent iterated local search approach for Euclidean TSPs has been suggested by Codenotti et al. [44]. Here, rather than restarting local search on a modi:ed tour, the problem is perturbed to escape local optima. This is achieved by applying small random perturbations, such as the change of inter city distances or the removal of cities [44], to the problem instance. A local optimum is then determined for the perturbed problem which then serves as new local search starting tour for the original problem. 3.3. An extension to incorporate 2-opt for ATSPs As mentioned earlier the inability of the most local search approaches for the ATSP to perform tour segment reversals may be a drawback for some instances of ATSPs, e.g. for instances for which symmetry holds absolutely or approximately between most pairs of cities. For this type of instance and others, even a simple local search procedure based on tour segment reversals as 2-opt may lead to considerable improvements in tour quality. However, searching a tour for an improving 2-opt move is computationally expensive for the ATSP, being of complexity O(n2 ), which cannot be reduced by the use of candidate list as it is done for symmetric TSPs. We therefore incorporate 2-opt into our iterated search strategy and apply a basic 2-opt local search only when a new best tour is found. The aim of this strategy is to avoid expensive 2-opting for local optima of poor quality in order to save computing time. To search for improving 2-opt moves we suggest the following simple search technique. Starting from a city, move forward along the edges in the current tour evaluating the possible gain of a segment reversal. For our implementation the overall cost of this approach is reduced by performing a :rst decrement search and applying other well known speed ups strategies such as Don’t Look Bits [31].
474
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
4. Computational results In this section we present the computational results and implementation details for the methods described above applied to symmetric and asymmetric instances of the TSP. As a test bed for the heuristic approaches we chose Reinelt’s very popular Travelling Salesman Problem library (TSPlib) [45,46], which can be found on the web at http://softlib.rice.edu/softlib/tsplib, as well as other real world instances and generated instances for the ATSP considered in [40], which were kindly supplied to us by David S. Johnson (these instances are now also available for download at http://www.research.att.com/∼dsj/chtsp/atsp.html). We used the following abbreviations in the :gures and tables below. HO is used to abbreviate HyperOpt where the digit in front of it indicates the HyperOpt parameter h in use. The su4x i3 indicates that HyperOpt was hybridised with 3-opt, following the strategy we introduced for ATSPs in Section 2. We also used a range of techniques to make HyperOpt, as well as the Local Search approaches with which we compare our methods, computationally competitive with the work of others. All the Local Search approaches we implemented work in a >rst decrement manner, this means that once an improving move is found it is performed immediately. We made use of candidate lists to reduce CPU times. While candidate lists may only lead to approximate local optima, the speed up is usually considerable for large instances [27]. The candidates we consider, consist of the 10 nearest neighbours in each of the North, South, East and West quadrant for symmetric instances and the 20 nearest cities to and from each city in the asymmetric case. We further applied the so called Don’t Look Bits suggested by Bentley [31], a technique designed to avoid the reevaluation of moves which have proven unsuccessful in earlier iterations. In addition to the above techniques we suggest a rejection approach unique to HyperOpt based on an approximate lower bound for move evaluation which avoids expensive hypermove evaluation for non-promising hyperedges. Here, a hypermove between hyperedges H1 and H2 is only evaluated if for at least one city in H1 a city belonging to H2 is nearer than its immediate successor in the current tour. Although the evaluation of this rejection approach is of a computational complexity quadratic in h this is still much cheaper than a hypermove evaluation (which takes time exponential in h) and leads to a substantial speed up with only marginal loss of solution quality. The heuristics we refer to were run on a Pentium II 400 MHz PC with 256 Mb memory running a Linux 2.2.13 kernel. The algorithms are implemented in C++ using the GNU gcc compiler with -O2 compiler option. All the results reported represent average excess over optimum/best known lower bound and CPU times over :ve independent runs. The computational results we report for the iterated Kanellakis–Papadimitriou approach for asymmetric TSPs are taken from the recent survey by Cirasella et al. [40]. In order to allow the comparison of CPU times between this approach and ours the CPU times reported in [40] have been normalized using a benchmark code and times which were kindly supplied to us by David S. Johnson. 4.1. The asymmetric TSP In this section we present the computational results we gained for our HyperOpt and hybrid local search approaches when applied to the ATSP, in comparison to 3-opt as well as the Kanellakis– Papadimitriou implementation of Cirasella et al. [40]. The heuristic methods have been applied to
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
475
CPU Time: Local Search 25 3-opt 3HOi3 4HOi3
CPU Time in sec.
20
15
10
5
0 0
200
400
600
800
1000
Number of Cities N
Fig. 7. Real-world instances: CPU-times Local Search.
a selection of 47 real world instances including all asymmetric instances of Reinelts TSP library [45,46]. The asymmetric problems of TSPlib represent real world instances of problems occurring in TSP applications, such as delivery problems (ftv), stacker crane problems (rbg), as well as randomly perturbated symmetric problems (ry48p, kro124p). The additional real world instances include a coin collection problem (big702), instances of a table compression application (dc) and robot motion planning problems (atex). Further information on the types of instances can be found in [40]. In order to investigate the performance of our heuristic approaches in more detail they have also been applied to randomly generated instances of [40]. Some of the results we gained for these instances as well as the type of instances considered will be discussed later in this section. The local search 3-opt as well as our hybrid local search approaches 3HOi3 and 4HOi3 are reported in columns 3 to 5 of Table 1. The hybridisation of 3- and 4-HyperOpt with 3-opt proved very successful. These methods :nd tours of better quality than the ones found by 3-opt for nearly all instances. The faster of these approaches, 3HOi3, is able to :nd tours which are on the average about 1:3% better than 3-opted tours and produces tours of better or same quality as 3-opt for all but 8 of the 47 instances. The 4-HyperOpt/3-opt hybrid performs even better creating tours which are on the average about 1:8% better than tours found by 3-opt alone. Particularly for problems ftv, the stacker crane problems rbg and the robot motion planning problems dc our hybrid approaches were able to :nd tours of considerably better quality. Although the computational results for our approaches are encouraging in terms of tour quality, these approaches require more CPU time than 3-opt alone as illustrated in Fig. 7. The faster of our hybrid approaches 3HOi3 is on average about two times slower than 3-opt alone while 4HOi3 local search needs between 3 and 15 times as much CPU time. It does appear that we are gaining some of the bene:ts of the exact approach in terms of solution quality, while the running time remains reasonable as this exact approach is only used for small subproblems.
476
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
Table 1 ATSP: Real-world instances Problem
br17 ft53 ft70 ftv33 ftv35 ftv38 ftv44 ftv47 ftv55 ftv64 ftv70 ftv90 ftv100 ftv110 ftv120 ftv130 ftv140 ftv150 ftv160 ftv170 kro124p p43 rbg323 rbg358 rbg403 rbg443 ry48p atex1 atex3 atex4 atex5 atex8 big702 code198 code253 dc112 dc126 dc134 dc176 dc188 dc563 dc849 dc895 dc932 td100.1
N
17 53 70 34 36 39 45 48 56 65 71 91 101 111 121 131 141 151 161 171 100 43 323 358 403 443 48 16 32 48 72 600 702 198 253 112 126 134 176 188 563 849 895 932 101
Local search
Iterated local search
VNS
3-opt
3HOi3
4HOi3
3-opt
itKP
3HOi3
4HOi3
3-opt
3HOi3
4HOi3
0 8.49 3.05 4.43 2.08 5.22 5.53 5.05 8.3 9.34 6.61 17.94 16.32 14.28 11.4 14.58 11.77 13.57 13.61 13.25 8.1 0.19 2.22 3.87 0.6 0.63 3.73 5.76 0.16 8.34 9.49 10.67 3.15 31.02 0 1.51 3.73 1.81 2.83 2.69 2.74 0.7 2.62 1.45 0
0 6.72 1.89 4.71 1.31 5.41 3.39 5.64 5.92 7.18 6.05 12.29 12.18 11.67 7.99 8.08 10.73 8.28 13.88 12.62 7.32 0.28 0.96 1.96 0.16 0.06 2.58 6.51 0.24 8.24 10.22 10.12 3.17 19.69 0 1.24 2.04 1.48 1.72 2.30 2.50 0.66 2.24 1.34 0.02
0 2.64 1.71 4.19 2.67 5.05 3.71 5.56 5.85 3.93 5.98 9.98 11.72 9.28 7.87 12.59 8.38 11.69 11.59 11.94 4.47 0.08 0.64 0.99 0.16 0.16 2.62 2.47 0.24 8.39 8.64 9.06 2.59 16.87 0 1.21 1.85 1.30 1.20 2.00 2.25 0.65 2.08 1.13 0.01
0 0.12 0.02 1.66 0.29 0.13 0.78 0.33 0 0 0.65 0.41 0.13 2.21 1.07 1.14 1.75 2.33 2.78 1.37 1.14 0.01 0.2 0.74 0.08 0.09 0.81 0 0 0.02 2.96 2.63 0.46 0 0 0.77 1.62 1.32 1.98 1.85 2.67 0.7 2.51 1.41 0
0 0.01 0.02 3.02 0.03 0 1.3 0.49 0 0.33 0.08 0.08 0.13 0.12 0.33 0.27 0.21 0.4 0.24 0.45 1.28 0.01 0.3 0.79 0.3 0.11 1.92 0.44 0.03 0 0.15 0.99 0.21 0 0 0.31 0.64 0.55 0.59 0.52 0.78 0.61 0.58 0.17 0
0 0.27 0.26 0.59 0.26 0.54 0.51 0.33 0.27 0.16 0.49 0.46 0.43 1.8 0.72 0.51 1.8 0.88 1.71 0.66 0.01 0.01 0.02 0.21 0.02 0.02 0.25 0 0 0.99 5.79 3.14 0.3 8.43 0 0.39 0.12 1.05 0.74 0.41 2.01 0.66 1.49 0.59 0
0 0.03 0.09 1.34 0 0.13 0.26 0.32 0.44 0.16 0.18 0.41 0.27 0.94 1.03 1.11 0.99 1.51 1.7 0.66 0.77 0.01 0 0.03 0.03 0.02 0.12 0 0 0 3.09 2.73 0.21 2.81 0 0.26 0.06 0.6 0.39 0.34 1.68 0.65 1.31 0.52 0
0 0.03 0 0 0 0.1 0.78 0 0 0.16 0.09 0 0.2 0.08 0.07 0.05 0.1 0.4 0.22 0.62 0 0.01 0.14 0.41 0.03 0.02 0.34 0 0 0 1.2 1.9 0.28 0 0 0.9 2.71 1.64 2.3 1.63 2.72 0.7 2.62 1.41 0
0 0.09 0.16 0 0.11 0.43 0.63 0 0.01 0.29 0.42 0.25 0.27 0.65 1.01 0.58 0.85 0.82 0.68 0.68 0.51 0 0 0.05 0 0 0 0 0 0 3.91 3.65 0.18 0 0 0.53 0.18 1.01 0.65 0.4 2.13 0.66 1.59 0.59 0
0 0.07 0.07 0 0 0.08 0 0.07 0 0 0.09 0.3 0.22 0.42 0.74 0.95 0.7 0.75 0.6 0.49 0.33 0 0 0.02 0 0 0.1 0 0 0 2.48 3.38 0.04 0 0 0.2 0.09 0.66 0.36 0.24 1.82 0.65 1.48 0.54 0
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
477
Table 1 (continued) Problem
N
Local search 3-opt
td316.10 td1000.20 Average excess:
317 1001
Iterated local search
VNS
3HOi3
4HOi3
3-opt
itKP
3HOi3
4HOi3
3-opt
3HOi3
4HOi3
0.02 0.87
0 0.17
0 0.10
0 0.01
0 0
0 0
0 0
0 0
0 0
0 0
6.24
4.96
4.41
0.87
0.39
0.83
0.57
0.50
0.51
0.38
HyperOpt move ratio in % 65
% of improving HyperOpt moves
60
55
50
45
40
35
30 ft
ftv33..90
ftv100..170
kro p43
rbg
ry48p
TSPlib Problems
Fig. 8. Percentage of improving moves which were HyperOpt moves (the remaining improving moves are 3-opt moves).
It is interesting to investigate how much each part of the hybrid contributes. The chart in Fig. 8 illustrates the number of improving HyperOpt moves performed as a percentage of the total number of improving moves performed for the 26 TSPlib instances (the :rst 26 problems listed in Table 1 ignoring problem br17 which proved to be trivial). This ratio seems very dependent on the type of ATSP the hybrid is applied to ranging from 32% to 61% for the instances considered. While for problems of type ftv, which represent real life delivery problems, 3-opt is responsible for the majority of improving moves, this is reversed for the stacker crane instances rbg for which HyperOpt :nds a larger number of improving moves. In the former case it seems that 3-opt’s ability to reorganise the tour on a large scale dominates, whereas in the latter HyperOpt’s ability to optimise the order of short tour segments is more important. We now consider whether our hybrid approaches can compensate for large CPU times in an iterated framework. We therefore use the HyperOpt/3-opt hybrid as a local search engine in iterated local searches of LSMC and VNS type and compare our results to iterated 3-opt and the results for iterated Kanellakis–Papadimitriou [13], abbreviated itKP in the tables and charts, reported in [40]. The Kanellakis–Papadimitriou implementation uses, in addition to a Lin–Kernighan type variable-depth search, a second search process for an improving 4-opt move as suggested in [13], implemented in
478
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490 CPU Time: Iterated Local Search
CPU Time: Variable Neighbourhood Search
5000
9000 3-opt itKP 3HOi3 4HOi3
4500
3-opt itKP 3HOi3 4HOi3
8000
4000
7000
3500 CPU Time in sec.
CPU Time in sec.
6000 3000 2500 2000
5000 4000 3000
1500 2000
1000
1000
500 0
0 0
200
400
600
Number of Cities N
800
1000
0
200
400
600
800
1000
Number of Cities N
Fig. 9. Real-world instances: CPU-times Iterated Local Search.
the eCective manner Glover suggested in [47]. The local searches are started, as for our approaches, from nearest neighbour tours, random double bridge 4-opt moves are used for restart and the same nearest neighbour sets are used as candidate lists. For more information we refer to the original paper by Kanellakis and Papadimitriou [13] and [40]. The computational results we report for our iterated hybrid approaches 3HOi3 and 4HOi3 all include the 2-opt strategy as described in Section 3.3. This feature is included for all our iterated HOi3 approaches to keep the comparisons of computing times fair, even if 2-opting did not yield any improvement. 2-opt gives no further improvement for many of the real world problems for which we report results in Table 1. However, for some of the randomly generated instances we will discuss below improvements are considerable. To make up for 3-opts lower CPU-time per iteration we allow 3-opt 10 times more iterations than our hybrid approaches and iterated Kanellakis–Papadimitriou. The number of iterations we chose are 10N for iterated and VNS 3-opt and N for any other iterated approach, where N is the number of cities of each problem. For VNS we limited the maximal shake strength kmax to 6 for problems of less than 100 cities and to 8 otherwise. The minimum shake strength is 4. A more throughout investigation on how this parameter inJuences the search process can be found in the next section on the symmetric TSP. The computational results in terms of excess over optimum are shown in the last 7 columns of Table 1 and CPU-times illustrated in Fig. 9. We can see in the :gure that for the number of iterations chosen, CPU times for iterated 3-opt and our fasted hybrid approach, 3HOi3, are similar. However, in terms of tour quality iterated 3HOi3 outperforms iterated 3-opt in many less iterations. Particularly for the TSPlib problems ftv110 to ftv170, dc and the stacker crane instances rbg our approaches :nds substantially better tours. However, the average excess over optimum over all instances for iterated 3-opt and 3HOi3 are very close. This is mainly due to problem code198 for which iterated 3HOi3 is on the average only able to determine solutions of very poor quality, although the optimal solution was found for this problem for 2 out of the 5 independent runs. We cannot identify which feature of the code198 instance causes this, but since we have found the optimum 2 out of 5 times
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
479
given limited CPU time, and in all cases for the VNS approach in columns 10 –12, it is clear that either more iterations, or larger shakes, can overcome this feature. When leaving problem code198 out, 3HOi3 superiority becomes more apparent with an average excess rate of 0:66% over optimum for the rest of the problems in comparison to an excess rate of 0:88% achieved in this case by 3-opt. The second of our local search engines, 4HOi3, can again substantially improve on this performance at the cost of much higher CPU times. However, although both HyperOpt approaches are superior to iterated 3-opt for many of the real-world instances they are outperformed by itKP in terms of tour quality except for the stacker crane instances rbg. The comparison of CPU times to our approaches is hard since the running times of iterated Kanellakis–Papadimitriou are not only dependent on the problem size but also on its structure, see Fig. 9. While iterated 4HOi3 is faster for many of the larger instances, sometimes up to 10 times, it is about 6 times slower for the largest instance considered. The results we gained for VNS using the 3-opt and HOi3 local searches, as presented in Table 1 and the second chart in Fig. 9. We can see that the excess rates are better than for iterated local searches of LSMC type. For this approach 4HOi3 is able to close the gap to iterated Kanellakis–Papadimitriou and achieve on the average very similar excess rates. The CPU times for VNS approaches are about three times those for LSMC approaches since the local searches are restarted on more modi:ed parts of the tour. However, for many of the larger instances, VNS using 4HOi3 as local search engine is still competitive in terms of CPU time to iterated Kanellakis– Papadimitriou. Since some of the computational results we gained for the real world instances are encouraging we further investigated the performance of our heuristic approaches for some of the generated instances of [40]. In [40] Cirasella et al. introduce a variety of instance generators many of which are modelled on real world applications of the asymmetric TSP. The instances we are going to consider here are: • • • • •
rect: Random two-dimensional rectilinear instances, rtilt: Tilted drilling machine instances with additive norm, coins: Pay phone coin collection instances, crane: Random Euclidean stacker crane instances, shop50: No-wait Jowshop instances.
For more information on the instance generators considered see [40]. Each instance set consists of ten 100 city problems, ten 316 city problems, three 1000 city problems and one 3162 city problem. We are not going to report the computational results for all instance types introduced in [40], choosing to present results for the classes of instances where our approaches proved competitive. Especially for instances for which the triangle inequality does not hold, e.g. for random asymmetric matrices amat [40] among others, the performance of our approaches was relatively poor, achieving tour qualities similar to the applications of N iterations of 3-opt at the cost of much higher CPU times. To investigate the performance of our local search approaches in a similar environment to the results for itKP reported in [40] and because of large CPU times for the 3162 city problems we only report results for iterated approaches of LSMC type. The computational results for these instance sets are reported in Table 2, giving the % excess over the Held–Karp lower bound in each case. We can see that especially for the tilted drilling machine
480
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
Table 2 Generated instances: iterated local search Class
N
Excess over Held–Karp LB (%)
CPU-time (s)
3-opt
itKP
3HOi3
4HOi3
3-opt
itKP
3HOi3
4HOi3
rect
100 316 1000 3162
1.79 2.41 2.53 2.83
1.75 2.16 2.2 2.32
1.41 1.78 2.32 3.26
1.11 1.7 2.22 2.94
1.68 9.82 63.61 492.48
0.2 1.7 20.91 258.28
1.75 12.47 95.69 853.56
9.93 73.65 441.21 3014.23
rtilt
100 316 1000 3162
1.74 3.32 10.77 16.77
1.8 4.12 7.29 8.89
1.3 2.15 4.62 8.1
1.08 1.75 3.61 6.04
5.74 44.37 283.54 1490.02
15.83 512.12 9235 77716.04
5.19 50.7 441.17 3450.37
38.38 426.13 3394.56 29779.2
crane
100 316 1000 3162
2.02 2.36 2.06 2.27
1.46 1.79 1.27 1.36
1.82 2.46 2.66 3.28
1.55 2.5 2.39 2.99
1.69 9.82 61.2 470.12
0.27 2.74 29.23 372.19
1.89 13.64 111.38 1253.8
10.21 73.28 489.01 3646.99
coins
100 316 1000 3162
3.09 3.55 3.45 3.79
2.71 2.99 2.66 2.87
2.22 3.09 3.83 4.59
1.85 2.76 3.63 4.33
1.68 9.44 62.28 467.97
0.21 1.83 21 289.83
1.72 11.98 91.94 919.43
8.31 61.15 427.49 2968.27
shop50
100 316 1000 3162
0.51 3.28 8.63 10.51
0.49 2.36 3.66 4.51
0.46 1.85 5.05 9.35
0.36 1.27 3.28 7.00
14.62 65.36 239.19 843.54
134.04 1235.11 9270.7 39902.1
12.07 90.45 441.36 1467.77
103.17 973.88 4768.14 17651
rtilt and no-wait Jowshop shop50 instances, our approaches are highly competitive to iterated 3-opt using 10N iterations and iterated Kanellakis–Papadimitriou. Our approaches are particularly eCective for the rtilt instances, where both iterated HyperOpt local searches, 3HOi3 and 4HOi3, gain substantially better tours than itKP and any other approach considered in [40] over all instances sizes. It appears that our approaches may be the best known for problems of this class. Considering all generated instances of [40], our approach were nearly or actually the best for all the small problems of less than 1000 cities. For larger problems our approaches remain competitive. It is interesting that while for the real world stacker crane problems our approaches performed well, see instances rbg in Table 1, for the randomly generated instances crane our approaches do not perform as well as 3-opt while using much more computing time. It seems likely that there are substantive diCerences between real and randomly generated problem instances. The CPU times of each approach are given in the last four columns of Table 2. As for the real world instances the times are again very instance dependent. While itKP is about four times faster for the coin and crane instances than iterated 3HOi3 it is much slower than iterated 4HOi3 for the no-wait Jow shop instances.
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490 Iterated Local Search with and without 2-opt: rtilt
481
Iterated Local Search with and without 2-opt: rtilt 4000 3HOi3 3HOi3 with 2-opt
14
3HOi3 3HOi3 with 2-opt 3500
12
CPU Time in sec.
Excess over HK in %
3000 10
8
6
2500
2000
1500
4
1000
2
500
0
0 500
1000
1500 2000 Number of Cities N
2500
3000
500
1000
1500 2000 Number of Cities N
2500
3000
Fig. 10. rtilt instances with and without 2-opting.
Finally, we would like to investigate brieJy the advantage of incorporating 2-opt in the local search process for the ATSP. The average excess rates and CPU times for iterated 3HOi3 of LSMC type with out without 2-opting are illustrated in Fig. 10 for the rtilt instance class. We can see that the improvements gained are up to 2% for the instances of 1000 cities and more while the CPU-times increase by approximately 50%. When using 4HOi3 as local search engine this increase is much lower since overall CPU times are dominated by the time intensive dynamic programming approach to move evaluation. Improvements, although on a smaller scale, were also achieved for the rectilinear, no-wait Jowshop, coin collection as well as some real world problems. However, for highly asymmetric problems and problems for which the triangle inequality does not hold improvements were rare and usually very small when present. Overall, we can conclude that the hybrid approaches 3HOi3 and 4HOi3 are competitive with the best approaches known for a broad range of ATSP instances, in terms of solution quality and CPU time, and yield the best known results for some instances and instance classes. Overall, we can conclude that the hybrid approaches. 4.2. The symmetric TSP To test the performance of our algorithms on symmetric instances we applied them to all the large symmetric TSPlib instances of at least 1000 cities [45,46] but one (our code for STSPs uses geometric techniques which are not applicable to the only non-Euclidean problem si1032). We compare our results with fast implementations of 2- and 3-opt (although note that more successful Lin–Kernighan type approaches have been suggested in the literature [39,22,25]). We report results for basic k-HyperOpt with k in the range 2–4. All approaches use candidate lists consisting of the 10 nearest neighbours for each city out of each Euclidean quadrant. To determine the candidate sets we exploited k-d trees as suggested by Bentley in [48,49]. We start the local search engines from the tour produced by the Multiple Fragment (sometimes referred to as Greedy) algorithm [31]. Empirical evidence presented in [31] suggests that Multiple Fragment tours are good starting
482
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
Table 3 Local Search STSP: Average excess rate over all problems Local search
Average excess
Excess in % over optimum 2-opt
3-opt
2HO
3HO
4HO
5.68
3.32
4.82
4.20
3.78
CPU Time: Local Search 2-opt 3-opt 2HO 3HO 4HO
1400
1200
CPU Time in sec.
1000
800
600
400
200
0 10000
20000
30000
40000
50000
60000
70000
80000
Number of Cities N
Fig. 11. Symmetric TSP: CPU times Local Search.
solutions for improvement using k-opt type local search heuristics. The computational results for each individual TSPlib instance for 2-, 3-opt as well as 2- to 4-HyperOpt can be found in Table 5 (see appendix). The average excess rate over all instances is given in Table 3. We can see that although tour quality improves for our HyperOpt approaches with increasing h the algorithms are not able to generate tours of average quality competitive with 3-opt. 3-opt found the best tours for 22 out of the 33 instances considered while 4-HyperOpt found best tours for 10 of the problems. In addition 4-HyperOpt uses substantially more CPU time than 3-opt, as illustrated in Fig. 11. While 3-opt is about 4 times faster than 3-HyperOpt, using the same speed up techniques, it produces tours of a better quality for most of the instances. This may be due to basic HyperOpts inability to reorder tour segments of length larger than h−1 and the larger neighbourhood searched by 3-opt. However, tests with hybrid approach of HyperOpt and 3-opt, as suggested for the asymmetric TSP, proved unsuccessful gaining tour qualities inferior to 3-opt while taking much more CPU time. Comparing our HyperOpt approaches to 2-opt, all HyperOpt approaches outperform 2-opt in terms of tour quality. With similar CPU times, 2-HyperOpt is able to determine tours which are on the average 0:8% better than 2-opted tours and :nds tours of superior quality for all but 5 problems.
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
483
Table 4 Iterated Local Search STSP: Average excess rate over all problems Local Search
Iterated Local Search
Average excess
VNS
2-opt
3-opt
2HO
3HO
2-opt
3-opt
2HO
3HO
3.67
2.25
2.77
2.43
3.35
1.87
2.51
2.09
CPU Time: Iterated Local Search
CPU Time: Variable Neighbourhood Search
600
700 2-opt 3-opt 2HO 3HO
2-opt 3-opt 2HO 3HO
600
500
500 CPU Time in sec.
CPU Time in sec.
400
300
400
300
200 200
100
100
0
0 10000
20000
30000
40000
50000
60000
70000
80000
10000
20000
Number of Cities N
30000
40000
50000
60000
70000
80000
Number of Cities N
Fig. 12. Symmetric TSP: iterated Local Search.
We now compare CPU times and tour quality for our iterated local search of LSMC and VNS types, as discussed previously. Both heuristics were allowed 1000 iterations and were started from Multiple Fragment tours. The average excess rates gained by each method using diCerent local search engines are shown in the Table 4. We do not consider 4-HyperOpt as a local search engine for iterated search methods since it requires too much CPU time. We decided to include VNS results in this table using k-opt shakes with k in the range of 4–8 since this approach led to reasonable CPU times. Computational results for stronger shakes are discussed later in this section. The computational results for each individual problem of these methods can be found in Tables 6 and 7 in the appendix of this paper. Charts illustrating the CPU times for iterated local search are given in Fig. 12. For the LSMC type approaches as well as for VNS, 3-opt signi:cantly outperforms the other local search approaches for which we report results. Although 3-HyperOpt is able to close the gap to 3-opt and even :nds better tours for 10 out of the 33 test problems, this is achieved at the cost of much higher CPU times. As for pure local search, iterated 2-HyperOpt signi:cantly outperforms iterated 2-opt leading to better tours for nearly all instances in a similar amount of CPU time. When comparing LSMC type heuristics with VNS, the Variable Neighbourhood Search methods tend to be slightly slower for the same number of iterations, see Fig. 12, as restarts are carried
484
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490 Iterated Local Search: k-opt shakes
Iterated Local Search: k-opt shakes
4
1000 2-opt 3-opt 2HO
2-opt 3-opt 2HO 900
Total CPU Time in sec.
Average Excess in %
3.5
3
2.5
800
700
600
2 500
1.5
400 10
15
20
25
30
10
kmax
15
20
25
30
kmax
Fig. 13. Symmetric TSP: investigating VNS.
out on larger parts of the modi:ed tour. However, VNS approaches signi:cantly outperform LSMC approaches in terms of tour quality for each local search heuristic. The double bridge kick used by LSMC is not easily repaired by the local search heuristics considered here, but the kicks may not be strong enough to escape large basins in the solution space. This supports an observation by Hong et al. [43] who investigated diCerent k-opt kicks, with :xed k, for 2-, 3-opt and Lin–Kernighan. Their results suggest that better solutions may be obtained by applying stronger tour modi:cations than the double bridge 4-opt kick. The eCectiveness of the VNS shaking routine may be further enhanced since it adapts the strength of the tour modi:cations during the iterative process. For VNS methods we also investigated the dependency of the heuristic on the maximal shake strength kmax , i.e. the size of the largest k-opt shake in use. To do this we ran VNS on all instances using various local search engines with kmax taking values {8; 10; 15; 20; 25; 30}. Fig. 13 illustrates the outcome of this experiments in terms of average excess over optimum for each parameter kmax as well as total CPU time for all 33 instances. We can see that the VNS approach is surprisingly insensitive to this parameter in terms of both tour quality and CPU time. Best excess rates for the test problems considered here were gained for a value of kmax = 15 while the total CPU time increased for larger kmax by at most 10%. 5. Conclusions In this paper we have presented a new class of embedded neighbourhood search methods for the TSP, inspired by local search algorithms which we developed for scheduling printed circuit board assembly [3,4]. This embedded local search algorithm, which we called HyperOpt, is a hybrid method where an exact algorithm is used to solve small, embedded subproblems of the original problem to optimality which are then re-embedded to give an improved solution. In this paper we have compared the performance of our heuristic approaches to other simple and variable neighbourhood searches for the symmetric and asymmetric TSP. We further introduced an interesting hybrid of HyperOpt and
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
485
3-opt for the ATSP to bene:t from the complementary advantages of both methods. In particular this hybrid approach proved very promising, :nding substantially better tours than its component local search heuristics. Since single local search heuristics alone do not usually yield high quality solutions we investigated the performance of our approaches in an iterated local search environment. We consider iterative local search approaches inspired by the LSMC approach as proposed by Martin et al. [32–34] and the VNS approach suggested by Hansen and MladenoviOc [35–37] for the TSP. The experiments suggest that an iterated approach with a variable restart mechanism yield tours of a superior quality than the very popular double bridge 4-opt kick, when given the same number of iterations. While for the symmetric TSP our approaches cannot compete to a fast 3-opt heuristic, for asymmetric instances our hybrid local search approaches seem capable of compensating for larger CPU times spent on a local search in an iterative local search framework. When given similar CPU times our iterated approaches are able to compensate for large CPU time spend on local searches often outperforming a fast implementation of iterated 3-opt in terms of tour quality. When comparing our approaches to iterated Kanellakis–Papadimitriou [13], arguably the most successful local search approach for the ATSP, our approaches prove competitive for real world instances as well as some of the generated instances classes introduced in [40], yielding for one class the best results reported so far. A possible extension of our approaches for asymmetric TSPs to improve the solution quality may be achieved by the incorporation of 4-opt searches which proved very valuable [50] for the iterated Kanellakis–Papadimitriou implementation of [40]. This as well as a thorough investigation of the reasons why our approaches perform well on particular instances of the asymmetric TSP is the subject of ongoing and future research. An interesting and practically useful feature of the HyperOpt approaches is that some types of side constraints, such as time windows, may be handled by the outer local search while we may genuinely optimize the small subproblems and still meet the side constraints. Thus HyperOpt-like approaches have the potential to provide very good solutions to a range of complex practical problems. We have already shown this to be the case in the case of electronics assembly scheduling.
Acknowledgements We would like to thank David S. Johnson for supplying us with the instance generators, additional real world instances and benchmark data of [40]. These instances and benchmark code are now available for download at http://www.research.att.com/∼dsj/chtsp/atsp.html.
Appendix. Tables listing the computational results for individual TSPlib problems. The best tour found is shown in bold for each problem (Tables 5–7).
486
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
Table 5 Local Search: symmetric TSP Problem
Excess in % over optimum 2-opt
CPU times (s)
3-opt
2HO
3HO
4HO
2-opt
3-opt
2HO
3HO
4HO
dsj1000
6.86
2.89
6.42
4.87
4.38
0.22
0.45
0.26
2.64
15.2
pr1002 u1060
4.91 4.92
3.17 2.30
4.90 5.49
4.27 3.27
3.30 3.98
0.20 0.25
0.43 0.51
0.24 0.3
2.87 3.35
14.16 17.46
vm1084
5.94
3.63
2.06
3.12
3.37
0.26
0.48
0.3
2.81
13.84
pcb1173
5.12
3.15
4.13
3.73
3.72
0.25
0.5
0.32
3.18
18.87
11.47 6.73 4.13
5.14 2.90 1.74
8.15 4.28 3.03
7.53 4.89 4.90
4.97 5.52 2.47
0.26 0.28 0.28
0.53 0.53 0.46
0.29 0.31 0.29
1.78 1.68 1.54
10.96 9.01 9.17
4.27
2.65
3.34
2.77
2.59
0.31
0.59
0.35
4.42
22.71
J1400
4.98
2.14
3.50
2.48
2.13
0.34
0.77
0.43
4.69
19.59
u1432
5.93
4.17
4.91
4.41
4.58
0.29
0.64
0.35
4.08
22.23
J1577
8.96
3.55
9.70
8.30
7.79
0.37
0.69
0.38
2.65
13.55
d1655
5.80
4.35
4.99
4.94
3.85
0.35
0.75
0.41
3.98
20.31
vm1748
6.92
3.54
5.88
4.71
3.73
0.45
0.82
0.51
4.14
21.76
u1817
4.52
3.58
4.66
3.79
3.93
0.37
0.74
0.41
3.42
21.06
rl1889
6.74
4.60
7.58
5.89
4.29
0.42
0.82
0.46
2.59
15.69
d2103
6.22
3.75
5.37
4.41
2.04
0.46
0.77
0.54
3.18
15.11
u2152
5.82
4.07
5.71
4.69
5.17
0.47
0.99
0.56
3.99
24.2
u2319
2.58
1.81
2.47
1.91
1.46
0.5
1.05
0.64
7.69
37.05
pr2392
6.41
3.92
5.18
4.89
4.39
0.58
1.05
0.65
7.03
35.92
pcb3038
5.44
2.91
4.15
3.68
3.15
0.7
1.35
0.83
9.36
49.47
J3795
10.37
8.68
6.88
4.22
5.89
1.02
1.89
1.13
fnl4461
3.90
2.45
3.39
2.81
2.76
1.2
2.08
1.41
rl5915
4.51
2.08
5.02
5.74
3.47
1.41
2.41
1.54
7.27
46.66
rl5934
6.10
3.03
6.07
4.66
6.74
1.52
2.46
1.59
9.31
51.86
pla7397
4.27
2.89
4.15
3.19
3.29
2.01
3.76
2.31
20.21
89.2
rl11849
3.88
2.41
3.70
2.92
3.62
3.6
5.57
3.84
22.19
124.49
usa13509
4.50
2.61
3.53
3.28
2.85
4.57
7.33
5.01
38.66
200.83
brd14051
5.03
2.80
3.77
3.63
2.73
4.16
6.9
4.82
42.54
237.89
d15112
4.57
2.81
3.91
3.50
2.60
4.68
7.83
5.44
48.43
267.79
d18512
4.29
2.44
3.36
2.97
2.58
5.84
9.53
6.67
57.05
327.59
d1291 rl1304 rl1323 nrw1379
8.53 15.8
38.09 81.84
pla33810
5.94
3.68
4.87
4.16
3.66
10.68
17.98
12.45
87.67
427.6
pla85900
5.54
3.69
4.69
4.02
3.87
27.79
48.56
33.82
230.00
1203.25
Average
5.68
3.32
4.82
4.20
3.78
—
—
—
—
—
excess
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
487
Table 6 Iterated Local Search using double bridge kicks: symmetric TSP Problem
Excess in % over optimum
CPU times (s)
2-opt
3-opt
2HO
3HO
2-opt
3-opt
2HO
3HO
dsj1000
2.95
1.44
1.46
1.38
3.39
9.83
6.67
153.78
pr1002 u1060
2.23 2.02
1.92 0.98
2.24 1.54
1.47 0.98
3.40 4.13
10.27 10.81
6.66 6.58
145.92 134.77
vm1084
2.16
1.16
0.85
1.16
3.72
9.35
5.74
100.55
pcb1173
3.03
2.20
2.39
1.99
3.72
9.94
6.71
135.92
d1291 rl1304
6.72 2.56
3.40 1.33
5.23 1.47
3.82 3.47
3.66 3.68
10.77 10.06
5.62 5.21
77.03 69.89
rl1323
2.56
1.21
1.44
2.43
3.51
9.94
5.26
75.34
nrw1379
2.27
1.41
1.53
1.08
3.82
10.15
6.74
174.22
J1400
1.72
0.94
1.93
0.99
3.95
13.05
6.93
145.27
u1432
3.04
2.06
2.08
1.75
3.83
11.75
6.55
140.68
J1577
6.52
2.75
4.61
4.83
3.95
11.01
5.99
98.56
d1655
3.42
2.81
2.60
2.61
3.98
10.80
6.22
111.43
vm1748
2.92
2.01
2.21
2.27
4.12
10.54
6.09
109.68
u1817
2.30
1.14
1.50
1.65
4.14
11.17
6.05
97.28
rl1889
4.05
3.02
3.73
2.86
4.28
11.31
6.28
76.99
d2103
3.05
2.33
3.72
3.74
4.34
10.88
6.68
85.39
u2152
3.80
2.30
3.23
2.41
4.57
12.05
6.69
105.13
u2319
1.36
0.92
1.21
0.80
4.77
14.46
8.25
191.11
pr2392
3.89
2.56
2.95
2.55
5.12
12.02
7.52
134.50
pcb3038
3.84
1.79
2.50
1.96
5.72
12.73
8.54
167.02
J3795
9.23
7.49
4.51
3.43
6.29
14.01
8.32
100.23
fnl4461
2.91
1.75
2.23
1.74
7.65
15.36
10.82
196.38
rl5915
3.47
2.08
2.87
2.60
8.26
16.13
9.97
89.42
rl5934
4.77
2.07
3.79
3.37
8.62
16.66
10.57
96.84
pla7397
3.39
2.19
2.90
2.14
10.85
20.22
13.37
151.02
rl11849
3.39
2.15
3.10
2.27
17.04
24.04
18.94
142.37
usa13509
4.00
2.39
2.97
2.68
20.28
27.55
23.32
209.02
brd14051
4.45
2.53
3.24
2.64
19.66
27.71
23.90
236.98
d15112
4.15
2.31
3.29
2.88
22.43
30.58
26.89
247.45
d18512
3.92
2.58
2.99
2.49
25.85
35.06
30.16
255.16
pla33810
5.55
3.36
4.49
3.77
44.70
57.59
49.76
260.12
pla85900
5.36
3.49
4.51
3.77
119.16
139.39
126.05
472.32
Average
3.67
2.25
2.77
2.43
—
—
—
—
excess
488
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
Table 7 Iterated Local Search (VNS) using expanding k-opt kicks: symmetric TSP Problem
Excess in % over optimum
CPU times (s)
2-opt
3-opt
2HO
3HO
2-opt
3-opt
2HO
3HO
dsj1000
2.57
1.28
1.25
1.20
5.17
15.02
9.31
265.04
pr1002 u1060
1.93 1.67
1.32 0.65
1.84 1.20
1.03 0.61
5.37 5.38
15.35 16.33
9.53 9.44
249.98 239.89
vm1084
1.87
0.77
0.68
1.10
4.92
14.09
8.15
181.76
pcb1173
2.60
1.82
1.79
1.45
6.38
14.98
9.86
244.6
d1291 rl1304 rl1323
6.17 2.00 2.50
2.66 1.04 0.89
4.33 1.33 1.26
3.63 2.64 1.62
5.09 5.22 5.24
15.59 14.84 14.79
7.84 8.06 8.17
129.15 121.07 128.50
nrw1379
2.04
1.02
1.37
0.85
5.64
14.71
10.60
310.58
J1400
1.49
0.86
1.77
0.81
5.66
18.46
10.11
242.56
u1432
2.77
1.52
1.73
1.43
5.72
17.79
10.24
250.33
J1577
5.71
1.93
4.96
4.28
5.48
15.46
8.49
156.32
d1655
3.13
2.65
2.51
2.32
5.83
15.99
9.34
191.25
vm1748
2.51
1.44
1.72
1.32
5.57
15.14
8.88
183.40
u1817
1.99
0.69
1.36
1.27
5.74
16.54
9.25
169.83
rl1889
3.70
2.67
2.97
2.57
5.77
16.56
9.14
130.91
d2103
2.61
1.91
2.74
3.24
6.03
16.36
10.11
149.94
u2152
3.37
1.68
2.50
1.97
6.16
17.28
9.85
182.46
u2319
1.33
0.76
0.94
0.66
6.77
21.53
12.86
345.61
pr2392
3.45
2.02
2.74
2.03
6.57
16.41
10.98
220.43
pcb3038
3.46
1.33
2.14
1.70
7.45
16.80
12.17
281.61
J3795
8.07
6.01
4.45
2.83
7.68
18.51
11.02
158.88
fnl4461
2.67
1.51
2.08
1.54
9.36
19.16
14.97
320.25
rl5915
3.18
1.65
2.95
2.46
10.31
20.02
13.38
144.75
rl5934
4.51
1.78
3.63
3.01
10.99
20.67
13.83
157.57
pla7397
3.00
1.97
2.82
1.79
13.06
24.57
16.55
228.29
rl11849
3.15
1.91
2.88
2.11
19.98
29.52
23.43
218.44
usa13509
3.89
2.18
2.85
2.43
23.64
34.16
27.97
313.752
brd14051
4.29
2.37
3.07
2.42
23.72
33.74
28.28
357.39
d15112
4.02
2.17
3.21
2.73
24.89
36.74
31.79
359.63
d18512
3.82
2.42
2.89
2.33
27.94
40.87
35.38
360.87
pla33810
5.42
3.26
4.39
3.63
47.97
63.85
54.80
328.56
pla85900
5.31
3.43
4.47
3.69
119.78
148.30
132.78
553.96
Average
3.35
1.87
2.51
2.09
—
—
—
—
excess
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
489
References [1] Ahmadi J, Grotzinger S, Johnson D. Component allocation and partitioning for a dual delivery placement machine. Operations Research 1998;36(2):176–91. [2] Ball M, Magazine M. Sequencing of insertion in printed circuit board assembly. Operations Research 1989;36(2): 192–210. [3] Burke EK, Cowling P, Keuthen R. New models and heuristics for component placement in printed circuit board assembly. In: Proceedings of the 1999 International Conference on Information, Intelligence and Systems (ICIIS99), 1999. p. 133– 40. [4] Burke EK, Cowling P, Keuthen R. ECective heuristic and metaheuristic approaches to optimize component placement in printed circuit board assembly. In: Proceedings of the Congress on Evolutionary Computation CEC2000, 2000. p. 301–8. [5] Shevell SF, Buzacott JA, Magazine MJ. Simulation and analysis of a circuit board manufacturing facility. In: Proceeding of the 1986 Winter Simulation Conference (December), Washington, DC, 1986. p. 686 –93. [6] Hong JM, Lee SH, Lee BH. Hierarchical optimization method in the PCB assembly for surface mounting machines. In: Proceedings of the 1997 IEEE International Symposium on Industrial Electronics, ISIE, 1997. p. 129 –34. [7] Lee SH, Hong JM, Kim DW, Lee BH. ECective algorithm for a surface mounting machine in printed circuit board assembly. In: Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems, 1997. p. 932–7. [8] Burke EK, Cowling P, Keuthen R. The printed circuit board assembly problem: heuristic approaches for multi-headed placement machinery. In: Proceedings of the International Conference on Arti:cial Intelligence 2001 (IC-AI2001), Las Vegas, Nevada, USA: C.S.R.E.A. Press; 2001. [9] Lawler EJ, Lenstra JK, Rinnoy Kan AHG, Shmoys DB. The travelling salesman problem: a guided tour of combinatorial optimization. New York: Wiley; 1985. [10] Reinelt G. The travelling salesman: computational solutions for TSP applications. In: Lecture notes in computer science. Berlin: Springer; 1994. [11] Garey MR, Johnson DS. Computers and intractability: a guide to the theory of NP-completeness. San Fransisco, CA: Freeman; 1979. [12] Pekney JF, Miller DL. Exact solution of the no-wait Jowshop scheduling problem with a comparison to heuristic methods. Computers & Chemical Engineering 1991;15(11):741–8. [13] Kanellakis PC, Papadimitriou CH. Local search for the asymmetric traveling salesman problem. Operations Research 1980;28(5):1086–99. [14] Cowling P. Optimization in steel hot rolling. In: Optimization in industry, Chichester, England: Wiley; 1995. p. 55 – 66. [15] Laporte G, Semet F, Dadeshidze VV, Olsson LJ. A tiling and routing heuristic for screening of cytological samples. Journal of the OR Society 1998;49:1233–8. [16] SankoC D, Blanchette M. Multiple genome rearrangement and breakpoint phylogeny. Journal of Computational Biology 1998;5(4):555–70. [17] Lin KH, Chen CS, Twang TT. Layout-driven chaining of scan Jip-Jops. IEE Proceedings–Computers and Digital Techniques 1996;143(6):421–5. [18] Wade G, Roberts A. Ordering of cascade FIR :lter structures. Electronics Letters 1994;30(17):1393–994. [19] Bland R, Shallcross DF. Large travelling salesman problems arising from experiments in X-ray crystallography: a preliminary report on computation. OR Letters 1989;8:125–8. [20] Kolohan F, Liang M. A tabu search approach to optimization of drilling operations. Computers in Engineering 1996;31(1):371–4. [21] Walas RA, Askin RG. An algorithm for NC turret punch press tool location and hit sequencing. IIE Transactions 1984;16(3):280–7. [22] Johnson DS. Local optimization and the traveling salesman problem. In: Proceedings of the 17th Colloquium on Automata, Language and Programming, Lecture notes in computer science, vol. 443. Berlin: Springer, 1990. p. 446 – 61. [23] Applegate D, Bixby R, ChvOatal V, Cook W. Finding tours in the TSP. Technical Report 99885, Research Institute for Discrete Mathematics, Universitaet Bonn, 1999.
490
P.I. Cowling, R. Keuthen / Computers & Operations Research 32 (2005) 465 – 490
[24] Helsgaun K. An eCective implementation of the Lin–Kernighan traveling salesman heuristic. European Journal of Operational Research 2000;126:106–30. [25] Rego C. Relaxed tours and path ejections for the traveling salesman problem. European Journal of Operational Research 1998;106:522–38. [26] Lin S, Kernighan BW. An eCective heuristic algorithm for the travelling salesman problem. Operational Research 1973;21:498–516. [27] Johnson DS, McGeoch LA. The travelling salesman problem: a case study. In: Local search in combinatorial optimization. New York: Wiley; 1997. p. 215 –310. [28] Bellman RE. Dynamic programming treatment of the traveling salesman problem. Journal of the Association for Computing Machinery 1962;9:61–3. [29] Held M, Karp RM. A dynamic programming approach to sequencing problems. Journal of the Society for Industrial and Applied Mathematics 1962;10(1):196–210. [30] Burke EK, Cowling P, Keuthen R. ECective local and guided variable neighborhood search methods for the asymmetric travelling salesman problem. In: Proceedings of the EvoWorkshops 2001, Lecture notes in computer science, Berlin: Springer; 2001. p. 203–12. [31] Bentley JL. Fast algorithms for geometric traveling salesman problems. ORSA Journal on Computing 1992;4(4): 387–411. [32] Martin O, Otto SW, Felten EW. Large-step markov chains for the traveling salesman problem. Complex Systems 1991;5(3):299–326. [33] Martin O, Otto SW, Felten EW. Large-step Markov chains for the TSP incorporating local search heuristics. Operations Research Letters 1992;11(4):219–24. [34] Martin O, Otto SW. Combining simulated annealing with local search heuristics. Annals of Operations Research 1996. [35] Hansen P, MladenoviOc N. An introduction to variable neighborhood search. In: Voss S. et al., editor. Proceeding of the 2nd International Conference on Metaheuristics-MIC97, 1998. [36] Hansen P, MladenoviOc N. Variable neighborhood search: principles and applications. European Journal of Operational Research 2001;130:449–67. [37] MladenoviOc N, Hansen P. Variable neighborhood search. Computers & Operations Research 1997;24:1097–100. [38] Kirpatrick S, Gelatt CD, Vecchi MP. Optimization by simulated annealing. Science 1983;220:671–80. [39] Applegate D, Cook W, Rohe A. Chained Lin-Kerningham for large travelling salesman problems. Technical Report 99887, Research Institute for Discrete Mathematics, Universitaet Bonn, 1999. [40] Cirasella J, Johnson DS, McGeoch LA, Zhang W. The asymmetric traveling salesman problem: algorithms, instance generators, and tests. In: ALENEX 2001 Proceedings, Lecture notes in computer science. Berlin: Springer; 2001. [41] Baum EB. Iterated descent: a better algorithm for local search in combinatorial optimization problems. Manuscript 1986. [42] Baum EB. Towards practical ‘neural’ computation for combinatorial optimization problems. In: Neural Networks for Computing, AIP Conference Proceedings, 1986. p. 151. [43] Hong I, Kahng AB, Moon BR. Improved large-step markov chain variants for the symmetric tsp. Journal of Heuristics 1997;3(1):63–81. [44] Codenotti B, Manzini G, Margara L, Resta G. Perturbation: an e4cient technique for the solution of very large instances of the Euclidean TSP. INFORMS Journal of Computing 1996;8:125–1330. [45] Reinelt G. TSPLIB-A travelling salesman problem library. ORSA Journal on Computing 1991;3:376–84. [46] Reinelt G. TSPLIB95. Technical report, Institut fuer Angewandte Mathematik, Universitaet Heidelberg, 1995. [47] Glover F. Finding a best traveling salesman 4-opt move in the same time as a best 2-opt move. Journal of Heuristics 1996;2(2):169–79. [48] Bentley JL. Multidimensional binary search trees used for associative searching. Communications of the ACM 1975;18(9):509–17. [49] Bentley JL. K-d trees for semidynamic point sets. In: Proceedings of the 6th Annual Symposium on Computational Geometry, 1990. p. 187–97. [50] Johnson DS. Personal Communication, 2001.
