Information Sciences 177 (2007) 1248–1264 www.elsevier.com/locate/ins
Exchange strategies for multiple Ant Colony System Issmail Ellabib
a,*
, Paul Calamai a, Otman Basir
b
a
b
Department of Systems Design Engineering, University of Waterloo, Waterloo, Ont., Canada N2L 3G1 Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ont., Canada N2L 3G1 Received 20 June 2005; received in revised form 18 September 2006; accepted 23 September 2006
Abstract In this paper we apply the concept of parallel processing to enhance the performance of the Ant Colony System algorithm. New exchange strategies based on a weighting scheme are introduced under three different types of interactions. A search assessment technique based on a team consensus methodology is developed to study the influence of these strategies on the search behavior. This technique demonstrates the influence of these strategies in terms of search diversity. The performance of the Multiple Ant Colony System algorithm, applied to the Vehicle Routing Problem with Time Windows as well as the Traveling Salesman Problem, is investigated and evaluated with respect to solution quality and computational effort. The experimental studies demonstrate that the Multiple Ant Colony System outperforms the sequential Ant Colony System. The studies also indicate that the weighting scheme improves performance, particularly in strategies that share pheromone information among all colonies. A considerable improvement is also obtained by combining the Multiple Ant Colony System with a local search procedure. 2006 Elsevier Inc. All rights reserved. Keywords: Parallel and distributed Ant Colony optimization; Ant Colony System; Information exchange strategies; Vehicle routing problem with time windows; Traveling salesman problem
1. Introduction Swarm Intelligence (SI) is a property exhibited by some mobile systems such as social insect colonies and other animal societies that have collective behavior [3]. Individuals of those systems such as ants, termites and wasps are not generally considered to be individually intelligent however they do exhibit a degree of intelligence, as a group, by interacting with each other and with their environment. Those systems generally consist of some individuals sensing and acting through a common environment to produce a complex global behavior. They share many appealing and promising features that can be exploited to solve hard problems. Furthermore, they are particularly well suited for distributed optimization, in which the system can be explicitly formulated in terms of computational agents.
*
Corresponding author. Tel.: +1 519 885 1211x6813; fax: +1 519 746 4791. E-mail address:
[email protected] (I. Ellabib).
0020-0255/$ - see front matter 2006 Elsevier Inc. All rights reserved. doi:10.1016/j.ins.2006.09.016
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
1249
One of the most popular swarm inspired methods in computational intelligence areas is the Ant Colony Optimization (ACO) method. This method is inspired by the foraging behavior of an ant system and has many successful applications to discrete optimization problems. The basic idea behind ACO algorithms is to simulate the foraging behavior of a swarm of real ants using a swarm of artificial ants working as cooperative agents to construct high quality solutions using a construction procedure. This procedure manages a colony of ants that concurrently and asynchronously constructs problem solutions by moving through neighbor nodes of the problem’s construction graph [14]. The ACO metaheuristic is particularly amenable to parallelization, not just as the parallel version of a sequential algorithm intended to provide speed gains, but as a new kind of metaheuristic of higher efficiency and efficacy. In this paper, the aim is to enhance the performance of the sequential ACO algorithms by organizing the interaction among a group of colonies (i.e., each colony is associated with a complete ACO algorithm) via an exchange module. This can be achieved by developing a multiple colony approach in which each colony interacts and works in parallel with other colonies to solve a common problem cooperatively. We carry out some experiments to evaluate the effectiveness of the proposed exchange strategies and demonstrate the potential of applying the multiple colony approach over the sequential one on solving both the Traveling Salesman Problem (TSP), as a very well documented problem of multimodal combinatorial optimization problems, and the Vehicle Routing Problem with Time Windows (VRPTW), as it has many applications in transportation systems. This paper is organized as follows. In Section 2 we describe the Ant Colony System and our Ant Colony System model for solving the Vehicle Routing Problem with Time Windows. In Section 3 we briefly review some existing parallel and distributed ACO approaches. In Section 4 the concept of parallel processing is applied to the Ant Colony System and new exchange strategies based on a weighting scheme are introduced. A search assessment technique based on a team consensus methodology is introduced in Section 5 to describe the level of search diversity. Experimental results from the Multiple Ant Colony System are presented in Section 6 along with a comparative performance analysis involving other existing approaches. Finally, Section 7 provides some concluding remarks. 2. Ant Colony System In ACO algorithms, simple artificial ants act as co-operative agents to generate high quality solutions to a combinatorial optimization problem via interaction between the ants and their environment. The ants use a stochastic construction heuristic that employs probabilistic decisions on the basis of artificial pheromone trails and problem-specific heuristic information. The stochastic component allows the ants to explore a greater number of solutions than greedy heuristics. At the same time, the use of heuristic information helps guide the ants toward promising search regions using the collective interaction of a population of ants. Pheromone is accumulated during the construction phase through a learning mechanism implied in the algorithm’s pheromone update rule. In particular, ants move from one node to another on a construction graph using a transition rule that favors shorter edges and edges with greater amounts of pheromone. They update the pheromone trail of their generated tours based on a pheromone update rule. This rule deposits a quantity of pheromone proportional to the quality (or length) of the corresponding tour. Pheromone evaporation (i.e., where pheromone intensity decreases over time) is applied to avoid the unlimited accumulation of pheromone and the possible resulting premature convergence toward a suboptimal solution region. This mechanism thus favors the exploration of the search space. Dorigo et al. [13] introduced the concept of applying the underlying properties of ant foraging behavior to create a new computational approach, called Ant System (AS), to solve combinatorial optimization problems. Computational experiments on the Traveling Salesman Problem (TSP) established that the original AS algorithm does not scale well to large-scale instances. Subsequently, several modifications to this algorithm have been introduced, typically with different choices for the construction rules and the pheromone representation, such as Ant-Q system [18], Max–Min Ant System [35], and Ant Colony System [12]. In this paper we focus on the Ant Colony System. ACO algorithms are different from other stochastic search algorithms in which a set of parallel artificial ants that iteratively build solutions using a probabilistic transition rule based mainly on the pheromone intensity on the edges. This pheromone is updated according to a learning rule. The Ant Colony System (ACS) algorithm features two major changes to the rules employed in the AS algorithm [12], namely:
1250
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
(1) A new transition rule is introduced that favors either exploitation or exploration. From node i, the next node j in the route is selected by ant k, among the unvisited nodes J ki , according to the following transition rule: 8 1 if fq 6 q0 & j ¼ j g > > > < 0 if fq 6 q0 & j 6¼ j g ð1Þ P kij ðtÞ ¼ b > ½sij ðtÞ½gij ðtÞ > P > otherwise : ½s ðtÞ½g ðtÞb u2J k i
iu
iu
where j ¼ arg maxu2J ki ð½siu ðtÞ½giu ðtÞb Þ identifies the unvisited node in J ki that maximizes P kij ðtÞ, sij(t) is the amount of pheromone on the edge joining nodes i and j, gij(t) is the heuristic information for the ant visibility measure (e.g., defined as the reciprocal of the distance between node i and node j for the TSP), b is a control parameter that represents the relative importance of the ant visibility value versus the amount of pheromone on the edge joining nodes i and j, q is a generated random number in the range [0, 1], and q0 is a given threshold parameter. Thus, when q is less than or equal to q0 the ant employs exploitation to select node j* as the next node in its tour, whereas if q exceeds q0 the ant uses probabilistic exploration to select the next node in its tour. (2) The pheromone trial is updated in two different ways • Local updating: As the ant moves between nodes i and j, it updates the amount of pheromone on the traversed edge using the following formula: sij ðt þ 1Þ ¼ ð1 qÞsij ðtÞ þ qs0
if fedgeði; jÞ 2 T k g
ð2Þ
where s0, the initial amount of pheromone,1 is calculated as s0 = (nCi)1, n is the problem size (i.e., the number of nodes) and Ci is the cost of the initial tour produced by a construction heuristic such as the Nearest Neighbor heuristic, and the evaporation rate, q, is a parameter in the range [0, 1] that regulates the reduction of pheromone on the edges. The effect of local updating is that each time an ant traverses an edge(i, j) its pheromone trail sij is reduced, so that edges becomes less desirable for the ants in future iterations. This encourages an increase in the exploration of edges that have not been visited yet. Local updating helps avoid poor stagnation situations.2 • Global updating: When all ants have generated their tours, the edges belonging to the best tour are updated using the following formula: sij ðt þ 1Þ ¼ ð1 qÞsij ðtÞ þ qð1=C b Þ if fedgeði; jÞ 2 T b g
ð3Þ
where Cb is the cost of the best tour Tb found since the start of the algorithm. It is important to note that global updating adjusts only the pheromone on the edges belonging to the best tour. This encourages ants in future iterations to search in the vicinity of this best tour. 2.1. An Ant Colony System model for the VRPTW The VRPTW is focused on the efficient use of a fleet of capacitated vehicles that must make a number of stops to serve a set of customers so as to minimize cost, subject to vehicle capacity constraints and service time restrictions imposed at the customer locations. Due to the complexity of the problem, metaheuristics are often used for analyzing and solving practical sized instances of this problem [20,16,17]. In this section, the construction procedure of the Ant Colony System algorithm is adapted to solve the VRPTW by considering the design structure and the constraints of this problem. The VRPTW involves the construction of minimal cost routes, each route serviced by one vehicle, such that all customers are serviced exactly once. A set of vehicles is available to satisfy customer requests. Each request
1 2
At the beginning of the search a small amount of pheromone is assigned to all the edges. Stagnation occurs when the algorithm reaches an equilibrium state (i.e., a single path is chosen by all ants).
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
1251
specifies the size of the load to be transported, the pickup or delivery location denoted by the XY coordinates, and the time window defining allowable pickup or deliver time period at customer node i. In the proposed Single Ant Colony System (S-ACS) model, the Nearest Neighbor construction heuristic (NN) is first applied to generate an initial feasible solution (set of vehicle tours) to this problem. Then, these tours are transformed into a single corresponding ant tour by assigning a number of depots (with the same coordinates) equal to the initial number of vehicles constructed by the Nearest Neighbor heuristic. An initial amount of pheromone s0, based on the quality of the initial solution, is provided to all the edges. Then, the procedure constructs a single tour by an ant according to the problem structure and constraints, and corresponding to the multiple vehicle tours. In the S-ACS algorithm shown in Fig. 1, an ant constructs a route by choosing which customers to add to the route in a step-by-step process. When no additional customers can be added, the ant returns to the depot. Each ant starts from a depot and moves to a feasible unvisited customer based on the transition rule defined by Eq. (1) until all the customers are visited. After each decision step, the amount of pheromone deposited on the selected edge is updated locally by Eq. (2). The hierarchal objective criteria is applied for minimizing the number of vehicles and the total travel distance by updating the best solution found so far based on minimizing the number of vehicles and then the total travel distance, i.e., a feasible solution with v vehicles is better than a feasible solution with v + 1 vehicles, even if the total travel distance for the v vehicles solution is greater than that for the v + 1 vehicles solution. When the ants have visited all customers and their solutions are evaluated, the pheromone trails belonging to the best solution are updated globally by Eq. (3). The cost Ck of tour Tk, constructed by ant k, is calculated using C k ¼ C f V k þ Dk
ð4Þ
where Vk is the number of vehicles and Dk is the total travel distance of Tk. A fixed cost Cf is added to the total travel cost for each vehicle launched from the depot. The value of Cf is sufficiently large so as to bias the optimization to primarily minimize the number of vehicles and secondarily minimize the total travel distance. The information that we consider in the heuristic function (gij) are the travel time, the waiting time, and the urgency of service between the current node i and the next candidate node j. We represent this information using an unweighed heuristic function that uses this data in the manner proposed by Gambardella et al. [19]. In the NN heuristic, gij is used to measure the closeness of customers in the construction of an initial feasible solution to be improved by the S-ACS construction procedure. In the S-ACS construction procedure, gij (i.e., ant visibility) is used with the amount of pheromone, sij, to direct the search using the transition rule of
Fig. 1. A pseudo-code description of the S-ACS algorithm.
1252
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
Eq. (1). In particular, this function represents the time difference between the completion of service at node i and the beginning of service at node j (given by Wij), and the urgency of service (given by Uij) at node j when the vehicle arrives from node i as given by gij ¼ ½W ij U ij 1
ð5Þ
where Wij and Uij are given by W ij ¼ bkj ðbki þ si Þ U ij ¼ lj
ðbki
þ si Þ
ð6Þ ð7Þ
si is the service time at node i, lj is the latest start of service time at node j, and bki is the start of service time by vehicle k at node i. 3. The existing parallel and distributed ACO approaches The parallel architecture of the ACO algorithms can be exploited through the acceleration of the construction procedure, the decomposition of the problem domain or search space, or through the design of a distributed colony approach (i.e., multiple colony approach). Dorigo and Di Caro [11] as well as Dorigo and Stu¨tzle [14] provide an overview of some parallel implementations. Most parallelization models can be classified into fine-grained and coarse-grained models. In the fine-grained model, the population of ants is separated into a large number of very small sub-populations. These sub-populations are maintained by different processors, and some information is exchanged frequently among those processors. This model is suitable for massively parallel architecture-systems with a huge number of processors connected with a specific high speed topology. In the coarse-grained model, the population of ants is divided into few sub-populations. These sub-populations are maintained by different processors and some information is exchanged among those processors at every defined exchange step. This model is suitable for clustered computers. Bullnheimer et al. [4] introduced two parallel implementations of the AS algorithm called the Synchronous Parallel Implementation (SPI) and the Partially Asynchronous Parallel Implementation (PAPI). SPI is based on a master–slave paradigm in which every ant finds a solution in the slave and sends the result to the master. When the solutions are available from all slaves the master updates the pheromone information and sends the updated information back to all slaves. This implementation parallelizes the construction phase, but it has the disadvantage that all ants have to wait for all others every iteration, which leads to a communication overhead between the master and the slaves. PAPI is based on the coarse-grained model in which information is exchanged among colonies every fixed number of iterations. The considerable cost of communication encountered in the two implementations is reported in [4], which indicates that PAPI performs better than SPI in terms of running time and speedup. Randall and Lewis [30] developed a parallel ACO to solve the TSP on a distributed memory architecture. Their approach of the ‘‘parallel ant’’ type is an internal parallelization composed of a master processor and multiple slave processors. The algorithm assigns only one ant to each processor. They demonstrated that one of the disadvantages to this approach is the large volume of communication required to maintain the pheromone matrix. Islam et al. [22] developed an on-demand routing algorithm for ad hoc networks using an ACO algorithm and parallelized this algorithm, on the basis of individual ants, using a distributed memory machine. They noticed that the percentage of communication among the ants is much higher than the percentage of computation by an ant. Piriyakumar and Levi [27] developed a multiple colony approach based on the ACS and the MMAS algorithms. In their approach, the global update is carried out after some predefined number of iterations instead of every iteration, and the best solution is exchanged among all colonies. This balances the overhead of the communication time and idle time and the sharing of the pheromone values. Stu¨tzle [34] studied the effect on solution quality of applying independent runs of an ACO algorithm without communication compared with one longer run of the algorithm. He demonstrated the effectiveness of using parallel independent runs over sequential runs of the Max–Min AS algorithm for solving the TSP. Middendorf et al. [25,26] introduced a parallel implementation, based on the coarse-grained model, of the AS algorithm. Their multiple colony approach employs best-exchange strategies in which best solution information is
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
1253
exchanged between colonies under two different topologies. They studied the performance of employing these strategies, and the results indicated that the multiple AS algorithm employing some strategies outperforms the single AS algorithm in certain instances. Chu et al. [5] introduced a parallel implementation, based on the coarse-grained model, of the ACS algorithm and used the same idea of the best solution exchange but under several communication topologies. Preliminary experiments indicated that the Parallel Ant Colony System algorithm outperforms both the AS and the ACS algorithms when solving a TSP. Rahoual et al. [29] presented two parallel ACO Ant System approaches combined with a local search for solving the Set Covering Problem. The first approach duplicates the algorithm on several processors units, which process independently of each other. In the second approach, each ant is considered a process and is assigned to one processor. Experimental tests confirm that these parallel approaches offer good performances comparing to sequential algorithms. Doerner et al. [9,10] introduced different parallelization strategies to solve the capacitated Vehicle Routing Problem (VRP). In [9], a Savings Based ACO algorithm that employs a rank based Ant System (ASrank) for pheromone management is parallelized on the basis of individual ants. In [10], a combination of coarse-grained and fine-grained parallelization for the D-Ant algorithm is introduced to speed up the search in which the ants solve only sub-problems rather than the whole problem. The goal of these approaches is to improve the execution time of the algorithms without altering their behavior. Delisle et al. [8] introduced a parallel ACO approach to solve an industrial scheduling problem in an aluminum casting center. This approach is implemented on the basis of individual ants in a shared memory model with OpenMP. The goal of this approach is also to improve the execution time of the sequential ACO algorithm without altering its behavior. Talbi et al. [37] introduced another master–slave paradigm to parallelize the AS algorithm in which a local search method based on tabu search is introduced in each slave to improve the solution constructed by the ant. Their results demonstrated the complementary gains brought by incorporating tabu search within the AS algorithm in the parallel implementation. Lee et al. [24] introduced a multiple colony approach to solve a Weapon–Target Assignment problem. In this approach, a heuristic process is used as an interaction process among parallel ant colonies. It is conducted only for the best solution through a greedy reformation. Alba et al. [1] introduced two parallel approaches to solve the Minimum Tardy Task problem. The first approach employs parallel independent ant colonies in which a number a sequential ACO searches are executed on a set of processors without any communication. Each colony is differentiated in the values of key parameters. The second approach employs parallel interacting ant colonies with an exchange of information between colonies. The global update is carried out after some predefined number of iterations instead of every iteration, and the best solution is exchanged among the colonies according to a ring topology. Jong and Wiering [6] developed a multiple ant colony approach to solve the Bus Stop Allocation Problem. In their approach each busline is represented by a separate ACS and the pheromone levels of each ACS are updated separately. This approach outperformed both a Greedy Algorithm and Simulated Annealing on all test problems. Kawamura et al. [23] developed a multiple ant colony approach to solve the TSP in which each colony corresponds to the AS colony. Positive and negative pheromone effects are controlled through a colony-level interaction between colonies. Their approach demonstrated that a positive pheromone effect is better than a negative pheromone effect in terms of algorithm performance. In [23], performance is improved by incorporating a 2-opt heuristic technique. Gambardella et al. [19] decomposed the search space for VRPTW among two heterogeneous colonies in such a way that the two objectives of the VRPTW are optimized simultaneously by coordinating the activities of the two colonies. The first colony is applied to minimize the total travel distance and the second colony is applied to minimize the number of vehicles. The two colonies collaborated when the best feasible solution is improved. This approach combined with the local search procedure of Taillard et al. [36] is shown to be competitive with other successful approaches. Different parallel approaches were proposed in the literature in order to improve the performance of the sequential ACO algorithms. Although a number of parallel versions of ACO have been implemented and tested in limited settings, it is still an open question as to how an efficient parallel version of an ACO algorithm should be implemented, and how much improvement can be obtained over the sequential ACO algorithm. As well, since performance can be enhanced not only by the parallel implementation but also by the efficiency of the communication strategies that are incorporated in the parallel implementation, these strategies need to be
1254
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
studied. In this paper, we develop a multiple colony system in which each identical colony interacts and works in parallel based on new communication strategies (i.e., exchange strategies). We also study both the search behavior and performance of the proposed approach. We develop a search assessment technique based on a team consensus methodology to study the influence of the exchange strategies on the search behavior. We carry out a performance study to evaluate the effectiveness of the exchange strategies and demonstrate the potential of applying the multiple colony approach over the single colony approach on both the VRPTW and the TSP. 4. Multiple Ant Colony System We believe that the parallel running of a group of colonies can be enhanced by organizing those colonies in such a way that they can share their information efficiently. This information can be utilized by colonies via an exchange module that defines the interaction between the group of connected colonies. The interaction between the colonies relies on their adopting an efficient communication architecture that facilitates cooperation between them, and an exchange strategy that defines the rules for collaboration among them. Accordingly, we define an exchange module, as shown in Fig. 2, in which every colony handles the mechanism of cooperation. In the proposed approach, a group of identical colonies search in parallel and communicate with each other via the exchange module. The ants in each colony are divided equally into several groups. Each colony is associated with a complete ACS construction procedure introduced in [12]. Fig. 3 illustrates the complete pseudocode description of the Multiple Ant Colony System (M-ACS) algorithm. In this algorithm, colony h provides its search information to other colonies and receives search information from other colonies at every fixed number of iterations (i.e., exchange interval) via the exchange module procedure. In this procedure, the interaction occurs under different topologies and according to one of the communication strategies described below. In our parallel implementation of the algorithm, each colony consists of a swarm of 10 ants managed by one machine. Machine 0 is responsible for initialization, spawning, and collection and display of the results, while all machines (including machine 0) are responsible for constructing solutions to the problem. A Beowulf cluster is used for this implementation. The Beowulf cluster consists of a collection of eight identical PC machines equipped with 700 MHz Pentium III processors and 128 MB of RAM and interconnected by a Local Area Network (LAN) running the Red Hat Linux operating system. We employed the Message Passing Interface (MPI) software to allow the Beowulf cluster machines to interact and implemented, in the C++ language, the M-ACS algorithm with the option of employing one of the exchange strategies described below. 4.1. Exchange strategies The search for high quality solutions can be improved, in terms of performance, by employing a communication strategy that can propagate current high quality solution information to the colonies and use that information in the pheromone representation.
Colony h Exchange module Search module
Pheromone module
Fig. 2. Colony-level interaction framework.
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
1255
Fig. 3. A pseudo-code description of the M-ACS algorithm.
In the proposed strategies, we adjust the pheromone matrix of each colony through different colony-level interactions and according to the solution information so as to reinforce search in the vicinity of high quality solutions. The behavior of ants in one colony will be influenced by the solution information received from other colonies, where pheromone is added to the colony edges that belong to the best solutions of the group of colonies. A weighting scheme, presented in [15], is used in the proposed multiple colony approach to assess the quality of the best solutions constructed by several colonies. In this scheme, the pheromone trails on the edges of the best solutions are updated adaptively in response to determined weights, and an extra amount of pheromone is deposited on the edges of these solutions accordingly. To identify whether the colonies are converging to one solution or scattered in the search space we calculate the difference between the current overall average cost of the best solutions produced by the selected colonies and the cost of the best solution found so far. The cost of the best solution foundPso far is given by h h 1 C b ¼ minh2S fC hb g. The overall average cost of the best solutions is given by C AVG ¼ jSj h2S C b , where C b is the cost of the best solution found by colony h, for h 2 S where S {1, . . .. , M}, and M is the total number of colonies. We note that the difference between the overall average cost CAVG and the best cost Cb is likely to be less when the selected colonies approach the best solution than it will be when these colonies are scattered in the search space. We therefore use the difference (CAVG Cb) as a yardstick for detecting the convergence of the selected colonies. The colony h 2 S that has a best solution C hb that is less than the overall average CAVG is assigned a weight wh 2 (0, 1], otherwise it is assigned a weight equal to zero. In particular, we use the following formula: ( C AVG C hb if fh 2 S; ðC AVG C hb Þ > 0g h ð8Þ w ¼ CAVG Cb 0 otherwise which depends not only on the measure of convergence previously discussed but also on how close the cost of the best solution for the colony, C hb , is to the cost of the best solution found so far, Cb , in such a way that the closer C hb is to Cb the closer wh is to 1. These weights are used to define the following colony-level interaction pheromone update formula that is applied to each edge h of the best tour T hb : sij ¼ ð1 qÞsij þ qðwh =C hb Þ
if fedgeði; jÞ 2 T hb g
ð9Þ
This update together with the local updating formula (Eq. (2)) is intended to achieve a trade-off between exploration and exploitation. In fact, this formula has the effect of increasing the quantity of pheromone on edges associated with some solutions according to the quality of these solutions and the current convergence state of the colonies.
1256
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264 2 M
3
1
h
Fig. 4. Interaction of colonies in a star topology.
7
6 2
3
2
3
2 5 1
1
1
1D
8
4 4
2D
3D
Fig. 5. Interaction of colonies in a hypercube topology.
1
2
h
M
Fig. 6. Interaction of colonies in a directed ring topology.
The interaction among colonies provides best solution information to each colony under three different topologies as shown in Figs. 4–6. Accordingly, the pheromone is updated based on the weighting scheme with the information provided by the selected topology. In particular, the exchange module exchanges the best solutions between a group of colonies (specified by the selected topology), weighs those best solutions using the weighting scheme, and then applies the colony-level interaction pheromone update formula (Eq. (8)). Accordingly, we define the weighting-exchange strategies as follows: • Strategy-1: the interaction is accomplished among M colonies organized as in Fig. 4. That is, colony 1 exchanges information with colonies 2 through M. • Strategy-2: the interaction is accomplished among a subset of colonies organized as in Fig. 5. That is, colonies 1 and 3 exchange information with colonies 2 and 4 in this 2D topology. • Strategy-3: the interaction is accomplished between two consecutive colonies organized as in Fig. 6. That is, colony 1 exchanges information with colony M and colony 2, colony 2 exchanges information with colony 1 and colony 3, etc.
5. Search assessment technique Several search assessment techniques are introduced to study the search diversity and convergence (i.e., stagnation situations) in different ACO algorithms. In general, a common assessment value is used to aggregate the measures among a group of nodes or ants to assess the search behavior of the ACO algorithms. Averaging is mainly used in the existing techniques to aggregate the individual assessments. These techniques have no reason to expect that there is a difference among the individual assessments. We refer the reader to Dorigo and Stu¨tzle [14] for a detailed description on different assessment techniques. In what follows we describe the consensus problem and how the team consensus methodology is adopted to determine the common assessment value. 5.1. Consensus problem When probability distributions represent the respective judgments of a number of individuals, the single resulting distribution may be thought of as a consensus among the individuals. Determining this single distri-
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
1257
bution is referred to as the consensus problem. In the S-ACS algorithm, jXj decisions have to be taken, based on a probability distribution at each node, by each ant to construct a feasible solution (where X is the set of all nodes, and jXj is the cardinality of X). Let fi(t) represent a probability distribution at decision node i at time t that satisfies the probability distribution requirements. Then, the problem is to determine, from the distributions of all nodes, a single distribution fc(t) that represents consensus among the group of nodes. One way of combining the distributions fi(t) is to take their weighted sum as given by: X fc ðtÞ ¼ pi fi ðtÞ ð10Þ i2X
P where pi P 0 and i2X pi ¼ 1. The problem is to determine appropriate values for those weights. It seems reasonable to assign more weight to fi(t), and hence to decision node i, than to fj(t), and hence to decision node j, only if an ant at the decision node i is more certain to select the candidate node (with regard to the selection of the next customer) than at node j. In fact, the existing assessment techniques assign equal weights (pi = 1/jXj for "i 2 X) for each fi(t) in order to determine the common assessment value. This common assessment makes sense if there isn’t much difference on the assessment of the decision nodes. For an alternative, an appealing idea is proposed in [2] for combining the performance of a group of sensors using the team consensus methodology to reach a consensus in identifying an object. The sensors revise their opinions in a consensus manner; each sensor updates its own utility in light of the revisions made by the others. We adopt this idea to determine the common assessment value in our assessment technique instead of using equal weights. 5.2. Consensus-based assessment technique In general, the main mechanism in the self-organization of the ACO algorithms is the positive feedback given through the pheromone update by the group of ants. In this mechanism, the better the ant’s tour, the larger the amount of pheromone the ant deposits on the edges of its tour. This in turn leads to the fact that these edges have a higher probability of being selected in the subsequent iterations of the algorithms via the state transition rule. Also, the pheromone values on the edges are further influenced by the evaporation process that avoids an unlimited accumulation of pheromone and decreases the pheromone values on edges that never, or very rarely, get visited by ants. In general, the pheromone information on the edges is iteratively updated based on a pheromone update rule, and represented in a pheromone matrix. The search behavior of the S-ACS algorithm, therefore, can be visualized and assessed through the distribution of pheromone trail values in this pheromone matrix as given by X Hi ðtÞ ¼ sij ðtÞ log sij ðtÞ; ð11Þ j2X
where Hi ðtÞ represents the entropy of node i at iteration t, and we interpret this as a measure of the (potential) diversity in the search from this node at that iteration. The decision nodes have different pheromone information distributions to their incident edges and hence they represent different types of uncertainties in constructing a high quality tour. Thus, the consensus methodology can be applied to determine a suitable weight for each decision node based on the distributions of pheromone. In what follows we develop an assessment technique based on this methodology to assess, in relative terms, the search diversity of the algorithm at different stages of the solution process. In doing so, the pheromone matrix, T ¼ ½sij , can be viewed as a one-step transition probability matrix of a homogeneous Markov chain with jXj states representing the converging process of the algorithm. This interpretation enables us to use the limit theorems of Markov chains to determine whether or not the consensus is reachable. According to the convergence property of the algorithm, the pheromone matrix T is irreducible3 and aperiodic.4 Therefore, a consensus is reachable (consult Ref. [7] for the conditions of reaching a consensus). Let Eq. (10) denotes
3
A chain is irreducible if any state can be reached from any other state. A chain is aperiodic if all its states are aperiodic (i.e., the period of states is the largest integer k such that all possible recurrence times for the states are multiples of k, if k = 1 then state i is called aperiodic; if k > 1 then state i is periodic). 4
1258
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264 1 No-Strategy Strategy-1(I=50) Strategy-2(I=50)
0.95
Strategy-3(I=50)
Entropy
0.9
0.85
0.8
0.75 0
20
40
40
80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 440 460 480 500
Iteration
Fig. 7. Average cumulative entropy of the exchange strategies with I = 50, VRPTW test problem: r208.
the value of this consensus, and let p represent the corresponding unique stationary probability vector. Thus, the value of the vector p is determined by solving the linear equations Tp ¼ p
ð12Þ
together with the equation X pi ¼ 1;
ð13Þ
i2X
where p P 0. The stationary probability vector values are used to aggregate the node measures (Eq. (11)) to arrive at the cumulative entropy, Hh ðtÞ, of the S-ACS algorithm (i.e., colony h) at iteration t as follows: X Hh ðtÞ ¼ pi ðtÞHi ðtÞ ð14Þ i2X
5.3. Influence of the exchange strategies In this section, we study the influence of the exchange strategies on the search behavior of the M-ACS algorithm for solving both VRPTW and TSP test problems. In solving the TSP, the VRPTW construction procedure is replaced with the construction procedure of the ACS algorithm introduced in [12]. The search assessment technique described above is applied to the M-ACS algorithm while employing the exchange strategies. The cumulative entropy at different iterations are reported on the VRPTW test problem r208 [32] and the TSP test problem eil101 [31]. The cumulative entropies of the M colonies are averaged at every iteration t = 1, . . . , 500 for M-ACS without employing any strategy (No-Strategy)5 as well as employing the exchange strategies. In order to determine an appropriate exchange interval I (the number of iterations between exchanges), different values are considered for these strategies. In both cases, the appropriate value of I is selected according to the diversity level prior to the exchange (i.e., when the search is diversified), and the fluctuation range—the difference in the entropy prior to the exchange and immediately after the exchange (i.e., when the search is intensified). For the VRPTW, the diversity level and the fluctuation range for I = 50 are generally greater than for other exchange intervals. For this reason the exchange interval I = 50 was used for the remaining compu-
5
M colonies work in parallel without interaction.
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
1259
1 No-Strategy Strategy-1(I=30) Strategy-2(I=30)
Entropy
0.95
Strategy-3(I=30)
0.9
0.85
0.8
0.75 0
20
40
40
80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400 420 440 460 480 500
Iteration
Fig. 8. Average cumulative entropy of the exchange strategies with I = 30, TSP test problem: eil101.
tational tests of the VRPTW. Similarly, for the TSP, the diversity level and the fluctuation range for I = 30 are generally greater than for other exchange intervals. Thus, the exchange interval I = 30 was used for the computational tests of the TSP. Figs. 7 and 8 illustrate the average cumulative entropies of No-Strategy and the three exchange strategies (Strategy-1, Strategy-2, and Strategy-3) for the VRPTW and the TSP test problems, respectively. As illustrated, Strategy-1 exhibits the largest swing (largest fluctuation) in cumulative entropy about every exchange step. This strategy also intensifies the search more than the other strategies (i.e., has lower entropy at the exchange step) and contrasts with Strategy-3, which diversifies the search the most (i.e., has the highest entropy prior to the exchange step). This indicates that the exchange strategies produce searches that are different in their ability to intensify and diversify the search at every exchange step. 6. Computational experiments and comparisons A performance study was carried out to evaluate the effectiveness of the exchange strategies and demonstrate the potential of applying M-ACS over S-ACS on the VRPTW and the TSP. For the performance evaluations and comparisons, the results of all experiments were obtained using fixed parameter values for all problem instances. The parameter settings, as proposed in [12], were set to b = 2, q = 0.1, q0 = 0.9, and m = 10 ants. We demonstrated the potential of applying the multiple colony approach over the sequential approach to the extent of giving the S-ACS a computational time equal to the time given to each colony multiply by the number of colonies in the M-ACS as proposed in [34]. We have found that 500 iterations are large enough for the construction procedure (with m = 10 ants) to provide high quality solutions. We have also noticed that the construction procedure with 10 ants provide better performance than 80 ants in both problems (i.e., TSP and VRPTW). However, an investigation to find a better choice for parameter values is still needed. 6.1. Computational results on the Solomon data sets To examine the solution quality for any improvement, the performance is subjected to a comparative test on a wide range of VRPTW instances with different characteristics. Standard benchmark problem instances are available to support and facilitate meaningful comparison in performance [32]. Solomon introduced 56problem instances in six different data sets [33]. These instances vary in some characteristics such as fleet size, vehicle capacity, travel time, spatial and temporal customer distribution (position, time window density, time window width, and service time). Distances are measured using the Euclidean metric. Each instance involves a
1260
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
central depot of vehicles, and one hundred customers to be serviced. The instances are subdivided into six data sets (C1, C2, R1, R2, RC1, RC2). C1 and C2 are characterized by a clustered customer distribution. R1 and R2 are characterized by a random customer distribution. RC1 and RC2 are characterized by a semi-clustered customer distribution. We evaluate the performance in terms of the average of the best solutions for the instances of each data set (Mean Number of Vehicles, MNV, and Mean total Travel Distance, MTD) as well as the Cumulated Number of Vehicles, CNV, and the Cumulated Travel Distance, CTD of these solutions for all 56 instances. In the first experiment, a comparison on the performance of M-ACS employing the exchange strategies, with exchanges every 50 iterations, is performed. The number of iterations was set to 5000. A performance comparison is summarized in Table 1. Numbers in bold indicate the worst result when unique. We emphasize on the worst results to demonstrate the consistency not only for the best strategy but also for other strategies that can provide good results for all data sets. The results in this table illustrate that Strategy-1 exhibits better performance, in terms of MNV, MTD, CNV, and CTD than the other strategies. In the second experiment, a comparison on the performance of S-ACS and M-ACS (with and without employing an exchange strategy) was performed. We set 3 minutes of computation time for the M-ACS algorithm and 3 · 8 minutes of computation time for the S-ACS algorithm to demonstrate the potential of applying the multiple colony approach over the sequential approach. The M-ACS results are obtained without employing any exchange strategy, No-Strategy, and employing Strategy-1 (since best results are obtained by M-ACS employing this strategy). A performance comparison is summarized in Table 2. Numbers in bold indicate the worst result when unique. The results in this table illustrate a significant improvement in the performance, in terms of MNV and MTD, of M-ACS over S-ACS for data sets (R1, R2, and RC1) and quite similar performance for the other data sets. In terms of CNV and CTD, M-ACS also obtained better results than S-ACS. The improvements achieved by No-Strategy over S-ACS are in the total number of vehicle with five instances having a reduction of one vehicle each, and in the total travel distance of 26 instances with an average of a 3.67% reduction in tour length. The improvements achieved by Strategy-1 over S-ACS are in the total number of vehicle with seven instances having a reduction of one vehicle each and in the total travel distance of 43 instances with an average of a 4.66% reduction in tour length. Note that the difference in the improvements between No-Strategy and Strategy-1 reveals the synergetic effect caused by the cooperation process implied in Strategy-1. Furthermore, M-ACS required only 12.5% of the S-ACS computation time. This has led us to conclude that there is a potential advantage in running M-ACS for a proportionally shorter time than S-ACS.
Table 1 Comparison of results obtained by M-ACS employing the exchange strategies for the Solomon data sets Data set
Mean values
M-ACS employing
C1
MNV MTD
10.00 859.00
10.00 868.20
10.00 868.62
C2
MNV MTD
3.00 613.25
3.00 620.41
3.00 618.28
R1
MNV MTD
12.50 1281.65
12.50 1283.55
12.50 1293.56
R2
MNV MTD
3.00 1131.01
3.00 1152.45
3.00 1179.42
RC1
MNV MTD
12.13 1421.85
12.25 1427.62
12.25 1435.23
RC2
MNV MTD
3.38 1314.38
3.38 1320.77
3.38 1342.82
All
CNV CTD
Strategy-1
421 62,348
Strategy-2
422 62,844
Strategy-3
422 63,485
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
1261
Table 2 Comparison of results obtained by S-ACS and M-ACS for the Solomon data sets Data set
Mean values
S-ACS
M-ACS employing No-Strategy
Strategy-1
C1
MNV MTD
10.00 886.90
10.00 870.48
10.00 859.11
C2
MNV MTD
3.00 624.87
3.00 631.75
3.00 620.78
R1
MNV MTD
12.83 1314.76
12.58 1304.65
12.50 1282.40
R2
MNV MTD
3.09 1224.58
3.00 1199.08
3.00 1137.35
RC1
MNV MTD
12.38 1435.45
12.25 1455.02
12.13 1423.85
RC2
MNV MTD
3.38 1405.27
3.38 1363.06
3.38 1317.23
All
CNV CTD
423 64,279
428 64,954
421 62,526
6.1.1. Combining M-ACS with a local search In this section, we combine M-ACS with the 2-opt* exchange heuristic of Potvin and Rousseau [28] (M-ACS + 2opt*) to test the capability of M-ACS to generate appropriate solutions for a route improvement heuristic. Then, a performance comparison between M-ACS + 2opt* and the multiple colony approach of Gambardella et al. [19] (MACS-VRPTW) is conducted. The 2-opt* exchange heuristic is used to improve the route constructed by the ants, as it is particularly useful when the time constraints are the main concern [28]. We embedded the 2-opt* exchange heuristic in the construction phase of the M-ACS algorithm in order to progressively improve the current solutions constructed by the ants. Because of the high computational cost that would be required to improve the current solutions at every iteration, the 2-opt* exchange procedure was only called prior to every exchange step (i.e., at the highest levels of diversity) as in step 10 of the M-ACS algorithm.
Table 3 Performance comparison between M-ACS + 2opt* and MACS-VRPTW for the Solomon data sets Data set
Mean values
C1
MNV MTD
M-ACS + 2opt* 10.00 834.58
10.00 828.38
C2
MNV MTD
3.00 601.19
3.00 591.85
R1
MNV MTD
12.33 1234.33
12.38 1210.83
R2
MNV MTD
3.00 1072.19
3.00 960.31
RC1
MNV MTD
11.88 1385.51
11.92 1388.13
RC2
MNV MTD
3.38 1238.68
3.33 1149.28
All
CNV CTD
417 59,920
MACS-VRPTW
418 57,583
1262
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
Table 4 Performance comparison between ACS, PACS, and M-ACS employing Strategy-1, Strategy-2, and Strategy-3 Instance
PACS
Strategy-1
Strategy-2
Strategy-3
st70
Mn SD
678.8 3.3
675.6 1.1
676.1 1.0
676.7 1.4
eil101
Mn SD
646.3 4.3
638.1 3.6
639.6 5.3
642.1 3.14
tsp225
Mn SD
3887.0 12.9
3936.7 14.6
3942.4 17.4
3971.8 39.00
We set the number of runs and the computation time of our approach consistent with the MACS-VRPTW approach. Performance was evaluated on the basis of the best solution obtained in three runs on each of the 56problem instances, and the computation time was set to 30 minutes for each run. The MACS-VRPTW approach was executed on a Sun UltraSparc-1 167 MHz machine. The performance is summarized in Table 3. Numbers in bold indicate the worst result when unique. Although this comparison is imperfect, since results were obtained using different computer platforms, the following observations can be made. M-ACS + 2opt* obtained solutions requiring a fewer number of vehicles for data sets R1 and RC1 and requiring more vehicles in data set RC2. For data set R2, M-ACS + 2opt* obtained solutions requiring the same number of vehicles with relatively longer MTD. For the two clustered data sets both approaches produced relatively the same results. In terms of CNV and CTD, M-ACS + 2opt* obtained lower CNV with relatively large CTD. Note that MACS-VRPTW employed a more sophisticated route improvement heuristic, the CROSS-exchange heuristic of Taillard et al. [36], to improve solutions. In fact, this heuristic exchanges segments of consecutive customers between two routes up to the length of five customers, whereas the 2-opt* exchange heuristic exchanges two edges between two routes. Nevertheless, the combination of M-ACS and 2opt* exchange yields considerable improvement in performance and is relatively competitive with MACS-VRPTW on some data sets. Furthermore, the improvements of M-ACS + 2opt* over M-ACS indicate that M-ACS constructs solutions that can be improved by a local search procedure. 6.2. Computational results on TSP instances A performance study was carried out to evaluate the effectiveness of the strategies and compare the performance between the M-ACS and the Parallel Ant Colony System (PACS) of Chu et al. [5] using some TSP problem instances [31]. To ensure a fair comparison, we set the number of runs and the number of iterations consistent with the PACS approach. The performance was evaluated on the basis of the best solutions obtained in several runs on each instance. The number of iterations for problems st70 and eil101 was set to 1000 iterations, and for problem tsp225 was set to 2000. Table 4 summarizes the performance in terms of the Mean (Mn) and the Standard Deviation (SD) of the best solutions obtained over 10 runs. The results illustrate the potential of employing Strategy-1 over the other strategies. Furthermore, the SD of Strategy-1 is quite small compared to the others, which shows the consistency of M-ACS employing Strategy-1. Note that the results of PACS reported in this table were obtained from the best results of employing several exchange strategies. The authors also report 3887.0 for the Mn value of tsp225, but the best known solution is 3916 [31]. In general, the results obtained by M-ACS employing Strategy-1 demonstrate the effectiveness and the consistency of Strategy-1 compared with the other tested strategies for the three instances. 7. Conclusion and further work An important issue on improving the ACO performance is to find useful mechanisms for sharing information to improve search behavior. In this paper, the single colony approach is extended to a multiple colony approach. Exchange strategies are introduced under three different types of interactions. A weighting scheme
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
1263
is applied in these strategies for adapting the amount of pheromone based on the quality of solutions found by several colonies, considering the state of convergence. Experiments on both the VRPTW and the TSP test problems using the proposed assessment technique demonstrate that the exchange strategies have a considerable influence on the search diversity. Performance studies are performed on a wide range of VRPTW instances as well as on some TSP instances to confirm the effectiveness of the exchange strategies. The results indicate that the multiple colony approach outperforms the single colony approach. In particular, a significant improvement can be accomplished by employing the strategy executed under the star topology, in which the search information is shared among all colonies. The combination of M-ACS and 2-opt* exchange also yields an improvement in performance. It would be interesting to study the integration of more efficient local search procedures, such as the CROSS-exchange heuristic [36] and the Ejection Chains heuristic [21], with the proposed multiple colony approach. We postulate that the proposed multiple colony approach is a promising approach for solving the VRPTW and possibly other combinational optimization problems, however, further investigations are needed to determine, among other things, the optimal interaction scheme among colonies, the impact of using heterogeneous colonies, and the performance of this approach when applied to a ‘‘wide’’ range of TSP instances. Of particular importance is the development of an interaction scheme in which the colonies can interact asynchronously. Furthermore, it may be worth studying whether the proposed entropy-based diversity measure might be used as a diversity control measure. For example, the pheromone distributions can be considered in the weighting scheme by utilizing the entropy-based diversity measure, since the proposed scheme is absolutely dependent on the average cost of the current tours. References [1] E. Alba, G. Leguizamo´n, G. Ordo´n˜ez. Parallel ant algorithms for the minimum tardy task problem, in: CongresoArgentino de Ciencias de la Computacion, San Justo, Argentina, October 4–8, 2004, vol. CACIC’04, pp. 1835–1846. [2] O. Basir, H. Shen, New approach for aggregating multi-sensory data, Journal of Robotic Systems 10 (8) (1993) 1075–1093. [3] E. Bonabeau, M. Dorigo, G. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems, Oxford University Press, New York, 1999. [4] B. Bullnheimer, G. Kotsis, C. Strauss, Parallelization strategies for ant system, in: R. DeLeone, A. Murli, P. Pardalos, G. Toraldo (Eds.), High Performance Algorithms and Software in Nonlinear Optimization, Kluwer Series of Applied Optimization, vol. 24, Kluwer Academic Publishers, Dordrecht, Netherlands, 1998, pp. 87–100. [5] S.C. Chu, J.F. Roddick, J.S. Pan, Ant colony system with communication strategies, Information Sciences 167 (2004) 63–76. [6] J. de Jong, M. Wiering, Multiple ant colony system for the bus stop allocation problem, in: Proceedings of the Thirteenth Belgium– Netherlands Conference on Artificial Intelligence, vol. BNAIC’01, pp. 141–148, 2001. [7] M. DeGroot, Reaching a consensus, Journal of the American Statistical Association 69 (1974) 118–121. [8] P. Delisle, M. Krajecki, M. Gravel, C. Gagne´, Parallel implementation of an ant colony optimization metaheuristic with openmp, in: Third European Workshop on OpenMP, Barcelona, Spain, September 8–9, 2001, vol. EWOMP’01. [9] K.F. Doerner, R.F. Hartl, G. Kiechle, M. Lucka, M. Reimann, Parallel ant systems for the capacitated vehicle routing problem, in: J. Gottlieb, G.R. Raidl (Eds.), Evolutionary Computation in Combinatorial Optimization: 4th European Conference, EvoCOP 2004, Coimbra, Portugal, April 5–7, Lecture Notes in Computer Science, vol. 3004, Springer-Verlag, Heidelberg, Germany, 2004, pp. 72–83. [10] K.F. Doerner, R.F. Hartl, M. Lucka. A parallel version of the d-ant algorithm for the vehicle routing problem, in: R. Trobec, P. Zinterhof, M. Vajtersic, A. Uhl (Eds.), International Workshop on Parallel Numerics 2005, PARNUM’05, Portoroz, Slovenia, April 20–23, 2005, pp. 204–214. [11] M. Dorigo, G. Di Caro, The ant colony optimization meta-heuristic, in: D. Corne, M. Dorigo, F. Glover (Eds.), New Ideas in Optimization, Advances Topics in Computer Science, McGraw-Hill, London, 1999, pp. 11–32. [12] M. Dorigo, L. Gambardella, Ant Colony System: a cooperative learning approach to the traveling salesman problem, IEEE Transactions on Evolutionary Computation 1 (1997) 53–66. [13] M. Dorigo, V. Maniezzo, A. Colorni, The Ant System: optimization by a colony of cooperating agents, IEEE Transactions on Systems, Man, and Cybernetics 26 (Part B) (1996) 1–13. [14] M. Dorigo, T. Stu¨tzle, Ant Colony Optimization, MIT Press, Massachusetts, 2004. [15] I. Ellabib, O. Basir, P. Calamai, A new ant colony system updating strategy for the vehicle routing problems with time windows, in: Proceedings of the Fifth Metaheuristics International Conference (MIC2003), Kyoto, Japan, 2003, pp. 18.1–18.6. [16] D. Favaretto, E. Moretti, P. Pellegrini, Ant colony system approach for variants of the traveling salesman problem with time windows, Journal of Information and Optimization Sciences 27 (1) (2006) 35–54. [17] D. Favaretto, E. Moretti, P. Pellegrini, Ant colony system for a VRP with multiple time windows and multiple visits, Journal of Interdisciplinary Mathematics, in press.
1264
I. Ellabib et al. / Information Sciences 177 (2007) 1248–1264
[18] L. Gambardella, M. Dorigo. Ant-Q: A reinforcement learning approach to the TSP, in: Proceedings of the Twelfth International Conference on Machine Learning (ML-95), Palo Alto, CA, 1995, pp. 252–260. [19] L. Gambardella, E. Taillard, G. Agozzi, MACS-VRPTW: a multiple Ant Colony System for vehicle routing problems with time windows, in: D. Corne, M. Dorigo, F. Glover (Eds.), New Ideas in Optimization, Advances Topics in Computer Science, McGrawHill, London, 1999, pp. 63–76. [20] M. Gendreau, O. Bra¨ysy, Metaheuristic approaches for the vehicle routing problem with time windows: a survey, in: The 5th Metaheuristics International Conf. MIC2003, Kyoto, Japan, 2003. [21] F. Glover, New ejection chain and alternating path methods for traveling salesman problems, in: O. Balci, R. Sharda, S. Zenios (Eds.), Computer Science and Operations Research: New Developments in their Interface, Pergamon Press, Oxford, 1992, pp. 449– 509. [22] M.T. Islam, P. Thulasiraman, R.K. Thulasiram, A parallel ant colony optimization algorithm for all-pair routing in manets, in: Proceedings of the 17th International Symposium on Parallel and Distributed Processing (IPDPS’03), Washington, DC, IEEE Computer Society, 2003, pp. 259–267. [23] H. Kawamura, M. Yamamoto, K. Suzuki, A. Ohuchi, Multiple ant colonies algorithm based on colony level interactions, IEICE Transactions on Fundamentals of Electronics, Communications, and Computer Sciences E83-A (2) (2000) 371–379. [24] Z.-J. Lee, C.-Y. Lee, S.-F. Su. Parallel ant colonies with heuristics applied to weapon–target assignment problems, in: The Seventh Conference on Artificial Intelligence and Applications, TAAI2002, Taichung, Taiwan, November 15, 2002, pp. 201–206. [25] M. Middendorf, F. Reischle, H. Schmeck, Information exchange in multi colony ant algorithms, in: J. Rolim (Ed.), Parallel and Distributed Computing, Proceedings of the 15 IPDPS 2000 Workshops, Third Workshop on Biologically Inspired Solutions to Parallel Processing Problems (BioSP3), LNCS, vol. 1800, Springer-Verlag, Berlin, 2000, pp. 645–652. [26] M. Middendorf, F. Reischle, H. Schmeck, Multi colony ant algorithms, Journal of Heuristics: Special Issue on Parallel Metaheuristics 8 (3) (2002) 305–320. [27] D.A.L. Piriyakumar, P. Levi, A new approach to exploiting parallelism in ant colony optimization, in: International Symposium on Micromechatronics and Human Science (MHS), Nagoya, Japan, 2002. IEEE Xplore, pp. 237–243. [28] J. Potvin, J. Rousseau, An exchange heuristic for routing problems with time windows, Journal of Operational Research Society 46 (1995) 1433–1446. [29] M. Rahoual, R. Hadji, V. Bachelet, Parallel ant system for the set covering problem, in: M. Dorigo, G. Di Caro, M. Sampels (Eds.), Ants Algorithms – Proceedings of ANTS 2002, Third International Workshop on Ant Algorithms, Lecture Notes in Computer Science, vol. 2463, Springer-Verlag, Heidelberg, Germany, 2002, pp. 262–267. [30] M. Randall, A. Lewis, A parallel implementation of ant colony optimization, Journal of Parallel and Distributed Computing 62 (9) (2002) 1421–1432. [31] G. Reinelt, TSPLIB library, http://www.iwr.uni-heidelberg.de/ groups/ comopt/ software/tsplib95/.25. [32] M. Solomon, VRPTW benchmark problems, http://web.cba.neu.edu/ msolomon/problems.htm. [33] M. Solomon, Algorithms for the vehicle routing and scheduling problems with time window constraints, Operations Research 35 (1987) 254–265. [34] T. Stu¨tzle, Parallelization strategies for ant colony optimization, in: A.E. Eiben, T. Back, M. Schoenauer, H.P. Schwefel (Eds.), Proceedings of PPSN-V Fifth International Conference on Parallel Problem Solving from Nature, Lecture Notes in Computer Science, vol. 1498, Springer-Verlag, Berlin, 1998, pp. 722–731. [35] T. Stu¨tzle, H. Hoos, MAX–MIN ant system, Future Generation Computer Systems Journal 16 (8) (2000) 889–914. [36] E. Taillard, P. Badeau, M. Gendreau, F. Guertin, J. Potvin, A tabu search heuristic for the vehicle routing problem with soft time windows, Transportation Science 31 (2) (1997) 170–186. [37] E. Talbi, E. Roux, C. Fonlupt, D. Robillard, Parallel ant colonies for combinatorial optimization problems, in: J. Rolim et al. (Eds.), Parallel and Distributed Processing 11 IPPS/SPDP’99 Workshops, LNCS, vol. 1586, Springer, Berlin, 1999, pp. 239–247.
