Pheromone landscapes in numerical optimization

Pheromone landscapes in numerical optimization Urszula Boryczka Institute of Computer Science, University of Silesia, Będzińska 39, 41–200 Sosnowiec, Poland phone/fax: (+48 32) 3689756 e-mail: [email protected]

ABSTRACT: In this paper we present a new algorithm based on ant colony metaphor for examining numerical optimization problems. The ants’ search behavior consists of a parallel searches on distributed randomly points. Using synergetic effect and tandem running ants try to calibrate with another ants to exchange information about the fruitful positions of search space. The position of the searching point in each region is periodically moved to more fruitful regions according to the value of pheromone and agents try to analyse the neighborhood of this point. This algorithm is tested on several testable functions and finally we conclude the achieved results.

KEYWORDS: ant colony optimization, numerical optimization, abstract landscapes

1

INTRODUCTION

Complex dynamical systems show and provide emergent properties. As an example, biological metaphors offer insight into many aspects of cooperation among agents (for example ants). The behaviour of the system as a whole can be viewed as a side of their collective behaviour, but not as a simple superposition of the individual behaviours of its elements. We want to emphasize that properties of the system are not a priory predictable from the structure of the local interactions and that they are of functional significance. The computation to be performed is contained in the dynamics of the system, which in turn is determined by the nature of local interactions between many individuals (agents) via for example, pheromone values. Many of the dynamic systems that have been developed today find their equivalent in nature, and all of them shows important emergent properties. Originally developed for combinatorial optimization problems, the ant colony metaphor has recently been applied to continuous function optimization. In this paper we introduce a new approach for presenting a pheromone landscape (map) of testable functions in numerical optimization, which is presented as a possible extension of ACO algorithms in continuous search space. Our ACO approach is mainly based on the work of Bilchev et al [1], [2]. The algorithm utilises positive feedback to encourage local search in areas where improvement (positive reinforcement) continues to be made, resulting in promising presentation of the search space — pheromone landscapes. This article is organised as follows. Section 2 describes the inspirations of our approach. Section 3 outlines some previous research results on the ant colony optimisation method. Section 4 describes the numerical optimization and the idea of abstract landscapes. Section 5 presents the algorithm in detail. Section 6 shows experimental results for analysed problems. Conclusions are given in section 7.

2

BIOLOGICAL INSPIRATIONS

Algorithms inspired by phenomena of ant colony behavior are knowing increasing success among researchers in computer science and operational research. This approach presents a highly structured social organism. This model takes inspirations from the observations made by people, who studied real ants’ behavior. The main idea is that self–organizing principles which allow the highly coordinated behavior of real ants can be exploited to lead a population of artificial agents called ants cooperate to solve different computational problems. Biologists have shown how fruitful is to consider stigmergic, indirect communication to explain self–organization. The most important fact in ants’ behavior is the communication among individuals via the pheromone trails produced by the ants. 1

In real ant colonies, pheromone intensity decreases over time because of evaporation. In Simple Ant Colony Optimization (S-ACO) is simulated by applying an appropriately defined pheromone evaporation rule. Pheromone evaporation reduces the influence of build poor quality solutions. Pheromone trail evaporation can be seen as an exploration mechanism that avoids quick convergence of all the ants to the suboptimal solutions. A mechanism of evaporation, enable the forgetting of errors or of poor choices done in the past, allows a continuous improvement of the problem structure [11]. In this paper, we are interested in a model of the foraging strategy of ants and in its application to optimization. These ants use relatively simple principles to search the food, both from the local and global viewpoints. Starting from their initial location, they globally cover a given search space by partitioning it into many points. Each ant performs a local random exploration of its sites and its site choice is sensitive to the success previously met on the sites. This principles can be used to implement a new strategy for the search of a global optimum of a function f in a search space S.

3

ANT COLONY OPTIMIZATION AS A NEW METAHEURISTICS

In this paper we defined an ant algorithm to be a multi–agent system inspired by the observation of some real ant colony behavior exploiting the stigmergic communication paradigm. The optimization algorithm that we propose in this paper was inspired by previous works on Ant Systems and, in general, by the term — stigmergy. This phenomenon was first introduced by Grasse [13], [14]. An essential step in this direction was the development of Ant System (AS) by Marco Dorigo [4], [7], [10], a new type of heuristic inspired by analogies to the foraging behavior of real ant colonies, which has proven to work successfully in a series of experimental studies. Diverse modifications of AS have been applied to many different types of discrete optimization problems and have produced very satisfactory results [6]. Recently, the approach has been extended by [5] to a full discrete optimization metaheuristics, called the Ant Colony Optimization (ACO) metaheuristics. AS, which was the first ACO algorithm [8] was designated as a set of three ant algorithms differing in the way the pheromone trail was updated by ants. Their names were: ant–density, ant–quantity, and ant–cycle. A number of algorithms, including the metaheuristics, were inspired by ant–cycle, the best performing of the ant algorithms. The Ant Colony System (ACS) algorithm has been introduced by [7] to improve the performance of Ant System [9],[12], which allowed to find good solutions within a reasonable time for small size problems only. The ACS is based on 3 modifications of Ant System: • a different node transition rule, • a different pheromone trail updating rule, • the use of local and global pheromone updating rules (to favor exploration). The node transition rule is modified to allow explicitly for exploration. An ant k in city i chooses the city j to move to following the rule: arg maxu∈Jik {[τiu (t)] · [ηiu ]β }, if q ≤ q0 j= J, if q > q0 , where q is a random variable uniformly distributed over [0, 1], q0 is a tuneable parameter (0 ≤ q0 ≤ 1), and J ∈ Jik is a city that is chosen randomly according to a probability    τ (t) · [η ]β iJ PiJ , pkiJ (t) =  [τ (t)] · [ηil ]β il  k l∈Ji

which is similar to the transition probability used by Ant System. We see therefore that the ACS transition rule is identical to Ant System’s when q > q0 , and is different when q ≤ q0 . More precisely, q ≤ q0 corresponds to an exploitation of the knowledge available about the problem, that is, the heuristic knowledge about distances between cities (in TSP) and the learned knowledge memorized in the form of pheromone trails, whereas q > q0 favors more exploration. 2

In Ant System all ants are allowed to deposit pheromone after completing their tours. By contrast, in the ACS only the ant that generated the best tour since the beginning of the trail is allowed to globally update the concentrations of pheromone on the branches. The updating rule is: τij (t + n) = (1 − ρ) · τij (t) + ρ · ∆τij (t, t + n), where (i, j) is the edge belonging to T + , the best tour since the beginning of the trail, ρ is a parameter governing pheromone decay, and: ∆τij (t, t + n) =

1 , L+

where L+ is the length of the T + . The local update is performed as follows: when, while performing a tour, ant k is in city i and selects city j ∈ Jik to move to, the pheromone concentration on edge (i, j) is updated by the following formula: τij (t + 1) = (1 − ρ) · τij (t) + ρ · τ0 . The value of τ0 is the same as the initial value of pheromone trails and it was experimentally found that setting τ0 = (n · Lnn )−1 , where n is the number of cities and Lnn is the length of a tour produced by the nearest neighbour heuristic, produces good results [7], [12]. The ACO metaheuristics has been successfully applied to many discrete optimization problems [3]. We also applied this ACO algorithm in many different combinatorial optimization problems, such as: TSP, Bus Routing Problem — BRP, Multiple Knapsack Problem — MKP, Job Shop Scheduling Problem (JSP) and the clustering and searching problems (Data and Web Mining). The algorithm for continuous functions is based in part on the natural behavior of some special ant colonies. Combinatorial problems (analyzed before) always have a finite number of possible solutions, while this is not true for continuous problems. One cannot directly represent such problems as connected graphs, and other representations are needed. In order to implement the mechanism of cooperation in continuous search space, it is necessary to process a finite number of destinations in the problem space. It will be possible using a distributed searching mechanism.

4

NUMERICAL OPTIMIZATION AND ABSTRACT LANDSCAPES

Optimization is something that one has to deal with in every day life. It has applications in any field of science. Just to name a few: management, control theory, physics, engineering. An important step in optimization is the identification of some objective, i. e., a quantitative measure of the performance of the system. The objective can be any quantity or combination of quantities that can be represented by a single number. There is no universal algorithms to solve such optimization problems. Many of the problems arising in real-life applications are NP-hard. Because of the complexity of the searching the space in this scientific area we propose to use an abstract landscape to predict the direction of searching. The general numerical optimizatrion problem is defined as finding x ∈ S ⊆ Rn such that: f (x) = minf (y); y ∈ S, gj (x) ≤ 0, f orj = 1, ..., m. where f and gj are functions on S; S is a search space defined as a Cartesian product of domains of variables xi ’s (1 ≤ i ≤ n). The set of points that satisfying all the constraints gj is denoted F . The problem presented in formulas is often treated as finding the minimum value of a single fitness function eval(x) with the constrainthandling methods. Since eval(x) is static, it is easily to be mapped as a pheromone landscape created by the agents-ants in Ant Colony Systems. In complex environments, individuals are not fully able to analyze the situation and calculate their optimal strategy. Instead they can be expected to adapt their strategy over time based upon the pheromone landscape and choose the effective one. In ACO, strategies that have relatively effective in a population become more widespread, and strategies that have been less effective become less common in the population.

3

The idea of an abstract landscape has been widely used in the physical and natural sciences to characterize the dynamics of systems. It was originally developed to study potential energy in physical systems. Biologists have independently developed landscapes to characterize evolutionary movement in an abstract „fitness landscape” of genes. More recently, energy landscapes have been used in artificial intelligence to characterize the dynamics of complex systems as neural networks. The resulting landscape shows all possible configurations, and the dynamics of the system can be provided from the initial conditions and the shape of the landscape. Landscape theory predicts the overall configuration by explicitly taking into account sequences of state actions in reducing frustration until a local optimum is reached. It can be useful to analyze the effect of swarm intelligence in ACO. The theory is relevant to policy in that it illuminates where minor changes in the initial configuration can lead to major changes in the final configuration. If one begins near the boundary of two pairs of attraction, then a small movement at the start can lead to large changes in the final outcome. The created landscape therefore measures how difficult it would be to move the whole system from one local optimum to another. The resulting landscape helps us predict about the dynamics of the search process in numerical optimization.

5

DESCRIPTION OF OUR APPROACH

A population-based method is a good choice to provide robust searching. A population of searching agents-ants is introduced to social sharing of information about the search space. Some rules can be observed underlie animal social behavior, including ants, flocks, fish etc. E. O. Wilson suggests that social sharing of information among agents offers an interesting advantage, which has been made great success in learning. A proposal adaptation of ACO in continuous search problems is made by discretizing the continuous search space in some way. In this paper we use a discrete structure to represent a set of different points spread on the search space. In order to implement the Ant Colony Algorithm in continuous problem space, it is necessary to create a finite set of destinations in the search space (called regions), following Bilchev et al.’s proposal [1], [2]. These regions are initially distributed randomly in the problem space, and evolve over time due to the searching process of agents (ants), so that they gradually become located in promising area.The regions are updated (via pheromone values) as the result of two different search processes: locally and globally performed. During the global search process the trail diffusion has been employed. To avoid premature convergence and to improve the diversity it is necessary to decrease the amount of pheromone globally (according to standard global updating rule). The local search method utilize the recruitment mechanism: the search agents collaborate by using pheromones to focus attention on region likely to yield improved fitness. For each searching step, the searching agent will move to a new point in the analysed region at random, but with the greatest value of pheromone in comparison with the neighborhood of the particular position. Using a pheromone table shared by all agents the local information sharing is incorporated by the diffusion of the pheromone. In terms of choice the point with the highest value of pheromone as the current location of the ant, the nearest points are compared based on their pheromone values and some kind of heuristics. The realization of the algorithm may be presented as follows: • Create regions in search space, • Allocate each searching agents to occupy an interesting region in the search space (a random-proportional rule), • For each searching agent: – The agent will move to a new point based on the neighborhood strategy of choice, – Update the pheromone values according the evaluation function of searching points, • Repeat the previous steps until a stop condition (for example, a predefined iteration number I) is reached. The parameters of presented algorithm include: the number of regions in searching process, number of ants m, the coefficient of ρ. A general outline of the ACO algorithm is shown below. It is worth to remarking that the original proposal ([1]) for ACO in continuous domains is used to proceed with the local exploration after a genetic algorithm has finished on the global search level. Our algorithm may be presented as follows: 1 2

procedure Continuous_ACO begin

4

3 4 5 6 7 8 9 10 11 12

t = 0, Initialize_regions R(t) Evaluate R(t) while Stop_conditions_not_met do begin t=t+1 Update_pheromone, Reallocate_antsR(t) Evaluate R(t) end end end procedure

The CACO algorithm can be seen as a trajectory approach which simultaneously searches on different regions and exploits the past experience to guide the search towards the most promising regions according to the quality of the results. Furthermore, the accumulated pheromone focusing the ants’ attention on more promising regions on the feasible search space.

6

EXPERIMENTS AND RESULTS

Test functions for numerical optimization problem are proposed by scientists in many papers. These test cases include objective functions of various types: quadratic, cubic, polynomial, etc., with various numbers of variables and different types. The topologies of feasible search space are also quite different. The chosen test cases are summarized in Table 1.

No. F1 F2 F3 F4 F5

Table 1: Testable functions. Testable function √ √ f (x1 , x2 ) = −x1 sin√ x1 − x2 sin x2 f (x1 , x2 ) = 10 · sin x1 2 + x2 2 f (x1 , x2 ) = cos(x1 + sin(x2 )) f (x1 , x2 ) = (sin(x1 ) − sin(x2 ))3 f (x1 , x2 ) = sin(sin(5 · x1 )) · cos(x2 )

The experiments were designed to study the behaviour of CACO by varying parameter setting which significantly affect the results. For CACO algorithm the number of ants was established to 100, 500, 1000 and 3000 for each experiment. For each ant the searching process with the following values of parameters was performed: 1. The evaporation factor is equal to 0.20. 2. The local search radius (the region size) was set to values: 32, 16 and 8. 3. The number of moves performed by ants in each local subspace is as following: 10, 5 or 2. 4. The precision of searching is established to: 8, 4 or 1 step in each subspace determined by the region size. The maximum number of evaluations per experiment was set to 100 000. Pictures 1, 4, 7 present the structure of search space for analyzed functions. The next pictures concern the pheromone landscapes achieved for testable functions. Similar situation — really good copies of the testable functions can be seen on Fig. 2, 3, 5, 6, 8 which show pheromone landscapes created by ants in 4 experiments.

7

CONCLUSIONS

The good performance of the algorithm for most presented functions using different parameter values suggests that little parameter–tweaking is necessary in general. In summary the ant colony paradigm is a useful optimization method, providing an effective framework for communication between agents via pheromone values in numerical optimization for determine pheromone landscapes. From the perspective of the global optimum found CACO reaches the appropriate values in all the problems considered. This algorithm is also powerful in that it allows local agents to cooperate with other during the tandem running, while at the same time communicating information to the global search. As it could be observed homogeneous parameters do not allow algorithm to 5

a)

b)

Figure 1: Isometric map of functions: a) F1 , b) F3 .

Figure 2: Pheromone landscapes created by ants in four experiments (function F1 ).


a)

b)

Figure 4: Isometric map of functions: a) F3 , b) F4 .

6



Figure 7: Isometric map of function F5 .


7

be really good adapted to the multi-modal functions, so we decided to use heterogeneous parameters for future experiments. Further avenues of research include studying other functions, utilizing presented algorithm or its modifications.

References [1] Bilchev, G., Parmee, I.: “The Ant Colony Metaphor for Searching Continuous Design Spaces”, Proceedings of the AISB Workshop on Evolutionary Computing, Springer-Verlag, Berlin, 1995, 25–39. [2] Bilchev, G., Parmee, I.: “Constrained Optimization with an Ant Colony Search Model”, Proceedings of the AECDC, Plymouth, UK, 1996, 145–151. [3] Bonabeau, E., Dorigo, M., Theraulaz, G.: “Swarm Intelligence. From Natural to Artificial Systems”, Oxford University Press, 1999. [4] Dorigo, M.: “Optimization, Learning and Natural Algorithms ”(in Italian), Ph.D. Thesis, Dipartimento di Elettronica, Politecnico di Milano, IT, 1992. [5] Dorigo, M., Caro, G. D.: “The Ant Colony Optimization meta-heuristic”, in: New Ideas in Optimization (D. Corne, M. Dorigo, F. Glover, Eds.), McGraw-Hill, London, UK, 1999. [6] Dorigo, M., Caro, G. D., Gambardella, L. M.: “Ant Algorithms for Discrete Optimization”, Technical Report IRIDIA/98-10, Université Libre de Bruxelles, Belgium, 10, 1999. [7] Dorigo, M., Gambardella, L. M.: “A Study of Some Properties of Ant-Q”, Proceedings of Fourth International Conference on Parallel Problem Solving from Nature, PPSNIV, Lecture Notes in Computer Science, Lecture Notes in Computer Science, Springer-Verlag, Berlin, 1996, 656–665. [8] Dorigo, M., Gambardella, L. M.: “Ant Colonies for the Traveling Salesman Problem”, Biosystems, 43, 1997, 73–81. [9] Dorigo, M., Gambardella, L. M.: “Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem”, IEEE Trans. Evol. Comp., 1 (1), 1997, 53–66. [10] Dorigo, M., Maniezzo, V., Colorni, A.: “Positive Feedback as a Search Strategy”, Technical Report 91–016, Politechnico di Milano, Włochy, 1991. [11] Dorigo, M., Stützle, T.: “Ant Colony Optimization”, The MIT Press, 2004. [12] Gambardella, L. M., Dorigo, M.: “HAS-SOP: Hybrid Ant System for the Sequential Ordering Problem”, Technical Report 11, IDSIA, 1997. [13] Grasse, P.-P.: “La Reconstruction du Nid et les Coordinations Inter-Individuelles chez Bellicositermes Natalensis et Cubitermes sp. La Theorie de La Stigmerie”, Insects Soc., 6, 1959, 41–80. [14] Grasse, P.-P.: “Termitologia”, vol. II, Paris, Masson, 1984.

8