Niching for Dynamic Environments using Particle Swarm Optimization Isabella Schoeman1 and Andries Engelbrecht2 1
Department of Computer Science, University of Pretoria, Pretoria 0002, South Africa
[email protected] 2 Department of Computer Science, University of Pretoria, Pretoria 0002, South Africa
[email protected]
Abstract. Adapting a niching algorithm for dynamic environments is described. The Vector-Based Particle Swarm Optimizer locates multiple optima by identifying niches and optimizing them in parallel. To track optima effectively, information from previous results should be utilized in order to find optima after an environment change, with less effort than complete re-optimization would entail. The Vector-Based PSO was adapted for this purpose. Several scenarios were set up using a test problem generator, in order to assess the behaviour of the algorithm in various environments. Results showed that the algorithm could track multiple optima with a varying success rate and that results were to a large extent problem-dependent.
1 Introduction Currently Particle Swarm Optimization can be considered to be a well-established population-based optimization technique. Since its inception by Kennedy and Eberhart [1], various developments resulted in an algorithm that proved effective, especially in the case of complex optimization problems. The initial purpose of the strategy was to find the overall best value of a function in a search space, the algorithm being designed in such a way that the population of particles would be discouraged from settling on a suboptimal solution. A balance had to be struck between exploration and exploitation of the search space, an issue addressed elegantly by incorporating both a personal and a social component when updating particle positions. The kind of difficult and complex optimization problems for which PSO proved to be effective, often contains many suboptimal solutions. In some cases it can be advantageous to find the positions of all the optima in the search space. It might be necessary to know the ‘second best’ solution, or a parameter not included in the objective function might necessitate a choice between several good solutions. Research has therefore also been directed towards developing PSO algorithms to locate multiple optima. Such algorithms are also called niching algorithms. Various approaches have been followed and promising results were reported [2],[3],[4],[5],[6],[7],[8],[9]. Although PSO has been successfully applied to static problems, most real-world optimization has to be carried out in a dynamic environment, that is, the objective
function changes over time. Locations and values of optima may change while some optimal solutions disappear altogether and others appear in new positions. To locate an optimum in PSO, particles have to converge on a single point. Keeping track of dynamically changing optima therefore requires frequent re-optimization. The best results would be obtained if a good solution has been found before the next objective function change. This implies that particles have already converged on the optimal solution and diversity should therefore have to be increased in order to find the optimum that has moved away. For optimal results a balance must be found between exploitation and exploration. PSO in dynamic environments proved to be very effective when locating and tracking a single optimal value [10],[11]. In a highly multimodal environment, however, temporal changes may result in a suboptimal solution taking on the best overall value. In such circumstances it would be advantageous if an algorithm would locate and track multiple optima in parallel. In this paper an initial and explorative study of the ability of a niching PSO method to track multiple dynamically changing optima is undertaken. The vector-based PSO [7],[8],[9] is adapted to locate and track those optima in a changing environment. A limited number of functions is set up and used to test this approach. The paper is organized as follows: section 2 describes PSO techniques to locate multiple optima in a static environment, while research on PSO models to locate a single optimum in a dynamic environment is explored in section 3. Section 4 addresses attempts to locate and track multiple dynamic optima. The experimental setup is described and results reported in section 5 while section 6 contains the conclusion.
2 Niching Algorithms for Multimodal Optimization Several niching algorithms have been developed to find multiple optima in a static environment. Examples are Kennedy’s PSO using k-means clustering [2], the “stretching” technique of Parsopoulos and Vrahatis to transform a function once an optimum has been found in order to set the stage for locating subsequent optima sequentially [3],[4], as well as the NichePSO of Brits et al [5]. In all these approaches the initial identification of candidate solutions proved to be a difficult and problematic issue. In his species-based PSO (SPSO) Li devised an ingenious strategy to overcome this challenge [6]. Initial neighbourhood best values – the so-called species seeds – were determined by first sorting all particles in decreasing order of fitness. Particles are checked in turn from best to least-fit against species seeds found so far and assigned to the seed if it falls within a pre-specified radius. The particle becomes a new seed if it does not fall within the radius of any of the seeds identified so far. Particles are now adjusted over a number of iterations to converge on the best position in each particle’s neighbourhood. The optimization process therefore takes place in parallel. While some of these algorithms yielded very good results, it remained a challenge to devise a strategy where multiple optima could be located and optimized with no prior knowledge of the objective function landscape, the number of optima to be found in a designated search area and the niche radii.
Another objective was to find a solution that would be simple, but powerful and where the principles driving PSO would be utilized to its fullest extent. Concepts like the tendency to move towards personal best as well as global best are implemented using vectors, as well as the concept of a velocity associated with a particle. These vectors are manipulated to find a new position. If these vectors could also be manipulated to facilitate niching, the result would be an elegant as well as a powerful solution. Schoeman and Engelbrecht developed sequential and parallel vector-based particle swarm optimizers implementing these principles [7],[8],[9]. In the original PSO, the velocity vector associated with a particle is updated during each iteration. If the position vector towards a particle’s personal best position points roughly in the same direction as the position vector towards the best position found so far in the entire neighbourhood, it means that the particle’s position will be adjusted towards the swarm’s best position, and unless some other obstacle in the function’s landscape is encountered, the particle is probably moving towards an optimal solution. If the two vectors are pointing roughly in opposite directions, it is an indication that, without the influence of the current neighbourhood best position, the particle would be adjusted towards another optimal solution. Thus, when identifying niches, this knowledge can be used to identify particles that are not in the niche surrounding the current neighbourhood best position. Not all particles where both vectors point in the same direction would of course be moving towards the current best position, as there may be other optimal solutions between those particles and the current neighbourhood best. Vector addition is used extensively in the original PSO paradigm. In the vectorbased PSO, another vector operation is used, namely the dot product. In general the dot product of two vectors will be positive when they move in the same direction and negative when moving in opposite directions. Although the stochastic nature of the PSO algorithm is such that the above conjecture will not always hold, it can be used to identify the niche to which a particle belongs. Niches are identified sequentially by first finding the particle with the overall best fitness – the neighbourhood best value of the first niche and calculating the dot product of all particles. A niche radius is then assigned to that niche by calculating the distance between the neighbourhood best particle and the nearest particle with a negative dot product, that is, a particle which will probably converge on a neighbouring niche. A niche identification number starting at one is also assigned to each particle where the dot product is positive and the distance between the particle and its neighbourhood best is smaller than the niche radius. The process is then repeated for particles where niche identification numbers have not yet been assigned, until all particles have been identified. This strategy will yield a number of niches, each with its own neighbourhood best and unique niche radius. Each particle will also be numbered to show to which niche it belongs. Unlike most other niching algorithms developed so far, the niche radius need not be specified in advance. This approach also facilitates niching in functions with irregular landscapes as niche radii are calculated individually and differ from one another. Additional niches may be identified as the landscape is not necessarily symmetrical. Once all niches have been identified, the particles belonging to each niche are treated as subswarms to be optimized. Optimization takes place in parallel. During this process niches can be merged if they are observed to be converging on the same optimum. A parameter indicating the distance between optima that will trigger merg-
ing, is set in advance. This parameter, a small value, is known as the granularity. The same value is used to find an initial personal best value when spawning new particles. One of the advantages of this PSO model is therefore that very few parameters have to be set in advance.
3 Particle Swarm Models for Tracking a Single Optimum in a Dynamic Environment One of the first investigations into the modification of the Particle Swarm Optimizer to locate changing extrema, was by Carlisle and Dozier [10]. The use of a sentry particle was proposed to detect any change in the environment. To prevent a return to outdated positions after an environment change, resetting of the personal best positions to the current position of each particle, was proposed. It should, however, only be reset if the current position is better than the previous personal best position. It must be noted that such a strategy would only be effective when the swarm has not yet converged to a solution. If it has converged, the velocity updates would also be very small. Therefore partial re-initialization of the swarm could be combined with the resetting of personal best positions to increase diversity. Eberhart and Shi experimented with tracking and optimizing a single optimum in a dynamic system [11]. Successful tracking of a 10-dimensional parabolic function with a severity of up to 1.0 was demonstrated. PSO seemed to be naturally suited to such problems and was perceived to perform better than genetic algorithms on similar problems. Dynamic environments can, however, vary considerably. It was reasoned that searching from the current position works well for small environmental changes, or when the swarm has not yet reached an equilibrium state. For severe environmental changes, reinitialization of the entire swarm could be more effective as previous optimal positions would contribute very little or nothing at all to the effort. A combination of these two approaches, namely retaining the global best position and reinitializing a percentage of the particle positions, was suggested in order to retain potentially good positions as well as increase diversity. Blackwell and Branke [12] developed a new variant of PSO that would work well in dynamic environments. They constructed interacting multi-swarms by extending the single population PSO as well as the charged particle swarm optimization method. In charged particle swarm optimization a roaming swarm of “charged” particles, that is, particles that are repelled by a subswarm converging on a peak, is maintained to detect new peaks in the search space. Although the purpose of the multiswarms that had to be maintained on different peaks, was to track changing optima, the creation of subswarms per se is relevant to multimodal optimization. Two forms of swarm interaction were proposed: exclusion and anti-convergence. Exclusion prevents swarms from settling on the same peak, by re-initializing the lesser swarm if two swarms move too close to one another. Anti-conversion re-initializes the worst swarm in order to track down any new peak that may appear. The latter is of course more relevant in a dynamic environment.
4
Particle Swarm Models for Multiple Dynamic Optima
Most techniques using PSO to track moving optima have been developed for single optimum functions. The SPSO described in section 2 has, however, been modified for operation in dynamic environments [13]. The fact that multiple optima are found in parallel provides a natural mechanism for extending the algorithm. To track optima, each particle’s personal best fitness value is re-evaluated before being compared with its current fitness. A strategy for preventing crowding at known optima entails introducing a maximum species population parameter. Only the best candidate members are allocated as members of a species. Redundant particles are reinitialized at random positions in the solution space. The SPSO was extensively tested on functions generated by Morrison and De Jong’s dynamic test function generator [14]. Results indicated that the algorithm can successfully track optima in a dynamic two-dimensional environment. The testing, however, focused mainly on the effect of different values of the population maximum and the species radius on the performance of the algorithm. The proposed dynamic vector-based PSO uses the parallel vector-based PSO to find initial multiple optima. For a strategy to be effective, all optima should be found after each change with less effort than re-optimization would entail. The only data that would be useful after the objective function has been modified, would be the position where the previous optima were located, and only if the changes were not too severe. For severe changes it could be argued that no benefit can be derived from previous optimal positions and that complete re-optimization would be preferable. The proposed algorithm was designed to retain only those values and then spawn a few particles in the immediate vicinity. Only those particles are then optimized. During this process, niches may be merged if they converge on the same position. New optima, may, however, appear when the objective function changes. To locate them will not be so simple, as the initial strategy to find niches must basically be repeated to find new candidate solutions. The algorithm does, however, deactivate particles falling within the niche radii of the existing optima by marking them not to be updated. The remaining particles are then grouped around candidate solutions and optimized. Some of these niches will form around existing niches and be absorbed during the optimization process. Some will, however, converge on new optima. The vector-based PSO algorithm for multiple dynamic optima can now be presented: 1. Find niches and optimize by using the parallel vector-based PSO. See section 2. 2. Repeat: If function has been modified, then Stage 1: 1. Retain particle with the best value in each niche. 2. Create a small number of new particles in vicinity of previous best particles. 3 additional particles at random positions within 80% of the niche radius from the neighbourhood best particle in each niche, works well. 3. Optimize to find new best value in each niche. 4. Merge niches as described in section 2.
Stage 2: 5. Clear vector of particles but retain best particles 6. Create additional particles over entire search space 7. Consolidate particles in existing niches as follows (see section 2): - Find niche radii - Set identification numbers of particles inside each niche radius. 8. Optimize remaining particles if any - Repeat until all particles have been included in a niche (see section 2): - Find best value (neighbourhood best for that niche) - Set neighbourhood best of particles to best value - Set particles with positive dot product to next niche - Optimize niches in parallel and merge if necessary
5
Experimental Setup and Results
To test the above algorithm exhaustively for all possible moving optima is a major undertaking. For this study Morrison and De Jong’s test problem generator was used [14]. The test problem generator was devised to be used in the study of EA performance in changing environments. An environment in two dimensions consisting of a number of cone shapes is generated by the following equation:
(
)
f ( X , Y ) = max i =1, N H i − Ri . ( X − X i ) 2 + (Y − Yi ) 2 .
(1)
where the height of each cone is given by H, the slope by R and the position by (Xi, Yi). The number of optima, their positions, shapes and heights can be specified. An environment with three cones is illustrated in Figure 1. When the environment changes, optima may be obscured and appear again at a later stage.
Fig. 1. An environment with three peaks, generated by Morrison and De Jong’s test problem generator.
Six separate scenarios were implemented; in each case 30 runs were performed in a two-dimensional environment with range [-1.0, 1.0]. The initial number of particles was set to 30 to find the optima for stage 1 of the algorithm. For stage 2 it was in-
creased to 60. The granularity parameter was set to 0.05 for all the experiments. When updating, parameter w of the equation given in [1], is set to 0.8, while c1 and c2 are both set to 1. Vmax was not used in this case [7],[8],[9]. Settings for the six experiments that have been carried out, as well as the results obtained, are summarized in Table 1. Table 1. Experimental setup and results
Initial environment Modification Three peaks Temporal changes over six steps Five peaks Temporal changes over six steps Three peaks Spatial changes over six steps Five peaks
Spatial changes over six steps
Three peaks
Spatial changes over six steps. One peak is obscured after three steps
Three peaks
Spatial changes over eight steps. One peak is obscured after three steps and appears again after another three steps
Result All three solutions found after each step All five solutions found after each step All three solutions found after each step All three solutions found after each step All three solutions found after three steps and two solutions after next three steps All three solutions found after three steps and two solutions after next three steps. In 23 of 30 experiments three optima were located in last two steps, in 5 experiments two optima and in 2 experiments only one. All optima were located in 73% of experiments
6 Conclusion In this paper it was demonstrated that the parallel vector-based particle swarm optimizer could be modified to track multiple moving optima in a few selected dynamic environments. Good results were found when existing optima were tracked and the spatial severity was low. Results are, however, highly problem-dependent, and locating new peaks is not always successful. It is also computationally more expensive to search the entire problem space for these peaks, and in some cases re-optimization of the search space might have to be considered. Future research holds many challenges. More complicated test functions will have to be devised to facilitate thorough testing in a variety of environments. The technique should also be refined in order to obtain more conclusive results.
References 1. Kennedy, J., Eberhart, R.C.: Particle Swarm Optimization. In: Proceedings of the IEEE Int. Conf. On Neural Networks. Piscataway N.J. (1995) 1942–1948 2. Kennedy, J.: Stereotyping: Improving Particle Swarm Performance with Cluster Analysis. In: Proceedings of the 2000 Congress on Computational Intelligence. Piscataway N.J. (2000) 1507–1512 3. Parsopoulos, K.E., Plagianakos, V.P., Magoulas, G.D., Vrahatis, M.N.: Stretching Techniques for Obtaining Global Minimizers through Particle Swarm Optimization. In: Proceedings of the Particle Swarm Optimization Workshop. Indianapolis USA (2001) 22-29 4. Parsopoulos, K.E., Vrahatis, M.N.: Modification of the Particle Swarm Optimizer for Locating all the Global Minima. In: Kurkova, V., Steele, N.C., Neruda, R., Karny, M. (eds.): Artificial Neural Networks and Genetic Algorithms, Springer (2001) 324-327 5. Brits, R., Engelbrecht, A.P., van den Bergh, F.: A Niching Particle Swarm Optimizer. In: Proceedings of the 4th Asia-Pacific Conference on Simulated Evolution and Learning (SEAL 2002). Singapore (2002) 692-696 6. Li, X.: Adaptively Choosing Neighbourhood Bests using Species in a Particle Swarm Optimizer for Multimodal Function Optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference 2004 (GECCO 2004). 105-116 7. Schoeman, I.L., Engelbrecht, A.P.: Using Vector Operations to Identify Niches for Particle Swarm Optimization. In: Proceedings of the Conference on Cybernetics and Intelligent Systems. Singapore (2004) 8. Schoeman, I.L., Engelbrecht, A.P.: A Parallel Vector-Based Particle Swarm Optimizer. In: Proceedings of the International Conference on Artificial Neural Networks and Genetic Algorithms. Coimbra Portugal (2005) 9. Schoeman, I.L., Engelbrecht, A.P.: Containing Particles inside Niches when Optimizing Multimodal Functions. In: Proceedings of SAICSIT2005. White River South Africa (2005) 78-85 10.Carlisle, A., Dozier, G.: Adapting Particle Swarm Optimization to Dynamic Environments. In: Proceedings of the International Conference on Artificial Intelligence. Las Vegas Nevada USA (2000) 429-434 11.Eberhart, R.C, Shi, Y.: Tracking and Optimizing Dynamic Systems with Particle Swarms. In: Proceedings of the 2001 Congress on Evolutionary Computation (CEC2001). 94-100 12.Blackwell, T., Branke, J.: Multi-Swarm Optimization in Dynamic Environments. In: Raidl, G.R. (ed.): Applications of Evolutionary Computation, Vol. 1281. Springer (2004) 489-500 13.Parrott, D., Li, X.: A Particle Swarm Model for Tracking Multiple Peaks in a Dynamic Environment using Speciation. In: Proceedings on the 2004 Congress of Evolutionary Computation (CEC2004). 98-103 14.de Jong, K.A., Morrison, R.W.: A Test Problem Generator for Non-Stationary Environments. In: Proceedings of the Congress on Evolutionary Computation. IEEE Press. (1999) 2047-2053
