Multi-swarm hybrid for multi-modal optimization

WCCI 2012 IEEE World Congress on Computational Intelligence June, 10-15, 2012 - Brisbane, Australia

IEEE CEC

Multi-swarm hybrid for multi-modal optimization Antonio Bolufé Röhler

Stephen Chen

Dep. Artificial Intelligence and Computer Systems University of Havana Havana, Cuba [email protected]

School of Information Technology York University Toronto, Canada [email protected] swarm systems thus provide a useful framework for the development of hybrid algorithms.

Abstract—Multi-swarm systems base their search on multiple sub-swarms instead of one standard swarm. The use of diverse sub-swarms increases performance when optimizing multi-modal functions. However, new design decisions arise when implementing multi-swarm systems such as how to select the initial positions and initial velocities, and how to coordinate the different sub-swarms. Starting from the relatively simple multiswarm system of locust swarms, ideas from differential evolution and estimation of distribution algorithms are used to address the new design considerations that are specific to multi-swarm systems. Experiments show that the new hybrid system can perform better than each of the individual components.

In this paper, a new hybrid multi-swarm system is presented. The new system effectively combines components from particle swarm optimization (PSO) [1], estimation distribution algorithms (EDA) [5], and differential evolution (DE) [6] into a multi-swarm hybrid. Experimental results using the Black Box Optimization Benchmark (BBOB) functions [7] show that the new system performs better than each of the individual optimization methods.

Keywords-particle swarm optimization; multi-swarm system; hybridization; estimation distribution algorithms; differential evolution; exploration; exploitation

I.

INTRODUCTION

Multi-swarm systems are based on the utilization of different sub-swarms, and they provide an effective approach to address one of the main concerns of heuristic search: the balance of exploration and exploitation. In multi-swarm systems, it is possible to implement search strategies where the mechanisms for diversifying and intensifying the search can be clearly separated. This is achieved by using each sub-swarm as an intense region-focused search sub-process and a separate diversification strategy which will use a more general procedure to determine where and when to launch the subswarms. The development of multi-swarm systems leads to new decisions which did not exist during the original development of particle swarm optimization [1]. Some of these decisions have been previously analyzed – e.g. the number of particles to use in each sub-swarm [2], the effects of non-random initial positions and initial velocities [3], and the optimal value for the constriction factor [4]. From this previous work, several design decisions have well-established guidelines – e.g. the use of non-random initial positions and initial velocities leads to improved results in multi-swarm systems (which is not the case for standard single swarms) [4]. Other design decisions, such as which diversification strategy to use or which specific search method will be used to select the initial positions and initial velocities of a sub-swarm, have less established guidelines. Some of these decisions can be addressed by relatively independent sub-components which allow different optimization techniques to be inserted. Multi-

U.S. Government work not protected by U.S. copyright

1759

The presented system is built upon locust swarms, a multiswarm system specifically designed for multi-modal problems [8]. In locust swarms the search is guided by a “devour and move on” strategy – after a sub-swarm “devours” a relatively small region of the search space (to find a local optimum) scouts are deployed to look for new promising regions to “move on”. The scouting process (random search) and the selection of the initial velocities (directed away from the previous optima) are key components of locust swarms which the hybrid system aims to improve. Random scouting is substituted by a more methodical search technique, i.e. the Univariate Marginal Distribution Algorithm for continuous domains (UMDA) [5], which does some optimization during the scouting process. To better exploit the gradient information gathered by UMDA, difference vectors from differential evolution are used to select the initial velocities. The paper is organized as follows. In Section II, some background about particle swarm optimization and multiswarm systems is presented. Locust swarms are introduced in Section III and UMDA in Section IV. The new UMDA-PSO hybrid is described in Section V. The benchmark functions and the computational results are presented in Section VI. An analysis of the key components of the new multi-swarm system is done in Section VII, and a comparison with a previous EDAPSO hybrid system is performed in Section VIII. Section IX provides some discussion about the new multi-swarm system, and a summary is given in Section X. II. BACKGROUND Particle swarm optimization is based on the principles that guide the behavior of swarms in nature, such as flocks of birds and schools of fish [1]. Each particle represents a member of the swarm and describes a possible solution to the optimization problem. The particles move through the search space taking into account personal and social experience. The personal

experience is embodied in the best location found by an individual particle, while the social experience is represented by the best location found by a group of neighboring particles. An individual particle maintains a position and a velocity. It moves throughout the search space updating its position x using (1), where v is the velocity and d is a given dimension. The velocity update (2) uses uniform random values є1 and є2 on the range [0,1], and the weights c1 and c2 to combine the attractions to the personal best and global best positions: pbest and gbest, respectively. The constriction factor, represented by χ, is a parameter that affects the balance of global and local search [9]. xd = xd + vd 



vi 1, d   vi , d  c11  pbesti , d  xi , d   c2 2 gbesti , d  xi , d  





A key feature of PSO is the progressive focus of the search around the best found locations. The communication topology (e.g. ring or star) influences the convergence rate of the search. However, independent of the used topology, the search trajectory of each particle has a convergent nature (due to the attraction of the best locations), and therefore an overall exploitative behavior. Achieving a balance between exploration and exploitation is fundamental when optimizing in multi-modal search spaces. Compared to modifications which seek to improve the balance between exploration and exploitation in PSO (e.g. [10][11][12][13]), an alternative approach is provided by multiswarm systems. Instead of seeking a compromise which may weaken both the exploitative and explorative mechanisms of the search process, multi-swarm systems separate them into distinct phases which focus primarily on either exploration or exploitation. The underlying sub-swarms focus more on exploitation, and an additional mechanism coordinates the subswarms focusing more on exploration. How these sub-swarms are coordinated depends on the specific diversification method(s) implemented by the multi-swarm system. Waves of Swarm of Particles (WoSP) [14], for example, bases its diversification mechanism on the “collision” of particles. When particles get too close, a short range force leads to the expulsion of the particles into new waves/sub-swarms, and this avoids a complete convergence. The Dynamic MultiSwarm Particle Swarm Optimizer (DMS-PSO) [15] periodically regroups the particles of the sub-swarms (after they have converged) into new sub-swarms, and the new swarms are started with particles from previous swarms. Locust swarms [8] are based on a “devour and move on” strategy. III.

LOCUST SWARMS

Locust swarms are a multi-swarm system that has been specifically designed for multi-modal problems [8]. The subswarms in locust swarms each use a predefined number of function evaluations (FEs). These evaluations are distributed among a scouting process and the sub-swarm execution. The

1760

scouting process aims to find good initial positions for the particles. The first sub-swarm (and scouting) is done over the entire search space, subsequent sub-swarms start “around” the optimum found by the previous sub-swarm. The search is guided by a “devour and move on” strategy. After a sub-swarm has “devoured” the area around a (local) optimum (to the point that further improvements are unlikely), scouts are deployed around this location to search for new promising regions to “move on”. To ensure exploration of new parts of the search space, the scouting process is done at a given “gap” from the optimum. Scouts are generated through a random search aiming to promote exploration. The best solutions from the scouting process are then used as the initial positions for the next sub-swarm, and the initial velocities are directed away from the optimum. The “devour and move on” strategy presents a useful metaphor for the design of optimization algorithms. The main idea is the separation of the search process into two different phases: an intensive search phase (devour) and a highly exploratory phase (move on). This approach is related to the concepts of diversification and intensification proposed by Glover and Laguna [16]. Instead of achieving a compromise between exploitation and exploration, intensification and diversification refer to medium- and long-term strategies that address each of these aspects of the search process using more independent mechanisms. In locust swarms, random scouting and directing the initial velocities away from the previous optima are the main components of the “move on” strategy. As a result, the system allocates about half of the total function evaluations to this random search. This key component may be improved by substituting random search by a more methodical search technique. In addition to achieving an effective exploration of the area around the previous optimum, the technique used for diversification can also support the overall search by conducting some optimization during the scouting process. Initial velocities play an important role in the effectiveness of sub-swarms [4]. The ability to direct the initial velocities of the particles allows gradient information to be exploited. In locust swarms, the initial velocities are directed away from the optimum in an attempt to diversify the search. If the scout position is better than the previous optimum, the particle will follow an underlying gradient in the search space. Alternative methods for selecting the initial velocities have been studied [3], and one of the most effective methods uses a differential evolution (DE) based technique to determine the velocities. IV. ESTIMATION DISTRIBUTION ALGORITHMS Estimation of distribution algorithms (EDA) [5] are stochastic optimization algorithms which explore the search space by sampling an explicit probabilistic model. The model is constructed from previously found promising solutions and it can be adapted to learn the structure of the problem. However, building an explicit probabilistic model is often more time consuming than using an implicit one, especially when the model takes into account interactions within the variables.

Among EDAs, the Univariate Marginal Distribution Algorithm for continuous domains [5] is a promising candidate to replace the random search used for scouting in locust swarms. UMDA is based on the assumption that the joint density function follows an n-dimensional normal distribution which is factorized by a product of unidimensional and independent normal densities. Because of this assumption, UMDA doesn't try to learn a complex probabilistic model based on multiple variable interactions. Although learning the variable dependencies can lead to improvements in the search process (when such dependencies exist), it is time consuming and requires a large amount of function evaluations to adapt the model. However, the scouting process in locust swarms is performed with very few function evaluations, so an efficient method is required. For each variable in every generation of UMDA, a set of statistical tests are carried out to find the density function that best fits the variable. (See Algorithm 1.) The first population is randomly initialized with a uniform distribution over the entire search space. During each iteration, the best solutions are selected from the current population to generate the probabilistic model. In the particular case of Gaussian UMDA, the mean value and the standard deviation are estimated. Using these estimators, the inverse Gaussian function is generated and sampled to obtain the next population. A search using UMDA with Gaussian density functions can be slightly biased to the center of the solution space [17]. This characteristic may become a drawback when the optimal solution is near the borders of the feasible region. However, if UMDA is used as a substitute for random scouting, then the initial solutions will be generated around the optimum previously found by the sub-swarm. Considering this optimum as an indicator of a good search region, a slight bias towards it could improve the overall search process. V.

UMDA-PSO MULTI-SWARM SYSTEM

The UMDA-PSO multi-swarm system is built upon locust swarms, and the main difference is to substitute the random scouting by Gaussian UMDA. (See Algorithm 2.) The system exploits the ability of UMDA to methodically explore the search space to find promising new regions and the strong convergence ability of sub-swarm optimization to perform an intense regional search. The initial velocities of the sub-swarm particles are initialized using a differential evolution based technique. Algorithm 1 UMDA Algorithm 1 UMDA Initialize population P0 randomly Initialize population P0 randomly Evaluate P0 Evaluate P0 t=0 t=0 Repeat Repeat Select set of best solutions St from Pt Select set of best solution St from Pt For each variable i in St For each variable i in St Estimate mean value µi and standard deviation σi Estimate mean value µi and standard deviation σi , σ) Sample Pt+1 from Gaussian_inverse_function(µ Sample Pt+1 from Gaussian_inverse_function(µii, σii) Evaluate Pt+1 Evaluate Pt+1 t=t+1 t=t+1 Until Stoping Criteria

Algorithm 2 UMDA-PSO (LB, UB) [init_posit, sample_set] = UMDA(LB, UB) [init_veloc] = DE(sample_set) [prev_best] = PSO(init_posit, init_veloc) search_range= (UB - LB) / 4 For cycle = 1:max_cycles For phase = “diversification” : “intensification” If (phase = = “diversification”) bounds = search_range / (cycle) Else bounds = search_range / (cycle2) newLB =max( prev_best – bounds, LB) newUB =min( prev_best + bounds, UB) [init_posit, sample_set] = UMDA(newLB, newUB) [init_veloc] = DE(sample_set) [prev_best] = PSO(init_posit, init_veloc) End End

The UMDA-PSO system uses the “devour and move on” strategy from locust swarms. It also adds a new strategy which follows a diversification-intensification approach by dividing each cycle into two distinct phases. The intensification phase, initialized in a narrower search region around the previously found optimum, aims to deliver a more exploitative search around a known good solution. The diversification phase starting on a broader search region around the previous optimum promotes an exploration of promising new regions. Each phase involves a sub-swarm with UMDA initial positions and DE-based initial velocities. A. Initialization Step The first UMDA-PSO cycle is executed over the entire search space. This initialization step aims to provide a nonbiased search before starting a more regional exploration. B. Initial Velocities Given the much smaller number of function evaluations, sub-swarms benefit from non-random initial velocities [4]. The effectiveness of a selection method for initial velocities based on differential evolution has been reported in [3]. The creation of a new solution in DE involves a “perturbation vector”, which is generated by adding a difference vector to a base solution (3) [6]. This equation uses a scaling factor F and three distinct and randomly selected solutions x1, x2, and x3 drawn from the population. x = x1 + F(x2  x3 ) 





Difference vectors are essentially slopes between two points, and they provide an effective way to identify

1761

exploitable gradients. To capture this gradient, the initial velocity vector is developed in (4) using xa and xb which are randomly chosen solutions from a set of selected UMDA solutions, and xa has a better fitness than xb. F is the scaling factor in DE, and it has a typical range of [0,1], as reported in [6]. v0 = F(xa  xb ) 





C. Intensification-Diversification Strategy After the initialization step, the system follows a search strategy with repeated cycles. Every cycle is divided into distinct phases for intensification and diversification. Each phase consists of using UMDA to find the initial positions, the DE-based technique to determine the initial velocities, and a sub-swarm search with these initial positions and initial velocities. Every phase is started around the optimum found by the previous one. The difference between the intensification and diversification phases is the range for the lower and upper bounds of the region on which UMDA is started. These limits are determined by the bounds vector which establishes the interval around the previous optimum to be explored. Although these bounds constrain the generation of the UMDA solutions, they do not restrict PSO particles from “flying” beyond these limits. Looser bounds favor exploration (5) while more focused bounds around the previous optimum favor an intense exploitation (6). The search_range represents, for each variable, the initial distance used to establish the bounds around the optimum. This value is set to be one fourth of the variable's feasible interval (See Algorithm 2, line 4). The cycle variable represents the current cycle of the algorithm. The bounds are not only modified according to the search phase (intensification or diversification) but also continuously reduced in size. Smaller bounds initialize subsequent cycles in more restricted regions than the previous ones, and this guarantees a finer search as the algorithm advances. bounds = search _ range / cycle 



bounds = search _ range /(cycle 2 ) 



This narrowing of the search promotes the convergence of the general search process. Metaheuristics tend to converge as the search advances, but they may also suffer from premature convergence (around a sub-optimal solution) if exploration and exploitation are not well balanced. The risk of a premature convergence is reduced in the UMDA-PSO system through the continuous restart of new search cycles around new regions. Conversely, the frequent restarts and the execution of UMDA and PSO with a highly limited number of function evaluations decrease the chances of achieving complete convergence. Although a complete convergence may not be achieved, the continuous narrowing of the general search process allows the

1762

multi-swarm system to deliver a focused exploration on the best found regions. D. Particle Swarm Implementation The PSO implementation is built from the benchmark published by El-Abd and Kamel [18]. The source code for this benchmark implementation was slightly modified to use a ring communication topology instead of a star topology. After this modification, the benchmark becomes an implementation of standard PSO [9], where the positions and velocities of each particle are updated by equations (1) and (2). E. UMDA Implementation The UMDA inserted into the hybrid multi-swarm system is a standard implementation using Gaussian density functions and truncation selection [5]. For each iteration, the mean value and the standard deviation are estimated for each variable and used to generate the inverse Gaussian function which is then sampled to completely replace the previous population, as described in Section IV. F. Parameters The UMDA-PSO system produced its best results with the following parameter configuration (although exhaustive parameter testing was not performed). The PSO parameters of c1=c2=1.494, constriction factor χ=0.634, and population size p=15 are set according to recommendations in [4]. For UMDA, the population size is p=100 and the selection coefficient is c=0.2, i.e. the best 20 solutions are selected in each iteration to learn the probabilistic model. The DE-based velocities use a scaling factor of F=1.0. With these parameters, the best results were obtained with 12 cycles of intensification and diversification (not including the initialization step). The initialization is done with a fixed amount of 4000 FEs which are evenly divided among UMDA and PSO. Subsequent cycles, each consisting of intensification and diversification phases, are assigned 8000 FEs which is 4000 FEs for each phase, or 2000 FEs for each execution of UMDA and PSO. The total number of FEs is 4000 for the initialization plus 96,000 for the 12 cycles of intensification and diversification. The total of 100,000 FEs = 5000*D is consistent with previous results (D = 20 dimensions) [3][4][10][13]. VI.

COMPUTATIONAL RESULTS

The following experiments have been performed using the Black-Box Optimization Benchmarking (BBOB) functions [7]. The BBOB consists of 24 functions broken into five sets which are 1-separable functions, 2-functions with low or moderate conditioning, 3-unimodal functions with high conditioning, 4multi-modal functions with adequate global structure, and 5multi-modal functions with weak global structure. In Table 1, some key attributes of the functions (fn) are indicated, e.g. whether or not they are separable (s), unimodal (u), or have (adequate) global structure (gs). For each function different instances can be generated, e.g. each instance has a different optimal value (shifted in f-space) [7].

TABLE I BBOB FUNCTIONS Set

fn

Function Name

s X X X X X

TABLE II COMPUTATIONAL RESULTS UMDALocust BBOB fn PSO UMDA PSO swarms 10 3.30e+3 6.85e+3 3.24e+4 1.01e+3 11 2.84e+1 6.54e+1 1.37e+2 8.74e+0 12 1.79e+1 1.53e+0 5.40e+3 3.18e−2 13 3.49e+0 1.50e+0 5.14e+1 1.54e−1 14 3.35e−3 1.34e−3 1.37e−1 9.40e−4 15 3.05e+1 6.05e+1 1.66e+1 6.09e+0 16 2.91e+0 5.37e+0 1.72e+0 4.66e−2 17 1.81e−1 6.61e−1 1.18e−1 1.24e−3 18 1.00e+0 2.87e+0 5.55e−1 1.24e−2 19 8.65e−1 3.16e+0 3.61e+0 3.71e−1 20 6.95e−1 1.14e+0 1.86e+0 6.35e−1 21 7.28e+0 2.85e+0 1.41e+0 7.22e+0 22 3.72e+0 4.92e+0 1.69e+0 5.62e+0 23 6.43e−1 1.33e+0 5.41e−1 1.44e−1 24 7.23e+1 1.13e+2 5.51e+1 3.21e+1 Mean error from known optimum for each function.

Attribute u gs X X X X X X X X X

1 Sphere 2 Ellipsoidal, original 3 Rastrigin 1 4 Büche-Rastrigin 5 Linear Slope 6 Attractive Sector 7 Step Ellipsoidal 2 8 Rosenbrock, original 9 Rosenbrock, rotated 10 Ellipsoidal, rotated X X 11 Discus X X 12 Bent Cigar X 3 13 Sharp Ridge X 14 Different Powers X 15 Rastrigin, rotated X 16 Weierstrass X 4 17 Schaffers F7 X 18 Schaffers F7, moderately ill-conditioned X 19 Composite Griewank-Rosenbrock F8F2 X 20 Schwefel 21 Gallagher’s Gaussian 101-me Peaks 5 22 Gallagher’s Gaussian 21-hi Peaks 23 Katsuura 24 Lunacek bi-Rastrigin Names and selected attributes of the 24 functions in the BBOB problem set – separable (s), unimodal (u), global structure (gs).

find promising regions, and PSO should be used as the more exploitative technique tasked with finding the best solution in these sub-regions.

Multi-swarm systems are specifically designed to solve multi-modal problems, so the current experiments focus on sets 4 and 5 (multimodal functions with adequate and weak global structure respectively). Set 3 (functions with high conditioning) is also included to study the system’s behavior when optimizing difficult unimodal problems. To be consistent with previous results (e.g. [3][4][10][13]), the following experiments perform 25 independent trials (5 trials on each of the first 5 instances) on each function. The first set of experiments (see Table II) compares the results obtained by the hybrid system with locust swarms, PSO, and UMDA. The presented results are the mean errors from known optima. Locust swarms and PSO results are from [3], the UMDA results correspond to the same implementation used in the hybrid system, but executed independently and for the full 100,000 FEs. In Table II, the UMDA-PSO multi-swarm algorithm obtains the best results for 12 of the 15 tested functions. UMDA-PSO tends to achieve large improvements on sets 3 and 4, the illconditioned unimodal and the multi-modal functions with adequate global structure. The multi-modal functions without global structure present a harder challenge for the UMDA-PSO system. Comparing the independent sub-components of UMDA and PSO, the results in Table II show that UMDA outperforms PSO in every function of set 4 (multimodal) while PSO dominates UMDA in set 3 (unimodal). These results confirm that UMDA provides a more explorative search while PSO has a stronger convergence. Thus, to exploit the strengths of both techniques, UMDA should be used as the exploratory search method to

1763

It can also be noticed that locust swarms outperform PSO on 8 of the 10 multi-modal functions. The “devour and move on” strategy was specifically designed to provide an advantage for locust swarms on multimodal functions [8]. By building upon locust swarms and improving the random scouting process with UMDA, UMDA-PSO is able to improve even further upon the results for locust swarms on multi-modal search spaces. A pair-wise comparison is presented in Table III. The reported values are the relative improvements (%-diff = (a-b)/a) achieved by UMDA-PSO versus locust swarms, PSO, and UMDA. A positive value indicates that UMDA-PSO (b) outperforms the given algorithm (a). The mean improvements for each set of functions are also presented and the last row shows the mean difference over all of the 15 tested functions. TABLE III COMPARATIVE RESULTS Locust BBOB fn PSO UMDA swarms 10 69.4% 85.3% 96.8% 11 69.2% 86.6% 93.6% 12 99.8% 97.9% 100.0% 13 95.5% 89.7% 99.7% 14 71.9% 29.8% 99.3% Mean set 3 81.2% 77.8% 97.9% 15 80.0% 89.9% 63.3% 16 98.4% 99.1% 97.2% 17 99.3% 99.8% 98.9% 18 98.7% 99.5% 97.7% 19 72.6% 76.0% -133.2% Mean set 4 89.8% 92.9% 44.8% 20 8.6% 44.3% 65.8% 21 -155.5% -416.6% -0.8% 22 24.3% -120.2% 33.7% 23 77.5% 89.1% 73.3% 24 55.6% 71.6% 41.8% Mean set 5 2.1% -66.3% 42.7% Total mean 57.7% 34.8% 61.8% Relative improvement of UMDA-PSO with respect to locust swarms, PSO, and UMDA.

UMDA-PSO is outperformed by another algorithm in only 5 of the 45 comparisons (negative %-diff values). When compared to locust swarms and PSO, the highest levels of improvement are achieved for the multi-modal functions with adequate global structure. For the set of ill-conditioned unimodal functions, UMDA-PSO shows clear improvements over all of the other methods. The worst comparative results occur for the multi-modal functions without global structure, especially for functions 21 and 22 (Gallagher’s Gaussian functions). As reported in [7] the key property of these functions is the existence of optima “with position and height being unrelated and randomly chosen”. This stochastic nature presents a difficult challenge for the UMDA-PSO system which relies on an organized exploration of the search space. VII. SYSTEM COMPONENTS The UMDA-PSO multi-swarm system is based on three key components: the selection of initial positions for the subswarms through UMDA, the selection of the initial velocities using difference vectors from DE, and the diversificationintensification strategy. Each component has a specific design purpose. Starting the sub-swarms at UMDA positions exploits the convergent ability of PSO, the DE-velocities improve the performance of sub-swarms with reduced function evaluations, and the diversification-intensification strategy provides a mechanism for escaping from local optima. The following experiments show that these components are essential to achieve the observed performance improvements in the overall system. To establish how relevant each component is, and for what class of functions they provide the greatest benefit, three versions of the hybrid system were implemented. Each version removes one of the main components. The “random positions” version substitutes the UMDA search with random scouts such as in locust swarms, the “random velocities” version initializes the velocities randomly instead of using DE-velocities, and the “direct search” version doesn't apply the diversificationintensification strategy – instead, it executes every UMDAPSO cycle (with DE-velocities) over the entire search space. Table IV shows the relative improvements (%-diff=(a-b)/a) achieved by the full UMDA-PSO (b) system when compared with each of the previously described versions (a). The mean improvements for each set of functions are also presented and the last row shows the mean improvement over all of the 15 functions. The size of a positive %-diff value indicates how important the missing component is for the whole system. The total mean values show that each component increases the general performance of the system. The least important component is the use of DE-velocities which leads to a general improvement of approximately 10 percent. The selection of the initial positions through UMDA and the intensificationdiversification strategy each affect system performance by around 40 percent. Providing an improvement of 40.9 percent, the diversification-intensification strategy is the single most important feature of UMDA-PSO. However, this search

TABLE IV COMPONENT ANALYSIS Random Random BBOB fn Direct Search Velocities Positions 10 -3.3% 30.0% 86.6% 11 24.1% -122.1% 85.9% 12 32.8% 96.9% 98.9% 13 16.3% 84.3% 97.1% 14 4.9% 50.3% 80.3% Mean set 3 14.9% 27.9% 89.8% 15 8.3% 73.1% 84.5% 16 25.0% 90.1% 99.5% 17 20.1% 98.3% 98.9% 18 18.7% 95.2% 98.2% 19 3.0% 67.6% 71.1% Mean set 4 15.0% 84.8% 90.4% 20 7.2% 35.7% 64.1% 21 -6.9% -138.7% -392.9% 22 0.2% -0.2% -111.4% 23 4.6% 69.6% 89.2% 24 3.0% 37.2% 64.2% Mean set 5 1.6% 0.7% -57.3% Total mean 10.5% 37.8% 40.9% Relative improvement of UMDA-PSO with respect to the “random velocities”, “random positions”, and “direct search” versions.

strategy becomes a significant drawback when optimizing functions 21 and 22 – the random distribution of the local optima in these functions benefits from a more exploratory search process such as the one in the direct search version. Unlike the intensification-diversification strategy, direct search does not assume a global structure. Excluding functions 21 and 22, the improvement provided by the intensificationdiversification strategy rises to 86 percent on the remaining 13 functions. The large difference in performance for the “random positions” version, i.e. the removal of UMDA initial positions, when optimizing set 4 (multi-modal functions with adequate global structure) is consistent with the results from Section VI. The current result supports the use of UMDA over random scouting as a more methodical exploration mechanism for finding promising search regions. In general, all the components are important when optimizing set 4. It confirms that well selected initial positions and initial velocities, as well as a good balance between exploration and exploitation are benefits to multi-swarm systems in multi-modal search spaces. VIII. COMPARISON WITH AN EDA-PSO HYBRID SYSTEM A hybrid EDA-PSO system has been previously proposed by Zhou and Jin [19]. Based on the best half of the swarm, the system samples an independent univariate Gaussian distribution. The new particles can be updated using the normal PSO equations or sampled from the estimated distribution, this decision is made with a probability p (participation ratio). If p=0, the system will behave as a pure EDA algorithm and if p=1, it will be a pure PSO algorithm. Based on this system, an implementation for the BBOB functions is provided by El-Abd and Kamel [20]. The implementation improves the original results by incorporating the method used in [21] for updating the variance of the Gaussian model, as reported in [22]. The current experiments (see Table V) compare the EDAPSO hybrid (using the source code provided by El-Abd and

1764

Kamel [20]) with the UMDA-PSO multi-swarm system. The experiments are performed according to the specifications described in Section VI. In Table V, the mean errors from known optima and the relative improvement (%-diff=(a-b)/a) achieved with the UMDA-PSO multi-swarm system (b) versus the EDA-PSO hybrid (a) are presented. A p-value of less than 0.05 in the reported t-tests is used to determine if the %-diff is statistically significant. UMDA-PSO significantly outperforms the EDA-PSO hybrid on 13 of the 15 tested functions. IX.

DISCUSSION

Different heuristic search techniques have different strengths and weaknesses with regard to the key tasks of exploration and exploitation. Between UMDA and PSO, UMDA performs better on multi-modal functions (which require more exploration) and PSO performs better on unimodal functions (which require more exploitation). The developed UMDA-PSO hybrid combines the strengths of both search techniques into a new multi-swarm system that has better overall performance than either of the individual components. Thus, instead of simply mixing components of different heuristics as in the EDA-PSO hybrid (where a probability distribution function or PSO equations are randomly selected to produce the new population), the heuristics in UMDA-PSO are combined to complement each other. For example, the global search of the multi-swarm system is based on the exploratory abilities of Gaussian UMDA, while some of its weaknesses such as the slower convergence rate and the difficulty to model complex and non-symmetric distributions are compensated through the use of multiple sub-swarms and the restarts of the “devour and move on” strategy. The use of sub-swarms as an intensive (regional focused) search method provides, in multimodal problems, a benefit compared to standard local search. As population based methods, sub-swarms can simultaneously evaluate multiple optima in the promising region found by the TABLE V COMPARISON BETWEEN HYBRID SYSTEMS UMDA-PSO EDA-PSO %-diff p-value mean std dev mean std dev 10 5.24e+2 9.29e+3 6.99e+3 90.8% 0.00 8.53e+2 11 7.00e+0 2.74e+1 2.85e+1 64.9% 0.00 9.62e+0 12 4.13e−2 5.85e+3 1.68e+3 100.0% 0.00 2.49e−2 13 6.36e−1 1.27e+1 2.47e+0 98.1% 0.00 2.42e−1 14 2.93e−4 3.70e-3 1.18e−3 75.5% 0.00 9.07e−4 Mean set 3 85.9% 15 1.94e+0 1.58e+1 2.33e+1 59.3% 0.03 6.41e+0 16 1.09e−1 6.89e+0 5.02e+0 98.8% 0.00 8.35e−2 17 6.72e−4 7.92e−2 1.53e−1 98.0% 0.01 1.59e−3 18 5.40e−3 1.46e−1 9.64e-2 88.9% 0.00 1.63e−2 19 5.80e−1 3.60e+0 4.51e−1 72.4% 0.00 9.94e−1 Mean set 4 83.4% 20 2.06e−1 2.10e+0 8.02e−1 66.9% 0.00 6.94e−1 21 7.47e+0 8.13e+0 6.79e+0 -21.3% 0.27 6.16e+0 22 7.41e+0 7.41e+0 8.04e+0 20.2% 0.25 5.91e+0 23 3.78e−2 1.95e+0 3.31e−1 92.3% 0.00 1.49e−1 24 2.77e+0 1.08e+2 3.20e+1 71.0% 0.00 3.13e+1 Mean set 5 45.8% Total mean 71.7% Mean error from known optimum and relative improvements of UMDAPSO multi-swarm system versus the EDA-PSO hybrid.

global search. Moreover, the use of DE-velocities allow subswarms to exploit gradient information, while the communication between particles permit them to escape from local optima. The diversification-intensification strategy provides an effective framework to combine the search properties of UMDA and PSO. The short execution of each algorithm, i.e. the limited FEs for UMDA and PSO per cycle, disallow full convergence. A full convergence of UMDA would make unnecessary the execution of sub-swarms, but would require more FEs than if sub-swarms are used. When initialized on a specific region, sub-swarms do not need a large amount of FEs to find (near) optimal solutions. With fewer function evaluations allocated to each search phase, more restarts around new regions can be performed. These restarts avoid a premature convergence and provide a mechanism for balancing exploration and exploitation of the entire system. When the UMDA-PSO system is executed with a fixed amount of FEs, the number of diversification-intensification cycles determines the amount of FEs assigned to each of the search components. With fewer cycles, UMDA and PSO will have more FEs and therefore longer executions. Both components will then have greater chances to explore and converge on their starting regions, and this promotes a more exploitative overall search procedure. On the other hand, an increased number of cycles lead to more but shorter executions of UMDA and PSO. Shorter runs will decrease the opportunities to exploit the landscape's information, but will produce more restarts and therefore a more diversified search. An effective strategy for adapting dynamically the number of cycles according to a problem’s characteristics poses an interesting opportunity for future research. X. SUMMARY Multi-swarm systems provide a useful framework for the development of hybrid algorithms. Design decisions such as how to select the initial positions and initial velocities, and how to coordinate the different sub-swarms allow different optimization techniques to be combined through this framework. The presented system uses UMDA as a global search method to select initial positions for the sub-swarms. Each sub-swarm is thus expected to provide an intensive search around the region where it is initialized, and this initialization includes DE-based initial velocities. The general search mechanism of the new multi-swarm system adds a diversification-intensification strategy to the “devour and move on” strategy of locust swarms. This addition refines the balance between exploration and exploitation which leads to improved performance of the new UMDA-PSO hybrid compared to the individual components of UMDA and PSO. REFERENCES [1]

[2]

1765

J. Kennedy and R. C. Eberhart, “Particles Swarm Optimization,” in Proceedings of the IEEE International Conference on Neural Networks, 1995, pp. 1942–1948. S. Chen, “An Analysis of Locust Swarms on Large Scale Global Optimization Problems,” in Lecture Notes in Computer Science, Vol. 5865 : Proceedings of Fourth Australian Conference on Artificial Life,

[13] S. Chen and J. Montgomery, “A Simple Strategy to Maintain Diversity and Reduce Crowding in Particle Swarm Optimization,” in Lecture Notes in Computer Science, Vol. 7106: Proceedings of Joint Australasian Conference in Artificial Intelligence, D. Wang and M. Reynolds, Eds. Springer-Verlag, 2011, pp. 281–290. [14] T. Hendtlass, “WoSP: A Multi-Optima Particle Swarm Algorithm,” in Proceedings IEEE Congress on Evolutionary Computation, 2005, pp. 727–734. [15] S. Z. Zhao, J. J. Liang, P. N. Suganthan, and M. F. Tasgetiren, “Dynamic Multi-Swarm Particle Swarm Optimizer with Local Search for Large Scale Global Optimization,” in Proceedings IEEE Congress on Evolutionary Computation, 2008, pp. 3845–3852. [16] F. Glover and M. Laguna, Tabu Search. Kluwer Academic Publishers, Boston, 1997. [17] B. Yuan and M. Gallagher, “Convergence Analysis of UMDA with Finite Populations: A Case Study on Flat Landscapes,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2009, pp. 477–482. [18] M. El-Abd and M. S. Kamel, “Black-Box Optimization Benchmarking for Noiseless Function Testbed using Particle Swarm Optimization,” in Proceedings of the Genetic and Evolutionary Computation Conference. 2009, pp. 2269–2273. [19] Y. Zhou and J. Jin, “EDA-PSO – a new hybrid intelligent optimization algorithm,” in Proceedings of the Michigan University Graduate Student Symposium, 2006. [20] M. El-Abd and M. S. Kamel, “Black-Box Optimization Benchmarking for Noiseless Function Testbed using an EDA and PSO Hybrid,” in Proceedings of the Genetic and Evolutionary Computation Conference. 2009, pp. 2263–2268. [21] J. Grahl, P. A. N. Bosman, and F. Rothlauf, “The correlation-triggered adaptive variance scaling idea,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2006, pp. 397–404. [22] M. El-Abd and M. S. Kamel, “Preventing premature convergence in a PSO and EDA hybrid.,” in Proceedings of IEEE Congress on Evolutionary Computation, 2009, pp. 3060–3066.

K. Korb, M. Randall, and T. Hendtlass, Eds. Springer-Verlag, 2009, pp. 211–220. [3] S.Chen and J. Montgomery, “Selection Strategies for Initial Positions and Initial Velocities in Multi–optima Particle Swarms,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2011, pp. 53– 60. [4] A. Bolufé Röhler and S. Chen, “An Analysis of Sub-swarms in Multiswarm Systems,” in Lecture Notes in Computer Science, Vol. 7106: Proceedings of Joint Australasian Conference in Artificial Intelligence, D. Wang and M. Reynolds, Eds. Springer-Verlag, 2011, pp. 271–280. [5] P. Larrañaga and J. A. Lozano, Estimation of Distribution Algorithms, A New Tool for Evolutionary Computation. Kluwer Academic Publishers, 2011. [6] R. Storn and K. Price, “Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces,” Journal of Global Optimization, 11:341–359. [7] N. Hansen, S. Finck, R. Ros, and A. Auger, “Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions,” INRIA Technical Report RR-6829, 2009. [8] S. Chen, “Locust Swarms – A New Multi-Optima Search Technique,” in Proceedings of the IEEE Congress on Evolutionary Computation, 2009, pp. 1745–1752. [9] D. Bratton and J. Kennedy, “Defining a Standard for Particle Swarm Optimization,” in Proceedings of the IEEE Swarm Intelligence Symposium, 2007, pp. 120–127. [10] Y. Noa Vargas and S. Chen, “Particle swarm optimization with resets – improving the balance between exploration and exploitation,” in Lecture Notes in Computer Science, Vol. 6438 : Proceedings of 9th Mexican International Conference on Artificial Intelligence, G. Sidorov et al. Eds. Springer-Verlag, 2010, pp. 371–381. [11] L. Cartwright, T. Hendtlass: “A Heterogeneous Particle Swarm,” in Lecture Notes in Computer Science, Vol. 5865 : Proceedings of Fourth Australian Conference on Artificial Life, K. Korb, M. Randall, and T. Hendtlass, Eds. Springer-Verlag, 2009, pp. 201–210. [12] S. S. Pace and C. Woodward, “Diversity Preservation using Excited Particle Swarm Optimisation,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2011, pp. 61–68.

1766