An adaptive two-layer particle swarm optimization ... - Semantic Scholar

Information Sciences 273 (2014) 49–72

Contents lists available at ScienceDirect

Information Sciences journal homepage: www.elsevier.com/locate/ins

An adaptive two-layer particle swarm optimization with elitist learning strategy Wei Hong Lim, Nor Ashidi Mat Isa ⇑ Imaging and Intelligent System Research Team (ISRT), School of Electrical and Electronic Engineering, Engineering Campus, Universiti Sains Malaysia, 14300 Nibong Tebal, Penang, Malaysia

a r t i c l e

i n f o

Article history: Received 10 July 2013 Received in revised form 22 January 2014 Accepted 8 March 2014 Available online 18 March 2014 Keywords: Particle swarm optimization (PSO) Adaptive two-layer particle swarm optimization with elitist learning strategy (ATLPSO-ELS) Adaptive division of labor (ADL) Orthogonal experimental design (OED) Memetic computing (MC)

a b s t r a c t This study presents an adaptive two-layer particle swarm optimization algorithm with elitist learning strategy (ATLPSO-ELS), which has better search capability than classical particle swarm optimization. In ATLPSO-ELS, we perform evolution on both the current swarm and the memory swarm, motivated by the tendency of the latter swarm to distribute around the problem’s optima. To achieve better control of exploration/exploitation searches in both current and memory swarms, we propose two adaptive division of labor modules to self-adaptively divide the swarms into exploration and exploitation sections. In addition, based on the orthogonal experimental design and stochastic perturbation techniques, an elitist learning strategy module is introduced in the proposed algorithm to enhance the search efficiency of swarms and to mitigate premature convergence. A comprehensive experimental study is conducted on a set of benchmark functions. Compared with various state-of-the-art PSOs and metaheuristic search variants, ATLPSO-ELS performs more competitively in the majority of the benchmark functions. 2014 Elsevier Inc. All rights reserved.

1. Introduction Particle swarm optimization (PSO) algorithm was initially proposed by Kennedy and Eberhart in 1995 [25] to emulate the social behavior of a school of fishes or a flock of birds in search of food [1,9,10,25]. As a population-based algorithm, PSO uses a pool of individuals to perform the search on the solution space. Each individual (i.e., particle) represents the potential solution of the optimization problem, whereas the location of the food source denotes the corresponding global optimum. Unlike most existing metaheuristic search (MS) algorithms, the PSO particle is capable of remembering its current position in the search space and its personal best position, that is, the best position that it has ever achieved. In other words, PSO roams around the search space through the current swarm and simultaneously memorizes their personal best positions in the memory swarm [6]. Although each PSO particle searches for the food independently and stochastically, these particles also collaborate and share information with each other to enable them to move toward the food from different directions [1,25,26]. The PSO has a simple concept and is highly efficient, which is why it has been actively applied for many real-world problems with promising results [2,9]. Although PSO has a fast convergence rate, many experiments have shown that when solving complex problems, basic PSO tends to converge into the local optima in the early stage of optimization [33,52]. Another major concern in the control of PSO performance is exploration/exploitation balance, as excessive exploration wastes computational resources, while ⇑ Corresponding author. Tel.: +60 45996093; fax: +60 45941023. E-mail addresses: [email protected] (W.H. Lim), [email protected] (N.A. Mat Isa). http://dx.doi.org/10.1016/j.ins.2014.03.031 0020-0255/ 2014 Elsevier Inc. All rights reserved.

50

W.H. Lim, N.A. Mat Isa / Information Sciences 273 (2014) 49–72

excessive exploitation leads to local premature convergence [48]. In addition, although the global best particle is essential to guide the classical PSO swarm during the searching process, this particle has the least effective learning strategy to guide itself [28]. The global and personal best positions of the global best particle are the same, and this similarity inevitably nullifies the particle’s social and cognitive components in classical PSO’s velocity update equation. A substantial amount of PSO variants have been reported to address the aforementioned issues [3,5,7,18,20,31–33,37– 39,44,46,48,51,55,61,62]. Nevertheless, the universality and robustness of these new variants in tackling the diverse set of problems with different properties remain questionable, as no specific PSO variant is able to achieve the best solution for all optimization problems [21]. The dominant performance of these PSO variants in a certain class of problem could be related to the ‘‘No Free Lunch (NFL) theorem, which has mathematically proven that the average performance of any pair of optimization algorithms over all possible optimization problems is identical [58]. Accordingly, an optimization algorithm that performs well on a class of problems will lose dominance on the remaining set of optimization problems to equalize the average performance of the pair of algorithms across all optimization problems. The inability of the PSO to cope with a diverse set of problems may also be attributed to the fact that different problems have differently shaped fitness landscapes. The problem’s difficulty is compounded by the fact that in a certain benchmark, such as the composition function [50], the shape of the local fitness landscape in different subregions may be significantly different [32]. To effectively solve these complex problems, different PSO particles should play different roles (i.e., perform search with different exploration/ exploitation strengths) in different locations of fitness landscape and different evolutionary stages. However, most of the PSO variants that have been proposed so far use only one type of learning strategy and have limited choices of exploration/exploitation strengths to perform the search in different subregions of the search space [32]. This situation inevitably restricts the algorithms’ robustness in addressing different types of problems. Developing the PSO variants by using model fusion is a growing trend nowadays [21,32,57]. With the use of model fusion, these PSO variants are equipped with multiple learning strategies, and some intelligent selection techniques are developed to assign appropriate learning strategies to the particles in different subregions of the search space. As different learning strategies have different types of exploration/exploitation strengths, different particles can play different roles during the search process, and even the same particle can play different roles during the different search progress. Another promising line of research is memetic computing (MC) [42], which is a broad subject that studies complex and dynamic computing structures composed of interacting modules (memes) whose evolution dynamics is inspired by the diffusion of ideas [42]. Under this approach, various interacting algorithmic components (memes) are designed and harmoniously coordinated to solve diverse problems. More precisely, these algorithmic modules attempt to seek the optimum solution in the optimization search space by collaborating with each other through information sharing or exchanging mechanisms. Extensive experimental results suggested that some merit exists in applying model fusion or the MC approach to cope with different complex situations. Inspired by the general concept of MC as a composition of interacting modules, we proposed an adaptive two-layer PSO with elitist learning strategy (ATLPSO-ELS). In our approach, we employ a two-layer evolution framework in the ATLPSO-ELS to evolve (1) the current swarm that carries the particles’ current positions in the search space and (2) the memory swarm that memorizes the best knowledge (i.e., the cognitive and social experiences) of each particle. To ensure better control of exploration/exploitation during the swarms’ evolution, we develop two adaptive division of labor (ADL) modules in the ATLPSO-ELS to self-adaptively allocate different search tasks to each swarm members, based on the population diversity and fitness of swarm members. As learning strategy for global best particle is crucial to improve the PSO’s performance, we propose an elitist-based learning strategy (ELS) module to evolve the global best particle when predefined conditions are met. The ELS module consists of two learning strategies, derived by the orthogonal experiment design (OED) [17,40] and stochastic perturbation techniques. The remainder of this paper is organized as follows: Section 2 briefly presents related works. Section 3 explains the methodologies of ATLPSO-ELS. Section 4 provides the experimental settings, while Section 4.5 discusses the experimental results. Section 5 concludes the paper. 2. Related works In this section, we first discuss the mechanism of the basic PSO and some state-of-the-art PSO variants. To ensure completeness, we briefly describe the OED technique. Finally, some of the OED-based PSO variants are reviewed. 2.1. Basic PSO algorithm For a D-dimensional problems hyperspace, PSO particle represents a potential solution and is represented by two vectors to indicate its current state, i.e., current position Xi = [Xi1, Xi2, . . . , XiD] and velocity Vi = [Vi1, Vi2, . . . , ViD]. During the search process, the trajectory of each particle is stochastically adjusted according to the particle’s cognitive component, Pi, and the group’s social component, Pg [10,25,26]. Mathematically, the d-th dimension of particle i’s velocity, Vi,d(t + 1) and position Xi,d(t + 1) at (t + 1)-th iteration of searching process are updated as follows:

V i;d ðt þ 1Þ ¼ xV i;d ðtÞ þ c1 r 1 ðPi;d ðtÞ X i;d ðtÞÞ þ c2 r 2 ðPg;d ðtÞ X i;d ðtÞÞ

ð1Þ


X i;d ðt þ 1Þ ¼ X i;d ðtÞ þ V i;d ðt þ 1Þ

51

ð2Þ

where i = 1, 2, . . . , S is the particle’s index; S is the population size; Pi = [Pi1, Pi2, . . . , PiD] is the best previous position that produces best fitness value for particle i; Pg = [Pg1, Pg2, . . . , PgD] is the global best particle found by all particles so far; c1 and c2 are the acceleration factors that control the influences of cognitive and social components, respectively; r1 and r2 are two random numbers with the range of [0, 1]; and parameter x is the inertia weight that is used to balance the global and local searches of particles [48].

2.2. PSO variants and improvements The first area of research to improve PSO’s performance is the parameter adaptation strategy. Shi and Eberhart [48] developed PSO with linearly decreasing weight (PSO-LDIW) by using the parameter x to balance the exploration/exploitation searches. Ratneweera et al. [46] introduced a time-varying acceleration coefficient (TVAC) strategy into PSO, where c1 and c2 are varied with time to balance the global/local searches. Two PSO-TVAC variants, namely, PSO-TVAC with mutation (MPSO-TVAC) and self-organizing hierarchical PSO-TVAC (HPSO-TVAC) are developed in [46]. Zhan et al. [61] proposed an evolutionary state estimation (ESE) module in their adaptive PSO (APSO) to identify the evolutionary states. The output of the ESE module is then used to adaptively adjust the particles’ x, c1, and c2. Leu and Yeh [31] proposed grey PSO to adjust the particles’ x, c1, and c2 values by using grey relational analysis. Hsieh et al. [20] developed an efficient population utilization strategy for PSO to adjust the population size adaptively. Population topology also plays a major role in the performance of PSO as it decides the information flow rate of the best solution within the swarm [24,27]. Inspired by the social behavior of clans, Carvalho and Bastos-Filho [5] developed a clan topology (i.e., Clan PSO). Bastos-Filho et al. [3] and Pontes et al. [44] improved Clan PSO by hybridizing it with the migration mechanism and APSO, respectively. Mendes et al. [37] proposed that the movement of each particle is influenced by its neighborhood members rather than just the particle’s Pi and Pg positions and proposed the fully informed PSO (FIPSO). Qu et al. [45] developed a distance-based local informed PSO (LIPS) to compensate the drawback of FIPSO in multimodal problems. LIPS uses the local information contributed by the particle’s nearest distanced-based neighborhood to form different stable niches. Liang and Suganthan [34] proposed a dynamic multi-swarm PSO (DMS-PSO) with the dynamically changing neighborhood structure. Montes de Oca et al. [39] proposed Frankenstein PSO (FPSO) with time-varying population topology. FPSO is initialized with the fully connected topology. The connectivity of the topology decreases with a certain pattern and is eventually reduced into the ring topology [39]. Marinakis and Marinaki [36] introduced PSO with expanding neighborhood topology (PSOENT) by hybridizing PSO with the variable neighborhood search strategy. The neighborhood structure of PSOENT is expanding based on the quality of the produced solution. Other studies in this field focus on exploring the PSO’s learning strategies. Liang et al. [33] proposed a comprehensive learning PSO (CLPSO) in which each particle i is allowed to learn either from its Pi or from other particle’s historical best position in each dimension. Tang et al. [51] proposed an improved variant of CLPSO called feedback learning PSO with quadratic inertia weight (FLPSO-QIW). In their approach, each FLPSO-QIW particle generates potential exemplars from the first 50% of the fitter particle. Inspired by DMS-PSO [34] and CLPSO [33], Nasir et al. [41] proposed a dynamic neighborhood learningbased PSO (DNLPSO) in which each particle’s exemplar is selected from its neighborhood, which changes dynamically with time. Huang et al. [23] used an example set of multiple global best particles to update the particle’s velocity in their examplebased learning PSO (ELPSO). Behesti et al. [4] used the median position and median fitness value of a particle in their medianoriented PSO to address the swarm convergence issue. Gholizadeh [16] integrated a novel cellular automata-based velocity update scheme in their proposed sequential cellular PSO to optimize the layout of truss structure. Recently, an increasing body of evidence indicates that no single learning strategy is optimal for all problems. Moreover, the optimal learning strategy for a given problem will be time variant, as the strategy will depend on factors such as the population’s degree of convergence [49]. Therefore, multiple strategies or methods should be applied in one algorithmic framework, and this approach has emerged as a plausible line of research to enhance PSO’s performance. Li et al. [32] proposed a self-learning PSO (SLPSO), in which each particle in SLPSO has four strategies to cope with different types of search spaces. The selection of strategies for each particle is based on a probability value derived from a self-adaptively improved probability model. Wang et al. [57] and Hu et al. [22] introduced the self-adaptive learning-based PSO and adaptive PSO with multiple adaptive method, respectively. Both PSOs have similar working mechanisms as the SLPSO but adopt different search strategies and intelligent selection techniques. Wang et al. [53] proposed the diversity enhanced PSO with neighborhood search (DNSPSO), in which a total of three new search strategies are developed to enhance the algorithm’s diversity and local and global search abilities. Motivated by the tendency of memory swarm members to distribute around the vicinity of the problem’s optima, Lim and Mat Isa [35] proposed a two-layer PSO with intelligent division of labor (TLPSO-IDL), which aims to enhance the search capability of PSO by evolving the current best knowledge of each particle. After the current swarm evolution of TLPSO-IDL is performed, an intelligent optimization procedure is executed to provide additional evolution on the memory swarm to allow swarm members to either locate more promising regions around the optima or to fine-tune the regions of the already-found optima. An IDL module is used by TLPSO-IDL to self-adaptively allocate different search tasks (i.e., exploration and exploitation) to each swarm members during memory swarm evolution.

52


2.3. Orthogonal experimental design (OED) OED [17,40] has been developed as an effective mathematical tool to study the best combination of factor levels in design problems. This technique works on an orthogonal array (OA) [18], represented as LM(QM1), where L denotes the OA; Q is the number of the level of factors; and M ¼ Q jlogQ ðNþ1Þj is the number of test cases combination. For N factor experiments, only the first N columns (or arbitrary N columns) of LM(QM1) are used. The L4(23) OA for an experiment, which consists of three factors (N = 3) and each factor has two levels (Q = 2), is shown in Eq. (3). The algorithm used to construct the two-level OA for N factors is presented in Appendix A.

3 1 1 1 61 2 27 7 6 L4 ð23 Þ ¼ 6 7 42 1 25 2

ð3Þ

2 2 1 To identify the best combination of the LM(QM1) OA, factor analysis (FA) is performed based on the experimental results of all the M combinations of OA. FA determines the best level of each factor by estimating the effect of individual factors on the overall results independently [17]. To investigate which level has a significant effect on the corresponding factor, the main effect of factor j (1 6 j 6 N) with level k (1 6 k 6 Q), denoted as Sjk is calculated as follows:

PM Sjk ¼

m¼1 fm zmnq PM m¼1 zmnq

ð4Þ

where zmnq is set as 1 if m-th combination is assigned with q-th level for the n-th factor; otherwise, zmnq is set as 0. With all the Sjk calculated, the best combination of the levels can be determined by selecting the level of each factor that provides the highest-quality output. For a minimization problem, a smaller Sjk indicates better quality for the k-th level on factor j. The robust performance of OED is revealed by the fact that even if the best experimental combination does not exist in the OA, it can still be discovered by the FA process [18,62]. 2.4. OED-based PSO variants Recently, OED has been used by researchers of PSO for different purposes. Ko et al. [29] used OED to identify the optimal parameter settings of PSO. Zhao et al. [63] generated the initial population of their improved PSO by using OED. Yang et al. [59] developed an OED-based multi-parent crossover operator, which behaves as the local search among the m randomly selected particles, in their orthogonal design-based PSO (ODPSO). Wang and Chen [54] proposed an OED-based local search operator in their orthogonal test-based PSO (OT-PSO) to exploit the neighbor area around the global best solution. OED has a twofold role in the extrapolated PSO based on orthogonal design (ODEPSO) [13]. On one hand, it is used to initialize the ODEPSO swarm. On the other hand, OED is used to design a self-adaptive orthogonal crossover operator that could self-adaptively adjust the number of factors in OA and the segmentation location of solution vectors by setting similarity bounds. Ho et al. [18] developed an intelligent move mechanism (IMM) by using OED in their orthogonal PSO (OPSO) to predict the particle’s next position. Unlike the OED-based crossover operators proposed in [13,54,59], OPSO selects two temporary moves, H and R, that correspond to the particle’s cognitive and social parts, respectively, as the parent vectors of IMM module. In the orthogonal learning PSO (OLPSO) proposed by Zhan et al. [62], the OED is used to construct an effective exemplar to guide the particle search. OLPSO with local topology (OLPSO-L) was reported to eclipse its variant with global topology (OLPSO-G). 3. Adaptive two-layer PSO with elitist learning strategy In this section, we first provide the motivation used to develop the proposed ATLPSO-ELS, followed by a general description of the involved modules in ATLPSO-ELS. Next, we explain the methodologies used to calculate the two metrics of diversities used in the swarm evolution of ATLPSO-ELS, namely, population spatial diversity (PSD) and population’s fitness spatial diversity (PFSD). Finally, we provide a detailed description of each module used in ATLPSO-ELS. 3.1. Motivation According to [35], TLPSO-IDL achieved significant performance gains in solving conventional and rotated problems. Nevertheless, our early studies revealed that the searching performance of TLPSO-IDL deteriorates when dealing with shifted and complex problems that possess more complicated fitness landscapes. By revisiting the TLPSO-IDL framework, we observed that the mechanism of adaptive task allocation is performed on the memory swarm members only, where each of these members is self-adaptively assigned to perform either the exploration or exploitation searches during the memory swarm evolution. Conversely, all current swarm members perform the same search task, given that no such division of labor mechanism has been developed in the current swarm evolution. The absence of adaptive task allocation mechanism in current


53

swarm evolution might be one of the contributing factors for the inferior performances of TLPSO-IDL in solving problems with more complicated search spaces. Second, the current and memory swarm evolutions in TLPSO-IDL are performed sequentially during the searching process. Although such sequential implementation could improve the algorithm’s searching accuracy, it is anticipated to incur more unnecessary fitness evaluations (FEs), considering that the current and memory swarm evolutions do not always play equally important roles during the searching process. Specifically, in some evolutionary states, the current swarm evolution is more promising than the memory swarm evolution in improving the fitness of swarm members and vice versa. Excess consumption of FEs tends to compromise the searching efficiency of TLPSO-IDL. Finally, we also observed that no twoway interaction has been established between TLPSO-IDL swarm members and the global best particle Pg. As explained in [52], different particles may have good values in different dimensions of their positions. Without the two-way interaction, the current or memory swarm members that successfully improve their fitness are unable to transfer their useful information to the Pg particle. The absence of information exchange mechanism between the swarm members and the Pg particle tends to impair the convergence capability of TLPSO-IDL toward the optimal solutions of the problems. Motivated by the aforementioned observations, we opine that the optimization capability offered by the TLPSO-IDL framework is not fully exploited and that TLPSO-IDL could be enhanced further. We propose a new PSO variant, namely, ATLPSO-ELS, in which some new features are introduced. First, a new adaptive task allocation mechanism has been developed in the current swarm evolution of ATLPSO-ELS. Second, ATLPSO-ELS does not perform the current and memory swarm evolutions sequentially. Finally, an information exchange mechanism was established in ATLPSO-ELS to allow the swarm members to contribute their useful information to the Pg particle and to further evolve the latter particle. 3.2. General description of the proposed ATLPSO-ELS The proposed ATLPSO-ELS is developed based on the general concept of MC, i.e., the combination of various interacting modules under one algorithmic framework, given that this approach is one of the most prominent practices used to address the various types of problems in the computational intelligence community. The first component adopted in the ATLPSO-ELS involves the two-layer evolution framework, which is responsible for the evolutions of the current and memory swarms. The use of the two-layer framework in our approach is inspired by the dynamic behaviors of the current and memory swarms of PSO during its evolution. Specifically, the current swarm, which carries the particle’s current position, tends to explore the search space before gathering them around their best experiences (i.e., cognitive and social experiences) [11]. In turn, the memory swarm members that represent the particle’s best experiences are prone to congregating around the minimizers of the objective functions and exploiting these regions [11]. The strong clustering tendency of the memory swarm suggests that the algorithm can be effectively guided toward the search of the global optimum by evolving the particles’ best experiences with some intelligent optimization procedures. The idea of ADL is used to resolve the intense conflict between the exploration/exploitation searches of PSO. Two ADL modules are proposed to self-adaptively assign different search tasks to each swarm member during the current and memory swarms’ evolutions. Based on the swarm member’s fitness and diversity, the proposed ADL modules will either locate the swarm members to visit the unexplored region in the search space (i.e., exploration) or locate the swarm members to finetune the regions of already-known minima (i.e., exploitation). Although the concept of the ADL modules shares some similarities with the ESE module used by APSO [61], our approach has some differences that distinguish its ADL modules from those in APSO. First, our ADL modules divide the population into the exploration and exploitation sections according to the swarm member’s fitness and diversity, whereas the ESE module is used to determine the evolutionary states of APSO particles. Second, ADL modules are used to self-adaptively assign different search tasks to different swarm members with different fitness and diversity. Conversely, the outputs of the ESE module are used to adapt the APSO particle’s x, c1, and c2 parameters. Finally, the development of the ELS module is motivated by the fact that the learning strategy used by the Pg particle in majority of PSO variants (e.g., FIPSO, CLPSO, and FLPSO-QIW) is the same as those used by the remaining particles in the population. The Pg particle should be equipped with its own learning strategies given that the Pg particle is responsible for guiding the swarm to seek the optimum solution. We propose two learning strategies, namely, the OED-based learning strategy (OEDLS) and the stochastic perturbation-based learning strategy (SPLS) in the ELS module to specifically evolve the Pg particle when predefined conditions are met. The OEDLS is designed to extract the useful information from other particles to further evolve the Pg particle. This mechanism aims to rapidly guide the best knowledge of each particle toward the global optimum, thereby enhancing the algorithm’s convergence speed. Meanwhile, the SPLS module addresses swarm stagnation by providing extra momentum to the Pg particle to assist the particle’s escape from the local optima. 3.3. PSD metric PSD describes the population’s solution space diversity and is calculated based on the population spread in the solution space. PSD allows ATLPSO-ELS to adaptively divide the population into the exploration/exploitation sections and assign different search tasks to each cooperating particle during the memory swarm evolution. In other words, the PSD metric controls the relative size of the exploration and exploration sections during the memory swarm evolution.

54


The ATLPSO-ELS’s population consists of S particles with the personal best position of P = [P1, P2, . . . , PS]. To calculate each particle’s PSD, a hypothetical particle of Pave is first computed as the average over all S particles. P adv e represents the average of the d-th component of all P position and is calculated as

Padv e ¼

S 1X Pi;d S i¼1

ð5Þ

The PSDi of particle i is then calculated as the Euclidean distance between particle i and P adv e and expressed as

psdi ¼ PSDi ¼

rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi XD 2 ðPi;d Padv e Þ d¼1 psdi psdmin psdmax psdmin

ð6Þ

ð7Þ

where D denotes the dimensionality of search space; psdi is the particle’s i non-normalized spatial diversity; psdmax and psdmin represent the maximum and minimum non-normalized spatial diversity in the population; these values are obtained from the particles that are nearest and farthest from Pave, respectively. Eqs. (6) and (7) reveal that particle i with Pi position closer to P adv e has lower spatial diversity, and vice versa. 3.4. PFSD metric PFSD describes the population’s solution space diversity from the fitness perspective. Thus, each particle’s PFSD is influenced by its fitness. The computation of PFSD involves merging fitness and solution spaces, in which each particle’s distance contribution is weighted based on its fitness. In ATLPSO-ELS, the PFSD is used to adaptively regulate the particle’s exploration strength during the current and memory swarm evolutions. To calculate each particle’s PFSD, another hypothetical particle of PW.avg is computed. The influences of the particle i on the PW.avg is adjusted according to the weight contribution of Wi, which is calculated as follows:

wi ¼

fmax f ðPi Þ fmax fmin

wi W i ¼ PS

i¼1 wi

ð8Þ

ð9Þ

where wi represents particle i’s non-normalized weight value; fmax and fmin represent the worst and best fitness in the population, respectively. Next, the weighted average particle of PW.avg is computed. PW.avg represents the fitness-weighted average across the d-th component of all P position in the population and is calculated as vg PW:a ¼ d

S X W i P i;d

ð10Þ

i¼1

To calculate PFSDi, particle i’s Euclidean distance to the PW.avg particle is weighted according to the particle’s Wi, as shown in Eq. (11). Only the good (i.e., low) fitness particles that are far from the PW.avg are assigned a large PFSDi. For particles with worse (i.e., high) fitness or close to the PW.avg, a small value of PFSDi is obtained.

PFSDi ¼ W i

rffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi XD ve 2 ðPi;d PW:a Þ d d¼1

ð11Þ

Notably, the derivations of PSD and PFSD metrics in our proposed work are different from those of the mean distance and evolutionary factor in APSO. First, PSD metric is computed based on the distance between a target particle and a hypothetical particle Pavg, where the latter is absent in APSO. The mean distance of APSO is calculated as the distance of a target particle with all particles in the population. Second, the PSD and PFSD values of each ATLPSO-ELS particle are different, as these metrics are dependent on an individual particle’s fitness and its distance from the hypothetical particles Pavg and PW.avg. By contrast, the same value of evolutionary factor is assigned to all APSO particles. 3.5. Evolution of current swarm in ATLPSO-ELS During the current swarm evolution of ATLPSO-ELS, an ADLcurrent module was developed to adaptively adjust the particles’ exploration/exploitation strengths based on their respective PFSD values. The ADLcurrent module adjusts the velocity of each particle i (i.e., Vi) via a selected exemplar particle Pe,i. To generate the Pe,i of each particle i, the tournament size Tsize,i that is dependent on the particle i’s PFSDi values is first calculated

PFSDi T size;i ¼ max 2; T size;max PFSDmax

ð12Þ


55

where d e is a ceiling operator; PFSDmax denotes the maximum PFSD; Tsize,max = z S represents the maximum tournament size that is available for the current swarm member, with the parameter z that ranges from 0 to 1. The Tsize,i value of each particle i is crucial to self-adaptively adjust the particle’s exploration/exploitation strengths by determining the number of candidates that are eligible as the exemplar candidates for particle i. The particle with a larger Tsize,i value is more likely to select the fitter particle as exemplar and thus has stronger exploitation strength. Conversely, the particle with smaller Tsize,i value is more explorative as it tends to select outlying particles to guide the search. Eq. (12) shows that the Tsize,i of each particle varies linearly with their respective PFSD value. Accordingly, the particle i with better (i.e., lowve er) fitness f(Pi) and distant from P W:a has higher values of PFSDi and Tsize,i, and thus, is more exploitative. By contrast, the d ve particle i with inferior (i.e., higher) f(Pi) or near P W:a has stronger exploration strength, given that smaller PFSDi and Tsize,i d values are assigned to the particle. Tsize,max emerges as another key metric to change the value of Tsize,i, and therefore, it is also essential in tuning the particle’s exploration/exploitation strengths during the current swarm evolution of ATLPSO-ELS. Fig. 1 depicts the exemplar selection procedure for particle i. To obtain the d-th dimensional component of Pe,i, a total of Tsize,i candidates are first randomly selected from P = [P1, . . . Pi, . . . PS], by excluding the Pg particle. Among these Tsize,i candidates, the particle k with fittest (i.e., lowest) f(Pk) is used as the d-th dimensional component of Pe,i. In our implementation, the exemplar index of Pe,i is obtained instead of the real position of Pe,i to ensure that new information from the candidates can be used immediately once they successfully update their personal best position. The particle i in the current swarm updates the velocity of the particle as follows:

V i ¼ xV i þ cr3 ðPe;i X i Þ

ð13Þ

where c is the acceleration coefficient and is set to 2; r3 is a random number in the range of [0, 1]. To sustain the particle’s momentum during the search process, the Vi,d of particle i is randomly initialized when the value becomes zero. The fitness of the updated Xi of particle i, f(Xi) is then evaluated and compared with Pi. If f(Xi) < f(Pi), Xi will replace Pi. Similarly, Xi could replace Pg, if f(Xi) < f(Pg). A failure counter fc is used to record the number of times where the fitness of Pg particle fails to be improved. If the fitness of Pg is successfully improved, the fc counter is reset to zero; otherwise, the counter value is increased by one. The fc counter is used to check whether the condition for Pg particle to perform its unique learning strategy through ELS module is met. Fig. 2 depicts the complete procedure of the ADLcurrent module in the current swarm. To prevent frequent changing of guidance direction, the Pe,i is maintained as the exemplar of particle i until Pe,i fails to improve the Pi position of particle i. To achieve this objective, we define an improve_flagi to record the success experience of Pe,i in guiding the particle i. When

Fig. 1. Exemplar index selection in the current swarm’s ADLcurrent module.

Fig. 2. Overall framework of the ADLcurrent module adopted in the current swarm evolution.

56


the current Pe,i is no longer able to guide the particle in the current swarm toward the better solution, the improve_flagi is set as ‘no’ and a new exemplar index is then obtained to generate a new Pe,i, which could offer a new searching direction. The search mechanism proposed in the ADLcurrent module shares some similarities with that of CLPSO, FLPSO-QIW, and DNLPSO given that these PSO variants also use the exemplars that are derived from the non-global optimum solution to guide the search. Nevertheless, the working mechanisms used in the ADLcurrent module to obtain the exemplar are different from those of these PSO variants. For instance, the number of candidates used by the CLPSO, FLPSO-QIW, and DNLPSO to generate the exemplar is always set as two for all particles without considering the latter’s diversity and fitness. Compared with these PSO variants, the proposed ADLcurrent module is equipped with a sequence of systematic procedures to determine the Tsize,i of each particle during the derivation of the exemplar. Therefore, the latter is anticipated to be more efficient in tuning the particle’s exploration/exploitation strengths. 3.6. Evolution of memory swarm in ATLPSO-ELS An ADLmemory module is proposed to perform the memory swarm evolution of ATLPSO-ELS. ADLmemory module uses PSD and PFSD metrics to adaptively allocate different searching tasks to the memory swarm’s member to ensure these newly evolved cognitive and social experiences could either locate better regions around the problem’s optima (i.e., exploration) or to rapidly exploit the regions of the already-found minima (i.e., exploitation). In the following subsections, we explain the mechanism of the ADLmemory module in detail. 3.6.1. Adaptive task allocation by PSD metric A heuristic rule is used by ADLmemory to adaptively divide the memory swarm into the exploration and exploitation regions based on the PSD metric. Particle i with higher spatial diversity (i.e., higher PSDi) tends to emphasize the exploitation to refine its solution, whereas exploration is more favorable for particle i with lower spatial diversity (i.e., lower PSDi) to prevent premature convergence. A learning probability Pc,i is proposed to calculate the likelihood of particle i to engage in an exploitation search

Pc;i ¼

PSDi ðK 2 K 1 Þ þ K 1 PSDmax

ð14Þ

where PSDmax represents the maximum PSD; K1 and K2 are parameters that range from 0 to 1, used to determine the maximum and minimum probabilities of a particle to perform exploration and exploitation searches in the memory swarm, respectively. Eq. (14) indicates that a particle with higher PSD,i is more likely to be assigned into the exploitation region because of its higher Pc,i value, whereas a particle with smaller Pc,i value, which is associated with lower PSD,i, has a higher chance to perform the exploration search. A random number Rand is subsequently generated to assign the appropriate search task of particle i. If Rand is smaller than Pc,i, particle i is assigned into the exploitation section and vice versa, as shown below

taski ¼

exploitation; Rand < Pc;i exploration;

otherwise

ð15Þ

3.6.2. Exploitation section of memory swarm Each exploitation particle i in the memory swarm adjusts its Pi based on the social experience Pg as follows:

Pi;temp ¼ Pi þ cr4 ðP g Pi Þ

ð16Þ

where Pi,temp is the adjusted cognitive experience of particle i, and r4 is the random number in the range of [0, 1]. The fitness of Pi,temp is then evaluated and compared with Pi. If f(Pi,temp) < f(Pi), the updated Pi,temp will replace Pi. 3.6.3. Exploration section of memory swarm Inspired by FIPSO [37], we propose a new learning strategy to evolve the memory swarm members that engage into the exploration sections. Unlike the original FIPSO, which considers all particles in the neighborhood, our proposed method ranm m m domly selects Ni particles from the memory swarm, P m i ¼ ½P i;1 ; P i;2 ; . . . P i;N , by excluding the exploration Pi particle itself, to update the particle i’s position

Pi;temp ¼ Pi þ

X

ck r k ðPm i;k P i Þ

ð17Þ

Pm 2Ni i;k

where P m i;k is the personal best position of the randomly selected Pm,i members; Ni is the number of selected Pm,i members required by the particle i to guide the search and is set as Tsize,i to ensure that each exploration particle has varying exploration strength; rk is the random number in the range of [0, 1]; and ck is the acceleration coefficient that is equally distributed among the Ni randomly selected particles and calculated as

ck ¼ call =jN i j;

where call ¼ 4:1

ð18Þ


57

3.6.4. Complete framework of ADLmemory module Fig. 3 presents the overall framework for the ADLmemory module. Similar to the ADLcurrent module, the Ni randomly selected Pm,i members that are used to guide the exploration particle i are maintained as long as they are able to improve the particle i’s fitness f (Pi). When the current Pm,i members no longer improve the f (Pi), a new set of Pm,i members are randomly selected again to provide a new searching direction for particle i. Furthermore, to prevent a nullified effect during the evolution of Pg particle in the ADLmemory module, the Pg particle is evolved through SPLS, which will be explained in the following section. 3.7. Elitist learning strategy module The ELS module consists of two learning strategies, namely, the OEDLS and the SPLS, which will be triggered to evolve the Pg particle when the predefined conditions are met. Fig. 4 presents the overall framework of the ELS module, and the implementations of the OEDLS and SPLS will be presented in the following subsection. 3.7.1. OEDLS OED is capable of identifying the best combination levels with different factors within a reasonably small number of experimental samples. In addition, according to [52], different particles may have good values in different dimensions of their Pi position. Given this situation, we propose the OEDLS to enhance the learning efficiency of the Pg particle. When a particle successfully improves its Pi position during the current or memory swarm evolutions and Pi is different from the Pg particle, OEDLS is triggered to investigate which dimension of the Pi position contains the useful information and extract this information to further evolve the current Pg particle. In OEDLS, each d-th dimension is assigned as a factor. Thus, D-experimental factors for a D-dimensional problem exist. For each factor (dimension), two levels (i.e., Q = 2) are contributed by both the Pg and improved Pi, which are assigned as levels 1 and 2, respectively. When OEDLS is executed, the LM(2D) OA is generated first. A total of M combinations of candidate solutions, Xj(1 6 j 6 M), are then constructed according to the contents of LM(2D) OA, Pg, and improved Pi positions. Specifically, for d-th dimension, Pg,d is selected when the level in OA is 1, while improved Pi,d is selected for level 2. The fitness value of each Xj is then evaluated and recorded as fj(1 6 j 6 M). To identify the most significant level for a particular dimension, we calculate the Sjk for each k-th level and j-th factor by using Eq. (4), and FA is then used to derive the predictive solution Xp.

Fig. 3. Overall framework of the ADLmemory module adopted in memory swarm evolution.

Fig. 4. ELS module in ATLPSO-ELS.

58


Fig. 5. OEDLS in the ELS module.

The fitness of Xp, f(Xp), is then compared with f(Pg). If f(Xp) < f(Pg), it implies that OEDLS produces a more superior particle Xp and will replace the existing Pg. The procedure of OEDLS is presented in Fig. 5. Although the OEDLS module is designed based on the OED technique, this module is different from existing OED-based PSO variants, in terms of the purpose, involved particles, and condition to execute the module. Unlike Zhao et al. [63] and Ko et al. [29] who used the OED for population initialization and parameter selection, respectively, OEDLS aims to establish a two-way information exchange paradigm between the improved swarm members and Pg particle. In ATLPSO-ELS, the factor level assigned in the OEDLS is Q = 2, as only the improved swarm members and Pg particle are involved. ATLPSO-ELS is different from ODPSO [59], OT-PSO [54], and ODEPSO [13] considering that these variants randomly select two or more particles into their respective OA. Moreover, the OED-based operators of these variants perform search space quantization, and their Q values are set larger than two. For OPSO [18], the IMM module uses the temporary moves H and R to predict the particle’s next position, followed by the particle’s velocity. Unlike OEDLS, the IMM module is executed in every iteration. Finally, ATLPSO-ELS is different from OLPSO [62] in term of the conditions required to execute the OED-based module. Specifically, OLPSO triggers orthogonal learning only when its particle’s exemplar is no longer effective in guiding the search. By contrast, ATLPSO-ELS executes the OEDLS immediately once the fitness improvement is identified in the swarm members. Compared with OLPSO, our approach is more effective in extracting useful information from the non-global best particles and to further improve the fitness of the Pg particle. 3.7.2. SPLS To prevent the Pg particle from being trapped in the local optima, we proposed the SPLS to perform perturbation on the Pg particle, if the fitness of the particle is not improved for m successive FEs. The parameter m, which ranges from 1 to 10, should not be set too large or too small, as the former wastes computational resources, while the latter deteriorates the algorithm’s speed. In SPLS, one of the d-dimensions of the Pg particle, i.e., Pg,d is first randomly selected and is then perturbed by a normal distribution as follows:

Pper gd ¼ P gd þ sgnðr 5 Þr 6 ðX max;d X min;d Þ

ð19Þ

where P per gd is the perturbed Pg,d; sgn() is the sign function; r5 is a random number in the range of [1, 1] with uniform distribution; r6 is a random number generated from the normal distribution of N (l, r2) with a mean value of l = 0 and r = R. R denotes the perturbation range that linearly decreased with the numbers of FEs shown as

R ¼ Rmax ðRmax Rmin Þ

fes FEmax

ð20Þ

where Rmax = 1 and Rmin = 0.1 are the maximum and minimum perturbation ranges, respectively; fes denotes the number of consumed FEs; FEmax represents the maximum FEs. A perturbed Pg particle, P per will be produced through SPLS and replace g the Pg particle f ðP per g Þ < f ðP g Þ. Fig. 6 shows the implementation of SPLS. 3.8. Complete framework of ATLPSO-ELS Fig. 7 illustrates the complete implementation of the ATLPSO-ELS. The population divisions of current and memory swarms are not performed frequently to save computational resource. The current population division is maintained until f(Pg) is not improved for S successive FEs. If fc > S, new exploration and exploitation sections are generated, and all particles in the current and memory swarms are assigned new search tasks.


59

Fig. 6. SPLS in the ELS module.

Fig. 7. Complete framework of the ATLPSO-ELS algorithm.

Fig. 7 shows that unlike TLPSO-IDL, the current and memory swarm evolutions of ATLPSO-ELS are not performed sequentially, as this approach tends to demand unnecessary numbers of FEs to converge the algorithm toward the optimum. Instead, only one type of swarm evolution will be performed in every generation of ATLPSO-ELS. ATLPSO-ELS will first initiate the evolution of current swarm. As long as the current swarm evolution is able to improve the fitness of the Pg particle with at least one of the particle’s swarm members, we assume that these current swarm members are on the right track to locate the global optimum, and no intervention from the memory swarm evolution is required. When the current swarm evolution no longer improves the Pg particle, the memory swarm evolution of ATLPSO-ELS will then take over the search. Similarly, the memory swarm evolution will be replaced by the current swarm evolution if the former fails to achieve any fitness improvement on the Pg particle. Another issue that is worth mentioning is that although the ideas behind the ADLcurrent and ADLmemory modules are similar, i.e., some individuals in the current and memory swarms focus on exploration and others on exploitation, we designed these two population division strategies with different search dynamics. This necessity is driven by the fact that the current and memory swarms have exhibited different degrees of clustering tendency during the evolution, as validated by the Hopkins test [19] in [11]. Accordingly, the H-measure of the memory swarm is higher than that of the current swarm, which implies that the former appears to have an exploitative behavior, whereas the latter appears to exhibit a more explorative nature. As different clustering tendencies have been demonstrated by the current and memory swarms, using the population division strategies with similar search dynamics on these two distinct swarms tends to introduce a conflicting effect on the evolution process. The poor performance of using the strategies with similar search dynamics is proven in [11,43], and these findings suggest that using the population division strategies with different search dynamics might be plausible to enhance the performance of ATLPSO-ELS. 4. Simulation results This section describes the experimental settings that were used to evaluate the ATLPSO-ELS and presents the experimental results.

60


4.1. Benchmark function Twenty-five scalable benchmark functions [50,60] are used to thoroughly investigate the performance of ATLPSO-ELS and its competitors thoroughly. We perform the evaluation with 50 variables, i.e., D = 50. Table 1 lists these benchmark functions, brief descriptions of their formulae, their feasible search range RG, the fitness value of their global minimum Fmin, and their accuracy level e. The benchmark functions that were used are divided into four classes, namely, conventional problems, rotated problems, shifted problems, and complex problems.

4.2. Simulation settings for the involved PSO algorithms We compared 10 state-of-the-art PSO variants with the ATLPSO-ELS. Table 2 describes the parameter settings for all PSO variants extracted from their respective literature. For ATLPSO-ELS, the values of x, c, and call are set based on the recommendation in [33,37]. A parameter sensitivity analysis is also conducted in the following subsection to investigate the effect of parameters K1, K2, z, and m on the searching performance of ATLPSO-ELS. All PSO variants are run independently 30 times to reduce random discrepancy. For a fair comparison, the same maximum FEs of FEmax = 3.00E+05 are used to terminate the algorithms [50]. The calculations are also stopped if the exact solution is found. Except for SLPSO and TLPSO-IDL, none of the compared PSO variants had been reported by their corresponding literature to solve 50-D problems. Thus, the optimal population sizes of these algorithms in solving the 50-D problems

Table 1 Twenty-five benchmark functions used in this study (Note: fbiasj,"je[1,11] denotes the shifted fitness value applied to the corresponding functions). No.

Function name

Category I: Conventional Problems F1 Sphere F2

Schwefel 1.2

F3

Rosenbrock

F4

Rastrigin

F5

Noncontinuous Rastrigin

F6

Griewank

F7

Ackley

F8

Weierstrass

RG

Fmin

e

PD

[100, 100]D

0

1.0e–6

PD

[100, 100]D

0

1.0e–6

Formulae

F 1 ðX i Þ ¼

2 d¼1 X i;d

2 P F 2 ðX i Þ ¼ d¼1 ð dj¼1 X i;j Þ PD1 2 F 3 ðX i Þ ¼ d¼1 ð100ðX 2i;d X i;dþ1 Þ þ ðX i;d 1Þ2 Þ PD 2 F 4 ðX i Þ ¼ d¼1 ðX i;d 10 cosð2pX i;d Þ þ 10Þ P 2 F 5 ðX i Þ ¼ D d¼1 ðY i;d 10 cosð2pY i;d Þ þ 10Þ X i;d ; jX i;d j < 0:5 where Y i;d ¼ roundð2X i;d Þ=2; jX i;d j P 0:5 pffiffiffi P QD 2 F 6 ðX i Þ ¼ D d¼1 X i;d =4000 d¼1 cosðX i;d = dÞ þ 1 qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi PD 2 F 7 ðX i Þ ¼ 20 expð0:2 d¼1 X i;d =DÞ P expð D cosð2 p X Þ=DÞ þ 20 þ e i;d d¼1 P Pk max k k ð ½a cosð2 p b ðX F 8 ðX i Þ ¼ D i;d þ 0:5ÞÞ d¼1 k¼0 P max k k ½a cosðpb Þ D kk¼0 a ¼ 0:5; b ¼ 3; k max ¼ 20

Category II: Rotated Problems F9 Rotated Sphere F10 Rotated Schwefel 1.2 F11 Rotated Rosenbrock F12 Rotated Rastrigin F13 Rotated Noncontinuous Rastrigin F14 Rotated Grienwanks

F9(Xi) = F1(Zi), Zi = M Xi F10(Xi) = F2(Zi), Zi = M Xi F11(Xi) = F3(Zi), Zi = M Xi F12(Xi) = F4(Zi), Zi = M Xi F13(Xi) = F5(Zi), Zi = M Xi F14(Xi) = F6(Zi), Zi = M Xi

Category III: Shifted Problems F15 Shifted Sphere F16 Shifted Rosenbrock F17 Shifted Rastrigin F18 Shifted Noncontinuos Rastrigin F19 Shifted Griewank F20 Shifted Ackley F21 Shifted Weierstrass

F15(Xi) = F1(Zi) + fbias1, F16(Xi) = F3(Zi) + fbias2, F17(Xi) = F4(Zi) + fbias3, F18(Xi) = F5(Zi) + fbias4, F19(Xi) = F6(Zi) + fbias5, F20(Xi) = F7(Zi) + fbias6, F21(Xi) = F8(Zi) + fbias7,

Category IV: Complex Problems F22 Shifted Rotated Griewank F23 Shifted Rotated Ackley F24 Shifted Rotated High Conditioned Elliptic F25

Shifted Expanded Griewank’s plus Rosenbrock

Zi = Xi o, Zi = Xi o, Zi = Xi o, Zi = Xi o, Zi = Xi o, Zi = Xi o, Zi = Xi o,

fbias1 = 450 fbias2 = 390 fbias3 = 330 fbias4 = 330 fbias5 = 180 fbias6 = 140 fbias7 = 90

F22(Xi) = F6(Zi) + fbias8, Zi = (Xi o) M, fbias8 = 180 F23(Xi) = F7(Zi) + fbias9, Zi = (Xi o) M, fbias9 = 140 d1 P 6 D1 2 F 24 ðX i Þ ¼ D Z i;d þ fbias10 ; Z i ¼ ðX i oÞ M; d¼1 ð10 Þ f bias10 ¼ 450 F20 = F6(F3(Zi,1, Zi,2)) + F6(F3(Zi,2, Zi,3)) + + F6(F3(Zi,D1, Zi,D)) + F6(F3(Zi,D, Zi,1)) + fbias11 Zi = Xi o, fbias10 = 130

[2.048, 2.048]

D

0

1.0e–2

[5.12, 5.12]D

0

1.0e–2

[5.12, 5.12]D

0

1.0e–2

[600, 600]D

0

1.0e–2

[32, 32]D

0

1.0e–2

[0.5, 0.5]D

0

1.0e–2

[100, 100]D [100, 100]D [2.048, 2.048]D [5.12, 5.12]D [5.12, 5.12]D [600, 600]D

0 0 0 0 0 0

1.0e–6 1.0e–2 1.0e–2 1.0e–2 1.0e–2 1.0e–2

[100, 100]D [100, 100]D [5.12, 5.12]D [5.12, 5.12]D [600, 600]D [32, 32]D [0.5, 0.5]D

450 390 330 330 180 140 90

1.0e–6 1.0e–2 1.0e–2 1.0e–2 1.0e–2 1.0e–2 1.0e–2

[600, 600]D [32, 32]D [100, 100]D

180 140 450

1.0e–2 1.0e–2 1.0e–6

[5, 5]D

130

1.0e–2


61

Table 2 Parameter settings of the involved PSO algorithms. Algorithm

Year

Population topology

Parameter settings

MPSO-TVAC [46] APSO [61] FIPSO [37] CLPSO [33]

2004

Fully Connected

x: 0.9 0.4, c1: 2.5 0.5, c2: 0.5 2.5

2009 2004 2006

x: 0.9 0.4, c1 + c2: [3.0, 4.0], d = [0.05, 0.1], rmax = 1.0, rmin = 0.1 P v = 0.729, ci = 4.1 x: 0.9 0.4, c = 2.0, m = 7

FLPSO-QIW [51]

2011

FPSO [39] SLPSO [32] OPSO [18] OLPSO-L [62] TLPSO-IDL [35]

2009 2012 2008 2011 2013

Fully Connected Local URing Comprehensive Learning Comprehensive Learning Adaptive Adaptive Fully Connected Orthogonal Learning Adaptive

ATLPSO-ELS

–

Adaptive

x: 0.9 0.4, c = 2.0, call = 4.1, K1 = 0.4, K2 = 0.8, Rmax = 1.0, Rmin = 0.1, m = 5, z = 0.5

x: 0.9 0.2, c1: 2 1.5, c2: 1 1.5, m = 1, Pi = [0.1, 1], K1 = 0.1, K2 = 0.001, r1 = 1, r2 = 0 P

x: 0.9 0.4, ci = 4.1 x: 0.5 0.0, c = 1.496, a = [0, 1], R = 4 x: 0.9 0.4, c1 = c2 = 2.0 x: 0.9 0.4, c = 2.0, G = 5 x: [0.9 0.4], c = 4.1, c1: 2 1.5, c2: 1 1.5, K1 = 0.4, K2 = 0.8, Rmax = 1.0, Rmin = 0.1, m = 5, z = 0.5

are undetermined. To resolve this issue, the population size S used by these PSO variants to solve the 50-D problems is set based on the recommendation of [32], i.e., S = 30. 4.3. Performance metrics In this study, we evaluate the searching accuracy, searching reliability, and searching efficiency of PSO through the mean fitness value (Fmean), success rate (SR), and success performance (SP), respectively [50]. We use the Wilcoxon test [15] to perform rigorous comparisons between ATLPSO-ELS and its peers. The test is a non-parametric statistical procedure [15] that is used to perform pairwise comparison between the ATLPSO-ELS and its peers. The Wilcoxon test is conducted at 5% significance level (i.e., a = 0.05), and the values of h, R+, R, and p are reported. The h value indicates whether the performance of ATLPSO-ELS is better (i.e., h = ‘‘+’’), insignificant (i.e., h = ‘‘=’’), or worse (i.e., h = ‘‘’’) than that of the other 10 algorithms at the statistical level. R+ and R denote the sum of the ranks at which ATLPSO-ELS outperforms and underperforms compared with the other methods. The p-value represents the minimal significance level for detecting differences. If the p-value is less than a, it definitively proves that the better result achieved by the best algorithm in each case is statistically significant and was not obtained by chance. 4.4. Parameter sensitivity analysis Four parameters were introduced in ATLPSO-ELS, namely, K1, K2, z, and m. Nevertheless, a complete evaluation on all possible combinations of these parameters is impractical. The parameter tuning strategy, as reported in [30], is used to find an appropriate parameter combination that gives ATLPSO-ELS relatively good performances in solving conventional (F1 to F8), rotated (F9 to F14), shifted (F15 to F21), and complex (F22 to F25) problems. Two 10-D functions are selected from each problem category for parameter tuning, namely F1 and F8 for conventional problem, F10 and F14 for rotated problems, F16 and F17 for shifted problems, and F24 and F25 for complex problems. For each category of problems, we first initialize the parameter combination of [K1, K2, z, m] as the mean values of their respective upper and lower boundary values, i.e., [0.5, 0.5, 0.5, 5]. To summarize, the parameter values of K1, K2, and z range from 0 to 1, whereas m ranges from 0 to 10. Based on this initial combination, we adjust the parameters one at a time. With other parameters fixed, we first vary the value of K1 from 0 to 1 to find the K1 value that gives the best fitness value. Then, we update the value of K1 in the parameter combination of [K1, K2, z, m]. This new combination is used to tune K2. The process continues until we have updated z and m. Figs. 8–11 show the experimental findings of ATLPSO-ELS with varying K1, K2, z, and m values. The simulation results indicate that the searching accuracy of ATLPSO-ELS, as represented by Fmean, in solving the conventional and rotated problems is insensitive to parameters K1, K2, z, and m. ATLPSO-ELS successfully identifies the global optima of F1, F8, F10, and F14 regardless of the K1, K2, z, and m values. Conversely, the Fmean values of ATLPSO-ELS change along with parameters K1, K2, z, and m. Generally, ATLPSO-ELS obtained inferior Fmean values in the minimum or maximum values of K1, K2, z, and m. This result could be justified by the fact that in the extreme case of K1 = 0 and K2 = 1, particle i with the lowest diversity (i.e., PSDi = 0) has a probability of 0 (Pc,i) and 1 (1 – Pc,i) to perform exploitation and exploration searches, respectively. By contrast, the particle i with the highest diversity of PSDmax has a probability of 1 (Pc,i) and 0 (1 – Pc,i) to engage in exploitation and exploration sections, respectively. The opposite scenario could be observed in another extreme scenario of K1 = 1 and K2 = 0. Both scenarios have inevitably limited the searching flexibilities of particles with the lowest and highest diversity values, as they are allowed to perform only one type of search task. Meanwhile, the lower and upper boundary values of z produce Tsize,max = 2 and S, respectively. In these cases, the members of the current swarm tend to be overbiased to-

62


1.00E+01

1.00E+00 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1.00E+00 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1.00E-02 1.00E-03

1.00E-02

Fmean

Fmean

1.00E-01

1.00E-06

1.00E-04 1.00E-05

1.00E-04

1.00E-08

Value of K1

Value of K2

(a)

(b) 1.00E+00

1.00E-02

0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1.00E-04

2

3

4

5

6

7

8

9 10

1.00E-04 1.00E-06

1.00E-06 1.00E-08

1

1.00E-02

Fmean

Fmean

1.00E+00

1.00E-08

Value of z

Value of m

(c)

(d)

Fig. 8. Parameter tuning for F16, (a) K1, (b) K2, (c) z, and (d) m.

1.00E-11

1.00E-12

1.00E-12 1.00E-13 1.00E-14

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Fmean

Fmean

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1.00E-13 1.00E-14 1.00E-15

Value of K1

Value of K2

(a)

(b)

1.00E-12

1.00E-12

1.00E-13 1.00E-14 1.00E-15

0

Fmean

Fmean

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Value of z

1

2

3

4

5

6

7

8

9 10

1.00E-13 1.00E-14 1.00E-15

Value of m

(c)

(d)


ward the exploration or exploitation searches, given that the Tsize,i values that were produced are either too small or too large. Such extreme parameter settings tend to jeopardize the capability of the ADLcurrent module in regulating the particle’s exploration/exploitation strengths. Figs. 8–12 reveal that ATLPSO-ELS achieves the best Fmean values in functions F16, F17, F24, and F25 when K1, K2, z, and m are set as 0.4, 0.8, 0.5, and 5, respectively. The parameter settings of K1 = 0.4 and K2 = 0.8 allow the Pc,i of each particle i to vary from 0.4 to 0.8 according to its diversity value (0 < SPD < SPDmax). These settings could provide more flexibility to the particles with the lowest and highest diversities in performing the search. When particle i has PSDi = 0, particle i still has minimum probabilities of 0.4 (Pc,i) and 0.6 (1 – Pc,i) to perform exploitation and exploration searches, respectively. For particle i with highest diversity of PSDmax, the probability that it will engage in exploitation search is 0.8 (Pc,i), while the probability that it will perform exploration search is 0.2 (1 Pc,i). The aforementioned experimental findings suggest that we can set the parameter combination [K1, K2, z, m] as [0.4, 0.8, 0.5, 5] in the following performance evaluation, as the tested functions used in the parameter sensitivity analysis cover a wide range of situations. 4.5. Comparison of ATLPSO-ELS with other state-of-the-art PSO variants Table 3 shows the results of the Fmean, standard deviation (SD), and the Wilcoxon test (h) achieved by the 11 algorithms for all the tested problems. Boldface text in the tables indicates the best results among the algorithms. The Fmean comparison

63

2.50E+05 2.00E+05 1.50E+05 1.00E+05 5.00E+04 0.00E+00

Fmean

2.00E+05 1.50E+05 1.00E+05 5.00E+04 0.00E+00 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Value of K1

Value of K2

(a)

(b)

1.00E+05 8.00E+04 6.00E+04 4.00E+04 2.00E+04 0.00E+00

2.00E+05

Fmean

Fmean

Fmean


1.50E+05 1.00E+05 5.00E+04 0.00E+00

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

1

2

3

4

5

6

Value of z

Value of m

(c)

(d)

7

8

9 10

1.20E+00 1.00E+00 8.00E-01 6.00E-01 4.00E-01 2.00E-01 0.00E+00

Fmean

Fmean


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Value of K1

Value of K2

(a)

(b)

4.00E-01 3.00E-01

Fmean

Fmean

6.00E-01 5.00E-01 4.00E-01 3.00E-01 2.00E-01 1.00E-01 0.00E+00

2.00E-01 1.00E-01 0.00E+00 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1.00E+00 8.00E-01 6.00E-01 4.00E-01 2.00E-01 0.00E+00 0

1

2

3

4

5

6

Value of z

Value of m

(c)

(d)

7

8

9 10


results between ATLPSO-ELS and other peers are summarized as ‘‘w/t/l’’ and #BMF. w/t/l’’ means that compared with its contenders, ATLPSO-ELS wins in w functions, ties in t functions, and loses in l functions. #BMF is the number of best Fmean value achieved by each PSO variant. The Wilcoxon test results (h) is summarized as ‘‘+/=/,’’ to denote the number of functions in which ATLPSO-ELS performs significantly better than, almost the same as, and significantly worse than its peer algorithm, respectively. Table 5 presents the SR and SP values produced by all involved algorithms to compare their reliabilities and efficiencies. We summarize the SR and SP results as #S/#PS/#NS and #BSP, respectively. The former indicates the number of functions that are completely solved (i.e., SR = 100%), partially solved (i.e., 0% < SR < 100%), and never solved (i.e., SR = 0%) by a particular PSO variant, respectively. Conversely, the latter represents the number of best (i.e., lowest) SP values attained by the involved PSO variants. The SR and SP values of functions F11 and F23 to F25 are omitted from Table 5, given that none of the involved PSO variants are able to solve these functions within the predefined e in at least one run. 4.5.1. Comparison of the Fmean results Table 3 shows that ATLPSO-ELS has the best searching accuracy as ATLPSO-ELS outperforms its peers with large margin in majority of the problems. ATLPSO-ELS achieves 21 best Fmean values out of the 25 used benchmarks, i.e., 1.75 times better than the second-ranked TLPSO-IDL. Both ATLPSO-ELS and TLPSO-IDL successfully locate the global optima of all the

64


102

104

10−2

log[ObjV(x)−ObjV(x*)]


100

10−4 10−6 10-8 APSO

10−10

CLPSO FLPSO−QIW FPSO FIPSO

10−12

OLPSO−L MPSO−TVAC

103

102

101

APSO CLPSO FLPSO−QIW FPSO FIPSO

100

OLPSO−L MPSO−TVAC OPSO

OPSO

10−14

SLPSO

SLPSO

TLPSO−IDL

TLPSO−IDL

10

ATLPSO−ELS

−16

0

0.5

1

1.5

2

2.5

Function Evaluation Number

10

3 5 x 10

ATLPSO−ELS

−1

0

0.5

1

1.5

2

2.5

3 5


x 10

(b)

(a) 1011

102

1010

10−2 10

−4

10

−6

10

−8



100

APSO CLPSO FLPSO−QIW

10

−10

10

−12

FPSO FIPSO OLPSO−L MPSO−TVAC

APSO CLPSO FLPSO−QIW FPSO FIPSO

10

OLPSO−L

9

MPSO−TVAC OPSO SLPSO TLPSO−IDL ATLPSO−ELS

108

107

OPSO SLPSO TLPSO−IDL

10−14

ATLPSO−ELS

0

0.5

1

1.5

2


(c)

2.5

3 5

x 10

106

0

0.5

1

1.5

2


2.5

3 5

x 10

(d)

Fig. 12. Convergence curves of 50-D test functions. (a) F7, (b) F12, (c) F21, and (d) F24.

conventional and rotated problems, except for functions F3 and F11. For shifted problems, the searching accuracy of all involved algorithms is degraded as none of them can find the global optima of the tested functions, except for function F19. Nevertheless, ATLPSO-ELS is least affected by the shifting operation, given that it is the only algorithm that can locate the near-global optima of all shifted problems (except for F16) with a minimum accuracy level of 1013. Finally, the performance of all involved PSO variants degrades further in the complex problems. The inclusion of both rotating and shifting operations (F22 to F23) and expanded operations (F25) significantly increased the problems’ complexities. Although no algorithms could locate the global or near-global optima of the complex problems, ATLPSO-ELS is proven the best because it attained three best Fmean values in four of the tested problems.

4.5.2. Comparison of the non-parametric Wilcoxon test results The pairwise comparison results between ATLPSO-ELS and ATLPSO-ELS’s peers by using Wilcoxon test are summarized in Tables 3 and 4. Table 3 provides the pairwise comparison results in each used benchmark by using the h values, whereas Table 4 reports the R+ and R obtained in each comparison and the associated p-value. Table 3 shows that the h values obtained from the Wilcoxon test are consistent with the reported Fmean values. The number of problems in which ATLPSO-ELS significantly outperforms its peers is much larger than the number of problems in which it performs significantly worse than its peers. Table 4 confirms the significant improvements of ATLPSO-ELS over the 10 PSO variants, given that all p-values reported by the Wilcoxon test are less than a = 0.05. This experimental finding also successfully detects the performance differences between ATLPSO-ELS and TLPSO-IDL. The finding strongly suggests that the better results achieved by ATLPSO-ELS against TLPSO-IDL are statistically significant and were not obtained by chance.

65

W.H. Lim, N.A. Mat Isa / Information Sciences 273 (2014) 49–72 Table 3 Fmean, SD, and h values for 50-D problems. APSO

CLPSO

FLPSOQIW

FPSO

FIPSO

OLPSO-L

MPSOTVAC

OPSO

SLPSO

TLPSOIDL

ATLPSOELS

F1

Fmean SD h

2.50E01 1.81E01 +

3.29E47 1.28E46 +

2.90E81 5.97E81 +

7.02E+01 6.98E+01 +

2.96E01 8.06E01 +

4.86E33 5.15E33 +

0.00E+00 0.00E+00 =

6.55E56 1.04E55 +

3.22E28 8.50E28 +

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F2

Fmean SD h

1.46E+03 4.82E+02 +

5.13E+03 1.00E+03 +

2.62E+02 8.90E+01 +

3.44E+03 1.33E+03 +

8.13E+00 2.47E+01 +

5.71E+02 1.85E+02 +

2.54E02 2.83E02 +

2.44E+04 9.35E+03 +

2.79E+01 8.74E+01 +

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F3

Fmean SD h

4.62E+01 1.53E+00 +

4.35E+01 1.83E01 +

4.22E+01 2.39E01 +

5.68E+01 7.08E+00 +

4.77E+01 8.44E01 +

4.30E+01 3.18E+00 +

4.34E+01 5.10E01 +

5.57E+01 3.28E+01 +

6.11E26 2.33E25

3.79E+01 1.05E+00 +

1.70E+01 2.89E+00

F4

Fmean SD h

5.80E01 6.29E01 +

9.10E+01 1.08E+01 +

2.60E+00 1.52E+00 +

1.85E+01 1.02E+01 +

1.57E+00 3.71E+00 +

3.32E01 6.03E01 +

3.02E15 6.47E15 +

6.63E02 2.52E01 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F5

Fmean SD h

3.60E02 3.22E02 +

8.10E+01 9.76E+00 +

5.58E+00 2.36E+00 +

1.60E+01 9.56E+00 +

5.70E01 8.65E01 +

1.17E+00 1.15E+00 +

2.61E15 3.72E15 +

1.33E01 3.46E01 +

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F6

Fmean SD h

1.70E01 8.21E02 +

3.39E11 1.73E10 +

5.75E04 2.21E03 +

1.86E+00 9.28E01 +

1.93E01 3.47E01 +

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00 =

8.22E04 2.53E03 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F7

Fmean SD h

6.60E02 2.57E02 +

1.15E14 2.59E15 +

3.43E14 1.07E14 +

1.80E+00 1.10E+00 +

1.70E01 3.38E01 +

5.09E15 1.79E15 +

0.00E+00 0.00E+00 =

4.07E+00 6.39E+00 +

3.43E15 5.07E15 +

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F8

Fmean SD h

5.44E01 1.88E01 +

0.00E+00 0.00E+00 =

1.88E05 8.29E05 +

3.35E+00 2.35E+00 +

9.80E01 9.53E01 +

0.00E+00 0.00E+00 =

1.50E01 4.58E01 +

0.00E+00 0.00E+00 =

2.41E08 1.32E07 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F9

Fmean SD h

2.01E01 1.17E01 +

2.56E46 1.25E45 +

1.15E80 4.42E80 +

6.28E+01 6.96E+01 +

4.97E01 1.06E+00 +

2.89E33 2.19E33 +

0.00E+00 0.00E+00 =

1.62E55 3.84E55 +

1.69E27 6.75E27 +

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F10

Fmean SD h

1.26E+03 3.22E+02 +

5.77E+03 9.90E+02 +

2.62E+02 7.62E+01 +

3.23E+03 1.79E+03 +

8.45E+00 2.24E+01 +

1.92E+03 4.17E+02 +

1.08E01 1.93E01 +

1.92E+04 8.76E+03 +

6.67E05 3.57E04 +

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F11

Fmean SD h

5.15E+01 1.39E+01 +

4.48E+01 2.39E+00 +

4.55E+01 3.16E+00 +

5.62E+01 7.00E+00 +

4.85E+01 5.72E02 +

4.24E+01 3.73E+00 +

4.40E+01 4.64E01 +

3.16E+01 5.84E+00 +

3.65E+01 2.29E+00 +

3.92E+01 1.93E+00 +

2.40E+01 3.51E+00

F12

Fmean SD h

1.83E+02 5.61E+01 +

3.33E+02 2.34E+01 +

1.26E+02 1.76E+01 +

1.80E+02 5.01E+01 +

2.65E+01 3.39E+01 +

9.80E+01 5.16E+01 +

7.95E+01 5.80E+01 +

1.33E+02 6.23E+01 +

1.99E01 1.09E+00 =

4.71E+00 2.58E+01 =

0.00E+00 0.00E+00

F13

Fmean SD h

2.59E+02 6.15E+01 +

3.21E+02 2.52E+01 +

1.28E+02 2.13E+01 +

1.53E+02 3.74E+01 +

4.15E+01 5.13E+01 +

1.78E+02 4.94E+01 +

1.13E+02 7.17E+01 +

1.95E+02 3.36E+01 +

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F14

Fmean SD h

2.10E+02 1.01E+02 +

1.45E+00 4.50E01 +

1.52E+00 5.39E01 +

7.28E+00 5.62E+00 +

1.94E01 4.08E01 +

7.58E01 2.68E01 +

0.00E+00 0.00E+00 =

6.14E01 2.84E01 +

2.22E17 8.45E17 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

F15

Fmean SD h

2.27E01 9.70E02 +

5.68E14 0.00E+00

1.44E13 4.15E14

1.71E+04 1.47E+04 +

6.20E+00 3.90E+00 +

5.68E14 0.00E+00

1.16E11 6.14E11 +

8.80E+00 3.90E+01 +

1.36E04 8.39E05 +

2.66E08 1.26E07 +

2.27E13 4.72E14

F16

Fmean SD h

1.97E+03 3.83E+03 +

7.80E+01 4.23E+01 +

1.05E+02 4.86E+01 +

1.11E+09 2.65E+09 +

9.08E+03 4.88E+03 +

1.33E+01 1.94E+01 +

2.08E+02 3.68E+02 +

6.12E+04 3.34E+05 +

1.54E+02 5.97E+01 +

1.54E+02 1.82E+02 +

2.67E01 1.01E+00

F17

Fmean SD h

5.92E01 7.76E01 +

6.85E+01 1.01E+01 +

5.88E+00 2.51E+00 +

2.08E+02 4.59E+01 +

1.31E+02 2.93E+01 +

1.43E+00 1.10E+00 +

2.98E01 4.64E01 +

3.12E+00 2.20E+00 +

3.57E04 2.04E04 +

1.78E02 3.25E02 +

2.10E13 7.03E14

F18

Fmean SD h

7.20E03 1.06E02 +

6.99E+01 7.32E+00 +

1.20E+01 3.16E+00 +

1.63E+02 2.82E+01 +

1.48E+02 3.94E+01 +

3.00E+00 1.78E+00 +

3.01E01 4.67E01 +

2.74E+00 1.87E+00 +

5.29E04 3.03E04 +

5.46E03 1.46E02 +

1.93E13 5.08E14

F19

Fmean SD h

0.00E+00 0.00E+00 =

4.24E09 1.35E08 +

2.05E03 3.49E03 +

1.46E+03 4.63E+02 +

0.00E+00 0.00E+00 =

1.47E01 1.07E01 +

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00 =

0.00E+00 0.00E+00

(continued on next page)

66


Table 3 (continued) APSO

CLPSO

FLPSOQIW

FPSO

FIPSO

OLPSO-L

MPSOTVAC

OPSO

SLPSO

TLPSOIDL

ATLPSOELS

F20

Fmean SD h

6.04E02 1.71E02 +

4.52E08 2.47E07 +

2.19E13 5.74E14

9.29E+00 3.25E+00 +

4.47E+00 8.26E01 +

8.05E14 1.51E14

6.89E09 7.69E09 +

6.76E01 1.24E+00 +

2.42E03 4.49E04 +

1.69E05 3.35E05 +

3.32E13 7.95E14

F21

Fmean SD h

8.79E01 5.86E01 +

2.96E02 1.06E01 =

3.87E02 7.38E02 =

2.96E+01 5.70E+00 +

2.00E+01 3.86E+00 +

4.85E02 1.48E01 +

1.20E+01 4.21E+00 +

7.87E01 7.88E01 +

1.18E01 2.16E02 +

1.71E+00 5.60E01 +

2.61E14 1.24E14

F22

Fmean SD h

1.49E+00 1.47E01 +

2.95E02 5.28E02 +

6.02E03 1.04E02 +

2.06E+02 1.44E+02 +

4.81E+00 1.78E+00 +

3.76E02 4.00E02 +

1.50E02 1.64E02 +

6.33E+00 1.29E+01 +

1.17E01 3.73E02 +

8.16E03 1.11E02 +

3.86E03 7.15E03

F23

Fmean SD h

2.07E+01 1.65E01 =

2.11E+01 4.40E02 +

2.11E+01 4.16E02 +

2.11E+01 4.92E02 +

2.12E+01 5.03E02 +

2.12E+01 5.06E02 +

2.05E+01 1.56E01

2.13E+01 5.19E02 +

2.09E+01 1.04E01 +

2.06E+01 2.27E01 =

2.07E+01 2.17E01

F24

Fmean SD h

1.32E+07 4.09E+06 +

5.19E+07 8.32E+06 +

1.89E+07 4.92E+06 +

1.03E+08 8.26E+07 +

1.02E+07 3.44E+06 +

1.82E+07 5.14E+06 +

5.35E+06 2.55E+06 +

2.01E+07 1.97E+07 +

3.61E+06 8.97E+05 +

2.25E+07 1.14E+07 +

1.60E+06 6.08E+05

F25

Fmean SD h

4.13E+00 1.19E+00 +

2.03E+01 1.65E+00 +

4.01E+00 1.46E+00 +

4.35E+01 8.59E+00 +

2.76E+01 5.79E+00 +

2.98E+00 8.26E01 +

2.18E+00 6.84E01 +

2.65E+00 9.88E01 +

1.44E+00 3.73E01 +

2.54E+00 7.58E01 +

1.11E+00 5.89E01

23/2/0 1 23/2/0

23/1/1 2 22/2/1

23/0/2 0 22/1/2

25/0/0 0 25/0/0

24/1/0 1 24/1/0

21/2/2 4 21/2/2

19/5/1 7 18/6/1

23/2/0 2 21/4/0

19/5/1 6 16/8/1

12/12/1 12 11/14/0

21

w/t/l #BMF +/=/

Table 4 Wilcoxon test for the comparison of ATLPSO-ELS and 10 other PSO variants. ATLPSOELS vs.

APSO

CLPSO

FLPSOQIW

FPSO

FIPSO

OLPSO-L

MPSOTVAC

OPSO

SLPSO

TLPSO-IDL

R+ R p-Value

323.5 1.5 1.490E07

322.5 2.5 8.344E07

316 9 1.967E06

325 0 5.960E08

324.5 0.5 1.192E07

310.5 14.5 7.361E06

299.5 25.5 5.877E05

323.5 1.5 1.490E07

286.5 38.5 1.271E04

268 57 3.420E03

4.5.3. Comparisons of the SR results As shown in Table 5, ATLPSO-ELS is superior to its peers in term of searching reliability, as it is able to completely solve 18 (out of 25) of the tested benchmarks. ATLPSO-ELS and TLPSO-IDL have completely solved all conventional and rotated problems, except for functions F3 and F11. The searching reliability of ATLPSO-ELS in the shifted problems is notable, given that ATLPSO-ELS is the only algorithm that can solve these problems with SR values of not less than 90%. None of the involved algorithms are able to solve the complex problems completely or partially, except for function F22, where ATLPSO-ELS achieves the highest SR value of 86.67%. Although no algorithms are able to solve functions F23 to F25 completely or partially, ATLPSO-ELS is proven better because it achieved the best Fmean values in all the tested problems, as shown in Table 3. 4.5.4. Comparisons of the SP results As shown in Table 5, the algorithms are unable to completely or partially solve some problems (i.e., SR = 0%). In such a case, the SP value is set as an infinity value (‘‘Inf’’), and only the convergence curves as shown in Fig. 12 are used to justify the algorithm’s efficiency. Only four representative convergence curves—one from conventional, rotated, shifted, and complex problems—are presented because of space limitations. Table 5 shows that ATLPSO-ELS achieves seven and five best (i.e., smallest) SP values among the eight and five conventional and rotated problems, respectively. This finding implies that ATLPSO-ELS requires the least computation cost to achieve acceptable e in the tested problems. The rapid convergence characteristic of ATLPSO-ELS is validated by Fig. 12(a) and (b), where the convergence curves of ATLPSO-ELS in functions F7 and F12 drop off sharply at one point at the early stage of optimization. These observations indicate ATLPSO-ELS’s capability to break out of the local optima and its ability to obtain an optimal solution within a small number of FEs. For shifted problems, ATLPSO-ELS records six best SP values in functions F15 to F18, F20, and F21. The outstanding performance of ATLPSO-ELS in the shifted problems is due to its rapid convergence speed during the early stage of optimization, as shown in Fig. 12(c). For complex problems, we justify the algorithm’s convergence speed according to the functions’ convergence curves. From Fig. 12(d), we observe that the convergence speeds of ATLPSO-ELS and its peers are relatively similar at the early stage of optimization. Nevertheless, the convergence speed of ATLPSO-ELS in function F24 increased rapidly in the middle stage of optimization, which allowed ATLPSO-ELS to surpass all its peers.

67

W.H. Lim, N.A. Mat Isa / Information Sciences 273 (2014) 49–72 Table 5 SR and SP results for 50-D benchmark problems. APSO

CLPSO

FLPSO-QIW

FPSO

FIPSO

OLPSO-L

MPSO-TVAC

OPSO

SLPSO

TLPSO-IDL

ATLPSO-ELS

F1

SR SP

0.00 Inf

100.00 1.25E+05

100.00 6.04E+04

13.33 9.68E+04

80.00 9.86E+04

100.00 1.52E+05

100.00 4.49E+03

100.00 5.57E+04

100.00 1.44E+05

100.00 9.49E+03

100.00 2.84E+03

F2

SR SP

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

70.00 1.62E+05

0.00 Inf

10.00 2.66E+05

0.00 Inf

0.00 Inf

100.00 1.98E+04

100.00 8.46E+03

F3

SR SP

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

100.00 8.91E+04

0.00 Inf

0.00 Inf

F4

SR SP

0.00 Inf

0.00 Inf

6.67 3.46E+06

0.00 Inf

40.00 3.56E+05

73.33 2.97E+05

100.00 8.57E+04

93.33 1.13E+05

100.00 8.27E+04

100.00 1.97E+04

100.00 2.85E+03

F5

SR SP

6.67 2.60E+06

0.00 Inf

0.00 Inf

0.00 Inf

33.33 2.66E+05

40.00 6.75E+05

100.00 1.08E+05

86.67 1.35E+05

100.00 7.67E+04

100.00 2.14E+04

100.00 2.91E+03

F6

SR SP

0.00 Inf

100.00 1.11E+05

100.00 5.00E+04

6.67 1.60E+05

70.00 1.05E+05

100.00 1.24E+05

100.00 4.54E+03

100.00 4.63E+04

100.00 8.79E+04

100.00 1.01E+04

100.00 3.08E+03

F7

SR SP

0.00 Inf

100.00 1.05E+05

100.00 4.79E+04

3.33 3.36E+05

53.33 1.07E+05

100.00 1.25E+05

100.00 5.64E+03

70.00 1.01E+05

100.00 7.40E+04

100.00 9.92E+03

100.00 3.62E+03

F8

SR SP

0.00 Inf

100.00 1.42E+05

100.00 6.67E+04

0.00 Inf

16.67 4.59E+05

100.00 1.66E+05

90.00 1.73E+05

100.00 6.68E+04

100.00 1.72E+05

100.00 1.14E+04

100.00 2.55E+03

F9

SR SP

0.00 Inf

100.00 1.26E+05

100.00 6.04E+04

13.33 8.24E+04

73.33 9.70E+04

100.00 1.52E+05

100.00 4.49E+03

100.00 5.56E+04

100.00 1.47E+05

100.00 1.00E+04

100.00 2.72E+03

F10

SR SP

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

73.33 1.62E+05

0.00 Inf

20.00 3.13E+05

0.00 Inf

100.00 1.83E+05

100.00 2.20E+04

100.00 8.25E+03

F12

SR SP

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

53.33 2.60E+05

0.00 Inf

20.00 3.17E+04

0.00 Inf

96.67 1.15E+05

96.67 1.22E+05

100.00 5.72E+03

F13

SR SP

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

43.33 2.84E+05

0.00 Inf

16.67 1.83E+05

0.00 Inf

100.00 1.19E+05

100.00 7.83E+04

100.00 5.67E+03

F14

SR SP

0.00 Inf

0.00 Inf

0.00 Inf

6.67 2.59E+05

80.00 8.72E+04

0.00 Inf

100.00 6.71E+03

0.00 Inf

100.00 1.38E+05

100.00 7.01E+03

100.00 4.79E+03

F15

SR SP

0.00 Inf

100.00 1.16E+05

100.00 5.85E+04

0.00 Inf

0.00 Inf

100.00 1.37E+05

100.00 1.70E+05

76.67 7.53E+04

0.00 Inf

100.00 1.02E+05

100.00 2.65E+04

F16

SR SP

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

90.00 2.25E+05

F17

SR SP

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

13.33 1.27E+06

70.00 2.65E+05

6.67 1.39E+06

100.00 1.44E+05

66.67 3.04E+05

100.00 1.05E+05

F18

SR SP

86.67 2.51E+05

0.00 Inf

0.00 Inf

0.00 Inf

0.00 Inf

6.67 3.45E+06

70.00 2.74E+05

3.33 3.16E+06

100.00 1.58E+05

90.00 2.39E+05

100.00 1.04E+05

F19

SR SP

100.00 1.20E+04

100.00 7.44E+04

100.00 4.88E+04

0.00 Inf

100.00 2.47E+03

10.00 1.33E+06

100.00 4.42E+03

100.00 2.94E+03

100.00 7.96E+03

100.00 1.19E+03

100.00 4.78E+03

F20

SR SP

0.00 Inf

100.00 1.33E+05

100.00 4.72E+04

0.00 Inf

0.00 Inf

100.00 1.15E+05

100.00 1.18E+05

73.33 7.49E+04

100.00 1.92E+05

100.00 1.60E+05

100.00 2.33E+04

F21

SR SP

0.00 Inf

83.33 2.27E+05

70.00 1.03E+05

0.00 Inf

0.00 Inf

90.00 1.75E+05

0.00 Inf

23.33 5.27E+05

0.00 Inf

0.00 Inf

100.00 3.10E+04

F22

SR SP

0.00 Inf

56.67 4.50E+05

83.33 2.04E+05

0.00 Inf

0.00 Inf

13.33 2.20E+06

46.67 4.72E+05

76.67 2.12E+05

0.00 Inf

73.33 3.23E+05

86.67 6.36E+04

1/2/22 0

8/2/15 0

8/3/14 0

0/5/20 0

1/12/12 0

7/7/11 0

10/8/7 0

5/9/11 0

15/1/9 1

14/4/7 1

18/2/5 19

#S/#PS/#NS #BSP

4.6. Comparison of the computation complexity of algorithms The computational complexity of an MS algorithm can be qualitatively and quantitatively evaluated through theoretical analysis [8] and the procedures proposed in [50], respectively. The theoretical analysis does not consider hardware factors, such as the operating systems, programming language, and compiler used to execute the tested algorithm during the complexity evaluation process. In most cases, the worst-case complexity is reported for theoretical analysis, despite the fact that average-case complexity, if identified, is more practical [8]. As compared with the basic version of PSO, ATLPSO-ELS needs to perform extra computations on the proposed ADLcurrent, ADLmemory, and ELS modules with the complexities of O (S2.D), O (S.D), and O (S.D2), respectively. Given that the value of D is usually set higher than S, we could deduce that the run-time complexity of ATLPSO-ELS in the worst-case scenario is O (S.D2) for each iteration.

68


Table 6 Results of the computational complexity for D = 50. APSO

CLPSO

FLPSO-QIW FPSO

FIPSO

OLPSO-L

MPSO-TVAC OPSO

SLPSO

TLPSO-IDL ATLPSO-ELS

T0 T1 b2 T

1.88E01 1.88E01 1.88E01 4.19E+00 4.19E+00 4.19E+00 1.37E+03 1.93E+03 2.95E+03

1.88E01 1.88E01 1.88E01 1.88E01 4.19E+00 4.19E+00 4.19E+00 4.19E+00 5.83E+02 5.86E+02 4.60E+02 5.79E+02

1.88E01 1.88E01 1.88E01 4.19E+00 4.19E+00 4.19E+00 2.75E+02 1.04E+03 4.23E+02

1.88E01 4.19E+00 5.93E+02

b 2 T 1 Þ=T 0 ðT

7.27E+03

3.08E+03

1.44E+03

3.31E+03

1.03E+04

1.56E+04

3.09E+03

2.42E+03

3.06E+03

5.51E+03

2.23E+03

Conversely, the quantitative complexity evaluation through the procedures proposed in [50] is platform dependent, and the computational times of all algorithms need to be measured under homogenous platform conditions [8]. The experimenb 2 denote the computing time required to evaluate the mathematical opertal results are presented in Table 6. T0, T1, and T ation explained in [50], the computing time required to perform 2.00E+05 evaluation of function F24 in 50-D without running the algorithm, and the mean computing time for a particular algorithm to perform 2.00E+05 evaluations of function b 2 values are measured in CPU seconds. F24 in 50-D, respectively. All T0, T1, and T Table 6 shows that OPSO has the least computational complexity at D = 50. We also observed that the complexity value of ATLPSO-ELS is slightly higher than that of TLPSO-IDL. Such observation is reasonable given that ATLPSO-ELS is improved with the ADLcurrent and ELS modules, which would inevitably consume additional computational resources. Despite having slightly higher complexity, previous experiment results reveal that our proposed ATLPSO-ELS significantly outperforms OPSO and TLPSO-IDL in terms of searching accuracy (Fmean), searching reliability (SR), and searching efficiency (SP). Another notable observation is that the complexity value of ATLPSO-ELS is much lower than that of SLPSO with relatively good searching accuracy. These observations suggest that compared with its peer algorithms, our proposed ATLPSO-ELS achieves better tradeoff between performance improvement and computational complexity. 4.7. Performance improvement analysis In this section, we first evaluate the effectiveness of each proposed strategy in ATLPSO-ELS, which includes two-layer evolution framework, ADL, and ELS modules, against the PSO-LDIW. We then investigate the performance improvement of ATLPSO-ELS over the previously proposed TLPSO-IDL in each tested problem category. 4.7.1. Effect of different strategies We study the effectiveness of each proposed strategy by comparing the searching performance of (1) PSO with two-layer evolution framework (TLPSO), (2) PSO with two-layer evolution and ELS module (TLPSO-ELS), (3) PSO with two-layer evolution framework and ADL modules (ATLPSO), and (4) the complete ATLPSO-ELS. For TLPSO and TLPSO-ELS, the Tsize,i and Pc,i values are set as 2 and 0.5, respectively, which implies that the population divisions in both of their current (ADLcurrent) and memory (ADLmemory) swarms are non-adaptive. The Fmean values produced by these ATLPSO-ELS variants are compared with those produced by the PSO-LDIW. The comparison results are expressed in terms of percentage improvement (%Improve) as follows [30]:

%Improv e ¼

F mean ðPSO LDIWÞ F mean ðnÞ 100% jF mean ðPSO LDIWÞj

ð21Þ

where n denotes TLPSO, TLPSO-ELS, ATLPSO, or ATLPSO-ELS. If n has better performance (i.e., smaller Fmean) than PSO-LDIW, %Improve will be positive; otherwise, %Improve is negative. The comparison results of all the ATLPSO-ELS variants in each problem category are presented in Table 7 and summarized as #BMF and average %Improve. Table 7 shows that the #BMF values produced by TLPSO, TLPSO-ELS, ATLPSO, and ATLPSO-ELS outperform that of PSO-LDIW, which indicates that any of these strategies helps to improve the searching accuracy of PSO. Among these ATLPSO-ELS variants, TLPSO exhibits the most inferior performance because of its least average %Improve values in almost all problem categories. ATLPSO performs particularly well in the rotated problems, given that it successfully obtains five

Table 7 Comparison of ATLPSO-ELS variants with PSO-LDIW in 50-D problem. #BMF (Average %Improve) PSO-LDIW

TLPSO

TLPSO-ELS

ATLPSO

ATLPSO-ELS

Conventional problems (F1 to F8) Rotated problems (F9 to F14) Shifted problems (F15 to F21) Complex problems (F22 to F25)

0 0 0 0

2 2 1 0

4 0 3 1

7 5 1 0

7 6 7 3

Overall results (F1 to F25)

0 (–)

(–) (–) (–) (–)

(86.058) (68.152) (86.600) (72.883)

5 (79.804)

(99.467) (65.650) (100.000) (75.648)

8 (87.689)

(97.190) (92.603) (75.798) (72.916)

13 (86.215)

(98.987) (96.305) (100.000) (75.299)

23 (94.837)

69


(out of six) #BMF values. TLPSO-ELS exhibits superior searching accuracy in solving the shifted problems by achieving more promising average %Improve values. Based on the aforementioned observations, we can deduce that the hybridization of the two-layer evolution framework and ADL strategies produces the desired rotationally invariant property to ATLPSO, and this hybridization enables ATLPSO to focus effectively on the fitness landscapes with non-separable characteristics. Conversely, the combination of both two-layer evolution framework and ELS strategies is a promising countermeasure against stagnation, given that this strategy introduces TLPSO-IDL with adequate diversity to locate the shifted optima in problems with a shifted fitness landscape. Finally, ATLPSO-ELS achieves the best searching accuracy in all types of problems compared with the other ATLPSO-ELS variants. ATLPSO-ELS successfully obtains seven (out of eight), six (out of six), seven (out of seven), and three (out of four) best #BMF values in the conventional, rotated, shifted, and complex problems, respectively. The excellent performance of ATLPSO-ELS in solving all tested problem categories implies that the three proposed strategies, namely, two-layer evolution framework, ADL, and ELS modules, are integrated effectively in the ATLPSO-ELS. None of the contributions of these proposed strategies are compromised when ATLPSO-ELS is used to solve different types of problems. 4.7.2. Performance improvement of ATLPSO-ELS against TLPSO-IDL As ATLPSO-ELS is considered the improved variant of TLPSO-IDL, we are interested in investigating the performance improvement of the former algorithm against the latter one by using Eq. (22).

%Improv e ¼

F mean ðTLPSO IDLÞ F mean ðATLSO ELSÞ 100% jF mean ðTLPSO IDLÞj

ð22Þ

The comparison results between ATLPSO-ELS and TLPSO-IDL in each problem category are presented in Table 8 and summarized as #BMF and average %Improve. Table 8 shows that the proposed ATLPSO-ELS achieves the overall performance gain of 39.802% against TLPSO-IDL. In addition, the searching accuracies of ATLPSO-ELS in solving the shifted and complex problems are enhanced substantially according to its impressive average %Improve values against TLPSO-IDL, which are 85.636% and 50.440%, respectively. By comparing the experimental results in Tables 6 and 8, we can conclude that the searching accuracy of ATLPSO-ELS, especially in solving the problems with more complicated fitness landscapes, is significantly improved from TLPSO-IDL by incurring a reasonable increment of computation cost. 4.8. Comparison with other state-of-the-art MS algorithms We compared our ATLPSO-ELS with five OED-based MS algorithms, namely, OLPSO [62], differential evolution with orthogonal crossover operator (OXDE) [56], biogeography-based optimization with orthogonal crossover operator (OXBBO) [12], orthogonal learning-based artificial bee colony (OCABC) [14], and orthogonal teaching learning-based optimization (OTLBO) [47]. The orthogonal crossover operators used by the OXDE and OXBBO perform the search space quantization, and their Q values are set larger than two. OCABC randomly select one bee to perform orthogonal learning in every iteration. Conversely, OTLBO randomly selects m learners to perform orthogonal learning through a multi-parent crossover operator during the teacher and learner phases. We perform a simulation of various 30-D conventional problems. The Fmean and SD values of all algorithms are recorded in Table 9. The results of the compared MS variants are extracted from their respective literature [12,14,47,56]. We assign the Fmean and SD values of the MS peer as ‘‘NA’’ if the result in a particular benchmark is unavailable. Table 9 shows that ATLPSOELS works best in almost all cases as it can solve eight out of ten problems, i.e., two times better than the third-ranked OCABC. In addition, ATLPSO-ELS is the only algorithm that successfully locates the global optima of Schwefel 2.22, Schwefel 2.21, and Ackley functions. 4.9. Discussion From previous subsections, the proposed ATLPSO-ELS has been proven to have more superior searching accuracy, searching reliability, and convergence speed compared with the previously proposed TLPSO-IDL and state-of-art PSO variants and MS algorithms. The competitive performance of ATLPSO-ELS against its contenders, especially TLPSO-IDL, is attributed to the three proposed improvements, namely, (1) the adaptive task allocation in current swarm, (2) the selective execution of current and memory swarm evolutions for each iteration of ATLPSO-ELS, and (3) the ELS module. Table 8 Comparison of ATLPSO-ELS with TLPSO-IDL in 50-D problem.

#BMF (Average %Improve) Conventional problems (F1 to F8) Rotated problems (F9 to F14) Shifted problems (F15 to F21) Complex problems (F22 to F25) Overall results (F1 to F25)

TLPSO-IDL

ATLPSO-ELS

7 (–) 4 (–) 1 (–) 1 (–) 13 (–)

7 (6.893) 5 (23.117) 7 (85.636) 3 (50.440) 22 (39.802)

70


Table 9 Comparisons between ATLPSO-ELS and other MS variants on optimizing 30-D problems. Function

OLPSO

OXDE

OXBBO

OCABC

OTLBO

ATLPSO-ELS

Sphere

Fmean SD

1.55E41 (3.32E41)

1.58E16 (1.41E16)

8.84E64 (8.92E64)

4.32E43 (8.16E43)

0.00E+00 (0.00E+00

0.00E+00 (0.00E+00)

Schwefel 2.22

Fmean SD

5.76E31 (5.46E31)

4.38E12 (1.93E12)

5.25E46 (3.50E46)

1.17E22 (7.13E23)

2.11EE221 (0.00E+00)

0.00E+00 (0.00E+00)

Schwefel 1.2

Fmean SD

2.17E04 (2.72E04)

6.41E07 (4.98E07)

1.81E11 (1.77E11)

NA

0.00E+00 (0.00E+00

0.00E+00 (0.00E+00)

Schwefel 2.21

Fmean SD

2.81E+00 (1.47E+00)

1.49E+00 (9.62E01)

2.39E01 (2.58E01)

5.67E01 (2.73E01)

4.01E215 (0.00E+00)

0.00E+00 (0.00E+00)

Rosenbrock

Fmean SD

4.78E01 (1.32E+00)

1.59E01 (7.97E01)

2.61E+01 (9.85E01)

7.89E01 (6.27E01)

NA

4.67E+01 (1.98E+00)

Step

Fmean SD

0.00E+00 (0.00E+00)

0.00E+00 (0.00E+00)

0.00E+00 (0.00E+00)

0.00E+00 (0.00E+00)

0.00E+00 (0.00E+00

0.00E+00 (0.00E+00)

Quartic

Fmean SD

1.35E03 (1.40E03)

2.95E03 (1.32E03)

9.41E04 (3.76E04)

4.39E03 (2.03E03)

1.69E05 (1.22E05)

8.47E+00 (4.13E01)

Rastrigin

Fmean SD

1.99E01 (4.06E01)

4.06E+00 (1.95E+00)

1.23E01 (4.05E01)

0.00E+00 (0.00E+00)

0.00E+00 (0.00E+00

0.00E+00 (0.00E+00)

Ackley

Fmean SD

3.52E15 (1.55E15)

2.99E09 (1.54E09)

2.66E15 (0.00E+00)

5.32E15 (1.82E15)

1.11E15 (1.00E15)

0.00E+00 (0.00E+00)

Grienwank

Fmean SD

0.00E+00 (0.00E+00)

1.48E03 (3.02E03)

0.00E+00 (0.00E+00)

0.00E+00 (0.00E+00)

0.00E+00 (0.00E+00)

0.00E+00 (0.00E+00)

6/2/2 2

7/1/2 1

6/2/2 2

4/3/2 4

3/5/1 6

8

w/t/l #BMF

One of the main deficiencies of TLPSO-IDL is its incompetence to solve problems with more complicated fitness landscapes. Despite achieving the best ranks in conventional and rotated problems, the searching accuracy of TLPSO-IDL in solving shifted and complex problems is less promising, as shown in Table 3. The inferior performances of TLPSO-IDL in these two problem categories imply that the use of adaptive task allocation on memory swarm alone is not sufficient for TLPSO-IDL to handle shifted and complex problems, which generally have a more challenging fitness landscape. Introducing a new adaptive task allocation mechanism into the current swarm of ATLPSO-ELS provides more diversity to the current swarm members. This additional diversity offers more potential exploration moves that allow the current swarm members to have a better chance of locating the shifted global optima of problems with more complicated fitness landscapes. As reported in Tables 3 and 8, the searching accuracies of ATLPSO-ELS in solving shifted and complex problems are significantly enhanced compared with TLPSO-IDL. The excellent searching accuracies demonstrated by ATLPSO-ELS in solving these two problem categories shows that using the adaptive task allocation mechanisms on both current and memory swarm evolutions is viable to improve the searching accuracy of TLPSO-IDL. Meanwhile, the selective execution of the current and memory swarm evolutions as implemented in the ATLPSO-ELS allows the algorithm to focus on one type of swarm evolution for each iteration. Unlike TLPSO-IDL, which sequentially performs both types of swarm evolutions, ATLPSO-ELS switches one swarm evolution to another only if the previous swarm evolution is no longer able to improve the fitness of Pg particle. Such implementation enables the ATLPSO-ELS to locate the global optima of the tested benchmarks without consuming unnecessary FEs. As shown in Table 5 and Fig. 12, the SP values achieved by the ATLPSO-ELS are significantly lower than those of TLPSO-IDL in most of the employed benchmarks. This observation validates the excellent searching efficiency of ATLPSO-ELS against the TLPSO-IDL, given that the former requires fewer FEs than the latter to solve the tested functions within the predefined e. Finally, the ELS module, which consists of OEDLS and SPLS, is specifically designed to evolve the Pg particle. OEDLS provides an efficient learning mechanism to the Pg particle. Specifically, it extracts and fully exploits any useful information that is available in the newly updated Pi particle to further improve the fitness of the Pg particles. The excellent prediction capability of OEDLS ensures that the Pg particle has a more promising search direction toward the global optimum, thereby enhancing the algorithm’s searching accuracy (see Tables 3 and 7) and convergence speed (see Table 5 and Fig. 12). Meanwhile, SPLS has effectively addressed the premature convergence issue of the basic PSO by providing the Pg particle some extra momentum to escape from the local optima. Despite having impressive searching performance, many metrics (i.e., PSD, PFSD, Pc, and Tsize) and parameters (i.e., K1, K2, z, and m) have been used by ATLPSO-ELS to perform adaptive task allocations on both current and memory swarms. The presence of some metrics such as PSD and PFSD might be redundant for the swarm evolutions and could be reduced into one universal metric. Moreover, the parameter tuning process used to achieve the reasonably good performance of ATLPSOELS is time consuming if too many parameters are involved. For future work, we aim to simplify the algorithmic framework of ATLPSO-ELS by reducing the metrics and parameters of the algorithm.


71

Fig. A1. Algorithm for constructing a two-level OA for N factor.

5. Conclusions This paper aims to address the main deficiencies of the previously proposed TLPSO-IDL. Therefore, an ATLPSO-ELS algorithm is developed to perform adaptive task allocations on the current and the memory swarms during the optimization process. Unlike TLPSO-IDL, the current and the memory swarms’ evolutions of ATLPSO-ELS are performed alternately. The previous swarm evolution should only be replaced when it is no longer able to improve the global best particle. Conversely, the ELS module is specifically designed to evolve the global best particle when predefined conditions are met. The module consists of OEDLS and SPLS, which are used to establish a two-way information exchange mechanism and to alleviate swarm stagnation, respectively. Extensive experimental studies indicate that the ATLPSO-ELS has superior searching performance compared with the previously proposed TLPSO-IDL and other state-of-the-art PSO and MS algorithms. Acknowledgments The authors express their sincere thanks to the associate editor and the reviewers for their significant contributions to the improvement of the final paper. The authors also thank Abdul Latiff Abdul Tawab, Ahmad Ahzam Latib, Nor Azhar Zabidin, and Amir Hamid for their technical support. This research was supported by the Universiti Sains Malaysia (USM) Postgraduate Fellowship Scheme and the Postgraduate Research Grant Scheme (PRGS) entitled ‘‘Development of PSO Algorithm with Multi-Learning Frameworks for Application in Image Segmentation.’’ Appendix A. Construction of a two-level orthogonal array (OA) for N factors See Fig. A1. References [1] A. Banks, J. Vincent, C. Anyakoha, A review of particle swarm optimization. Part I: background and development, Nat. Comput. 6 (2007) 467–484. [2] A. Banks, J. Vincent, C. Anyakoha, A review of particle swarm optimization. Part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative applications, Nat. Comput. 7 (2008) 109–124. [3] C.J.A. Bastos-Filho, D.F. Carvalho, E.M.N. Figueiredo, P.B.C. de Miranda, Dynamic clan particle swarm optimization, in: Ninth International Conference on Intelligent Systems Design and Applications (ISDA ’09) Pisa, 2009, pp. 249–254. [4] Z. Beheshti, S.M.H. Shamsuddin, S. Hasan, MPSO: median-oriented particle swarm optimization, Appl. Math. Comput. 219 (2013) 5817–5836. [5] D.F. Carvalho, C.J.A. Bastos-Filho, Clan particle swarm optimization, in: IEEE Congress on Evolutionary Computation (CEC 2008), 2008, pp. 3044–3051. [6] M. Clerc, Particle Swarm Optimization, ISTE Publishing Company, 2006. [7] M. Clerc, J. Kennedy, The particle swarm – explosion, stability, and convergence in a multidimensional complex space, IEEE Trans. Evol. Comput. 6 (2002) 58–73. [8] S. Das, S. Biswas, S. Kundu, Synergizing fitness learning with proximity-based food source selection in artificial bee colony algorithm for numerical optimization, Appl. Soft Comput. 13 (2013) 4676–4694. [9] Y. del Valle, G.K. Venayagamoorthy, S. Mohagheghi, J.C. Hernandez, R.G. Harley, Particle swarm optimization: basic concepts, variants and applications in power systems, IEEE Trans. Evol. Comput. 12 (2008) 171–195. [10] R.C. Eberhart, Y. Shi, Particle swarm optimization: developments, applications and resources, in: Proceedings of the 2001 Congress on Evolutionary Computation, vol. 81, 2001, pp. 81–86. [11] M.G. Epitropakis, V.P. Plagianakos, M.N. Vrahatis, Evolving cognitive and social experience in particle swarm optimization through differential evolution: a hybrid approach, Inf. Sci. 216 (2012) 50–92. [12] Q. Feng, S. Liu, G. Tang, L. Yong, J. Zhang, Biogeography-based optimization with orthogonal crossover, Math. Problems Eng. 2013 (2013) 20.

72


[13] Q. Feng, S. Liu, J. Zhang, G. Yang, Extrapolated particle swarm optimization based on orthogonal design, J. Convergence Inf. Technol. (JCIT) 7 (2012) 141–152. [14] W.-F. Gao, S.-Y. Liu, L.-L. Huang, A novel artificial bee colony algorithm based on modified search equation and orthogonal learning, IEEE Trans. Cybern. 43 (2013) 1011–1024. [15] S. García, D. Molina, M. Lozano, F. Herrera, A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization, J. Heuristics 15 (2009) 617–644. [16] S. Gholizadeh, Layout optimization of truss structures by hybridizing cellular automata and particle swarm optimization, Comput. Struct. 125 (2013) 86–99. [17] A.S. Hedayat, Orthogonal Arrays: Theory and Applications, Springer-Verlag, New York, 1999. [18] S.-Y. Ho, H.-S. Lin, W.-H. Liauh, S.-J. Ho, OPSO: orthogonal particle swarm optimization and its application to task assignment problems, IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 38 (2008) 288–298. [19] B. Hopkins, J.G. Skellam, A new method for determining the type of distribution of plant individuals, Ann. Bot. 18 (1954) 213–227. [20] S.-T. Hsieh, T.-Y. Sun, C.-C. Liu, S.-J. Tsai, Efficient population utilization strategy for particle swarm optimizer, IEEE Trans. Syst. Man Cybern. B Cybern. 39 (2009) 444–456. [21] M. Hu, T. Wu, J.D. Weir, An intelligent augmentation of particle swarm optimization with multiple adaptive methods, Inf. Sci. 213 (2012) 68–83. [22] M. Hu, T. Wu, J.D. Weir, An adaptive particle swarm optimization with multiple adaptive methods, IEEE Trans. Evol. Comput. 17 (2013) 705–720. [23] H. Huang, H. Qin, Z. Hao, A. Lim, Example-based learning particle swarm optimization for continuous optimization, Inf. Sci. 182 (2012) 125–138. [24] J. Kennedy, Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance, in: Proceedings of IEEE Congress on Evolutionary Computation, vol. 1933, 1999, pp. 1938. [25] J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings IEEE International Conference on Neural Networks, vol. 1944, 1995, pp. 1942– 1948. [26] J. Kennedy, R.C. Eberhart, Y. Shi, Swarm Intelligence, Morgan Kaufmann Publishers, 2001. [27] J. Kennedy, R. Mendes, Population structure and particle swarm performance, in: Proceedings of IEEE Congress on Evolutionary Computation (CEC ’02), 2002, pp. 1671–1676. [28] S. Kiranyaz, T. Ince, A. Yildirim, M. Gabbouj, Fractional particle swarm optimization in multidimensional search space, IEEE Trans. Syst. Man Cybern. B Cybern. 40 (2010) 298–319. [29] C.-N. Ko, Y.-P. Chang, C.-J. Wu, An orthogonal-array-based particle swarm optimizer with nonlinear time-varying evolution, Appl. Math. Comput. 191 (2007) 272–279. [30] A.Y.S. Lam, V.O.K. Li, J.J.Q. Yu, Real-coded chemical reaction optimization, IEEE Trans. Evol. Comput. 16 (2012) 339–353. [31] M.-S. Leu, M.-F. Yeh, Grey particle swarm optimization, Appl. Soft Comput. 12 (2012) 2985–2996. [32] C. Li, S. Yang, T.T. Nguyen, A self-learning particle swarm optimizer for global optimization problems, IEEE Trans. Syst. Man Cybern. B Cybern. 42 (2012) 627–646. [33] J.J. Liang, A.K. Qin, P.N. Suganthan, S. Baskar, Comprehensive learning particle swarm optimizer for global optimization of multimodal functions, IEEE Trans. Evol. Comput. 10 (2006) 281–295. [34] J.J. Liang, P.N. Suganthan, Dynamic multi-swarm particle swarm optimizer, in: IEEE Proceedings of Swarm Intelligence Symposium (SIS 2005), 2005, pp. 124–129. [35] W.H. Lim, N.A. Mat Isa, Two-layer particle swarm optimization with intelligent division of labor, Eng. Appl. Artif. Intell. 26 (2013) 2327–2348. [36] Y. Marinakis, M. Marinaki, A hybridized particle swarm optimization with expanding neighborhood topology for the feature selection problem, in: M. Blesa, C. Blum, P. Festa, A. Roli, M. Sampels (Eds.), Hybrid Metaheuristics, Springer, Berlin Heidelberg, 2013, pp. 37–51. [37] R. Mendes, J. Kennedy, J. Neves, The fully informed particle swarm: simpler, maybe better, IEEE Trans. Evol. Comput. 8 (2004) 204–210. [38] A. Mohais, R. Mendes, C. Ward, C. Posthoff, Neighborhood re-structuring in particle swarm optimization, AI 2005, Adv. Artif. Intell. (2005) 776–785. [39] M.A. Montes de Oca, T. Stutzle, M. Birattari, M. Dorigo, Frankenstein’s PSO: a composite particle swarm optimization algorithm, IEEE Trans. Evol. Comput. 13 (2009) 1120–1132. [40] D.C. Montgomery, Design and Analysis of Experiments, third ed., Wiley, New York, 1991. [41] M. Nasir, S. Das, D. Maity, S. Sengupta, U. Halder, P.N. Suganthan, A dynamic neighborhood learning based particle swarm optimizer for global numerical optimization, Inf. Sci. 209 (2012) 16–36. [42] F. Neri, C. Cotta, A primer on memetic algorithms, in: F. Neri, C. Cotta, P. Moscato (Eds.), Handbook of Memetic Algorithms, Springer, Berlin Heidelberg, 2012, pp. 43–52. [43] K.E. Parsopoulos, M.N. Vrahatis, Recent approaches to global optimization problems through Particle Swarm Optimization, Nat. Comput. 1 (2002) 235– 306. [44] M.R. Pontes, F.B.L. Neto, C.J.A. Bastos-Filho, Adaptive clan particle swarm optimization, in: IEEE Symposium on Swarm Intelligence (SIS), 2011, pp. 1–6. [45] B.Y. Qu, P.N. Suganthan, S. Das, A distance-based locally informed particle swarm model for multimodal optimization, IEEE Trans. Evol. Comput. 17 (2013) 387–402. [46] A. Ratnaweera, S.K. Halgamuge, H.C. Watson, Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients, IEEE Trans. Evol. Comput. 8 (2004) 240–255. [47] S. Satapathy, A. Naik, K. Parvathi, A teaching learning based optimization based on orthogonal design for solving global optimization problems, SpringerPlus 2 (2013) 130. [48] Y. Shi, R. Eberhart, A modified particle swarm optimizer, in: Proceedings IEEE World Congress on Computational Intelligence, 1998, pp. 69–73. [49] J.E. Smith, T.C. Fogarty, Operator and parameter adaptation in genetic algorithms, Soft. Comput. 1 (1997) 81–87. [50] P.N. Suganthan, N. Hansen, J.J. Liang, K. Deb, Y.P. Chen, A. Auger, S. Tiwari, Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real Parameter, Optimization, in, 2005. [51] Y. Tang, Z. Wang, J.-A. Fang, Feedback learning particle swarm optimization, Appl. Soft Comput. 11 (2011) 4713–4725. [52] F. van den Bergh, A.P. Engelbrecht, A cooperative approach to particle swarm optimization, IEEE Trans. Evol. Comput. 8 (2004) 225–239. [53] H. Wang, H. Sun, C. Li, S. Rahnamayan, J.-S. Pan, Diversity enhanced particle swarm optimization with neighborhood search, Inf. Sci. 223 (2013) 119– 135. [54] S. Wang, L. Chen, A PSO algorithm based on orthogonal test design, in: Fifth International Conference on Natural Computation (ICNC’09), Tianjin, 2009, pp. 190–194. [55] W. Wang, H. Wang, S. Rahnamayan, Improving comprehensive learning particle swarm optimiser using generalised opposition-based learning, Int. J. Model. Ident. Control 14 (2011) 310–316. [56] Y. Wang, Z. Cai, Q. Zhang, Enhancing the search ability of differential evolution through orthogonal crossover, Inf. Sci. 185 (2012) 153–177. [57] Y. Wang, B. Li, T. Weise, J. Wang, B. Yuan, Q. Tian, Self-adaptive learning based particle swarm optimization, Inf. Sci. 181 (2011) 4515–4538. [58] D.H. Wolpert, W.G. Macready, No free lunch theorems for optimization, IEEE Trans. Evol. Comput. 1 (1997) 67–82. [59] J. Yang, A. Bouzerdoum, S.L. Phung, A particle swarm optimization algorithm based on orthogonal design, in: IEEE Congress on Evolutionary Computation (CEC2010), Barcelona, 2010, pp. 1–7. [60] X. Yao, Y. Liu, G. Lin, Evolutionary programming made faster, IEEE Trans. Evol. Comput. 3 (1999) 82–102. [61] Z.-H. Zhan, J. Zhang, Y. Li, H.S.H. Chung, Adaptive particle swarm optimization, IEEE Trans. Syst. Man Cybern. B Cybern. 39 (2009) 1362–1381. [62] Z.-H. Zhan, J. Zhang, Y. Li, Y.-H. Shi, Orthogonal learning particle swarm optimization, IEEE Trans. Evol. Comput. 15 (2011) 832–847. [63] B. Zhao, C.X. Guo, B.R. Bai, Y.J. Cao, An improved particle swarm optimization algorithm for unit commitment, Int. J. Electr. Power Energy Syst. 28 (2006) 482–490.

An adaptive two-layer particle swarm optimization ... - Semantic Scholar

An adaptive two-layer particle swarm optimization ... - Semantic Scholar

Suggest Documents

Adaptive range particle swarm optimization - Semantic Scholar

Adaptive range particle swarm optimization - Semantic Scholar

An Adaptive Particle Swarm Optimization Applied ... - Semantic Scholar

An improved Particle Swarm Optimization ... - Semantic Scholar

An Improved Particle Swarm Optimization ... - Semantic Scholar

Particle Swarm Optimization - Semantic Scholar

Self Adaptive Particle Swarm Optimization for ... - Semantic Scholar

Chaos Adaptive Improved Particle Swarm ... - Semantic Scholar

Particle Swarm Optimization versus Genetic ... - Semantic Scholar

Particle Swarm Optimization For Quadratic ... - Semantic Scholar

Parallel Particle Swarm Optimization Clustering ... - Semantic Scholar

Selectively-informed particle swarm optimization - Semantic Scholar

rithm and Particle Swarm Optimization - Semantic Scholar

Particle Swarm Optimization Based Feature ... - Semantic Scholar

Particle-swarm optimization of broadband ... - Semantic Scholar

Particle Swarm Optimization with Cognitive ... - Semantic Scholar

Turbulent Particle Swarm Optimization Using ... - Semantic Scholar

Particle Swarm Optimization Algorithm for ... - Semantic Scholar

A FAST PARTICLE SWARM OPTIMIZATION ... - Semantic Scholar

Globally Convergent Particle Swarm Optimization ... - Semantic Scholar

Hybrid particle swarm optimization: Evolutionary ... - Semantic Scholar

Distributed Particle Swarm Optimization for ... - Semantic Scholar

Particle swarm optimization performance on ... - Semantic Scholar

Biases in Particle Swarm Optimization - Semantic Scholar