Artificial bee and differential evolution improved by clustering search ...

Soft Comput DOI 10.1007/s00500-014-1500-9

FOCUS

Artificial bee and differential evolution improved by clustering search on continuous domain optimization Tarcísio Souza Costa · Alexandre César Muniz de Oliveira

© Springer-Verlag Berlin Heidelberg 2014

Abstract Clustering Search (*CS) has been proposed as a generic way of combining search metaheuristics with clustering to detect promising search areas before applying local search procedures. The clustering process may keep representative solutions associated with different search subspaces. In this paper, new approaches are proposed, based on *CS, as an Artificial Bee Colony-based one, which detects promising food sources alike *CS approaches. The other new *CS approach is based on Differential Evolution (DE) algorithm. The DE is just a CS component (the evolutionary algorithm), different from ABC-based approach, called Artificial Bee Clustering Search (ABCS). ABCS tries to find promising solutions using some concepts from CS. The proposed hybrid algorithms, performing a Hooke and Jeeves-based local, are compared to another hybrid approaches, exploring an elitist criteria to apply local search. The experiments show that the proposed ABCS and Differential Evolutionary Clustering Search (DECS) are competitive for the majority continuous optimization functions benckmarks selected in this paper. Keywords Clustering Search · Artificial Bee Colony · Promising areas · Hybrid approaches

Communicated by I. R. Ruiz. T. S. Costa (B) Federal Institute of Maranhão, Pacas Street 5, Pinheiro Campus, Pinheiro, Brazil e-mail: [email protected] A. C. M. de Oliveira Federal University of Maranhão, Portugueses Avenue, No Number, Bacanga Campus, São Luís, Brazil e-mail: [email protected]

1 Introduction The modern metaheuristics attempt to balance exploration and exploitation movements explicitly by employing local and global search strategies. Exploration movements play the role of a generator of reference points to diversified search areas, which are more intensively inspected by exploitation components further. The early hybrid metaheuristics have demonstrated concern about rationally employing local search. Instead of applying always local search procedures on an entire population, the global search procedure runs normally until a promising area can be detected, according to specific criteria, for example, when a better new individual is found (elitist criterium). In a general way, the following action must be better to exploit the expected promising area, in a reduced search domain, by a local search procedure. Clustering Search (*CS) has been proposed as a generic way of combining search metaheuristics with clustering, aiming to detect promising search areas before applying local search procedures (Oliveira and Lorena 2004). Its generalized nature is achieved both by the possibility of employing any metaheuristic and also by applying to combinatorial and continuous optimization problems. Clusters of mutually close solutions provide reference points to relevant areas of attraction in the most of search metaheuristics. Furthermore, the cluster center itself is always updated by a procedure applied over inner solutions, called assimilation (Oliveira and Lorena 2004). Artificial Bee Colony (ABC) (Karaboga and Akay 2009) and Differential Evolution (DE) (Price and Storn 1995) are optimization algorithms inspired by biology and by the evolution of species. Both are powerful population-based stochastic search techniques, being efficient and effective global optimizers in the continuous search domain. ABC is based on the intelligent behavior of honey bees, providing a

123

T. S. Costa, A. C. M. de Oliveira

population-based search procedure in which candidate solutions, called foods positions, are modified by the artificial bees, aiming to discover new promising food sources and, finally, the one with the optimal amount of nectar (Karaboga and Akay 2009). DE works with a population of candidate solutions and some operations, similar to genetic algorithm mutation and crossover, are made at each generation trying to find new promising solutions. This paper intents to investigate two new *CS approaches, based on ABC and DE, applied to the unconstrained continuous optimization. The proposed approaches are called, respectively, ABCS and DECS, with highlights to the former in which promising search areas are associated with promising food sources and the bee movement is performed by the intrinsic *CS operation known as assimilation (Oliveira and Lorena 2004). ABCS and DECS are hybrid optimization approaches, when they combine the global search strategy with local search, both guided by the cluster process. The local search strategy is always implemented by Hooke and Jeeves (1961) local search procedure, a well-succeeded direct search ranking evaluation method of objective function. Computational experiments are presented, involving several optimization algorithms, including other *CS approach, ECS (Oliveira and Lorena 2004), and two plain hybrid versions ABC and DE that do not consider the promising clusters to employ local search over them. The remainder of this paper is organized as follows. The Sect. 3 describes the concepts behind *CS, ABC and DE. In Sect. 4, new proposed *CS approaches based on ABC and DE are described. Experimental results are presented in Sect. 5. The findings and conclusions are summarized in Sect. 6.

running time. The CS found the best-known solution in the literature for 84 % of the instances tested. The CS algorithm was also very robust, producing low deviations. The same approches, SA, GA and ILS, without the clustering process gave solutions with a poorer quality than CS. In Oliveira et al. (2012) CS with Simulated Annealing for Berth Allocation Problem (BAP) is considered, dealing with the assignment of ships to mooring positions, using the maximum space of the quay and minimizing service time. Computational results are compared against recent methods found in the literature. Riberio et al. (2011) present a CS to the workover rig routing problem that is a variant of the vehicle routing problem with time windows. The objective is to minimize the total lost production in the operations of onshore oil fields. The computational results show that CS is a good heuristic for large instances. The authors illustrate a typical clustering process provided by CS, show active clusters, poor clusters, and promising clusters. More about CS and its applications can be reviewed in Oliveira et al. (2013).

3 Algorithms foundations In this section, the foundations of Clustering Search , Artificial Bee Colony and Differential Evolution are briefly described, introducing concepts needed to explain further how to combine them to allow detecting promising search areas before applying the Hooke and Jeeves (1961) local search procedure. 3.1 Clustering search

2 Related works The original ideas behind CS was validated by the Evolutionary Clustering Search (ECS), applied to unconstrained numerical optimization Oliveira and Lorena (2004). The ECS was implemented by a steady-state genetic algorithm hybridized by local search mechanism based on Hooke and Jeeves (1961) direct search. In the computational experiments, the method was compared against other approaches, including the well-known Genocop III and the OptQuest Callable Library. The first combinatorial version of the CS was later applied to instances of sequencing problems that arise in scenarios involving arrangement of a set of client orders, gates in VLSI circuits and cutting patterns Oliveira and Lorena (2006). In Oliveira et al. (2013), two CS approaches, based on Simulated Annealing and Genetic, are presented to the Capacitated Centered Clustering Problem. The results show the efficacy of the CS, observing that all CS approaches had similar behavior, both in terms of solution quality and in

123

The *CS attempts to locate promising search areas by framing them by clusters. A cluster can be defined as a tuple G = {c, r, s}, where c and r are the center and the radius of the area, respectively. The radius of a search area is the distance from its center to the edge. There could exist different search strategies s associated with the clusters. Initially, the center c is obtained randomly and progressively it tends to slip along really promising sampled points in the close subspace. The total cluster volume is defined by the radius r and can be calculated, considering the problem nature. It is important that r must define a search subspace suitable to be exploited by the search strategies s associated with the cluster (Oliveira and Lorena 2007). *CS can be splitted off in four conceptually independent parts: a search metaheuristic (SM); an iterative clustering (IC) component; an analyzer module (AM); and a local searcher (LS). SM component works as a full-time solution generator, according to its specific search strategy, a priori, performing independently of the remaining parts, and manip-

Artificial bee and differential evolution

Fig. 1 CS conceptual design

ulating a set of |P| solutions. Figure 1 brings the *CS conceptual design, in this case the component SM is an evolutionary algorithm (EA), that is an ECS (Oliveira and Lorena 2004). The IC component aims to gather similar solutions into groups, maintaining a representative cluster center for them. To avoid extra computational effort, IC is fed, progressively, by solutions generated in each regular iteration of SM. A maximum number of clusters N C are an upper bound value that prevents an unlimited cluster creation, initially. A distance metric, ℘, must be defined, allowing a similarity measure for the clustering process. Solutions sk generated by SM are passed to IC that attempts to group as known information, according to ℘. If the information is considered sufficiently new, it is kept as a center in a new cluster, cnew . Otherwise, redundant information activates the closest center c j (cluster center that minimizes ℘ (sk , c j=1,2,..., )), causing an assimilation of the information represented by sk . Considering G j ( j=1,2,...) as all current detected clusters: cnew = sk cj

if ℘ (sk , c j ) > r j , ∀G j , or

= c j ⊕ β(sk  c j ), otherwise.

(1) (2)

where ⊕ and  are abstract operations over cj and sk meaning, respectively, addition and subtraction of solutions. The operation (sk  cj ) means the difference between the solutions sk and cj , considering the distance metric. A certain percentage, β, of this difference is used to update c j , giving cj . According to β, the assimilation can assume different forms: simple, crossover or path assimilations (Oliveira and Lorena 2007). To cover all the search space, a common radius rt must be defined, working as a threshold of similarity. For the Euclidean space, for instance, rt has been defined as (Oliveira and Lorena 2004): rt =

xsup − xinf √ 2 · n |Ct |

(3)

where |Ct | is the current number of clusters (initially, |Ct | = N C, in general, about 20–30 clusters), xsup and xinf are, respectively, the known upper and lower bounds of the

domain of variable x, considering that all variables xi have the same domain. The assimilation process is applied over the closest center c j , considering the new generated solution sk . According to β in Eq. 2, the assimilation can assume different forms. The three types of assimilation, proposed in Oliveira and Lorena (2004), are: simple, crossover and path relinking. In simple assimilation, β ∈ [0, 1] is a constant parameter, meaning a deterministic move of c j in the direction of sk . Only one internal point is generated more or less closer to c j , depending on β, to be evaluated afterwards. The greater β, the less conservative the movement is. This type of assimilation can be employed only with real-coded variables, where percentage of intervals can be applied to. The crossover assimilation means any random operation between two candidate solutions (parents), giving other ones (offsprings), similarly as a crossover operation in EAs. In this assimilation, β is an n-dimensional random vector and cj can assume a random point inside the hyper plane containing sk and c j . Since the whole operation is a crossover or other binary operator between sk and c j , it can be applied to any type of coding or even problem (combinatorial or continuous − → one). The β parameter is resulting from the type of crossover employed, not the crossover parameter itself. The path assimilation can generate several internal points or even external ones, holding the best evaluated one to be the new center. These exploratory movements are commonly referred in path relinking theory. In this assimilation, β is a η-dimensional vector of constant and evenly spaced parameters, used to generate η samples taken in the path connecting c j and sk . Since each sample is evaluated by the objective function, the path assimilation itself is an intensification mechanism inside the clusters, allowing yet sampling of wellsucceeded external points, extrapolating the sk –c j interval. AM component examines each cluster, indicating a probable promising cluster or a cluster to be eliminated. A cluster density, δi , is a measure that indicates the activity level inside the cluster i. For simplicity, δi counts the number of solutions generated by SM (selected or updated solutions in the EA case) (Oliveira and Lorena 2004; Ribeiro et al. 2007). Whenever δi reaches a certain threshold, meaning that some information template becomes predominantly generated by SM, such information cluster must be better investigated to accelerate the convergence process on it. Considering that each cluster should receive the same number of individuals, a cluster i is promising whether: δi >

NC |Pnew |

(4)

On the other hand, clusters with lower δi are eliminated, as part of a mechanism that allows creating other centers of information, keeping framed the most active of them. At the

123

T. S. Costa, A. C. M. de Oliveira

end of each generation, the AM performs the cooling of all clusters, resetting the accounting of δi . A cluster is considered inactive when no activity has occurred in the last generation. At last, the LS component is an internal searcher module that provides the exploitation of a supposed promising search area, framed by a cluster. This process can happen after AM having discovered a target cluster. LS can be considered as the particular search strategy s associated with the cluster, i.e., a problem-specific local search to be employed into the cluster. The component LS has been activated, at once, if δi ≥ PD ·

NS |Ct |

(5)

where the pressure of density, PD, allows controlling the sensibility of the component AM. The component LS has been implemented by a Hooke–Jeeves direct search in continuous optimization (Oliveira and Lorena 2004). 3.2 Artificial bee colony The Artificial Bee Colony (ABC) algorithm is a relatively new optimizer based on swarm intelligence. It is inspired on the cooperative foraging behavior of a swarm of honey bees (Karaboga and Akay 2009). The colony of artificial bees contains three groups of bees: employed bees, onlookers and scouts. The bees waiting on the dance area for making decision to choose a food source are called onlooker bees. The bees going to the food source, previously found, are named employed bees. The bees carrying out random search are called scout bees. A first half of the colony consists of employed bees and the second half constitutes the onlookers. For each food source, there is only one employed bee. In other words, the number of employed bees is equal to the number of food sources around the hive. The employed bees whose food source are exhausted become scout ones. At each cycle, ABC performs three basic operations: place employed bees on their food sources and evaluate their nectar amounts; after sharing the nectar information of food sources, onlookers select food source proportionally to their nectar amounts; and scout bees are sent to explore new sources randomly on the search space. These main steps are repeated through a predetermined Maximum Cycle Number (MCN ) or until a termination criterion is satisfied. Employed bees share the nectar amount of food sources on the dancing area in the colony. Onlooker bees go forth from dancing area, exploring the more promising source foods. The decision to be done by onlooker bees is governed by Eq. 6: fiti pi = SN

n=1 fit n

123

(6)

where pi is the probability associated with food source i, fiti is the fitness value of solution i and SN is the total number of food sources which is equal to the number of employed bees or onlooker bees. Both employed and onlooker bees modify their food sources, if the modification increase nectar amounts of those sources, onlooker and employed bees memorize the best source and forget previous one. Thus, to produce a candidate food position from the old one in memory is done by following equation: vi j = xi j + φi j (xi j − xk j )

(7)

where k ∈ {1, 2 . . . SN } and j ∈ {1, 2 . . . D} are randomly chosen indexes, and D is the dimension of variables. A food source is assumed to be abandoned if its fitness cannot be improved through certain number of cycles. The value of predetermined number of cycles is called limit for abandonment (Karaboga and Akay 2009). An abandoned food source should be replaced by another. In this case, the scout bees are responsible for substituting old food souces by new ones, that are new positions found randomly in the search space. This substitution is done through simple modification of current abandoned food source’s position, taking into account minimum (xmin ) and maximum (xmax ) of search space limited by variables, as can be seen in Eq. 8: j

j

j

j xi = xmin + rand[0, 1](xmax − xmin )

(8)

j ∈ {1, 2 . . . D} The balance between exploration and exploitation of search space, that is a desired characteristic of metaheuristics, can be seen in ABC, considering that employed and onlooker bees perform exploitation of search space, specially onlooker bees that explore especific food sources (possible promissing search subspaces). The exploration of search space is done by scout bees, that try to find new positions to food sources.

3.3 Differential evolution DE algorithm try to find the optimal solution x∗ in Ddimensional space, where a population P with individuals xi , i = 1, . . . , N P is randomly selected at the first step of DE (Price and Storn 1995). At each iteration, for every individual xi , i = 1, . . . , N P, three other distinct parameter vectors xa , xb , and xc are randomly chosen from the population P. The mutation operation consists in generating a new vector ui , described by the Eq. 9: ui = xa + F(xb − xc )

(9)

Artificial bee and differential evolution

where F is a scaling parameter. The mutated vector ui and the target (initial) vector xi form two parents. The crossover operation that is done between ui and xi can be seen by trial vector (offspring) yi :  j yi

=

j

ui j xi

if rand(0, 1) j ≤ CR or j = jrand , otherwise

(10)

where CR is a predefined crossover parameter, jrand is a random integer variable from [1, D] and rand(0, 1) j is a uniform random real variable from range [0, 1]. If f (yi ) ≤ f (xi ) then yi replaces xi and this operation is called selection. Several works have been investigating different values to DE parameters, as CR and F, and different DE strategies. One of these works that test several different values for CR and F and several DE strategies can be seen in MezuraMontes et al. (2006), the initial version defined above was named D E/rand/1/bin.

4 New clustering search approaches In this work, two new *CS approaches are presented, based on ABC and DE metaheuristics, Artificial Bee Clustering Search (ABCS) and Differential Evolutionary Clustering Search (DECS), respectively. The main motivation behind the new approaches lays down in the need of investigation of recently proposed competitive algorithms, rationally hybridized by Clustering Search , in continuous domain optimization. Clustering Search combined with Evolutionary Algorithms has presented good performance solving a large variety of optimization problems, as can be seen in Sect. 2. Such technique combined with efficient algorithms, as ABC and DE, presenting newer ways for this combination, has produced competitive results. The ABCS algorithm is based on potential characteristic of detecting promising areas presented by ABC algorithm. Such potential characteristic can be seen on basic operations of ABC, such as the exploration and abandonment of a food source. The exploration performed by employed bees can be seen as an activation of a cluster on *CS-based algorithms, considering that a food source is a central region of a cluster. The abandonment of food sources can be seen as a deactivation of a cluster with few assimilations for a long period of generations. An onlooker bee can be seen as a solution that must be assimilated by a cluster on a *CS-based algorithms, because this bee will increase a source’s fitness after it has explored and found a better position to that food source. ABCS is proposed to represent all food sources as central regions of

clusters and onlooker bees are solutions that can be or cannot be assimilated by these clusters. The Algorithm 1 illustrates the few differences between ABCS and pure ABC. The accounting of δi (density of food source i) is done only at onlooker bee phase, because in this phase the food sources are selected based on its probabilities of selection. Thus δi is a parameter to perform local search, that will inform which food sources are most promising. To detect these promising areas, δi increases whenever an onlooker bee access the food source and gets a new better position. If a food source reach a certain threshold there is a time to perform local search in such source to find better solutions. This threshold is known in *CS as cluster density (Sect. 3). δi ≥ PD ·

NP |SN |

(11)

where PD is a pressure of density, N P is the colony size and SN is the total number of food sources. All original ideas from CS defined in Oliveira and Lorena (2004) had to be implemented in the works cited in Sect. 2, in other words, the complete CS framework is available, the same is true for DECS, the another approach implemented in this work. However, to design ABCS, it was not necessary to implement the complete CS framework, because ABC incorporates functions from CS. For example, thinking the assimilation process more abstractly, when a bee from ABC comes to recruiting another bees in the search space looking for the most promising food sources, a set of bees can learn paths for good food sources previously found and explore the best places of all search space. In other words, that set of bees assimilate the information sent from another bees. This process implement the same idea used in CS-based algorithms described in Sect. 3.1, the Clustering Search. This process of assimilation did not need to be implemented in the new ABCS proposal, as in the algorithms mentioned in Sect. 2 and DECS, once this process is already indirectly being held in recruitment. The motivation is to create the ABCS to enjoy its intrinsic potential of clustering solutions, adding a few features to increase its ability to detect best solutions. Thus ABCS is proposed to add to ABC some clustering elements without modifying the main structure of original algorithm. This work proposes to add two elements in ABCS: the amount of activations of each food source and local search routines. The algorithm used to perform local search in ABCS is the Hooke and Jeeves’ algorithm (Hooke and Jeeves 1961), the same local searcher used in another algorithms shown in this work. By adding few components to ABCS increasing its clustering potential, such proposal is more intuitive and easier to implement, if compared to cited CS works (Sect. 2) and DECS.

123

T. S. Costa, A. C. M. de Oliveira Initialize the population of solutions xi ,i = 1, 2, . . . , SN Evaluate xi ,i = 1, 2, . . . , SN and memorize the best one cycles ← 1 repeat for i ← 1 to SN do vi j = xi j + φi j (xi j − xk j ) //candidate solution by employed bee if f (vi ) < f (xi ) then xi ← vi else ti ← ti + 1 end end //calculating the probabilities xi for i ← 1 to SN do f it pi = SN i n=1 f itn

end i ←0 t ←0 while t < SN do if r < pi then t ←t +1 vi j = xi j + φi j (xi j − xk j ) //candidate solution by onlooker bee if f (vi ) < f (xi ) then δi ← δi + 1 //updating density of food source xi ← vi else ti ← ti + 1 end end P if δi ≥ PD · N SN then Local Sear ch(xi ) //exploitation of promising food source δi ← 1 end i ←i +1 if i = SN then i ←0 end end for i ← 1 to SN do if ti > Limit then j j j j xi = xmin + rand[0, 1](xmax − xmin ) end end until cycles = MCN ;

Fig. 2 ABC and ABCS

Fig. 3 DECS (adapted from Oliveira and Lorena 2004)

The basic differences between ABC and ABCS can be seen by Fig. 2. The Fig. 2 illustrates the scout bee (S) searching for food sources and the recruitment (R). On one hand, in pure ABC, there is no action over the number of times that a food source was explored and there is no local search in these promising (more activated) food sources. On the other hand, ABCS computes the top activated food sources (a CS function added) over each iteration, and performs local search (another CS function added) in these areas, so called promising areas. DECS is similar to ECS. The main difference between ECS and DECS is the component EA, as illustrated in the Fig. 3. In first ECS version (Oliveira and Lorena 2004), EA is a genetic algorithm, while in DECS, this component is a DE, all other parts of DECS are equal to ECS.

Algorithm 1: Artificial Bee Clustering Search Table 1 Parameters of ABC, HABC and ECS Algorithm

Parameters NP

Limit

MCN

PD

80

200

10,000

PopSi ze

Max Gen

NC

200

8,000

40

ABC HABC ABCS ECS

10 2.5

Table 2 Parameters of DE, HDE and DECS Algorithm

Parameters NP

Max Gen

F

CR

300

8,000

0.4

0.9

DE HDE DECS

Strategy rand/1/bin rand/1/bin best/1/bin

5 Experimental results In this section, we present the numerical study done on the newly proposed approaches to evaluate their performance.

123

We compared the newly proposed approaches with the original ABC, an hybrid Artificial Bee Colony (HABC), the original DE, an Hybrid Differential Evolution (HDE) and Evolu-

Artificial bee and differential evolution Table 5 First set of functions (ABCS and ECS)

Table 3 First set of functions used in the experiments Abbreviation

Function

Range

D

Min

Func f mic

ABCS

ECS

f zak

Zakarov

[–5, 10]

30

0

f mic

Michalewicz

[0, 3.1415]

10

–9.6601

Mean

–9.6600

–9.6355

f ros

Rosenbrock

[–30, 30]

30

0

StdDev

5.8031E–005

0.0214

f sch

Schwefel7

[–500, 500]

30

0

H/A

30/30

6/30

FE

97,753.20

1,745,612.73

f ras

Rastrigin

[–5.12, 5.12]

30

0

f rid

Ridge

[–64, 64]

30

0

f col

Colville

[–10, 10]

0

Mean

0.0001

6.4467E–005

f lan

Langerman

[0, 10]

10

–1.4999

StdDev

2.5765E–005

4.8001E–005

f sph

Sphere

[–5.12, 5.12]

30

0

H/A

30/30

30/30

f gri

Griewank

[–600, 600]

30

0

FE

171,625.75

508,843.9

f ack

Ackley

[–15, 30]

30

0

0.0001

1.9567

4

f ros

f ras Mean

Table 4 First set of functions (ABC variations) Func

ABC

HABC

f mic Mean

–9.6600

–9.3207

StdDev

6.8215E–005

0.29288

H/A

30/30

10/30

FE

35,874.48

911,211.31

Mean

0.0343

0.0001

StdDev

0.0499

0.00002

H/A

0/30

30/30

FE

800,081.10

185,090.24

Mean

0.0001

0.0001

StdDev

7.1006E–005

1.1056E–005

H/A

30/30

30/30

FE

70,597.24

43,929.13

Mean

7.8033E–005

1,295.28

StdDev

6.2561E–005

252.88

H/A

30/30

0/30

FE

109,758.65

2 × 106

f ros

f ras

f sch

f lan Mean

–0.9551

–0.8450

StdDev

0.2802

0.1331

H/A

4/30

1/30

FE

733,114.20

1,952,654.20

tionary Clustering Search (ECS Oliveira and Lorena 2004). The results were obtained allowing all algorithms to perform up to 2 × 106 Maximum Function Evaluations in each one of 30 trials, an stopping criterion. Such parameters were used trying to solve eleven selected functions (Table 3). To continue the experiments, we also selected the 8 first functions from CEC 2005 (Suganthan et al. 2005) problem set.

StdDev

3.0449E–005

0.8046

H/A

30/30

2/30

FE

30,982.41

1,928,062.2

Mean

0.0001

1,075.82

StdDev

6.1813E–005

416.58

H/A

30/30

0/30

FE

733,098

2 × 106

f sch

f lan Mean

–1.0715099

–0.8445

StdDev

0.2954

0.1781

H/A

7/30

2/30

FE

1,001,020.17

1,918,115.96

For functions in Table 3, an execution is considered well succeeded if the best solution found satisfies an error rate, that was set to 10−4 . In other words, whenever the difference between the best solution found and the known optimal solution is less or equal than 10−4 , the execution is well succeeded, that is another stopping criterion adopted for all algorithms. For selected functions from CEC 2005 benchmark, the error rate was set to 10−8 and MAXFE (Maximum Function Evaluations) was set to D × 105 , where D is the dimension of the problem. The list of parameters of all algorithms are summarized in Tables 1 and 2, where the value 80 of N P was used based on Karaboga and Akay (2009), the other values were empirically tested. The parameter Limit was fixed at 200, that means a food source is assumed to be abandoned if its fitness could not be improved until 200 cycles. The component EA of ECS implemented in this work is a steady-state real-coded genetic algorithm employing well-known genetic operators as roulette wheel selection (Goldberg 1989), blend crossover (BLX0.25) (Eshelman and Schawer 1993), and non-uniform mutation (Michalewicz 1996) with simple assimilation. The specific parameters used

123

T. S. Costa, A. C. M. de Oliveira Table 7 Second set of functions (ABCS and ECS)

Table 6 Second set of functions (ABC and HABC) Func

ABC

HABC

Mean

100,000

100,000

Mean

100,000

100,000

StdDev

3.67E–005

8.66E–006

StdDev

3.08E–005

2.2968E–006

H/A

30/30

30/30

H/A

30/30

30/30

FE

30,326.89

57,343.65

FE

11,508.96

3,274.2

f sph

Func

ABCS

ECS

f sph

f gri

f gri

Mean

0.0001

0.0001

Mean

0.0001

0.0001

StdDev

4.82E–005

9.07E–006

StdDev

0.00002

3.574E–005

H/A

30/30

30/30

H/A

30/30

30/30

FE

55,186.20

39,759

FE

11,105.58

4,943

0.0001

8.646E–005

f ack Mean

f ack 0.0001

0.0001

Mean

StdDev

3.33E–005

1.46E–005

StdDev

2.38E–005

7.1095E–005

H/A

30/30

30/30

H/A

30/30

30/30

FE

61,055.17

40,015.55

FE

32,731.55

366,911.866 2.5389

f zak

f zak

Mean

136.45

0.023

Mean

0.014

StdDev

23.76

0.025

StdDev

0.0172

1.555

H/A

0/30

0/30

H/A

0/30

0/30

FE

800,040.58

2 × 106

FE

2 × 106

2 × 106

Mean

417.33

0.0003

Mean

0.0002

0.766

StdDev

170.63

0.00007

StdDev

1.98E–005

0.538

H/A

0/30

2/30

H/A

25/30

0/30

FE

800,040.10

1,938,631.93

FE

1,447,802.55

2 × 106

Mean

0.17965

0.0001

Mean

0.0001

0.00006

StdDev

0.13612

2.86E–005

StdDev

2.96E–005

5.5197E–005

H/A

0/30

30/30

H/A

30/30

30/30

FE

800,218.517

22,002.65

FE

15,069.03

3,123.6

f rid

f rid

f col

f col

by the ECS algorithm can be seen in Table 1, where PopSi ze denotes the GA’s population size, max Gen is the total number of generations and N C is the total number of clusters. Different from traditional *CS-based algorithms, ECS implementation of this work was designed to perform local search in certain cluster center cj only if the last call to Hooke and Jeeves in this center cj succeeded, as an attempt to minimize the total function evaluations performed by local search algorithm. The test suite consists of eleven benchmark functions commonly used for algorithms’ performance evaluation found in the literature and the CEC 2005 functions’ set. Table 3 shows this first benchmark, where D is the number of variables (Dimension), Min is the optimal solution and Range is the known upper and lower bounds of the domain of functions’ variables. HABC and HDE algorithms were not designed to detect promising areas, the Hooke and Jeeves local search is

123

activated only whenever the better current solution has been improved (elitist strategy). The variables mean, hits/attempts (H/A), standard deviation (StdDev) and function evaluations (FE) are necessary to analyze the performance of algorithms. The results of overall experiment was divided into eight tables: initial four Tables 4, 5, 6, and 7 contemplate results of overall functions to ABC and its hybrid versions (HABC and ABCS) and ECS. In Tables 8, 9, and 10 the results to DE and its hybrid versions (HDE and DECS) are shown. Table 11 shows the results for the first eight selected functions from CEC 2005 Benckmark, where f 110 means that the function number is 1 from CEC and D = 10. This table also shows the results for D = 10 and D = 30 for each function, the error rate and standard deviation within 25 runs for each function. Considering each function f ∈ F (Table 3), the mean is calculated over 30 executions for each value of the objective function f found by each algorithm, in other words, the best

Artificial bee and differential evolution Table 10 Third set of functions (DE variations)

Table 8 First set of functions (DE variations) Func

DE

HDE

DECS

Mean

–9.6570

–9.6568

–9.6600

StdDev

0.0105

0.0105

0

H/A

27/30

25/30

30/30

FE

699,994.0

789,470.8

53,637.7

f mic

Func

DE

HDE

DECS

Mean

0.0001

0.0001

0.0001

StdDev

0

0

0

H/A

30/30

30/30

30/30

FE

565,631.4

472,566.4

1,597,636.8

f zak

f ros

f rid

Mean

0.0001

0.1331

0.4524

Mean

0.0001

0.0001

12.4608

StdDev

0

0.7278

1.0437

StdDev

0

0

21.4445

H/A

30/30

29/30

0/30

H/A

30/30

30/30

0/30

FE

1,007,680.8

292,264.0

1,649,142.3

FE

610,822.0

439,338.0

1,629,141.2

0.0001

0.0001

f ras

f col

Mean

102.6892

39.2439

0.0001

Mean

0.0001

StdDev

10.7306

H/A

0/30

17.6983

0

StdDev

0.0001

0.0001

0.0001

0/30

30/30

H/A

30/30

30/30

30/30

FE

2 × 106

2 × 106

840,849.5

FE

25,281.5

19,450.7

236,202.3

Mean

4,828.1128

3,787.5247

0.0001

StdDev

719.0559

1,060.6500

0

H/A

0/30

0/30

30/30

FE

2 × 106

2 × 106

256,162.0

Mean

–1.4604

–1.4426

–1.3989

StdDev

0.1502

0.1750

0.2156

H/A

28/30

27/30

3/30

FE

281,709.5

306,508.5

1,589,190.7

f sch

f lan

Table 9 Second set of functions (DE variations) Func

DE

HDE

DECS

Mean

100,000

100,000

100,000

StdDev

0

0

0

H/A

30/30

30/30

30/30

FE

110,461.1

52,357.5

88,267.1

f sph

f gri Mean

0.0001

0.0079

0.0001

StdDev

0

0.0093

0.0001

H/A

30/30

12/30

30/30

FE

169,774.5

1,242,650.9

119,629.3

Mean

0.0001

0.0001

0

StdDev

0

0.0001

0

H/A

30/30

30/30

30/30

FE

203,759.6

129,076.9

148,216.3

f ack

solution found at each iteration of any algorithm is included in the calculation of mean. The H/A variable is the number of times that a certain algorithm has found the best solution

of f (hits) and the number of attempts (A) to find or not the best solution of a f. FE is a mean between total function calls in 30 executions for each function f. The efficiency of an algorithm is measured looking for the quantity of hits by the quantity of attempts of each function f , considering that a good algorithm can find the best solution without reaching the top number of FE, if was not possible to find the best solution, the mean has to be as close as possible from the best value. For example, from Table 5, the algorithm ABCS reaches the best performance, because for each function f , the total number of hits was greater than ECS and the number of FE was lower than ECS. The ABCS algorithm presented superior performance for most of all benchmark functions. Only in three functions, DE and its variations presented better results than ABCS: f lan , f zak and f rid . When ABCS presented similar performance of HABC (the same quantity of hits) the number of function evaluations is less than HABC. Minimizing Langerman function ( f lan ), which provided great difficulty for most of all algorithms, ABCS presents good performance with greater amount of hits and fewer function evaluations if compared against HABC and ECS. This comparison is analog at f rid (Table 7). The ABCS algorithm reaches the optimal known solution of f ros and f ras (Table 5) and reaches best mean of f col (Table 7) with fewer FE if compared against ABC and HABC. Considering the quantity of hits, the algorithm DECS presents better performance than DE and HDE at f mic , f ras , f sch and f gri , observing that at f gri , DECS reached the smallest quantity of FE.

123

T. S. Costa, A. C. M. de Oliveira Table 11 Best value, Mean error and standard deviation of pure ABC and ABCS Function Algorithms Best f 110 f 130 f 210 f 230 f 310 f 330 f 410 f 430 f 510

Mean error

StdDev

ABC

−450.0

0

0

ABCS

−450.0

0

0

ABC

−450.0

0

0

ABCS

–450.0

0

0

ABC

–4.4734E+02

1.5681E+01

8.7166E+00

ABCS

−4.4975E+02

1.42281E+01 7.8587E+00

ABC

1.7119E+03

4.5602E+03

1.5431E+03

ABCS

6.3005E+11

1.1751E+12

2.8966E+11

ABC

3.4812E+13

6.1297E+13

2.7674E+13

ABCS

2.3254E+11

5.7514E+11

1.9698E+11

ABC

4.8583E+06

8.2501E+06

2.6070E+06

ABCS

1.0359E+06

6.0966E+06

8.5759E+05

ABC

8.9342E+08

1.6532E+11

6.1047E+10

ABCS

7.6112E+09

2.2343E+11

5.6688E+7

ABC

4.1458E+04

4.4614E+04

2.4674E+03

ABCS

2.4783E+04

3.1113E+04

8.3146E+03

ABC

−3.0619E+02 8.9474E+01 1.1437E+02

ABCS

−310

0

ABC

5.9053E+03

9.9655E+03

1.4111E+03

ABCS

−310

0

0

ABC

3.9001E+02

2.4529E+00

4.5603E+00

ABCS

3.9001E+02

2.8554E+00

3.2635E+00

ABC

3.9002E+02

3.2748E+00

7.0525E+00

ABCS

3.9010E+02

2.0123E+00

2.6557E+00

ABC

−1.7993E+02 2.5629E−01 1.1187E−01

ABCS

−1.7985E+02 3.3935E−01 9.6315E−02

ABC

4.5163E+03

4.6963E+03

9.2825E−13

ABCS

−1.7891E+02

1.3602E+00

1.7329E-01

f 810

ABC

−1.1985E+02 2.0299E+01 6.7596E−02

ABCS

−1.1982E+02

f 830

ABC

−1.1933E+02 2.0866E+01 6.4689E−02

ABCS

−1.1925E+02

f 530 f 610 f 630 f 710 f 730

2.0308E+01

123

6.7199E−02

2.0880E+01 5.5297E−02

Bold values indicate better result obtained Fig. 4 Convergence curve for f zak

0

HABC and HDE got bad performance at f sch , because this function is extremely multimodal and these algorithms performs many calls to Hooke and Jeeves, without information about which regions of search space are promising, getting parked in local minimum points. ECS get better results to f sch when the parameter PD is less than 2.5; 2.5 was used because is the best one to overall functions. Figure 5 illustrates the progress of all algorithms minimizing Langerman function. The mean of objective function (known as fitness value in the chart) was calculated between 30 trials at intervals of 5,000 function evaluations. The Fig. 5 shows the velocity of convergence reached by ABCS, that with fewer quantity of FE, ABCS algorithm reached best means of fitness (y axis). In other Figs. 4, 6 and 7, the ABCS algorithm presents good convergence if compared against another approaches, always getting best mean of fitness, although in Figs. 4 and 7 ECS converges first, but with higher average fitness (y axis), at the end, ABCS shows better fitness values. Only DE and its variations found the optimal solution of f zak function, ABCS got best approximation (about 0.014) with best standard deviation, 0.0172 (Table 7) among ABC, HABC and ECS. We showed results for ABCS and ABC in Table 11 because these methods reach the best results for this benckmark. ABCS reach the best results for most of all selected functions from CEC 2005 and the best-known optimal solutions were found only for functions f 1 and f 5 .

6 Conclusion In this work new approaches are proposed combining Clustering Search with popular metaheuristics as Artificial Bee Colony and Differential Evolution. The proposed hybrid metaheuristics are called ABCS and DECS. Two elements of *CS are added to ABCS aiming to improve inherent characteristics of detecting promising food sources by ABC: quantity of activations of each food source and the local search

Artificial bee and differential evolution Fig. 5 Convergence curve for f lan

Fig. 6 Convergence curve for f sch

Fig. 7 Convergence curve for f mic

123

T. S. Costa, A. C. M. de Oliveira

component. DECS consists basically in using a DE algorithm as EA component. Some unconstrained continuous optimization functions are minimized and ABCS presented competitive results compared against algorithms: Pure ABC, HABC, ECS and variations of DE. DECS approach presented the second best result. When compared to recent works found in the literature, ABCS presents similar performance to Kang et al. (2011) and Wu et al. (2012), specifically for functions: Rosenbrock, Colville, Sphere, Ridge, Zackarov and Ackley. Regarding future work, some *CS components could be tested, as the distance metric and different types of bee assimilation by food sources, since the natural bee assimilation used by ABC algorithm is double cluster assimilation, because k is a neighbor food source that is selected randomly to contribute to perturbation caused a given food source i (Sect. 3.2).

References Chaves AA, Correa FA, Lorena LAN (2007) Clustering search heuristic for the capacitated p-median problem. Springer Adv Softw Comput Ser 44:136–143 Costa TS, Oliveira ACM, Lorena LAN (2010) Advances in clustering search. Adv Soft Comput 73:227–235 Eshelman LJ, Schawer JD (1993) Real-coded genetic algorithms and interval-schemata. In: Darrell Whitley L (ed) Foundation of genetic algorithms-2, Morgan Kaufmann Publisher, San Mateo 187–202 Feo T, Resende M (1995) Greedy randomized adaptive search procedures. J Glob Optim 6:109–133 Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Longman Publishing Co., Inc. Boston, MA, USA Hooke R, Jeeves TA (1961) Direct search solution of numerical and statistical problems. J Assoc Comput Mach 8:212–229 Kang F, Li J, Ma Z (2011) Rosenbrock artificial bee colony algorithm for accurate global optimization of numerical functions. Inf Sci 181(16):3508–3531

123

Karaboga D, Akay B (2009) A comparative study of artificial bee colony algorithm. Appl Math Comput 214:108–132 Mezura-Montes E, Velazques-Reyes J, Coello CAC (2006) A comparative study of differential evolution variants for global optimization. In: Proceedings of genetic and evolutionary conference (GECCO), pp 485–492 Michalewicz Z (1996) Genetic algorithms + Data structures = Evolution programs. Springer, New York Mladenovic N, Hansen P (1997) Variable neighborhood search. Comput Op Res 24:1097–1100 Oliveira ACM, Lorena LAN (2004) Detecting promising areas by evolutionary clustering search. In: Proceedings of Brazilian symposium on artificial intelligence SBIA2004, LNAI, vol 3171. Springer, New York pp 385–394 Oliveira ACM, Lorena LAN (2006) Pattern sequencing problems by clustering search. In: Advances in artificial intelligence IBERAMIA-SBIA 2006. Lecture notes in computer science, vol 4140, Springer, Heidelberg, pp 218–227 Oliveira ACM, Lorena LAN (2007) Hybrid evolutionary algorithms and clustering search. Hybrid Evolut Syst Stud Comput Intell 75:81– 102 Oliveira RM, Mauri GR, Lorena LAN (2012) Clustering search for the berth allocation problem. Expert Syst Appl 39:5499–5505 Oliveira ACM, Chaves AA, Lorena LAN (2013) Clust Search Pesquisa Operacional 33(1):105–121 Price K, Storn R (1995) Differential evolution: a simple and efficient adaptive scheme for global optimization over continuous spaces. In: Proceedings of technical report TR- 95–012, International Computer Sciences Institute, Berkeley, CA, USA Ribeiro Filho G, Nagano MS, Lorena LAN (2007) Evolutionary clustering search for flowtime minimization in permutation flow shop. Lecture Notes Comput Sci 4771:69–81 Ribeiro GM, Laporte G, Mauri GR (2011) A comparison of three metaheuristics for the workover rig routing problem. Eur J Op Res 220(1):2836 Suganthan PN, Hansen N, Liang JJ, Deb K, Chen YP, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization Wu B, Qian C, Ni W, Fan S (2012) Hybrid harmony search and artificial bee colony algorithm for global optimization problems. Comput Math Appl 64(8):2621–2634