A Clustering Algorithm applied to the Binarization of Swarm Intelligence Continuous Metaheuristics Jos´e Garc´ıaa,b , Broderick Crawforda , Ricardo Sotoa , Gino Astorgaa,c a
Pontificia Universidad Cat´ olica de Valpara´ıso, 2362807 Valpara´ıso, Chile. b Telef´ onica Investigaci´ on y Desarrollo, Santiago, Chile. c Universidad de Valpara´ıso, 2361864 Valpara´ıso, Chile.
Abstract The binarization of Swarm intelligence continuous metaheuristics is an area of great interest in operations research. This interest is mainly due to the application of binarized metaheuristics to combinatorial problems. In this article we propose a general binarization algorithm called K-means Transition Algorithm (KMTA). KMTA uses K-means clustering technique as learning strategy to perform the binarization process. In particular we apply this mechanism to Cuckoo Search and Black Hole metaheuristics to solve the Set Covering Problem (SCP). A methodology is developed to perform the tuning of parameters. We provide necessary experiments to investigate the role of key ingredients of the algorithm. In addition, with the intention of evaluating the behavior of the binarizations while the algorithms are executed, we use the Page’s trend test. Finally to demonstrate the efficiency of our proposal, Set Covering benchmark instances of the literature show that KMTA competes clearly with the state-of-the-art algorithms. Keywords: Bio-inspired Computation, Swarm Intelligence, Combinatorial Optimization, Machine Learning, Unsupervised Learning, Clustering. Email addresses:
[email protected] (Jos´e Garc´ıa),
[email protected] (Broderick Crawford),
[email protected] (Ricardo Soto),
[email protected] (Gino Astorga)
Preprint submitted to Swarm and Evolutionary Computation
August 28, 2018
1. Introduction In recent years, the areas of physics and swarm intelligence have generated a large number of metaheuritic algorithms. These metaheuristics are suitable for solving a broad class of complex optimization problems. Examples of these algorithms are Ant Colony Optimization (ACO) Glover and Kochenberger (2003), Firefly Algorithm (FA) Yang (2010a), Magnetic Optimization Totonchi and Reza (2008), Gravitational Search Algorithm (GSA) Rashedi et al. (2009), Cuckoo Search Algorithm (CSA)Yang and Deb (2009), Particle Swarm Optimization (PSO) Kennedy (2011). Part of the success of modern metaheuristics is because they are simple to understand and to implement given their nature inspired behaviour. However many of these algorithms are specifically designed to solve continuous problems. On the other hand, combinatorial problems arise in many areas of computer science and application domains. For example in protein structure prediction Jana et al. (2017); Lalwani et al. (2016), grouping routing Alvarez and Munari (2017); Iqbal et al. (2015), planning Canca et al. (2017), scheduling Gao et al. (2017) and timetabling Sch¨obel (2017) problems to mention only some examples. It is natural to try to apply these algorithms inspired by physics and swarm intelligence in combinatorial problems. 20
In the process of adaptation, a series of difficulties arise when moving from continuous spaces to discrete spaces. Among the main difficulties, we find spacial disconnect, hamming cliffs, loss of precision and curse of dimension Leonard et al. (2015); Piotrowski and Napiorkowski (2016). The inappropriate management of these situations has the consequence that binarizations are not always effective and efficient Lanza-Gutierrez et al. (2017). Moreover, it is known that the performance of the Swarm Intelligence algorithms depends in important way on the choice of its parameters. Furthermore, when
2
binarization is performed, these parameters that were calibrated for the continuous case are altered by the binarization mechanism. In many cases this alteration is carried out without being clear how the exploration/exploitation capacities generated in the continuous space are translated. This motivates the design of new algorithms, where a learning or adaptive mechanism is incorporated in the binarization stage in order to allow an adequate translation of the exploration/exploitation properties. During the last decade, there has been an important tendency to explore the hybridization of machine learning and metaheuristics techniques with the goal of improving the robustness of the algorithms. When analyzing the way in which these set of techniques are integrated, we observe that there is a group where machine learning techniques improve results of metaheuristics 40
and another where metaheuristics improve the results of machine learning algorithms. A state of the art with respect to these hybridizations will be developed in the section 4. Our work focuses on exploring the use of unsupervised learning techniques to strengthen metaheuristic algorithms. We carry out a specific integration to address the binarization problem of continuous metaheuristics. Specifically, our proposal aims to generate robust binary versions of continuous swarm intelligence algorithms. Additionally, with the incorporation of widely known techniques such as K-means, our proposal simplifies the binarization process so that any researcher or developer habituated to work with a continuous metaheuristic can easily generate robust binary versions. Moreover, a methodology is proposed using the velocity distribution of the solutions in the continuous and discrete space together with the page’s trend test to adjust the parameters used in the binarization process. This methodology allows identifying under what conditions the binarization process works properly and therefore have better control of this binarization process. In this article considering the research topic: Hybrid learning schemes 3
jointly embracing ingredients from swarm/evolutionary computation and machine learning, a K-means Transition Algorithm (KMTA) is proposed as a general mechanism to binarize continuous Swarm Intelligence metaheuris60
tics. This algorithm is composed of three operators. The main operator corresponds to the K-means transition operator. This operator translates the velocities obtained in the continuous space to transition probabilities in binary spaces. This translation is done by performing the clustering of solutions considering the solution velocity module in each dimension as metric. Because the clustering is executed at each iteration, the value of the transition probabilities of each solution is adapted depending on the clusters generated by the K-means Transition operator. In addition to the K-means transition operator, perturbation and repair operators are used. The main goal of this work corresponds to evaluate and understand the contribution of the different operators in the process of binarization as the system evolves when solving a well-known NP-hard combinatorial optimization problem such as the SCP. To develop the evaluation, we used the Cuckoo Search and Black Hole metaheuristics. Both are new bio-inspired optimization schemes that have been used to solve the set covering problem Soto et al. (2017); Garc´ıa et al. (2017c). This allows to have a reference of the result of our binarization with respect to other algorithms. The choice of these metaheuristics is based on two features. The first one points the fact that these metaheuristics have few parameters to configure and have been tested in several types of problems, therefore, they are quite simple to tune and allows us to focus on the
80
tuning and analysis of binarization. On the other hand, when analyzing the equations of the movements of these metaheuristics, they have totally different mechanisms. The black hole is a simplified version of PSO and cuckoo search is inspired by levy flight, therefore, the conclusions obtained regarding the binarization algorithm have greater support. A method was designed to estimate the parameters used by metaheuris4
tics and in the clustering binarization. Experiments were developed that shed light on the contribution of the different operators to the effectiveness of the algorithm. In addition, experiments were designed to evaluate the binarization and to give us insights on the convergence of the algorithms through the analysis of their intermediate results. Moreover, our binarizations were compared with an algorithm that uses a specific Teaching-learning binarization Lu and Vasko (2015) technique, with Binary Cuckoo Search (BCS), and Binary Black Hole (BBH) that use a general Transfer Function binarization method Soto et al. (2017), and with a specific Jumping Particle Swarm Optimization (JPSO) method Balaji and Revathi (2016). For this purpose we use tests problems from the OR-Library1 . Finally, an application to a crew scheduling real-world problem was studied. The remainder of this paper is organized as follows. Section 2 briefly introduces the Set Covering problem. In the section 3 other binarization works 100
are presented. Hybridization between machine learning and metaheuristics techniques is developed in Section 4. In Section 5 we explain the transition ranking binarization algorithm. The results of numerical experiment are presented in Section 6. In Section 7, a real-world application is detailed. Finally we provide the conclusions of our work in section 8. 2. Set Covering Problem The SCP is one of the oldest and most studied optimization problems. SCP is well-known to be NP-hard Gary and Johnson (1979). Nevertheless, different algorithms for solving it have been developed. There exist exact algorithms who generally rely on the branch-and-bound and branch-andcut methods to obtain optimal solutions Balas and Carrera (1996); Beasley (1987). These methods, however, need an effort for solving an SCP instance 1
OR-Library: http://www.brunel.ac.uk/ mastjjb/jeb/orlib/mknapinfo.html.
5
that grows exponential with the problem size. Then, even medium-sized problem instances often become intractable and cannot be solved any more using exact algorithms. To overcome this issue, the use of different heuristics have been proposed John (1990); Beasley and Chu (1996). For example, John (1990) presented a number of greedy algorithms based on a Lagrangian relaxation (called the Lagrangian heuristics), Caprara et al. Caprara et al. (1999) introduced relaxation-based Lagrangian heuristics applied to the set covering problem. Metaheuristics also have been applied to 120
solve SCP, some examples are genetic algorithm Yelbay et al. (2012), simulated annealing Brusco et al. (1999) and ant colony optimization Valenzuela et al. (2014). More recently swarm based metaheuristics as cat swarm Crawford et al. (2015), cuckoo search Garc´ıa et al. (2018a), artificial bee colony Karaboga (2005), and black hole Garc´ıa et al. (2017c) were also proposed. SCP has many practical applications in engineer, e.g., vehicle routing, railway, airline crew scheduling, microbial communities and pattern finding Balas and Padberg (1976); Borneman et al. (2001); Boros et al. (1997); Garfinkel and Nemhauser (1972). The SCP can be formally defined as follows. Let A = (aij ), be a n × m zero-one matrix, where a column j cover a row i if aij = 1, besides a column j is associated with a non-negative real cost cj . Let I = {1, ..., n} and J = {1, ...m}, be the row and column set of A, respectively. The SCP consists in searching a minimum cost subset S ⊂ J for which every row i ∈ I is covered by at least one column j ∈ J, i.e,: Minimize f (x) =
m X
cj x j
(1)
j=1
Subject to
m X
aij xj ≥ 1, ∀i ∈ I, and xj ∈ {0, 1}, ∀j ∈ J
j=1
where xj = 1 if j ∈ S, xj = 0 otherwise. 6
(2)
3. Binarization related work There is a set of metaheuristic techniques that were designed to operate in continuous spaces. Examples of these techniques are Artificial Bee Colony Karaboga (2005), Particle Swarm Optimization Shi et al. (2001), Black Hole 140
Hatamlou (2013), Cuckoo Search Yang and Deb (2009), Bat Algorithm Yang (2010b), FireFly Algorithm Yang (2009), FruitFly Pan (2012), Artificial Fish Swarm Li et al. (2002), Gravitational Search Algorithm Rashedi et al. (2009). Moreover, in operations research, there are a lot of problems that are combinatorial and non-polynomial type Kotthoff (2014). So naturally, it arises the idea of applying these continuous metaheuristics to combinatorial problems which are solved in discrete spaces. These adaptations are generally not trivial and have given rise at different lines of research. When a review is made in the literature Crawford et al. (2017) of binarization techniques, two main groups appear. A first group corresponds to general binarization frameworks. In these frameworks there is a mechanism that allows to transform any continuous metaheuristics in a binary one without altering the metaheuristics operators. In this category the main frameworks used are: Transfer Functions and Angle Modulation. The second group corresponds to binarizations developed specifically for a metaheuristic. Within this second group we found techniques such as Quantum Binary and Set based approach. Transfer Functions: The Transfer Function is the most used binarization method. It was introduced by J and Eberhart (1997). The Transfer Function is a very cheap operator, his range provides probabilities values
160
and try to model the transition of the solution positions. This function is responsible of the first step of the binarization method which corresponds to map the Rn solutions in [0, 1]n solutions. Two types of functions have been used in the literature, the S-shaped Yang et al. (2013), and V-shaped Crawford et al. (2014b). The Second Step is to apply a binarization rule 7
to the result of the Transfer Function. Examples of binarization rules are complement, roulette, static probability, and elitist Crawford et al. (2014b). In Khalil et al. (2006), this framework was used to optimize sizing of Capacitor Banks in Radial Distribution Feeders. In Robinson (2005), Transfer Functions were used for the analysis of bulk power system. This approach has also been used to solve the set covering problem using Binary Firefly Algorithm Crawford et al. (2014b),Soto et al. (2015) used Cuckoo Search Algorithm applied to the same problem. To solve the unit commitment problem yang et al. in Yang et al. (2013) used Firefly and PSO algorithms. The knapsack crystosystem it was approached in Palit et al. (2011). Network and reliability constrained problem solved in Chandrasekaran and Simon (2012) and the Knapsack problem solved by Zhang et al. Zhang et al. (2016) all using using Firefly algorithm. Angle Modulation: This method uses the trigonometric function shown in Equation 3. This function has four parameters which control the frequency 180
and shift of the trigonometric function. gi (xj ) = sin(2π(xj − ai )bi cos(2π(xj − ai )ci )) + di
(3)
This method was first applied in PSO, using a set of benchmark functions. Let a binary problem of n-dimension, and X = (x1 , x2 , ...xn ) a solution. We start with a four dimensional search space. Each dimension represents a coefficient of Equation 3. Then every solution (ai , bi , ci , di ) is associated to a gi trigonometric function. For each element xj the rule 4 is applied: ( bij =
1
if gi (xj ) ≥ 0
0
otherwise
(4)
Then for each initial 4-dimension solution (ai , bi , ci , di ), we get a binary n-dimension solution (bi1 , bi2 , ..., bin ). This is a feasible solution of our nbinary problem. The Angle modulate technique has been applied to network 8
reconfiguration problems Liu et al. (2007) using a binary PSO method, to multi-user detection technique Swagatam et al. (2013) using a binary adaptive evolution algorithm, and to the antenna position problem using a angle modulate binary bat algorithm Zakaria and Chaker (2015). Quantum binary approach: In the line of research that involves the areas of Evolutionary Computing (EC) and Quantum Computing, there are three categories of algorithms Zhang (2011) 1. Quantum evolutionary algorithms: These algorithms focus on the application of EC algorithms in a quantum computing environment. 2. Evolutionary-designed quantum algorithms: These algorithms try to automate the generation of new quantum algorithms using Evolution200
ary Algorithms. 3. Quantum-inspired evolutionary algorithms: These algorithms concentrate on the generation of new EC algorithms using some concepts and principles of Quantum Computing. In particular the Quantum Binary Approach belongs to Quantum-inspired evolutionary algorithms. In this sense these algorithms adapt the concepts of q-bits and superposition to work on normal computers. In the quantum binary approach method, each feasible solution has a position X = (x1 , x2 , .., xn ) and the quantum q-bits vector Q = [Q1 , Q2 , ..., Qn ]. Q represents the probability of xj take the value 1. For each dimension j, a random number between [0,1] is generated and compared with Qj , if rand < Qj , then xj = 1, else xj = 0. The upgrade mechanism of Q vector is specific to each metaheuristic. The Quantum Swarm optimization algorithm has been applied to a combinatorial optimization in Wang et al. (2008), cooperative approach in Zhao et al. (2005), knapsack problem in Shuyuan et al. (2004), and power quality monitor in Ibrahim et al. (2011). The Quantum Differential Evolution 9
algorithm was applied to the knapsack problem in Hota and Pat (2010), combinatorial problems Alegrıa and T´ upac (2014), and image threshold methods in Dey et al. (2015). Using Cuckoo search metaheuristic a Quantum algo220
rithm was applied to the knapsack problem Layeb (2011), and bin packing problem Layeb and Boussalia (2012). A Quantum Ant Colony Optimization was applied to image threshold in Dey et al. (2015). Using Harmony Search in Layeb (2013), and Monkey algorithm in Zhou et al. (2016) quantum binarizations were applied to the knapsack problem. The general binarization frameworks have the difficulty of producing Spacial Disconnect Leonard et al. (2015). The Spacial Disconnect, occurs when close solutions generated by metaheuristics in the continuous space, are not converted into close solutions in discrete space. Informally we can think in a loss of framework continuity. This phenomenon of Spacial Disconnect, has the consequence that the properties of exploration and exploitation are altered and therefore the precision and convergence of the metaheuristic worsen. A study of how Transfer Functions affect exploration and exploitation properties was developed in Saremi et al. (2015). For Angle Modulation the study was developed in Leonard et al. (2015). On the other hand, specific binarization algorithms, which modify the operators of the metaheuristic, are susceptible to problems such as Hamming cliffs, loss of precision, search space discretization and the curse of dimension Leonard et al. (2015). This was studied by Pampara in Pampara (2012) and for the particular case of PSO by Chen in Chen et al. (2011). In the
240
investigation of Chen, he observed that the parameters of the Binary PSO change the speed behavior of the original metaheuristic. In this article, a K-means binarization algorithm (KMTA) is proposed which does not modify the original metaheuristic. The main operator of this framework, establishes a relation between the displacement of solutions in the continuous space and the transition of probability in the discrete space. This 10
relationship is established through the clustering of the displacements. To each group generated by clustering a transition probability is assigned. With this mechanism, it is expected that the exploration and exploitation properties will not be altered, and therefore to observe good results of convergence and precision of the binarized algorithms in the resolution of combinatorial problems. 4. Hybridizing metaheuristics with machine learning When comes to integrate machine learning and metaheuristics, two large groups can be mainly indicated. The first group, corresponds metaheuristic techniques improve the performance of machine learning algorithms. The second one covers machine learning algorithms enhance the metaheuristic algorithms quality. For the first group, we find four main areas of application: improving clustering algorithms, feature selection applications, improving classification algorithms and strengthening regression algorithms. The sum260
mary of the integration between these two areas is shown in Figure 2. In the case of clustering, a variety of methods to solve it has been reported. One of the main problems that presents a greater algorithmic complexity, corresponds to the search of centroids for grouping in a better way the set of studied objects. Since this problem is NP-hard, approximate methods have been proposed to address it. There is a long list of studies in this area, however, in recent years the focus is to solve applied problems. In Mann and Singh (2017), an improved artificial bee colony algorithm was applied to solve the problem of energy efficient clustering in a wireless sensor network. The planning of helicopter transportation of employees to the production platforms of oil and gas was studied in de Alvarenga Rosa et al. (2016), using a cluster search metaheuristic. In the article, the metaheuristic approach was compared with CPLEX, finding that the former produces better, more stable and faster solutions. Another interesting application in logistics management 11
Figure 1: General scheme: Combining Machine Learning and Metaheuristics.
corresponds to the management policy of warehousing and item assignment. In Kuo et al. (2016a) a PSO algorithm was applied for item assignation problem in a synchronized zone order picking system. Finally, in Kuo et al. (2016b) a clustering method based on metaheuristics was proposed to solve a client segmentation problem. A major difficulty in the learning process of a machine learning algorithm 280
is related to the dimension of the dataset. The inadequate handling of the dimension of the dataset has as consequence problems of under or overfitting plus a greater amount of computation necessary for its training. Due to its nature, feature selection is a combinatorial problem and has been ef12
fectively addressed by metaheuristic algorithms. In Ahmad et al. (2015), metaheuristic algorithms were compared with traditional feature selection methods, applying them to datasets used in sentiment analysis. The selection of features is fundamental in real time data stream mining problems, in Fong et al. (2016) an accelerated PSO was proposed to efficiently address feature selection. The problems of classification and regression form an important group of problems that are usually addressed through supervised learning techniques. The constribution of metaheuristics in supervised learning algorithms in both classification and regression problems has been significant. Metaheuristics have contributed to the improvement of algorithms such as suport vector machine, artificial neural network, decision trees, logistic regression among others. In Chou and Thedja (2016) a classification system was proposed that integrates a firefly algorithm with the least squares support vector machine technique to apply it to geotechnical problems. Classification systems applied to health care using metaheuristic algorithms and big data techniques 300
are detailed in Tsai et al. (2016). In Fernandes et al. (2017), metaheuristics were used to design an enhanced probabilistic neural network algorithm. In regression problems, for example, by applying metaheuristics in time series using sliding-window, in Chou and Nguyen (2018), a model to predict the stock prices of Taiwan’s construction companies was designed. In Chou and Pham (2017) Using the Firefly version, the parameters of least squares support vector regression are optimized for enhancing prediction accuracy in engineering design. The shear strength prediction in reinforced concrete deep beams was addressed in Chou et al. (2015). In this article, they propose a firefly algorithm integrated with support vector machine algorithm for accurately predicting shear strength. In the other sense, the contribution of machine learning techniques to the strength of metaheuristic algorithms has been important. In this case, 13
we distinguish two large groups according to the way they are integrated. A first group corresponds to specific integrations where the techniques of machine learning are inserted through an operator in one of the metaheuristic modules. The second group corresponds to general integrations where the machine learning technique works as a selector of different metaheuristic algorithms, choosing the most appropriate for each instance. In the case of specific integration, we found integrations in different mod320
ules of the metaheuristic. For example in the initialization of solutions, performing the tuning of parameters, in the binarization of continuous metaheuristics, also machine learning is used in the population management, etc. In the tuning of parameters, in De Jong (2007) the authors apply a dynamic tuning that allows adapting the parameters depending on the instance and the evolution of the algorithm. A chess rating method was applied in Veˇcek et al. (2016). This method was compared with other techniques such as frace showing a good performance. In Ries and Beullens (2015) a decision tree was used to perform the tuning of parameters. Usually, the mechanism of solutions initialization of a metaheuristic is done in a random way or using some heuristic. However, there are attempts to use machine learning in the initialization of solutions. In Li et al. (2011) case-based reasoning was used to initialize a genetic algorithm and apply it to weighted circles layout problem. Hopfield neural networks were used in Yalcinoz and Altun (2001) to initiate solutions of a genetic algorithm that was used to solve the economic dispatch problem. In the population management, the main line of research is related to extracting information from the solutions previously visited in the search space and identifying the regions that have the greatest potential to exploit them. In the literature, we have found the use of clustering techniques to improve the exploration of the search space Streichert et al. (2003).
340
Also, the use of case-based reasoning techniques was investigated in Santos et al. (2006) in order to identify subspaces of searches to solve the single14
vehicle routing problem. Finally in Jin et al. (2014) an incremental learning technique was used to apply them to the constrained portfolio optimization problem. Finally, a research area with a lot of activity corresponds to design binary versions of algorithms that work naturally in continuous spaces to apply them to combinatorial problems. In this area we found the application of clustering techniques in Garc´ıa et al. (2018b) to perform the binarization. In Garc´ıa et al. (2017a, 2018a) the percentile concept and the ranking of the solutions were used to obtain binary algorithms from continuous algorithms. In reviewing the general integrations, we found in the literature three main groups: Algorithm selection, hyperheuristics, and cooperative strategies. In the case of the selection of algorithms, the objective is to choose from a portfolio of algorithms together with a set of characteristics associated with each instance of the problem, the algorithm that performs best for a given instance. The berth scheduling problem at bulk terminals was addressed in de Le´on et al. (2017) by algorithm selection techniques. In Smith-Miles et al. (2014) meta-learning techniques were used with the goal of proposing a methodology to measure the strengths and weaknesses of algorithms that solve optimization problems in different instances. The goal of hyper360
heuristics is to automate the design of heuristic or metaheuristic methods to address a wide range of problems. In Tyasnurita et al. (2015) an artificial neural network was used to improve the performance of hyperheuristics when solving different instances of the vehicle routing problem. The problem of nurse rostering was addressed in Asta et al. (2016) through a tensor-based hyperheuristic algorithm. Finally in Damaˇseviˇcius and Wo´zniak (2017) a hyperheuristic was designed which allows integrating different nature-inspired algorithms. Cooperative strategies consist of combining algorithms in a parallel or sequential way in order to obtain more robust methods. Cooperation can be complete by sharing the entire solution or partial where only one part of the solution is shared. In Cadenas et al. (2009) a centralized cooperative 15
strategy was developed where knowledge was modeled through fuzzy rules. In Martin et al. (2016) a distributed framework based on agents was proposed. Each agent corresponds to a metaheuristic, where the agent has the ability to adapt through direct cooperation. This framework was applied to the permutation flow shop problem. 5. K-means transition algorithm The Proposed KMTA algorithm has four modules. The first module corresponds to feasible solutions initialization which is described in section 5.1. Once the initialization of the solutions is performed, it is consulted if the 380
detention criterion is satisfied. This criterion includes a maximum of iterations. Subsequently if the criterion is not satisfied, the K-means transition operator shown in section 5.2 is executed. This module is responsible for performing the iteration of solutions. Once the iteration of solutions are performed these do not necessarily satisfy the conditions of the SCP, therefore a repair operator is used, it is shown in section 5.3. Once the transitions of the different solutions are made, we compare the resulting solutions with the best solution previously obtained. In the event that a superior solution is found, this replaces the previous one. Finally having completed a number of iterations in which there has not been a replacement for the best solution, a perturbation operator which is described in section 5.4 is used. The general algorithm scheme is detailed in figure 2. In the following subsections we will explain the detail of the operators that compose KMTA. 5.1. Initialization The generation of a new solution starts with the selection of a column randomly. It is then queried whether the current solution covers all rows. In case this does not happen, the Heuristic Operator 5.5 is invoked to add a new column, This operation is performed until all rows are covered. Finally it is 16
Figure 2: Flowchart of K-means transition algorithm.
17
verified if there are columns that all their rows are covered by more than one column, in this case proceed to delete the column. The solution initialization 400
procedure is shown in Algorithm 1. Algorithm 1 Initialization Algorithm 1: Function Initialization() 2:
Input
3:
Output Initialized solution Sout
4:
S ← SelecRandomColumn()
5:
while All row are not covered do
6:
S.append(Heuristic(S))
7:
end while
8:
S ← deleteRepeatedItem(S)
9:
Sout ← S
10:
return Sout
5.2. K-means transition operator Considering that our algorithm to binarize is a continuous swarm intelligence metaheuristic. Due to its iterative nature, it needs to update the position of solutions at each iteration. When the metaheuristic is continuous, this update is performed in Rn space. In Equation 5, the position update is presented in a general form. The x(t + 1) variable represents the x position of the solution at time t+1. This position is obtained from the position x at time t plus a ∆ function calculated at time t+1. The function ∆ is proper to each metaheuristic and produces values in Rn . For example in Cuckoo Search ∆(x(t)) = α ⊕ Levy(λ)(x), in Black Hole ∆(x(t)) = rand × (xbh (t) − x(t)) and in the Firefly, Bat and PSO algorithms ∆ can be written in simplified form as ∆(x(t)) = v(x(t)).
18
Algorithm 2 K-means transition Algorithm 1: Function K-meansTransition(ListX(t),M H) 2:
Input List solutions t (ListX(t)), M etaheuristic(M H)
3:
Output List solution t + 1 (ListX(t + 1))
4:
∆i List ← get∆i (ListX(t), M H)
5:
X i Groups ← K-means(∆i List, K)
6:
for X(t) in ListX(t) do
7:
for X i (t) in X(t) do
8:
X i Group ← getX i Group(i, X(t), X i Groups)
9:
Ptr (X i (t)) ← getTransitionProbability(X i Group)
10:
X i (t + 1) ← applyTransitionRule(M H,Ptr (X i (t)))
11:
end for
12:
end for
13:
for X(t + 1) in ListX(t + 1) do
14:
X(t + 1) ← Repair(X(t + 1))
15:
end for
16:
return ListX(t + 1)
x(t + 1) = x(t) + ∆(x(t))
(5)
In the K-means transition operator, we consider the movements generated by the metaheuristic in each dimension for all solutions. ∆i (x(t)) corresponds to the magnitude of the displacement ∆(x(t)) in the i-th position for the solution x at time t. Subsequently these displacement are grouped using abs(∆i (x(t))), the magnitude of the displacement. This grouping is done using the K-means technique where K represents the number of clusters used. Finally, a generic Ptr function given in Equation 6 is proposed to assign a
19
420
transition probability. Ptr : Z/kZ → [0, 1]
(6)
A transition probability through the function Ptr is assigned to each group. Naturally, this Ptr function is modelled as a cumulative probability function. For the case of this study as first approximation, we use the linear function given in Equation 7. Then a second adjustment described in section 6.2 will be performed to transition probability values of each cluster. In Equation 7, N(xi ) indicates the location of the group to which ∆i (x) belongs. The α coefficient, corresponds to the transition probability. The α is estimated in each metaheuristic. For our particular case, N(xi ) = 0 corresponds to elements belonging to the group that has the lowest ∆i values and therefore smaller transition probabilities will be assigned to them. Ptr (xi ) = Ptr (N (xi )) = α + N (xi )α
(7)
The K-means transition operator is described in Algorithm 2, and an illustration of the binarization process is shown in Figure 3. The K-means transition operator starts calculating ∆i for each solution. This step is specific to each metaheuristic. Subsequently the solutions are grouped using K-means clustering technique and the magnitude of ∆i as distance. With the group assigned to each solution we obtain the probability of transition using Equation 7. Afterwards the transition of each solution is performed. In the case of Cuckoo search the rule 9 is used to perform the transition, where xˆi is the complement of xi . For the Black Hole the rule 8 is used, where xibh is 440
the position of the best solution obtained after the last perturbation. Finally, each solution is repaired using the repair operator shown in algorithm 3. xi (t + 1) :=
( xibh (t), if rand < Ptr (xi ) xi (t), 20
otherwise
(8)
Figure 3: Mapping the continuous search space to a discrete search space.
xi (t + 1) :=
( xˆi (t), if rand < Ptr (xi ) xi (t),
(9)
otherwise
5.3. Repair Operator The repair operator aims to repair solutions generated by K-means transition and Perturbation operators. To perform the repairing process, the operator has as input parameter the solution Sin to repair and as output 21
parameter the repaired solution Sout . For the execution of the process, we iteratively use the Heuristic operator which indicates the column that must be added. Once all the rows are covered, we apply the deletion of columns that have all their rows covered by other columns. Algorithm 3 Repair Algorithm 1: Function Repair(Sin ) 2:
Input Input solution Sin
3:
Output The Repair solution Sout
4:
S ← Sin
5:
while needRepair(S) == True do
6:
S.append(Heuristic(S))
7:
end while
8:
S ← repeatedItem(S)
9:
Sout ← S
10:
return Sout
5.4. Perturbation Operator The K-means transition operator is responsible for performing the movements to find the optimum. However our algorithm can get trapped in to a local optimum. To escape from this local deep optimum, the K-means transition operator is complemented by a perturbation operator. This perturbation operator makes ην random deletions. Later the perturbed solution is completed using the repair operator. The number ην is obtained by considering the total length of the solution and multiplying by the factor ν. This factor ν is a parameter of the algorithm and must be estimated. This parameter controls the strength of the perturbation. This perturbation is applied 460
to the xbest and to the list of feasible solutions. The procedure is outlined in algorithm 4. 22
Algorithm 4 Perturbation Algorithm 1: Function Perturbation(Sin , ην ) 2:
Input Input solution Sin , strength of perturbation ην
3:
Output The perturbed solution Sout
4:
S ← Sin
5:
for i=1 to ην do
6:
Randomly remove a element of S
7:
end for
8:
Sout ← RepairOperator(S)
9:
return Sout
5.5. Heuristic Operator The Heuristic operator, aims to select a new column for the cases that a solution needs to be built or repaired. As input parameter the operator considers the solution Sin which needs to be completed. In the case of being a new solution Sin = ∅. With the list of columns belonging to Sin , we get the set of rows R not covered by the solution. With the set of rows not covered and using Equation 10 we obtain in line 4 the best 10 rows to be covered. With this list of rows (listRows) on line 5 we obtain the list of the best columns according to the heuristic indicated in Equation 11. Finally randomly in line 6 we obtain the column to incorporate.
W eightRow(i) =
W eightColumn(j) =
1 , Where Li is the sum of all ones in row i Li
(10)
cj , Where Mj is the set of rows covered by Col j |R ∩ Mj | (11)
23
Algorithm 5 Heuristic operator 1: Function Heuristic(Sin ) 2:
Input Input solution Sin
3:
Output The new column Cout
4:
listRows ← getBestRows(Sin , N =10)
5:
listcolumsnOut ← getBestColumns(ListRows, M =5)
6:
columnOut ← getColumn(listcolumnsOut)
7:
return columnOut
6. Results In this section, we present computational experiments with the proposed KMTA. For the execution of the instances we use a PC with windows 10, Intel Core i7-4770 processor with 16GB in RAM, and programmed in Python 2.7. Experiments were developed to analyze the key ingredients of our algorithm. To perform the statistical analysis in this study, the non-parametric test of wilcoxon signed-rank test, violin charts, and Page’s trend test are used. The analysis is performed by comparing the dispersion, median and interquartile 480
range of distributions. Finally, comparisons were made with the best binarizations that solve the SCP in the last years. A Binary Teaching-Learning Based Optimization Lu and Vasko (2015), Binary Black Hole (BBH) Soto et al. (2017), Binary Cuckoo Search BCS Soto et al. (2017), and Jumping Particle Swarm Optimization (JPSO) method Balaji and Revathi (2016) were selected. Benchmark Instances: We have performed an experimental evaluation of the proposed approaches on a set of 65 different well-known instances, organized in 11 sets from the Beasleys OR-library. Table 1 describes instance set, number of rows or constraints (m), number of columns or variables (n), range of costs and density (percentage of non-zeroes in the matrix).
24
Table 1: Instances from the Beasley’s OR-Library.
Instance
Cost
Density
range
(%)
1000
[1,100]
2
200
2000
[1,100]
2
6
200
1000
[1,100]
5
A
300
3000
[1,100]
2
B
300
3000
[1,100]
5
C
400
4000
[1,100]
2
D
400
4000
[1,100]
5
E
500
5000
[1,100]
10
F
500
5000
[1,100]
20
G
1000
10000
[1,100]
2
H
1000
10000
[1,100]
5
m
n
4
200
5
set
25
6.1. Parameter Setting As a starting point, we describe the methodology used to perform the estimation of parameters for each of the metaheuristics used. The parameters setting are shown in Tables 2 and 3. The Value column indicates the final value used by the parameter. The Range column indicates the scanned values to obtain the final setting. To perform the scan settings, the first problem of each instance was chosen. In each problem and configuration, the KMTA algorithm was executed 10 times for each metaheuristics and combination. Four measures were defined for the setting selection of the algorithm. 1. The percentage deviation of the best value obtained in the ten executions compared with the best known value, see Equation 12. bSolution = 1 −
KnownBestV alue − BestV alue KnownBestV alue
(12)
2. The percentage deviation of the worst value obtained in the ten executions compared with the best known value, see Equation 13. wSolution = 1 −
KnownBestV alue − W orstV alue KnownBestV alue
(13)
3. The percentage deviation of the average value obtained in the ten executions compared with the best known value, see Equation 14. aSolution = 1 −
KnownBestV alue − AverageV alue KnownBestV alue
(14)
4. The convergence time for the best value in each experiment normalized according to Equation 15. nT ime = 1 − 500
convergenceT ime − minT ime maxT ime − minT ime
(15)
Because we have four distinct measures, we used the area of the radar charts to evaluate the best performance configuration. Radar charts, are widely used in data mining and bioinformatic Thaker et al. (2016); Albo 26
Table 2: Parameter setting for Black Hole Algorithm
Parameters
Description
Value
Range
α
Transition
0.1
[0.08, 0.1, 0.12]
20%
[20, 25, 30]
15
[15, 20, 25]
5
[4,5,6,7]
700
[600,700,800]
probability coefficient ν
Coefficient for the perturbation operator
N
Number of particles
G
Number of transition groups
Iteration Number
Maximum iterations
27
Table 3: Parameter setting for Cuckoo Search Algorithm
Parameters
Description
Value
Range
α
Transition
0.1
[0.08, 0.1, 0.12]
20%
[15, 20, 25]
probability coefficient ν
Coefficient for the perturbation operator
N
Number of Nest
20
[20, 25, 30]
G
Number of
5
[4,5,6]
transition groups γ
Step Length
0.01
[0.009,0.01,0.011]
κ
Levy
1.5
[1.4,1.5,1.6]
700
[600,700,800]
distribution parameter Iteration Number
Maximum iterations
28
Figure 4: Radar Chart Examples for the Black Hole configuration.
et al. (2016). Each axis of the chart corresponds to one of the measures defined above. These measures take values between 0 and 1. Where 1 is the best value that can be obtained. Therefore the comparison between the different configurations is the area that contains the results of the four measures. The larger the area, the associated configuration performs better. In Figure 4 the four best configuration results are shown as an example for the KMTA-BH algorithm. 6.2. Insight of binarization quality and adjustment of transition probabilities This section aims to develop a methodology that allows to evaluate the quality of the binarization and to fine-tune transition probability estimates of
29
the different groups generated by K-means. The methodology is illustrated for the metaheuristic Cuckoo Search. From section 6.1 and also suggested by previous work Garc´ıa et al. (2017b), the Equation 7 was used with α = 0.1 to assign the transition probabilities. Two tests were defined: Comparison of distributions and Page’s trend, with the aim of understanding, evaluating and tuning effect of K-means unsupervised learning layer in the binarization process. The Comparison of distributions test aims to evaluate how the 520
binarization mechanism changes distribution of velocities from continuous space to binary space. Intuitively having a similarity between distribution of velocities in the continuous and discrete space, metaheuristics maintains its Diversification-Intensification properties. v − vmin (16) vmax − vmin As a starting point, the behavior of solution velocities in the continuous Nv =
and discrete space was studied and compared. To carry out this study, three configurations were defined: 1. Original: Corresponds to use the transition probabilities suggested by Equation 7 and that take values {0.1, 0.2, 0.3, 0.4, 0.5}. 2. Diversification: Put greater emphasis on the diversification of the system, modifying the probabilities to the following values {0.1, 0.2, 0.5, 0.5, 0.5}. 3. Intensification-Diversification: Putting additional emphasis on intensification and diversification. The system considers the following values {0.1, 0.1, 0.5, 0.5, 0.5}. Subsequently for each configuration, the solution velocities were collected in all iterations for both spaces, and then these were normalized using Equation 16. Results are shown in Figure 5 and Table 4. When analyzing the
30
figure 5, it is observed for the Initial configuration there is an important difference between continuous and discrete space. When comparing the value of 540
medians and interquartile ranges, these also have important variations. For the Diversification configuration the distributions, medians and interquartile ranges have greater similarity. However the configuration that presents the greatest similarities corresponds to Diversification-Intensification configuration. When we analyze the results of the three configurations shown in Table 4 against test instances defined in section 6.1, similar results are observed for best and average indicators in all three configurations. When we applied the Wilcoxon signed rank test to evaluate the significance of the differences in the results of the three configurations, the test indicates there is no significance. The values are shown in Table 4. The significance comparison was performed between the initial configuration and the proposed new configurations. For a deeper understanding of previous result, the page’s trend test was used. When only final results are analyzed as is the case of the Wilcoxon test, often interesting conclusions are hidden especially when the results are similar as in our case. The page’s trend test allows to compare the algorithms by analyzing their intermediate results. This test was used by Derrac et al. Derrac et al. (2014) for the analysis of evolutionary algorithms. Page’s trend test for ordered alternatives Page (1963) can be classified in the family of tests for association in multiple classifications. This test defines
560
the null hypothesis as the equality between the k treatments analyzed that can be rejected in favor of an ordered alternative (the ordered alternative is the main difference of this test with respect to the Friedman test, which only defines the alternative hypothesis as the existence of differences between treatments). The ordered alternative must be defined by the practitioner before starting the analysis. An order between the k treatments has to be provided, and it should reflect the expected order for the populations. Hence, 31
Figure 5: Comparison at continuous and discrete spaces of the normalized velocity distributions.
32
Table 4: Results for Original, Diversification and Intensification-Diversification Cases.
Instance
Original
Diversification
Intensi-Diversi
set
best
average
best
average
best
average
4.1
430
430.2
430
430.2
430
430.1
5.1
253
253
253
253
253
253
6.1
138
138
138
138
138
138
A.1
254
254.2
254
254.2
254
254.1
B.1
69
69
69
69
69
69
C.1
229
229.2
229
229.1
229
229.1
D.1
60
60
60
60
60
60
E.1
29
29
29
29
29
29
F.1
14
14
14
14
14
14
G.1
176
176.4
176
176.3
176
176.4
H.1
64
64
64
64
64
64
average
156
156.09
156
156.07
156
156.06
p-value
0.16
33
0.08
the treatments measures should be numbered from 1 to k, where treatment 1 has the smallest sum of ranks, and treatment k has the largest. Once such an order and the data are provided, the n samples can be ranked from the best to the worst, giving a rank of 1 to the best measure in the sample, a rank of 2 to the second, and a rank of k to the worst. If there are ties for a given sample, average ranks can be assigned. If the data is consistent with the initial ordering defined, then the sum of ranks values for each of the treatments will follow in increasing order. After obtaining the ranks, the Page’s L statistic can be computed using the expression shown in Equation 17.
L=
k X
jRj , where Rj =
Pn
i=1 ri
(17)
i=1
And rji i is the rank of the j-th of k measures on the i-th of n samples. In the case that larger values are required, a normal approximation should be considered. The normal approximation for the L statistic is given by the expression shown in Equation 18. Z=
12(L − 0.5) − 3N k(k + 1)2 p k(k + 1) N (k − 1)
(18)
The input data of the test corresponds to the differences between the two conditions (Original and Intensification-Diversification) recorded at 10 points (ci ) every 70 iterations for the cuckoo search binarization and 11 prob580
lem instances (4.1, 5.1, ...,H.1). For the calculation of the ranking it must be considered if any of the algorithms reaches the maximum before the end. The results of the page’s trend test are shown in Table 5. In this Table we computed Rj for the (Intesification-Diversification) - (Original) differences in fitness values. Note that ranks are assigned from 1 (greater absolute differences) to 10 (lower absolute differences), and midranks are assigned when necessary (hence, the sum of all the Rj values will always be 55). In 34
Table 5: Comparison of ranks for Page’s trend test A -B, where A = IntensificationDiversification and B = Original. Instance
c1
c2
c3
c4
c5
c6
c7
c8
c9
c10
4.1
1
2
3
4
5.5
5.5
7
8
9
10
5.1
1
3
2
5
4
6
7
8
9
10
6.1
2
1
4
3
5
7
6
8
9
10
A.1
1
2
4
3
5
6.5
6.5
8
9
10
B.1
1
3
2
4
5
6
7
8
9
10
C.1
1
2
3
5
4
7
6
8
9
10
D.1
2
1
3
4
5
6
7
8
9
10
E.1
1
3
3
4
5
6.5
6.5
8
9
10
F.1
1
2
3
5
4
7
6
8
9
10
G.1
1
2
3
5
4
6
7
8
9.5
9.5
H.1
1
2
3
4
5
6
7
8
9
10
Sum
13
23
33
46
51.5
69.5
73
88
99.5
109,5
L
4.222
p-value
< 0.01
the result we can consistently see increasing trends in the comparison of Intensification-Diversification configuration with respect to original configuration, the p-value indicates that this increase is significant. Because in Table 4 there is no significant difference between the results of the different configurations, we will analyze the convergence of the different configurations. Since minimum value for each instance is different and with the goal of simplifying the analysis, normalization shown in Equation 19 was performed. SolutionV alue − BestKnown (19) BestKnown For each instance in every ci , we calculate % -Gap. In the figure 6 the dif% − Gap = 100
35
ference between % -Gap of Intensification-Diversification configuration and % -Gap of initial configuration is shown. Figure 6 shows a clear superiority of Intensification-diversification configuration above Initial configuration (Note all values are negative). We also observe in the case of the IntensificationDiversification configuration, best value is reached in all cases before 500 600
iterations. Finally in Table 6 we generate results for the three configurations considering 500 iterations. In this Table when applying the test of Wilcoxon it is seen that when using 500 iterations there is a significant difference between the different configurations. If we only consider the intensificationdiversification configuration and we apply the Wilcoxon test for the cases 500 and 700 iterations, we see there is no significant difference between the results. The case of Black Hole metaheuristic obtains results equivalent to those presented in this section. Finally the configuration used in the following analyzes corresponds to intensification-diversification using 500 iterations. 6.3. KMTA operators analysis Using the configuration obtained in sections 6.1 and 6.2, in this section we will analyze the contribution of Perturbation and K-means transition operators to the effectiveness and efficiency of the algorithm. To compare the distributions of the results of the different experiments we use violin Chart. The horizontal axis corresponds to the problems. While Y axis uses the measure % - Gap defined in Equation 19. Furthermore, a non-parametric test, Wilcoxon signed-rank test is carried out to determine if the results of KMTA with respect to other algorithms have significant difference or not. The results were obtained by performing 30 experiments per instance using cuckoo metaheuristic.
620
6.3.1. Evaluation of K-means transition operator: To evaluate the contribution of the K-means transition operator to the final result, we designed a random operator. This random operator replaces 36
Figure 6: Convergence chart. Comparison between Intensification-Diversification and Original configurations.
37
Table 6: Results for Original, Diversification and Intensification-Diversification Cases using 500 iterations.
Instance
Original
Diversification
Intensi-Diversi
set
best
average
best
average
best
average
4.1
430
430.2
430
430.2
430
430.2
5.1
253
253
253
253
253
253
6.1
138
138
138
138
138
138
A.1
254
254.3
254
254.2
254
254.1
B.1
69
69
69
69
69
69
C.1
229
229.2
229
229.2
229
229.1
D.1
60
60
60
60
60
60
E.1
29
29
29
29
29
29
F.1
14
14
14
14
14
14
G.1
180
180.6
176
178.3
176
176.6
H.1
64
65.2
64
64.8
64
64
average
156.4
156.6
156.18
156.26
156
156.06
p-value
0.0034
38
0.0078
Figure 7: Evaluation of K-means transition operator with perturbation operator.
the clustering learning layer with an operator which assigns a fixed transition probability of 0.5 without considering the value of the solution velocity module in each dimension. In the evaluation we consider two cases. The first case the perturbation operator is included. In the second case perturbation operator is not included. KMTA corresponds to our standard algorithm. 05.pe is the random variant that includes the perturbation operator. KMTA-WPE corresponds to the algorithm with K-means operator without including the perturbation operator. Finally 05.wpe is the random variant that does not includes the perturbation operator. When we compared the best values and averages between KMTA and 05.pe algorithms shown in Table 7. KMTA outperforms to 05.pe in all problems. The comparison of distributions is shown in figure 7. We see the dispersion of the 05.pe distributions are bigger than the dispersions of KMTA. 39
Figure 8: Evaluation of K-means transition operator without perturbation operator.
Table 7: K-means transition operator comparison result. Instance
KMTA
KMTA
05.pe
05.pe
KMTA-
KMTA-
05.wpe
05.wpe
(best)
(avg)
(best)
(avg)
WPE
WPE
(best)
(avg)
(best)
(avg)
4.1
430
430.2
432
434.5
430
431.4
431
435.2
5.1
253
253
254
255.6
253
253.6
253
256.1
6.1
138
138
139
140.2
138
139.2
139
141.5
A.1
254
254.1
256
258.2
254
255.5
256
257.9
B.1
69
69
71
72.3
69
70.1
71
72.8
C.1
229
229.1
231
232.8
229
230.2
232
233.5
D.1
60
60
63
64.3
60
61.4
62
65.2
E.1
29
29
31
32.8
29
29.5
31
32.3
F.1
14
14
15
16.1
14
14.5
15
16.8
G.1
176
176.6
183
185.8
176
178.2
185
191.1
H.1
64
64
69
72.6
64
64.8
70
73.8
156.00
156.09
158.55
160.47
156.00
157.13
158.64
Average p-value
1.5e-08
40
161.47 1.2e-07
In particular, the dispersion of distributions increases in the case of larger problems. When the interquartile ranges are compared, it is observed that the KMTA ranges are closer to 0 than those of 05.pe. All this suggests that 640
the K-means transition operator together with perturbation operator, contribute to improve the precision and the quality of the results. When we evaluate the behaviour of the algorithms through the Wilcoxon test, this indicates that there is a significant difference between the two algorithms. The result comparison of KMTA-WPE and 05.wpe algorithms are shown in Table 7 and in Figure 8. From Table 7 we see a similar result to the previous case, the algorithm KMTA-WPE outperform in all the problems to 05.wpe algorithm. When we analyze the interquartile and median values of violin charts, we see that K-means transition operator improves both results and dispersion obtained in all problems. The Wilcoxon test again indicates that the difference is significant. These results indicate that the unsupervised learning layer implemented through K-means clustering has a relevant impact on improving the quality and dispersion of the results obtained. 6.3.2. Evaluation of Perturbation operator: To evaluate the contribution of the Perturbation operator, we designed an experiment that compares the results of our original KMTA algorithm with respect to the same algorithm but without using the perturbation operator KMTA-WPE. The results are shown in Table 7 and figure 9. When we compare the results of the Table, we see the best values in both cases are the same, however in the averages there is a better performance of KMTA com-
660
pared to KMTA-WPE. When we perform the analysis of the values obtained from violin chart, we see that the perturbation operator helps to reduce the dispersion of the data especially in the larger problems. The effect on dispersion is much lower than the Figures 7 and 8, however the Wilcoxon test
41
Figure 9: Evaluation of Perturbation operator.
indicates that the contribution of the perturbation operator is significant. 6.4. KMTA v/s Transfer Functions This section aims to evaluate KMTA against another general binarization mechanism. In order to carry out the evaluation, the Transfer Functions (section 5.1 and 6.2) were chosen due to their wide use, ease of implementation and results obtained. In the experimental design the K-means transition operator is replaced by an operator using Transfer Functions, see Figure 10. In order to perform the verification, the metaheuristics Cuckoo Search and Black Hole were binarized, generating, in case of the clustering technique, the KMTA-CS and KMTA-BH binarizations. In the Transfer Functions, it was considered as maximum number of iterations 500 and 2000, obtaining the binarizations TF-CS-500, TF-CS-2000, TF-BH-500, TF-BH-2000. Two 42
Figure 10: Transfer Function transition algorithm.
families of functions were evaluated. The S-shape family
1 d
eτ xi +1
and V-shape
d
family
eτ |xi | −1 d eτ |xi | +1
. The Transfer Function and the τ parameter were determined
following the methodology described in the 6.1 and 6.2 sections. In both d
metaheuristics 680
eτ |xi | −1 d eτ |xi | +1
the family was used. The value τ for metaheuristic
Black Hole was 1.5 and the case of metaheuristic Cuckoo Search was 2.5. From Tables 8, 9 and Figure 11 it is observed that KMTA-BH and KMTACS outperform the Transfer Functions binarizations in both 500 and 2000 iterations. For the case of 500 the difference is much more notorious indicating that the Transfer Functions are slower in their convergence than KMTA. Analyzing execution times, TF-500 algorithms are the ones that take less time closely followed by both KMTA binarizations. Finally the execution time of TF-2000 versions is quite longer. When we analyze the violin chart in Figure 12 and 13 we observe that quality and dispersion of solutions for 43
Table 8: K-means transition operator Black Hole comparison.
Instance
Best
KMTA-
time
TF-
time
TF-
time
Known
BH
(s)
BH-500
(s)
BH-
(s)
(avg)
(avg)
2000 (avg)
4
510
510.8
7.3
515.2
5.9
511.3
22.4
5
257.3
257.4
7.8
261.5
6.1
257.7
22.6
6
144.2
144.4
8.1
149.4
6.7
146.2
24.9
A
241.4
242.6
7.9
247.5
6.4
244.0
28.2
B
75.2
75.2
10.6
77.9
9.6
75.8
34.5
C
224.6
225.7
11.1
230.4
9.8
226.5
35.1
D
64.2
64.6
16.2
68.7
13.9
65,36
55.3
E
28.4
28,5
16.9
32,14
14.6
30,34
56.8
F
14
14.1
16.8
15.8
55.1
15.1
55.1
G
166.4
167.8
98.6
173.6
76.9
169.7
301.3
H
59.4
60.9
96.4
64.9
78.3
62.3
286.9
162.28
162.91
27.06
167.00
25.75
164.03
83.92
Average p-value
2.7e-10
44
3.1e-06
Table 9: K-means transition operator Cuckoo Search comparison.
Instance
Best
KMTA-
time
TF-CS-
time
TF-CS-
time
Known
CS
(s)
500
(s)
2000
(s)
(avg)
(avg)
(avg)
4
510
510.8
8.0
515.5
6.2
511.4
22.9
5
257.3
257.4
7.9
261.5
6.3
257.7
23.5
6
144.2
144.4
8.3
149.3
7.3
146,1
26.4
A
241.4
242.5
8.5
247,58
6.6
243.9
28.9
B
75.2
75,2
11.2
78.2
9.9
75.7
35.3
C
224.6
225.6
11.5
230.3
9.9
226.3
35.5
D
64.2
64.5
17.1
69.0
14.3
65.2
56.9
E
28.4
28.5
17.3
32.2
15.1
30.1
58,4
F
14
14.1
17.0
15.6
13.7
14.8
58.7
G
166.4
167.8
98.5
173.5
80.6
169,7
324.3
H
59.4
60.8
101.6
65.0
82.6
62.3
286.4
162.28
162.87
27.90
167.06
22.95
163.93
87.02
Average p-value
3.5e-10
45
2.4e-06
Figure 11: KMTA V/S Transfer Functions.
46
Figure 12: Comparison between KMTA-BH and TF-BH-500.
the case of binarizations with KMTA, they are superior to binarizations that use Transfer Functions. This difference increases in problems E, F, G and H which correspond to larger problems. When performing the Wilcoxon test, this indicates that differences between KMTA and TF algorithms are significant. Additionally, the page’s trend test was executed in order to see if the differences were significant during the execution of the algorithms. For this we compared KMTA-BH with TF-BH-500. For the generation of the data the values were captured every 50 iterations for all the evaluated problems. The value obtained L = 4235, indicates that the differences are significant throughout the execution.
47
Figure 13: Comparison between KMTA-BH and TF-BH-2000.
48
6.5. KMTA Comparisons 700
This section aims to evaluate our algorithm with respect to the state of the art of metaheuristics that have solved the set covering problem. The evaluation intends to have a reference of clustering approach performance against other binarization mechanisms. Three algorithms were selected. The first was TLBO-MOT-SCP algorithm. This algorithm corresponds to a specific binarization applied to Teaching-Learning Based Optimization Lu and Vasko (2015). The second algorithm corresponds to a PSO modification called Jumping Particle Swarm Optimization method (JPSO) Balaji and Revathi (2016). The third algorithm uses Transfer Functions as a generic binarization mechanism and applies it to the metaheuristics Cuckoo Search and Black Hole Soto et al. (2017). In order to carry out the comparison, the larger problems E, F, G and H of OR-library were used. The TLBO-MOT-SCP algorithm, adapts in a specific way the principles of the Teaching Learning metaheuristic which works naturally in a continuous space, to solve combinatorial problems such as set covering. Binarization adapts the iteration mechanism of metaheuristic with the purpose of generating values in the set {0,1}. TLBO-MOT-SCP was coded in C++ and executed on a Dell Optiplex 745 with an Intel Core 2 CPU 6600 2.40GH and 4 GB of memory and operating system CentOS Linux 7 64bit. For the case of JPSO, the binarization corresponds to a specific method
720
for PSO. We define 4 attractors xi , bi , gi , g ∗ and the new iteration is done using Equation 20. For the case of the set covering problem, ci = 0.25 was defined, but in general they must fulfil the condition 1 = c1 + c2 + c3 + c4 . A random number is used in the interval [0, 1] and depending on the value obtained one of the four attractors is selected. Subsequently through the jump algorithm (JPSO) a new solution is generated using the selected attractor. In addition, the jump algorithm is complemented by a local search
49
operator. xk+1 = c1 xki + c2 bi + c3 gi + c4 g ∗ i
(20)
For algorithm execution, an Intel Pentium Core2 Duo Processor PC having 1.6 GHz CPU and 1 GB RAM. All the procedures of the JPSO algorithm have been coded and implemented in MATLAB. Finally, the binarized Cuckoo search (BCS) and Black Hole (BBH) algorithms used Transfer Functions. The Transfer Functions described in the 3 section correspond to a general binarization mechanism. In particular to perform the binarization the function case and the function
1 d
1+e−xi (t+1)
1 −xd (t+1) i 3 1+e
was used for the Black Hole
in the case of Cuckoo Search. The maximum
number of iterations in each case was 4000. The implementation of both algorithms has been done in JavaTM and experiments have been launched on a 3.0 GHz Quad Core with 8 GB RAM machine running MS Windows 7 Ultimate. 740
Results obtained by all the six algorithms are shown in Table 10, in which first and second column represents instances name and their corresponding best known solution of optimum objective function value, correspondingly the instances where the proposed and compared algorithms reaches the optimal solution are highlighted by bold numbers in their columns. Best and avg represent the best and the average value obtained by the objective function. In the case of JPSO algorithm only the averages were reported. The algorithm JPSO and KMTA-BH were the ones that obtained the highest number of maximums with 14 of a total of 20. Then appears KMTA-CS with 13, the next was TLBO-MOT-SCP with 10 and finally BCS and BBH obtained five optimal. When we analyze the averages of the general binarization methods (KMTA and Transfer Function), KMTA has the average values close to the optimum where the maximum deviation is for the H group with 2 %, The
50
Table 10: OR-Library benchmarks E, F, G, H. Instance
Best
TLBO-
JPSO
BCS
BCS
BBH
BBH
Known
MOT-
(avg)
(best)
(avg)
(best)
(avg)
KMTACS
KMTACS(avg)
(best)
SCP
KMTA-
KMTA-
BH
CS
(best)
(avg)
29
(best) E.1
29
29
29
29
30
29
30
29
29
29
E.2
30
30
30
31
32
31
31
30
30
30
30
E.3
27
28
27
28
29
28
28
27
27.6
27
27
E.4
28
28
28
30
31
29
29
28
28
28
28.3
E.5
28
28
28
28
30
28
28
28
28
28
28.2
F.1
14
14
14
14
14
14
15
14
14
14
14
F.2
15
15
15
15
17
15
16
15
15
15
15
F.3
14
14
14
15
16
16
16
14
14
14
14
F.4
14
15
14
15
15
15
16
14
14
14
14
F.5
13
13
13
14
15
14
15
13
13.4
13
13.4
G.1
176
179
176
176
177
179
181
176
176.6
176
176.3
G.2
154
156
155
156
157
158
160
155
155.8
155
155.4
G.3
166
168
167.2
169
170
169
169
167
168.1
166
167.6
G.4
168
172
168.2
170
171
170
171
170
170.7
170
170.9
G.5
168
168
168
170
171
168
169.1
168
168.4
168
168.6
H.1
63
64
64
64
64
66
67
64
64
64
64.2
H.2
63
64
63
64
64
67
68
64
64.7
64
64.5
H.3
59
61
59.2
61
63
65
65
60
60.4
60
60.7
H.4
58
59
58.3
59
60
63
64
59
59.8
59
59.5
H.5
55
56
55
56
57
62
62
55
55.2
55
55.4
67.1
68.05
67.3
68.2
69.2
69.3
70.0
67.5
67.8
67.45
67.8
Average
51
Figure 14: KMTA comparisons.
52
Table 11: Wilcoxon test comparison result. 0 corresponds to no significance. KMTA-BH
KMTA-CS
JPSO
TLBO
BBH
BCS
KMTA-BH
0
0
1
1
1
1
KMTA-CS
0
0
1
1
1
1
results are shown in Figure 14. The quality and precision of the results are justified by the non-supervized learning layer of the K-means transition operator shown in Figure 8 n addition to the contribution of the perturbation operator shown in Figure 9. In the case of Transfer Functions their results are far from optimal, especially in problems G, H and E, see figure 14. When we analyzed the significance of the differences between the algorithms using the Wilcoxon test, we see in Table 11, the best performance 760
was for the JPSO algorithm with a significant difference regarding KMTABH and KMTA-CS. Subsequently KMTA-BH and KMTA-CS had significant differences regarding TLBO-MOT-SCP, BBH and BCS. 7. The crew scheduling problem This section aims to apply the proposed binarized algorithms to a realworld problem. For this, the crew scheduling problem was chosen. The crew scheduling problem (CSP) is a complex problem to solve because it consists of several stages and each of them with important algorithmic complexity. Today variations of the original problem appear incorporating new restrictions. In Hoffmann et al. (2017), a version of CSP with attendance rate was resolved. A CSP application with fairness preference was studied in J¨ utte et al. (2017). Applications are also found in bus or air transport. For example applications of fleet assignment and crew pairing problem was addressed ¨ in Ozener et al. (2017). Also, studies in public transport have been carried ¨ recently Oztop et al. (2017). In the CSP we start with a timetable of services that must be executed 53
with a certain frequency, for example, days, weeks, etc. In addition, the service must be executed in certain time windows within the period. Each service is divided into a sequence of trips. A trip is characterized by a departure station, a departure time, an arrival station, an arrival time and a crew 780
that delivers the service. When we consider a period of time and a crew, a roster must be executed, i.e a cyclical sequence of travel, where each trip is assigned a cost. The CSP then consists in finding a roster subset that covers all trips, satisfying the constraints imposed and at a minimal cost. The problem is broken down into two phases: 1. Pairing generation: a very large number of possible pairings is generated. A pairing is defined as a sequence of trips which can be assigned to a crew in a short working period. A pairing starts and ends in the same depot and is associated with a cost. 2. Pairing optimization: a selection is made of the best subset of all the generated pairings to guarantee that all the trips are covered at minimum cost. This phase follows quite a general approach, based on the solution of set-covering or set-partitioning problems. In this work we will assume that the pair generation phase has already been performed, therefore, we will focus efforts in resolving the pairing optimization phase. The pairing optimization phase requires the determination of a min-cost subset of the generated pairings covering all the trips and satisfying additional constraints. Usually, it is solved through the set covering problem.
800
To verify our algorithm, 7 datasets associated with real-world crew scheduling problems were used. These datasets come from an application in Italian railways and have been contributed by Paolo Nobili. In Table 12 the datasets and their results are shown. 54
Table 12: Railway crew scheduling problems. Instance
row
col
Density
Best
KMTA-
KMTA-
time
KMTA-
KMTA-
Known
CS
CS
(s)
BH
BH
(best)
(avg)
(best)
(avg)
time(s)
Rail507
507
63009
1.2%
174
175
178.1
176.4
174
177.9
187.4
Rail516
516
47311
1.3%
182
182
185.4
195.3
182
185.6
231.6
Rail582
582
55515
1.2%
211
213
215.5
231.7
214
216.3
256.1
Rail2536
2536
1081841
0.4%
690
691
697.2
1650.4
691
699.1
1879.3
Rail2586
2586
920683
0.4%
944
948
951.3
1456.2
950
950.4
1789.4
Rail4284
4284
1092610
0.2%
1062
1065
1071.7
3679.8
1067
1072.3
3905.1
Rail4872
4872
968672
0.2%
Average
1527
1534
1543.9
4105.2
1536
1545.1
4450.2
684.29
686.86
691.87
1642.14
687.71
692.39
1814.16
From the table, we observed that despite being problems of a much larger size than the previous ones, the performance of the KMTA-CS and KMTABH binarizations was adequate. In the case of KMTA-CS the % - Gap for the best value was 0.38 % and on the average was 1.11 %. In the case of KMTABH, the % - Gap for the best value was 0.50 % and on the average was 1.18 %. When we analyze the results obtained in the last 4 problems, which are on average 20 times larger than the first three with respect to the columns, we observe that the quality of the results obtained by the binarization is maintained. In the case of KMTA-CS the % - Gap for the best value was 0.36 % and for the average indicator, it was 0.97 %. In the case of KMTA-BH for the best value it was 0.50 % and for the average, it was 1.04 %. 8. Conclusion and Future Work In this article, we proposed an algorithm whose main function is to binarize Swarm-Intelligence continuous metaheuristics using an unsupervised K-means learning layer. To evaluate the performance of our algorithm, the set covering problem was used together with Cuckoo Search and Black Hole 820
metaheuristics. A methodology to find the configuration of metaheuristics was developed. Additionally, for the adjustment of K-means learning layer parameters we proposed velocity distribution test which provides informa55
tion on how exploration exploitation properties are translated into the binary space and page’s trend test which tells us if a consistently increasing trend in the ranks is found. The contribution of the different operators of the algorithm was evaluated, finding that K-means transition operator contributes significantly to improve precision and quality of solutions. An evaluation of our algorithm was performed using Transfer Functions. In the evaluation it was found that KMTA obtains better results and convergence than versions with Transfer Functions. Finally, in comparison with state of the art algorithms our algorithm showed a good performance. In the case of the SCP instances, the average deviation from the best known was 0.59 % for KMTA-CS and 0.52 % for KMTA-BH. In the case of the crew scheduling problem, the deviation was 0.38 % for KMTA-CS and 0.50 % for KMTA-BH with respect to the best value. As future work we want to investigate the behavior of other continuous metaheuristics when we generate binary versions through the K-means binarization method and applying these binarizations to other NP-hard combinatorial problems. Additionally, it is interesting to explore the use of machine 840
learning techniques in other stages of the metaheuristic algorithm, such as tuning of parameters, initialization of solutions and local search. In the research topic: Hybrid learning schemes jointly embracing ingredients from swarm/evolutionary computation and machine learning we must continue to obtain velocity space information to improve the decision making of evolutionary algorithms. In this sense an interesting line to develop is to use the velocity space information for autonomous search. 9. References References Ahmad, S. R., Bakar, A. A., Yaakub, M. R., 2015. Metaheuristic algorithms for feature selection in sentiment analysis. In: Science and Information 56
Conference (SAI), 2015. IEEE, pp. 222–226. Albo, Y., Lanir, J., Bak, P., Rafaeli, S., 2016. Off the radar: Comparative evaluation of radial visualization solutions for composite indicators. IEEE transactions on visualization and computer graphics 22 (1), 569–578. Alegrıa, J., T´ upac, Y., 2014. A generalized quantum-inspired evolutionary algorithm for combinatorial optimization problems. In: XXXII International Conference of the Chilean Computer Science Society SCCC, November. pp. 11–15. Alvarez, A., Munari, P., 2017. An exact hybrid method for the vehicle rout860
ing problem with time windows and multiple deliverymen. Computers & Operations Research 83, 1–12. ¨ Asta, S., Ozcan, E., Curtois, T., 2016. A tensor based hyper-heuristic for nurse rostering. Knowledge-based systems 98, 185–199. Balaji, S., Revathi, N., 2016. A new approach for solving set covering problem using jumping particle swarm optimization method. Natural Computing 15 (3), 503–517. Balas, E., Carrera, M. C., 1996. A dynamic subgradient-based branch-andbound procedure for set covering. Operations Research 44 (6), 875–890. Balas, E., Padberg, M. W., 1976. Set partitioning: A survey. SIAM review 18 (4), 710–760. Beasley, J. E., 1987. An algorithm for set covering problem. European Journal of Operational Research 31 (1), 85–93. Beasley, J. E., Chu, P. C., 1996. A genetic algorithm for the set covering problem. European Journal of Operational Research 94 (2), 392–404.
57
Borneman, J., Chrobak, M., Della Vedova, G., Figueroa, A., Jiang, T., 2001. Probe selection algorithms with applications in the analysis of microbial communities. Bioinformatics 17 (suppl 1), S39–S48. Boros, E., Hammer, P. L., Ibaraki, T., Kogan, A., 1997. Logical analysis of numerical data. Mathematical Programming 79 (1-3), 163–190. 880
Brusco, M. J., Jacobs, L. W., Thompson, G. M., 1999. A morphing procedure to supplement a simulated annealing heuristic for cost-andcoveragecorrelated set-covering problems. Annals of Operations Research 86, 611– 627. Cadenas, J. M., Garrido, M. C., Mu˜ noz, E., 2009. Using machine learning in a cooperative hybrid parallel strategy of metaheuristics. Information Sciences 179 (19), 3255–3267. Canca, D., De-Los-Santos, A., Laporte, G., Mesa, J. A., 2017. An adaptive neighborhood search metaheuristic for the integrated railway rapid transit network design and line planning problem. Computers & Operations Research 78, 1–14. Caprara, A., Fischetti, M., Toth, P., 1999. A heuristic method for the set covering problem. Operations research 47 (5), 730–743. Chandrasekaran, K., Simon, S. P., 2012. Network and reliability constrained unit commitment problem using binary real coded firefly algorithm. International Journal of Electrical Power & Energy Systems 43 (1), 921–932. Chen, E., Li, J., Liu, X., 2011. In search of the essential binary discrete particle swarm. Applied Soft Computing 11 (3), 3260–3269. Chou, J.-S., Ngo, N.-T., Pham, A.-D., 2015. Shear strength prediction in reinforced concrete deep beams using nature-inspired metaheuristic sup58
900
port vector regression. Journal of Computing in Civil Engineering 30 (1), 04015002. Chou, J.-S., Nguyen, T.-K., 2018. Forward forecast of stock price using sliding-window metaheuristic-optimized machine learning regression. IEEE Transactions on Industrial Informatics. Chou, J.-S., Pham, A.-D., 2017. Nature-inspired metaheuristic optimization in least squares support vector regression for obtaining bridge scour information. Information Sciences 399, 64–80. Chou, J.-S., Thedja, J. P. P., 2016. Metaheuristic optimization within machine learning-based classification system for early warnings related to geotechnical problems. Automation in Construction 68, 65–80. Crawford, B., Soto, R., Astorga, G., Garc´ıa, J., Castro, C., Paredes, F., 2017. Putting continuous metaheuristics to work in binary search spaces. Complexity 2017. Crawford, B., Soto, R., Berr´ıos, N., Johnson, F., Paredes, F., Castro, C., Norero, E., 2015. A binary cat swarm optimization algorithm for the nonunicost set covering problem. Mathematical Problems in Engineering 2015. Crawford, B., Soto, R., Monfroy, E., Paredes, F., Palma, W., 2014a. A hybrid ant algorithm for the set covering problem. International Journal of Physical Sciences 6 (19), 4667–4673.
920
Crawford, B., Soto, R., Olivares-Suarez, M., Palma, W., Paredes, F., Olguin, E., Norero, E., 2014b. A binary coded firefly algorithm that solves the set covering problem. Romanian Journal of Information Science and Technology 17 (3), 252–264.
59
Damaˇseviˇcius, R., Wo´zniak, M., 2017. State flipping based hyper-heuristic for hybridization of nature inspired algorithms. In: International Conference on Artificial Intelligence and Soft Computing. Springer, pp. 337–346. de Alvarenga Rosa, R., Machado, A. M., Ribeiro, G. M., Mauri, G. R., 2016. A mathematical model and a clustering search metaheuristic for planning the helicopter transportation of employees to the production platforms of oil and gas. Computers & Industrial Engineering 101, 303–312. De Jong, K., 2007. Parameter setting in eas: a 30 year perspective. In: Parameter setting in evolutionary algorithms. Springer, pp. 1–18. de Le´on, A. D., Lalla-Ruiz, E., Meli´an-Batista, B., Moreno-Vega, J. M., 2017. A machine learning-based system for berth scheduling at bulk terminals. Expert Systems with Applications 87, 170–182. Derrac, J., Garc´ıa, S., Hui, S., Suganthan, P. N., Herrera, F., 2014. Analyzing convergence performance of evolutionary algorithms: A statistical approach. Information Sciences 289, 41–58. Dey, S., Bhattacharyya, S., Maulik, U., 2015. New quantum inspired meta940
heuristic techniques for multi-level colour image thresholding. Applied Soft Computing 888, 999. Fernandes, S., Setoue, K., Adeli, H., Papa, J., 2017. Fine-tuning enhanced probabilistic neural networks using metaheuristic-driven optimization. In: Bio-Inspired Computation and Applications in Image Processing. Elsevier, pp. 25–45. Fong, S., Wong, R., Vasilakos, A. V., 2016. Accelerated pso swarm search feature selection for data stream mining big data. IEEE transactions on services computing 9 (1), 33–45.
60
Gao, K., Sadollah, A., Zhang, Y., Lentzakis, A., Su, R., 2017. Jaya, wca and hs algorithms for solving large-scale real-life urban traffic light scheduling problem. Swarm and Evolutionary Computation. Garc´ıa, J., Crawford, B., Soto, R., Astorga, G., 2017a. A percentile transition ranking algorithm applied to knapsack problem. In: Proceedings of the Computational Methods in Systems and Software. Springer, pp. 126–138. Garc´ıa, J., Crawford, B., Soto, R., Astorga, G., 2018a. A percentile transition ranking algorithm applied to binarization of continuous swarm intelligence metaheuristics. In: International Conference on Soft Computing and Data Mining. Springer, pp. 3–13. Garc´ıa, J., Crawford, B., Soto, R., Carlos, C., Paredes, F., 2017b. A k-means 960
binarization framework applied to multidimensional knapsack problem. Applied Intelligence, 1–24. Garc´ıa, J., Crawford, B., Soto, R., Castro, C., Paredes, F., 2018b. A k-means binarization framework applied to multidimensional knapsack problem. Applied Intelligence 48 (2), 357–380. Garc´ıa, J., Crawford, B., Soto, R., Garc´ıa, P., 2017c. A multi dynamic binary black hole algorithm applied to set covering problem. In: International Conference on Harmony Search Algorithm. Springer, pp. 42–51. Garfinkel, R. S., Nemhauser, G. L., 1972. Integer programming. Vol. 4. Wiley New York. Gary, M. R., Johnson, D. S., 1979. Computers and intractability. A Guide to the Theory of NP-Completeness. Glover, F., Kochenberger, G., 2003. The ant colony optimization metaheuristic: Algorithms, applications, and advances. Handbook of Metaheuristics, 250–285. 61
Hatamlou, A., 2013. Black hole: A new heuristic optimization approach for data clustering. Information sciences 222, 175–184. Hoffmann, K., Buscher, U., Neufeld, J. S., Tamke, F., 2017. Solving practical railway crew scheduling problems with attendance rates. Business & Information Systems Engineering 59 (3), 147–159. 980
Hota, A. R., Pat, A., 2010. An adaptive quantum-inspired differential evolution algorithm for 0–1 knapsack problem. In: Nature and Biologically Inspired Computing (NaBIC), 2010 Second World Congress on. IEEE, pp. 703–708. Ibrahim, A. A., Mohamed, A., Shareef, H., Ghoshal, S. P., 2011. An effective power quality monitor placement method utilizing quantum-inspired particle swarm optimization. In: Electrical Engineering and Informatics (ICEEI), 2011 International Conference on. IEEE, pp. 1–6. Iqbal, S., Kaykobad, M., Rahman, M. S., 2015. Solving the multi-objective vehicle routing problem with soft time windows with the help of bees. Swarm and evolutionary computation 24, 50–64. J, J. K., Eberhart, R., 1997. A discrete binary version of the particle swarm algorithm. IEEE 4105, 4104–4108. Jana, N. D., Sil, J., Das, S., 2017. Selection of appropriate metaheuristic algorithms for protein structure prediction in ab off-lattice model: a perspective from fitness landscape analysis. Information Sciences 391, 28–64. Jin, Y., Qu, R., Atkin, J., 2014. A population-based incremental learning method for constrained portfolio optimisation. In: Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), 2014 16th International Symposium on. IEEE, pp. 212–219.
62
1000
John, B., 1990. A lagrangian heuristic for set-covering problems. Naval Research Logistics (NRL) 37 (1), 151–164. J¨ utte, S., M¨ uller, D., Thonemann, U. W., 2017. Optimizing railway crew schedules with fairness preferences. Journal of Scheduling 20 (1), 43–55. Karaboga, D., 2005. An idea based on honey bee swarm for numerical optimization. Tech. rep., Technical report-tr06, Erciyes university, engineering faculty, computer engineering department. Kennedy, J., 2011. Particle swarm optimization. In: Encyclopedia of machine learning. Springer, pp. 760–766. Khalil, T., Youseef, H., Aziz, M., 2006. A binary particle swarm optimization for optimal placement and sizing of capacitor banks in radial distribution feeders with distorted substation voltages. Proceedings of AIML international conference, 137–143. Kotthoff, L., 2014. Algorithm selection for combinatorial search problems: A survey. AI Magazine 35 (3), 48–60. Kuo, R., Kuo, P., Chen, Y. R., Zulvia, F. E., 2016a. Application of metaheuristics-based clustering algorithm to item assignment in a synchronized zone order picking system. Applied Soft Computing 46, 143–150. Kuo, R.-J., Mei, C., Zulvia, F. E., Tsai, C., 2016b. An application of a metaheuristic algorithm-based clustering ensemble method to app customer seg-
1020
mentation. Neurocomputing 205, 116–129. Lalwani, S., Kumar, R., Gupta, N., 2016. An efficient two-level swarm intelligence approach for rna secondary structure prediction with bi-objective minimum free energy scores. Swarm and Evolutionary Computation 27, 68–79. 63
Lanza-Gutierrez, J. M., Crawford, B., Soto, R., Berrios, N., Gomez-Pulido, J. A., Paredes, F., 2017. Analyzing the effects of binarization techniques when solving the set covering problem through swarm optimization. Expert Systems with Applications 70, 67–82. Layeb, A., 2011. A novel quantum inspired cuckoo search for knapsack problems. International Journal of Bio-Inspired Computation 3 (5), 297–305. Layeb, A., 2013. A hybrid quantum inspired harmony search algorithm for 0–1 optimization problems. Journal of Computational and Applied Mathematics 253, 14–25. Layeb, A., Boussalia, S. R., 2012. A novel quantum inspired cuckoo search algorithm for bin packing problem. International Journal of Information Technology and Computer Science (IJITCS) 4 (5), 58. Leonard, B. J., Engelbrecht, A. P., Cleghorn, C. W., 2015. Critical considerations on angle modulated particle swarm optimisers. Swarm Intelligence 9 (4), 291–314. 1040
Li, X.-l., Shao, Z.-j., Qian, J.-x., 2002. An optimizing method based on autonomous animats: fish-swarm algorithm. System Engineering Theory and Practice 22 (11), 32–38. Li, Z.-q., Zhang, H.-l., Zheng, J.-h., Dong, M.-j., Xie, Y.-f., Tian, Z.-j., 2011. Heuristic evolutionary approach for weighted circles layout. In: Information and Automation. Springer, pp. 324–331. Liu, W., Liu, L., Cartes, D., 2007. Angle modulated particle swarm optimization based defensive islanding of large scale power systems. IEEE Power Engineering Society Conference and Exposition in Africa, 1–8.
64
Lu, Y., Vasko, F. J., 2015. An or practitioner’s solution approach for the set covering problem. International Journal of Applied Metaheuristic Computing (IJAMC) 6 (4), 1–13. Mann, P. S., Singh, S., 2017. Energy efficient clustering protocol based on improved metaheuristic in wireless sensor networks. Journal of Network and Computer Applications 83, 40–52. Martin, S., Ouelhadj, D., Beullens, P., Ozcan, E., Juan, A. A., Burke, E. K., 2016. A multi-agent based cooperative approach to scheduling and routing. European Journal of Operational Research 254 (1), 169–178. ¨ ¨ Mato˘glu, M. O., ¨ Erdo˘gan, G., Haouari, M., S¨ozer, H., 2017. Ozener, O. O., Solving a large-scale integrated fleet assignment and crew pairing problem. 1060
Annals of Operations Research 253 (1), 477–500. ¨ Oztop, H., Eliiyi, U., Eliiyi, D. T., Kandiller, L., 2017. A bus crew scheduling problem with eligibility constraints and time limitations. Transportation research procedia 22, 222–231. Page, E. B., 1963. Ordered hypotheses for multiple treatments: a significance test for linear ranks. Journal of the American Statistical Association 58 (301), 216–230. Palit, S., Sinha, S. N., Molla, M. A., Khanra, A., Kule, M., 2011. A cryptanalytic attack on the knapsack cryptosystem using binary firefly algorithm. In: Int. Conf. on Computer and Communication Technology (ICCCT). Vol. 2. pp. 428–432. Pampara, G., 2012. Angle modulated population based algorithms to solve binary problems. Ph.D. thesis, University of Pretoria, Pretoria. Pan, W.-T., 2012. A new fruit fly optimization algorithm: taking the financial distress model as an example. Knowledge-Based Systems 26, 69–74. 65
Piotrowski, A. P., Napiorkowski, J. J., 2016. Searching for structural bias in particle swarm optimization and differential evolution algorithms. Swarm Intelligence 10 (4), 307–353. Rashedi, E., Nezamabadi-Pour, H., Saryazdi, S., 2009. Gsa: a gravitational search algorithm. Information sciences 179 (13), 2232–2248. 1080
Ries, J., Beullens, P., 2015. A semi-automated design of instance-based fuzzy parameter tuning for metaheuristics based on decision tree induction. Journal of the Operational Research Society 66 (5), 782–793. Robinson, D., 2005. Reliability analysis of bulk power systems using swarm intelligence. IEEE, 96–102. Santos, H. G., Ochi, L. S., Marinho, E. H., Drummond, L. M. d. A., 2006. Combining an evolutionary algorithm with data mining to solve a singlevehicle routing problem. Neurocomputing 70 (1-3), 70–77. Saremi, S., Mirjalili, S., Lewis, A., 2015. How important is a transfer function in discrete heuristic algorithms. Neural Computing and Applications 26 (3), 625–640. Sch¨obel, A., 2017. An eigenmodel for iterative line planning, timetabling and vehicle scheduling in public transportation. Transportation Research Part C: Emerging Technologies 74, 348–365. Shi, Y., et al., 2001. Particle swarm optimization: developments, applications and resources. In: evolutionary computation, 2001. Proceedings of the 2001 Congress on. Vol. 1. IEEE, pp. 81–86. Shuyuan, Y., Min, W., Licheng, J., 2004. A quantum particle swarm optimization. IEEE Congress on Evolutionary Computation 1, 19–23.
66
Smith-Miles, K., Baatar, D., Wreford, B., Lewis, R., 2014. Towards objective 1100
measures of algorithm performance across instance space. Computers & Operations Research 45, 12–24. Soto, R., Crawford, B., Olivares, R., Barraza, J., Figueroa, I., Johnson, F., Paredes, F., Olgu´ın, E., 2017. Solving the non-unicost set covering problem by using cuckoo search and black hole optimization. Natural Computing, 1–17. Soto, R., Crawford, B., Olivares, R., Barraza, J., Johnson, F., Paredes, F., 2015. A binary cuckoo search algorithm for solving the set covering problem. In: Bioinspired Computation in Artificial Systems. Springer, pp. 88– 97. Streichert, F., Stein, G., Ulmer, H., Zell, A., 2003. A clustering based niching method for evolutionary algorithms. In: Genetic and Evolutionary Computation Conference. Springer, pp. 644–645. Swagatam, D., Rohan, M., Rupam, K., 2013. Multi-user detection in multicarrier cdma wireless broadband system using a binary adaptive differential evolution algorithm. Proceedings of the 15th annual conference on Genetic and evolutionary computation, GECCO, 1245–1252. Thaker, N., et al., 2016. Radar charts show value of prostate cancer treatment options. PharmacoEconomics & Outcomes News 762, 33–24. Totonchi, A., Reza, M., 2008. Magnetic optimization algorithms, a new syn-
1120
thesis. In: IEEE International Conference on Evolutionary Computations. Tsai, C.-W., Chiang, M.-C., Ksentini, A., Chen, M., 2016. Metaheuristic algorithms for healthcare: open issues and challenges. Computers & Electrical Engineering 53, 421–434.
67
¨ Tyasnurita, R., Ozcan, E., Shahriar, A., John, R., 2015. Improving performance of a hyper-heuristic using a multilayer perceptron for vehicle routing. Valenzuela, C., Crawford, B., Soto, R., Monfroy, E., Paredes, F., 2014. A 2-level metaheuristic for the set covering problem. International Journal of Computers Communications & Control 7 (2), 377–387. ˇ Veˇcek, N., Mernik, M., Filipiˇc, B., Crepinˇ sek, M., 2016. Parameter tuning with chess rating system (crs-tuning) for meta-heuristic algorithms. Information Sciences 372, 446–469. Wang, I., Zhang, Y., Zhou, Y., 2008. Discrete quantum-behaved particle swarm optimization based on estimation of distribution for combinatorial optimization. IEEE Evolutionary Computation, 897–904. Yalcinoz, T., Altun, H., 2001. Power economic dispatch using a hybrid genetic algorithm. IEEE power engineering review 21 (3), 59–60. Yang, X.-S., 2009. Firefly algorithms for multimodal optimization. In: International Symposium on Stochastic Algorithms. Springer, pp. 169–178. 1140
Yang, X.-S., 2010a. Firefly algorithm, stochastic test functions and design optimisation. International Journal of Bio-Inspired Computation 2 (2), 78– 84. Yang, X.-S., 2010b. A new metaheuristic bat-inspired algorithm. In: Nature inspired cooperative strategies for optimization (NICSO 2010). Springer, pp. 65–74. Yang, X.-S., Deb, S., 2009. Cuckoo search via l´evy flights. In: Nature & Biologically Inspired Computing, 2009. NaBIC 2009. World Congress on. IEEE, pp. 210–214. 68
Yang, Y., Mao, Y., Yang, P., Jiang, Y., 2013. The unit commitment problem based on an improved firefly and particle swarm optimization hybrid algorithm. In: Chinese Automation Congress (CAC), 2013. IEEE, pp. 718–722. ˙ B¨ Yelbay, B., Birbil, S¸. I., ulb¨ ul, K., 2012. The set covering problem revisited: an empirical study of the value of dual information. European Journal of Operational Research. Zakaria, D., Chaker, D., 2015. Binary bat algorithm: On the efficiency of mapping functions when handling binary problems using continuousvariable-based metaheuristics. Computer Science and Its Applications 456, 3–14. Zhang, G., 2011. Quantum-inspired evolutionary algorithms: a survey and 1160
empirical study. Journal of Heuristics 17 (3), 303–351. Zhang, X., Wu, C., Li, J., Wang, X., Yang, Z., Lee, J.-M., Jung, K.-H., 2016. Binary artificial algae algorithm for multidimensional knapsack problems. Applied Soft Computing 43, 583–595. Zhao, J., Sun, J., Xu, W., 2005. A binary quantum-behaved particle swarm optimization algorithm with cooperative approach. International Journal of Computer Science 10 (2), 112–118. Zhou, Y., Chen, X., Zhou, G., 2016. An improved monkey algorithm for a 0-1 knapsack problem. Applied Soft Computing 38, 817–830.
69
