A Binary Firefly Algorithm for Knapsack Problems
Kaushik Kumar Bhattacharjee∗ , S.P. Sarmah† Department of Industrial & Systems Engineering, Indian Institute of Technology Kharagpur, Kharagpur 721302, West Bengal, India. ∗ Email:
[email protected], †
[email protected] tance move and a greedy repair operator. Also we use the opposition-based learning technique to implement the proposed algorithm. The efficiency of the modified BFA is established by applying it to 01 KP.
Abstract—Knapsack problems are one of the classical NP-hard problem and it offers many practical applications in vast field of different areas. Several traditional as well as population based metaheuristic algorithms are applied to solve this problem. In this paper, we introduce the binary version of firefly algorithm (FA) for solving knapsack problems, specially 01 knapsack problem. The modified algorithm utilizes the advantages of the variable distance move along with the opposition-based learning mechanism to solve knapsack problems efficiently. So far FA is generally applied to continuous optimization problems. In order to investigate the performance of FA on combinatorial optimization problem, an attempt is made in this paper. To demonstrate the efficiency of the proposed algorithm an extensive computational study is provided with standard bench mark problem instances and comparison with particle swarm optimization is also carried out.
The work is organized as follows: first we will give a brief description of FA. Next we will discuss the details of proposed BFA. Section IV presents experimental results of BFA and finally the conclusions, based on the results, are discussed in the last section. II.
FA is one of the recently developed swarm intelligence based metehuristic. It is proposed by Yang in 2008 [5] and is inspired from the flashing lights of fireflies in nature. It is a kind of stochastic meta-heuristic algorithm that can be applied for solving the complex optimization problems, as well as NP-hard problems such as graph scheduling problem [6], quadratic assignment problem [7], job shop scheduling problem [8], traveling salesman problem [9]. A comprehensive survey of Firefly Algorithm was presented in [10]. In a typical FA, the brightness of a firefly means how well the objective of a solution is and brighter firefly (solution) attracts other fireflies. As the distance between fireflies increases, the attractiveness decreases exponentially because of the light absorption of medium. There are three basic assumption of FA [11]:
Keywords—Metaheuristic algorithms, binary firefly algorithm, knapsack problems, variable distance move, oppositionbased learning.
I.
I NTRODUCTION
The knapsack problems are well known strongly NPhard problem. Research has shown myriad applications which can be formulated as knapsack problems, such as project selection, resource distribution, investment decision making, capital budgeting, cargo loading, cutting stocks etc. The most common formulation of 01 knapsack problem (KP) is Maximize f (⃗x) = Sub ject to g(⃗x) =
1)
n
∑ c jx j
2)
j=1 n
∑ a j x j ≤ b,
(1)
j=1
x j ∈ {0, 1}, ∀ j = 1, 2, ..., n, c j > 0, a j ≥ 0, 0 < b
0.
In order to achieve sufficient diversification, the initial population is generated randomly from entire dimension. And we define the stopping criteria based on four basic criteria : reach the optimal solution, maximum run time, maximum number of iterations, predefined number of nonimprovement iterations. If any of this criteria meets, the algorithm will stop.
(4)
Here ε is a random number drawing from standard uniform distribution (i.e. ε ∼ U(0, 1)); and α ∈ [0, 1] is the scaling factor of the randomness. Pseudocode for FA is given in Algorithm 1.
B. Distance of two fireflies The distance between any two fireflies i and j is defined by the Hamming distance, i.e., the number of different elements between their permutations. Because the difference between the objective function values directly proportional to Hamming distance.
Algorithm 1: Pseudocode for FA Procedure setting the parameters of FA; generate the initial population of fireflies; evaluate the initial population; initialize the light intensity li ; while g < MaxGen do for i = 1 : n (all n fireflies) do for j = 1 : n (all n fireflies) do if l j > li then move firefly i towards firefly j; end attractiveness varies with the distance; evaluate new solution; update the light intensity li ; end end rank the fireflies and find out the current best; update the global best if required; if termination criteria fulfill then stop the evaluation process; end end return the global best solution;
III.
c max j=1,2,..,n a jj
1) Attractiveness of fireflies: A firefly i attracted to another firefly j, if the objective function value of i is smaller than the objective function value of j. The attractiveness of firefly i with respect to firefly j is given by 2 β (ri j ) = β0 e−γ ri j , (6) where ri j is the Hamming’s distance between fireflies i and j. Theoretical value of the light absorption coefficient γ is γ ∈ [0, ∞], we are considering γ in range of [0.01, 0.20]. C. Movement of fireflies In continuous problem the movement is defined by the Equation 4, which can be represented by Equation 7, like [7], xi = (1 − β (ri j ))xi + β (ri j )x j , xi = xi + α (ε − 0.5).
B INARY F IREFLY A LGORITHM
There are several studies available in literature of Firefly Algorithm for continuous optimization problems. Only few studies available for integer programming problem. In recent studies, Palit et al. [12] use a binary Firefly Algorithm to deduce the meaning of an encrypted message for cryptanalysis. Falcon et al. [13] used a binary adaptive Firefly Algorithm for fault identification in parallel and distributed system. Chandrasekaran and Simon [14] used binary real coded Firefly Algorithm for solving network and reliability constrained unit commitment problem. In all these works they use sigmoid function for transforming continuous variables to binary variables, and use euclidean distance to perform light intensity measuring. Here we use a different approach to solve knapsack problems similar to the approach used by Rahmaniani and Ghaderi [15] for solving facility location problem.
(7a) (7b)
So any firefly moves in solution space in two steps : in the first step β (ri j ) determines the movement of firefly i towards firefly j given by Equation 7a; then in the next step α determines the random movement of firefly i given by Equation 7b. It is to be noted that these two steps are not interchangeable. The first step is known as β -step, from the equation it is clear that xi will be equal to x j with a probability given by β (ri j ); and xi will be unchanged with a probability given by (1 − β (ri j )). The β -step procedure is shown in Algorithm 2. For the next step, we choose α value in the range of [0, 1] and randomize the movement of the firefly for continuous optimization problems. If the α value is high then xi will take large step and solution will be faraway from the current solution. Similarly for low α value new solution will be within the small neighborhood of the current solution. So α value determines the condition for global and local search, an appropriate value of α determines the balance between global search and local search procedure. However obtaining the appropriate level
A. Construction of individual firefly The individual firefly (solution) is represented by a nbit binary string, where n is the number of variables in knapsack problem. A bit string ⃗x ∈ {0, 1}n , may represent
74
Proceedings of the 2015 IEEE IEEM
TABLE I.
Algorithm 2: Pseudocode for β Step
Problem
evaluate the objective function values of firefly xi and x j , f (xi ) and f (x j ) respectively; if f (xi ) < f (x j ) then calculate the Hamming’s distance ri j = d(xi , x j ); calculate the attractiveness using Equation 6; for k = 1 : length(xi ) do if xik = x jk then x(new)ik = x jk ; else generate a random number r ∼ U(0, 1); if r < β (ri j ) then x(new)ik = x jk ; else x(new)ik = xik ; end end end end return x(new)i ;
f1 f2 f3 f4 f5 f6 f7 f8 f9 f10 8a 8b 8c 8d 8e 12a 12b
Capacity 2489815 3453702 2520392 3780355 4426945 4323280 4450938 3760429 5169647 4681373 5063791 4286641 4476000 6404180 5971071 5870470 5762284 6654569
op 5170626 6992404 5337472 7850983 9352998 9151147 9348889 7769117 10727049 9818261 10714023 8929156 9357969 13549094 12233713 12448780 11815315 13940099
For 01KP, variables are sorted and renumbered according to the decreasing order of their profit to weight ratios. The greedy algorithm is utilized for selection procedure which always chooses the last item for deletion. IV.
E XPERIMENTS AND C OMPUTATIONAL R ESULTS
In this section we implement the Binary Firefly Algorithm and performance of this algorithm is investigated with standard 01KP instances. All computational experiTM 2 R ments are conducted with python 2.7.6 in Intel⃝Core Duo CPU E8400 @ 3.00GHz with 2GB of RAM in Unix environment. Two sets of knapsack problems are considered here to test the efficiency of BCSA. First set of KP instances which contains 10 instances is taken from various literature, details can be found in [20]. Items in this set ranging from 4 to 23. Second set contains 25 instances is taken from http://www.math.mtu.edu/~kreher/cages/Data. html, used in [21], [22]. Number of items in this instance ranging between 8 and 24. Details of these test problems along with the best solution found are given in Table I.
D. Generating new firefly In the original FA if there is no better solution than a particular one, then we generate a solution randomly. Here we use opposition-based learning (OBL) introduced by Tizhoosh [16] to generate the new solution. It is proved that an opposite solution has a higher chance to be closer to the global optimal than a random solution [17]. Opposite point is defined as (8)
A. Effect of population size (n)
where X = (x1 , x2 , ..., xn ) be a point in n-dimensional space and x j ∈ [a j , b j ], j ∈ 1, 2, .., n [18]. The generalized OBL (GOBL) is defined as x´ = k(a + b) − x,
Problem 12c 12d 12e 16a 16b 16c 16d 16e 20a 20b 20c 20d 20e 24a 24b 24c 24d 24e
make infeasible solutions to feasible one. This procedure depends upon the problem type and the constraint.
of α is not an easy task. For binary variables this α -step only represents the change in bit values for a particular firefly. There are two ways by which we may change in bit values for binary variables : flip and swap procedure. We may use any of these two procedures as the α step for solving knapsack problem. And α represents the probability that a particular bit will change or not. The number of bits (nB) for which there is a change in bit value will depend on Hamming distance (i.e., ri j ). Because here our aim is to minimize the distance between firefly i and j, as firefly x j is brighter than firefly xi ; so xi attracts towards x j . So nB = α × ri j and α should be small; otherwise the Hamming distance will increase between xi and x j rather than decreasing. This is similar to variable distance move (k-distance or k-exchange); here we choose k = 1.
x´j = a j + b j − x j ,
S TANDARD TEST PROBLEMS OF 01 K NAPSACK .
Capacity op Problem Set 1 269 295 878 1024 20 35 11 23 375 481.0694 60 52 50 107 10000 9767 80 130 879 1025 Problem Set 2 1863633 3924400 1822718 3813669 1609419 3347452 2112292 4187707 2493250 4955555 2805213 5688887 3259036 6498597
For the experimental study we choose population size, n = {30, 40, 50, 60, 70, 80, 90, 100} and light absorption coefficient γ = 0.1. Here we consider total 50 independent runs for each case and the relation between number of iterations and population size is shown in Figure 1 for KP instance f8 . Anova table of Kruskal-Wallis test is given in Table II and p value is 2.91E − 12. So we can reject the null hypothesis and conclude that at least one sample median is significantly different from the others. Figure 2 shows the result of multiple comparison mean ranks. Two groups G1 = {n∣n = 30, 40} and G2 = {n∣n = 60, 70, 80, 90, 100} are significantly different from each other. As the computational complexity increases with the increase in population size, so we choose population size n = 60 for solving knapsack problem instances.
(9)
where k is a real number [19]. Here we consider k is a random variable generated from standard uniform distribution U(0, 1). This will transform the solution to a new search space, and by that provide more chance to find better solutions. E. Constraint handling When the binary string violates the constraint of the given problem, then repair algorithm is employed to
75
Proceedings of the 2015 IEEE IEEM
Light absorption co-efficient (γ) as Group
350
Number of Iteration
300
250
200
150
100
50
30
40
50
60
70
80
90
3
4
5
6
7
8
9
100
Population size (n)
100
150
200
250
300
350
9 groups have mean ranks significantly different from Group 1
The effect of population size n for knapsack instance f8 .
Fig. 4. Multiple comparison of mean ranks of light absorption coefficient γ for knapsack problem instance f8 . TABLE III.
1
Population size (n) as Group
2
10
0
Fig. 1.
1
Source Groups Error Total
2
3
4
K RUSKAL -WALLIS ANOVA TABLE
SS 624293.54 9788834.46 10413128
df 9 490 499
MS 69365.95 19977.21
χ2 29.92
p > χ2 4.53E-04
5
6
C. Results of second set of problem instances
7
8
100
150
200
250
300
Results of the second set of problem instances are given in Table IV. Here we consider population size n = 60, γ = 0.02 and maximum number of iterations iMax = 20 × n. Total 50 independent trails are conducted for each problem instance. Best and worst results found out between these runs are reported along with average, median and standard deviation among best solution found in each trial. Also average total time and average number of iterations are also reported. The average profit over 100 runs (AVPFT) for Binary Particle Swarm Optimization (BPSO) [23], Genotype-Phenotype Modified Binary Particle Swarm Optimization (GPMBPSO) [24], and Modified Binary Particle Swarm Optimization (MBPSO) [21] are also provided in Table IV.
350
2 groups have mean ranks significantly different from Group 4
Fig. 2. Multiple comparison of mean ranks of population size (n) for knapsack problem instance f8 . K RUSKAL -WALLIS ANOVA TABLE
TABLE II. Source Groups Error Total
SS 915846.64 4415699.36 5331546
df 7 392 399
MS 130835.23 11264.54
χ2 68.54
p > χ2 2.91E-12
B. Effect of light absorption coefficient (γ ) Here we consider population size n = 60 and light absorption co-efficient γ = {0.02, 0.04, 0.06, 0.08, 0.09, 0.10, 0.12, 0.14, 0.16, 0.18, 0.20}. Total 50 independent runs for each case and the relation between number of iterations and light absorption co-efficient is shown in Figure 3 for KP instance f8 . Anova table of Kruskal-Wallis test is given in Table III and result of multiple comparison mean ranks is shown in Figure 4. From the results it is clear that we must select light absorption co-efficient γ ≤ 0.02 for solving KP instances.
BFA easily find out the best solutions for all problem instances. Also the average values is much more better than the results found out by BPSO and GPMBPSO algorithms; and except from only one case (Problem Number 24b), BFA outperforms MBPSO for all other cases (total 19 cases). The average total time taking to solve a particular instance and average number of iterations for solving these problems are also very low.
V.
C ONCLUSION
250
In this work the performance of the Binary Firefly Algorithm has been extensively investigated on standard benchmark instances of knapsack problems. Proposed algorithm is also compared with recently developed different versions of particle swarm optimization algorithms. BFA outperforms most of cases. From the results it is clear that BFA is a powerful metaheuristic for solving knapsack problems. Also it produce much more better results compared to other standard algorithms presented in literature within very small time limit. BFA may be further modified and applied to other combinatorial problems for getting better performance, which is scheduled as future work.
Number of Iteration
200
150
100
50
0 0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
Light absorption co-efficient (γ)
Fig. 3. The effect of light absorption co-efficient γ for knapsack instance f8 .
76
Proceedings of the 2015 IEEE IEEM
TABLE IV. Problem 8a 8b 8c 8d 8e 12a 12b 12c 12d 12e 16a 16b 16c 16d 16e 20a 20b 20c 20d 20e 24a 24b 24c 24d 24e
best 3924400 3813669 3347452 4187707 4955555 5688887 6498597 5170626 6992404 5337472 7850983 9352998 9151147 9348889 7769117 10727049 9818261 10714023 8929156 9357969 13549094 12233713 12448780 11815315 13940099
worst 3924400 3813669 3347452 4187707 4955555 5688887 6498597 5170626 6992404 5337472 7850983 9352998 9151147 9348889 7769117 10717569 9818261 10712572 8929156 9357969 13514028 12196303 12448780 11783685 13929872
average 3924400 3813669 3347452 4187707 4955555 5688887 6498597 5170626 6992404 5337472 7850983 9352998 9151147 9348889 7769117 10725342.6 9818261 10713645.74 8929156 9357969 13544867 12208229.7 12448780 11810429.92 13933325.08
median 3924400 3813669 3347452 4187707 4955555 5688887 6498597 5170626 6992404 5337472 7850983 9352998 9151147 9348889 7769117 10727049 9818261 10714023 8929156 9357969 13549094 12207241 12448780 11810051 13930566
R ESULTS OF PROBLEM SET 2 std 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3679.074 0 642.92 0 0 9880.334 12987.548 0 4907.141 4702.448
R EFERENCES [1] A. Liu, J. Wang, G. Han, S. Wang, and J. Wen, “Improved simulated annealing algorithm solving for 0/1 knapsack problem,” in Sixth International Conference on Intelligence System Design and Applications (ISDA 2006), vol. 02. IEEE Computer Society, 2006.
[14]
[15]
[2] J. Thiel and S. Voss, “Some experiences on solving multiconstraint zeo-one knapsack problems with genetic algorithm,” INFOR, vol. 32, pp. 226–242, 1994.
[16]
[3] P. Zhao, P. Zhao, and X. Zhang, “A new ant colony optimization for knapsack problem,” in Seventh International Conference on Computer-Aided Industrial Design and Conceptual Design, November 17-19 2006, pp. 1–3.
[17]
[4] B. Ye, J. Sun, and W.-B. Xu, “Solving the hard knapsack problems with a binary particle swarm approach,” ICIC 2006, vol. LNBI 4115, pp. 155–163, 2006.
[18]
[5] X. Yang, “Firefly algorithm,” Nature-Inspired Metaheuristic Algorithms, vol. 20, pp. 79–90, 2008. [19]
[6] U. Honig, “A firefly algorithm-based approach for scheduling task graphs in homogeneous systems,” in Informatics. ACTA Press, 2010, pp. 24–33. [7] K. Durkota, “Implementation of a discrete firefly algorithm for the qap problem within the seage framework,” Master’s thesis, Electrical Engineering, Czech Technical University, Prague, 2011.
[20]
[8] A. Khadwilard, S. Chansombat, T. Thepphakorn, P. Thapatsuwan, W. Chainate, and P. Pongcharoen, “Application offirefly algorithm and its parameter setting for job shop scheduling,” in The First Symposium on Hands-On Research and Development, 2011, pp. 1–10.
[21]
[22]
[9] G. Jati, “Evolutionary discrete firefly algorithm for travelling salesman problem,” Adaptive and Intelligent Systems, vol. LNAI 6943, pp. 393–403, 2011. [10] I. Fister, I. Fister Jr., X.-S. Yang, and J. Brest, “A comprehensive review of firefly algorithm,” Swarm and Evolutionary Computation, vol. 13, pp. 34–46, 2013.
[23]
[11] X.-S. Yang, “Firefly algorithm for multimodal optimization,” Lecture Notes in Computer Science, vol. 5792, pp. 169–178, 2009.
[24]
[12] S. Palit, S. Sinha, M. Molla, A. Khanra, and M. Kule, “A cryptoanalytic attack on the knapsack crypto system using binary firefly algorithm,” in The Second International Conference on Computer and Communication Technology, ICCCT-2011, IEEE, 2011, pp. 428–432. [13] R. Falcon, M. Almeida, and A. Nayak, “Fault identification with binary adaptive fireflies in parallel and distributed systems,” in
77
ATT 0.066 0.302 0.064 0.04 0.043 1.04 0.41 0.208 0.222 0.066 0.469 0.467 0.291 1.674 1.897 8.897 6.338 8.698 0.648 1.64 10.458 24.743 1.31 22.288 21.021
ANI 1.78 8.98 1.82 1.04 1.2 27.26 10.64 5.32 5.64 1.58 10.86 10.76 6.7 38.42 44.34 182.22 126.12 176.84 13.3 33.82 172.74 406.38 22.2 367.7 348.1
BPSO 3921857.19 3807911.86 3328608.71 4186088.27 4932737.28 5683694.29 6478582.96 5166957.08 6989842.73 5316879.59 7834900.26 9334408.62 9118837.47 9321705.87 7758572.21 10707360.91 9791306.65 10703423.34 8910152.57 9349546.98 13510432.96 12205346.16 12427880.56 11792064.76 13922797.55
GPMBPSO 3922251.98 3807671.43 3326300.19 4184469.54 4921758.82 5678227.28 6476487.08 5162237.91 6988151.02 5301119.31 7826923.53 9326158.74 9114581.85 9317336.67 7757247.79 10702954.99 9786719.85 10695550.75 8905564.36 9343911.1 13506115.12 12202425.75 12419101.82 11791581.41 13921046.22
MBPSO 3924400 3813669 3347452 4187707 4954571.72 5688552.41 6493130.57 5170493.3 6992144.26 5337472 7843073.29 9350353.39 9144118.38 9337915.64 7764131.81 10720314.03 9805480.48 10710947.05 8923712.21 9355930.35 13532060.07 12223442.61 12443349.03 11803712.38 13932526.16
IEEE Congress on Evolutionary Computation, CEC-2011, IEEE, 2011, pp. 1359–1366. K. Chandrasekaran and S. Simon, “Network and realiability constrained unit commitment problem using binary real coded firefly algorithm,” International Journal of Electrical Power and Energy Systems, vol. 43, no. 1, pp. 921–932, 2012. R. Rahmaniani and A. Ghaderi, “A combined facility location and network design problem with multi-type of capacited links,” Applied Mathematical Modelling, vol. 37, pp. 6400–6414, 2013. H. Tizhoosh, “Opposition-based learning: a new scheme for machine intelligence,” in International conference on computational intelligence for modeling control and automation, 2005, pp. 695– 701. S. Rahnamayan, H. R. Tizhoosh, and M. M. A. Salama, “Opposition versus randomness in soft computing techniques,” Applied Soft Computing, vol. 8, pp. 906–918, 2008. ——, “Opposition-based differential evolution algorithms,” in IEEE congress on evolutionary computation, 2006, pp. 2010– 2017. H. Wang, Z. W., and S. Rahnamayan, “Enhanced opposition-based differential evolution for solving high-dimensional continuous optimization problems,” Soft Computing, vol. 15, pp. 2127–2140, 2011. K. Bhattacharjee and S. Sarmah, “Shuffled frog leaping algorithm and its application to 0/1 knapsack problem,” Applied Soft Computing, vol. 19, pp. 252–263, 2014. J. Bansal and K. Deep, “A modified binary particle swarm optimization for knapsack problems,” Applied Mathematics and Computation, vol. 218, pp. 11 042–11 061, 2012. K. Bhattacharjee and S. Sarmah, “A binary cuckoo serach algorithm for knapsack problems,” in Fifth International Conference of Industrial Engineering and Opration Management (IEOM), Singapore, March 2015, pp. 1–5. J. Kennedy and R. Eberhart, “A discrete binary version of particle swarm optimization,” in Proceedings of the World Multiconference on Systemics, 1997, pp. 4104–4109. S. Lee, S. Soak, S. Oh, W. Pedrycz, and M. Jeon, “Modified binary particle swarm optimization,” Progress in Natural Science, vol. 18, no. 9, pp. 1161–1166, 2008.
