2013 International Symposium on Computational and Business Intelligence
An Improved Random Inertia Weighted Particle Swarm Optimization Anupam Biswas1 , A. V. Lakra2 , Sharad Kumar3 and Avjeet Singh4 1234
Department of Computer Science & Engineering, Motilal Nehru National Institute of Technology Allahabad, Allahabad, India. Email: 1
[email protected], 2
[email protected], 3
[email protected], 4
[email protected] shown below:
Abstract—Interactive cooperation of local best and global best solution encourages particles to move towards them, with a hope that better solution may present in the neighboring positions around local best or global best. However, this encouragement does not guarantees that movements taken by particle will always be the suitable one (comparatively better solution). With the influence of three random parameters in PSO-RANDIW increases exploration power as well as probability of unsuitable movements (move towards comparatively worst solution). These unsuitable movement may delay in convergence. In this paper, we have introduced a noble method to avoid such move with cognition of particle’s own worst solution. Analysis on well known four benchmark functions shows proposed approach performance is comparatively better. Index Terms—Genetic Algorithm; Particle Swarm Optimization; Optimization; Heuristics; PSO-RCA
Xi (t) = (xi1 , xi2 , xi3 , ...xid ) Vi (t) = (vi1 , vi2 , vi3 , ...vid ) Yi (t) = (yi1 , yi2 , yi3 , ...yid ) , yi2 , yi3 , ...yid ) Yi (t) = (yi1
Velocity and position of ith particle in j th dimension for next iteration are evaluated and updated with the following two equations: Vij (t + 1) = Vij (t) + C1 × R1 × (Yij (t) − Xij (t)) +C2 × R2 × (Yij (t) − Xij (t)) Xij (t + 1) = Xij (t) + Vij (t + 1)
(2) (3)
Here, R1 and R2 are uniformly distributed random numbers in range [0,1]. C1 and C2 are the positive constants in range (0,2], known as acceleration coefficients. C1 regulates particles movement towards particle’s personal best and C2 regulates particles movement towards global best. The term C1 × R1 × (Yij (t) − Xij (t)) is associated with particle’s cognition of its own best solution. The term C2 × R2 × (Yij (t) − Xij (t)) is associated with particle’s collaborative interaction with its neighbours. These two terms are related with particle’s acceleration (rate of change in velocity) so these are often known as cognitive acceleration and social acceleration respectively. Numerous modifications to the original PSO are proposed to improve overall performance. A binary version of PSO is introduced by Kennedy and Eberhart [5], where particles make decisions in terms of either 0 or 1. The concept of inertia weight is developed in [4], [7], [8]. This inertia weight limits the exposition of the previous velocity into current velocity, is represented as follows:
I. I NTRODUCTION
Particle Swarm Optimization (PSO), a very successful algorithm of last two decades for solving optimization problems, was originally proposed by Kennedy and Eberhart [1]–[3]. PSO mimics social behavior of swarms such as birds flocking, fish schooling. Like other population based algorithms PSO also maintains a population of particles (swarm). Collective behavior of swarm directly influences the behavior of individual particle in the swarm to achieve goals. This kind of influences in decentralized or self organized system is termed as swarm intelligence. Particle Swarm Optimization is semi decentralized and self organized system, where individual particle interacts with one another and cumulatively learns from its own experience. Suitability of a particle is defined with their adaptability (fitness) in the solution domain. Particles with higher fitness put comparatively greater impact on its neighbours. Each particle is coupled with a velocity and a position in the solution domain. Particles maintain their best solution experienced so far local best (lbest) and best solution experienced by neighbour global best (gbest). Positions and velocities are updated in accordance with lbest and gbest experienced so far.
Vij (t + 1) = ω × Vij (t) + C1 × R1 × (Yij (t) − Xij (t)) (4) +C2 × R2 × (Yij (t) − Xij (t)) Here ω is the inertia weight which remains fixed for all iterations. Instead of using same value for all iterations [6] proposes PSO-RANDIW, the concept of random inertia weight. The randomness in Inertia Weight is defined as:
Position, velocity, lbest and gbest vector of ith particle at t iteration in d dimension can be represented respectively as
ω = 0.5 +
th
978-0-7695-5066-4/13 $26.00 © 2013 IEEE DOI 10.1109/ISCBI.2013.27
(1)
96
1 × rand() 2
(5)
TABLE I B ENCHMARK FUNCTIONS
In this paper we have introduced an additional cognitive component to PSO-RANDIW. This new addition stimulates particles to move in appropriate direction by avoiding probable improper movements. As cognitive acceleration defines particle’s own intelligence to decide upon which direction to move, this cognitive component defines particle’s own intelligence to decide upon which direction not to move. Remaining sections of the paper is organized as follows. Section II describes key motivational aspects of the proposal and the proposed approach. Section III provides details about experimental setups, benchmark functions and summarizes simulation results. Finally, we have concluded in section IV.
Function name
Definition f (x) =
Griewank Rosenbrock
f (x) =
n
x2 i i=1 4000
n−1
i=1
f (x) =
Schwefel
n i=1
100 xi+1 −
f (x) =
Sphere
−
n
n i=1
xi cos( √ )+1
2 x2i
i=1
+ xi − 1
x2i
xi sin
i
|xi |
2
TABLE II R ANGE AND O PTIMA
II. P ROPOSED APPROACH Awareness of particle’s own best position and neighbour’s best position helps particles to decide which direction to move and is suitable for convergence. These known best positions attracts particles towards themselve with a hope that neighbouring position of best solution will good or may better than the best one. Indeed sometime it may happen that, these attractions acts as traps for a particle. This is because solution nearby current lbest or gbest is not always good, as it is actualy depends on the solution domain. Any wrong movement of a particle may lead to diverse from optimal solution, which may result delay in convergence. Although fixed intertia weight improves performance by contolling the effect of previous wrong movement, but it actually reduces the explorative power of a particle. PSO-RANDIW increases exploration of solution domain by using randomly choosen inertia weights. This randomness in inertia weight with other two random parameter R1 and R2 increases the probability to take an unsuitable move. Which may hamper in overall perforance of the algorithm. We have proposed a noble approach to avoid such unsuitable movements without effecting the explorative power of PSO-RANDIW. We have introduced a new component to the velocity Equation 4 of PSO-RANDIW. We have redefined velocity equation as:
Function name
Range
Optimal solution
Griewank
[−600, 600]
f (x∗ ) = 0
Rosenbrock
[−2.048, 2.048]
f (x∗ ) = 0
Sphere
[−5.12, 5.12]
f (x∗ ) = 0
Schwefel
[−500, 500]
f (x∗ ) = −n × 418.9829
III. S IMULATION RESULTS Performance of the proposed approach is evaluated on four well known benchmark functions as shown in Table I. Initial ranges and optimal solution is listed in Table II All benchmark functions are tested with dimensions 30, 40 and 50. We have varied maximumm generations as 1000, 2000 and 3000 with corresponding dimensions 30, 40 and 50 respectively. Effectivenes of population size on performance of PSO is very less as shown by Eberhart and Shi [9]. Generally population size of PSO varies from 20 to 60. However, as shown in [10] increment of population results slight improvement in optimal solution. Hence, we have used population size 40 for all the experimental study. Acceleration coefficients C1 , C2 and newly added C3 kept as constant values 0.5, 1.5 and 0.7 respectively. Performance of proposed PSO-RCA is compared with PSORANDIW. We have considered five performance metrics to analyze the quality of the optimal solution of PSO-RCA: Mean, Median, Minimum value, Maximum value and Standard deviation.We have initialized population with uniformly distributed particles over the entire solution space. With the three dimensionality as described above we have carried out 50 trials for each function to present perforance metrics. We track the optimal solutions obtained over 50 trials and evaluated performance metrics. Results are presented in Table III. Bold figures as in the Table III represents comparatively better in contrast to other. For Griewank’s function in dimensions 30 and 40 PSO-RCA performs better than PSO-RANDIW in terms of mean, median and standard deviation, where as PSO-RANDIW reaches least optimal value. In dimension 50 PSO-RCA performance is comaparatively better. For sphere function PSO-RCA outperforms PSO-RANDIW in all dimensions. For Rosenbrock’s function in dimensions 30 and 40 PSO-RCA performs better,
Vij (t + 1) = ω × Vij (t) + C1 × R1 × (Yij (t) − Xij (t)) +C2 × R2 × (Yij (t) − Xij (t)) (6) −C3 × R3 × (Zij (t) − Xij (t)) Here Z is the vector of particle’s local worst position (lworst) it has attained so far. The local worst vector Z of particle i can be defined as Zi (t) = (zi1 , zi2 , zi3 , ...zid ). Since this new component is for avoiding unsuitable positions so have termed it as cognitive avoidance with similar notion of cognition as defined by cognitive acceleration and the algorithm as PSO-RANDIW with Cognitive avoidance (PSO-RCA). In this case, instead of attracting particles towards lworst, cognitive avoidance component prevents particles from taking movement towards it. The algorithm is initialized with uniformly generated random positions and velocities. Updates current lbest, lworst and gbest. Velocity, position and weight for next iteration are updated with Equations 6, 3 and 5 respectively. The process continues until the termination condition meet.
97
TABLE III P ERFORMANCE MEASURE OF PSO-RCA IN CONTRAST WITH PSO-RANDIW Objective function
Griewank function
Sphere function
Rosenbrock function
Schwefel function
Dimension
Generation
30
1000
40
2000
50
3000
30
1000
40
2000
50
3000
30
1000
40
2000
50
3000
30
1000
40
2000
50
3000
Measures
98
PSO-RANDIW
PSO-RCA
Mean Median Minimum Maximum Std. Deviation Mean Median Minimum Maximum Std. Deviation Mean Median Minimum Maximum Std. Deviation
4.090370 3.325210 1.300387 14.448410 2.878408 13.753858 12.974718 2.667439 32.192579 6.505127 34.371700 35.104581 11.948953 78.175656 14.060033
3.241581 2.771874 1.343269 10.774312 1.892029 12.161485 10.403020 1.511302 34.971454 5.887687 29.315429 29.577780 8.999661 53.915413 11.287730
Mean Median Minimum Maximum Std. Deviation Mean Median Minimum Maximum Std. Deviation Mean Median Minimum Maximum Std. Deviation
315.435270 262.198478 38.964297 1162.673768 245.423384 1509.015320 1217.593508 425.719492 4882.389019 915.218491 3280.504511 2786.031132 1572.248199 7560.849549 1432.826600
268.681462 216.139764 15.253190 962.551467 232.321019 1350.716320 1151.230063 345.258335 4075.670983 650.848038 3030.364902 2753.510258 1096.294612 5137.204230 1129.972400
Mean Median Minimum Maximum Std. Deviation Mean Median Minimum Maximum Std. Deviation Mean Median Minimum Maximum Std. Deviation
75.731301 72.195374 31.048944 147.695117 32.181622 152.408137 152.954044 58.198730 257.717569 49.739838 252.658764 230.307058 112.800783 422.658717 70.118699
70.069545 64.003528 29.819643 146.085355 29.064267 146.689820 153.465457 42.693853 241.702346 42.698884 254.871057 254.672020 123.006008 418.691451 64.134221
Mean Median Minimum Maximum Std. Deviation Mean Median Minimum Maximum Std. Deviation Mean Median Minimum Maximum Std. Deviation
-7299.419121 -7201.259483 -9152.273674 -5931.680369 784.849278 -8877.244050 -8847.429441 -10128.329887 -7422.720817 720.168711 -10454.535994 -10543.314750 -12624.742807 -8195.984377 1053.478428
-7188.665886 -7115.621378 -8655.370299 -5334.992230 700.668492 -8974.353653 -8885.422089 -10149.677360 -7372.940492 705.980290 -10560.528160 -10317.428142 -13832.258297 -8304.000972 1066.861034
Fig. 1. Covergence of PSO-RCA compared with PSO-RANDIW by comparing mean fitness value through each generation in Griewank, Rosenbrock, Sphere and Schwefel function
R EFERENCES
but for dimension 50 performance is poor. For Schwefel’s function in dimension 30 PSO-RCA perforance is poor. However, in dimensions 40 and 50 shows little improvement in performance than PSO-RANDIW. Covergence rate of PSO-RCA and PSO-RANDIW examined as in Figure 1. To compare convergence rate we have plotted mean fitness versus generation. As initial mean fitness is high so we have used log of it instead of using direct value. For Schwefel function, as mean fitness is negative so we have used negative of log of negative of mean fitness for plotting. Results shows convergence rate of PSO-RANDIW is initially higher, but at the end convergence rate of PSO-RCA overtakes.
[1] Kennedy J. and Eberhart R., Particle swarm optimization, in Procedings of IEEE International Conf. on Neural Networks, 1995, pp. 1942-1948. [2] Eberhart, R. C., and Kennedy, J. . A new optimizer using particle swarm theory. Proc. Sixth International Symposium on Micro Machine and Human Science (Nagoya, Japan), pp. 39-43, 1995. [3] Kennedy, J. (1997). The particle swarm: social adaptation of knowledge. Proceedings of IEEE International Conference on Evolutionary Computation. pp. 303-308. [4] Shi, Y.; Eberhart, R.C.,A modified particle swarm optimizer, in Procedings of IEEE International Conf. on Evolutionary Comp., 1998, pp. 69-73. [5] Kennedy, J and Eberhart, R.C. A discrete binary version of the particle swarm algorithm, IEEE Inter. Conf. on Sys., Man, and Cybernetics, 1997. [6] Eberhart R. C. and Shi Y., Tracking and optimizing dynamic systems with particle swarms, in Procedings 2001 IEEE International Congress on Evolutionary Computation, pp. 94-100. [7] Shi Y., and Eberhart R. A modified particle swarm optimizer. In Evolutionary Comp. Proc., 1998. IEEE World Congress on Comp. Intelligence., The 1998 IEEE International Conf. on, pp. 69-73, 1998. [8] A.Nikabadi, M.Ebadzadeh , Particle swarm optimization algorithms with adaptive Inertia Weight : A survey of the state of the art and a Novel method, IEEE journal of evolutionary computation , 2008. [9] Eberhart, R.C.; Shi, Y., Comparing inertia weights and constriction factors in particle swarm optimization, in Procedings of IEEE International Congress on Evolutionary Computation, vol. 1, 2000, pp. 84-88. [10] F. van den Bergh and A. P. Engelbrecht, Effect of swarm size on cooperative particle swarm optimizers, in Procedings of Genetic Evolutionary Comp. Conf. (GECCO-2001), San Francisco, CA, July 2001, pp.892-899.
IV. C ONCLUSION In this paper we have proposed a methodology to improve particles movement in random weighted particle swarm optimization. We have introduced a noble mechanism to restrain particles movements towards unsuitable solutions. As any wrong movement can cause particles to take extra unnecessary steps, which may dealy in convergence. So particles avoids any misguidance projected by lbest and gbest with cognition of worst solution. Execution with benchmark functions shows significant improvement in proposed approach.
99
