An Improved PSO Based Back Propagation Learning-MLP (IPSO-BP-MLP) for Classification D.P. Kanungo, Bighnaraj Naik, Janmenjoy Nayak, Sarada Baboo and H.S. Behera
Abstract Although PSO has been successfully used in much application, the issues of trapping in local optimum and premature convergence can be avoided by using improved version of PSO (IPSO) by introducing new parameter called inertia weight. The IPSO is based on the global search properties of the traditional PSO and focuses on the suitable balance of the investigation and exploitation of the particles in the swarm for effective solution. During IPSO iterations, with increase in possible generations, the search space is decreased. Motivated from successful use of IPSO in many applications, in this paper, it is an attempt to design a MLP classifier with a hybrid back propagation learning based on IPSO. The proposed method has been tested using benchmark dataset from UCI machine learning repository and performances are compared with MLP, GA based MLP and PSO based MLP.
Keywords Data mining Classification Improved particle swarm optimization Particle swarm optimization Multilayer perceptron Back propagation learning
D.P. Kanungo (&) B. Naik J. Nayak H.S. Behera Department of Computer Science Engineering and Information Technology, Veer Surendra Sai University of Technology, Burla, Sambalpur, Odisha 768018, India e-mail:
[email protected] B. Naik e-mail:
[email protected] J. Nayak e-mail:
[email protected] H.S. Behera e-mail:
[email protected] S. Baboo Department of Computer Science and Application, Sambalpur University, Burla, Sambalpur, Odisha 768018, India e-mail:
[email protected] © Springer India 2015 L.C. Jain et al. (eds.), Computational Intelligence in Data Mining - Volume 1, Smart Innovation, Systems and Technologies 31, DOI 10.1007/978-81-322-2205-7_32
333
334
D.P. Kanungo et al.
1 Introduction The PSO (Kennedy and Eberhart [1, 2]) is a meta-heuristic swarm based optimization technique, which has been successfully used in much application of science and engineering. The advantages of PSO like ease of implementation, less parameter settings, fast convergence and free from mathematical computations makes it more popular among other the optimization algorithm based on swarm intelligence. During the use of PSO for many applications, researchers found that, PSO may be trapped at some local optimum which leads to premature convergence. Many researchers addressed this issue to enhance the performance of the standard PSO. The method of improved PSO is basically based on the global search properties of the traditional PSO and focuses on the suitable balance of the investigation and exploitation of the particles in the swarm for effective solution. The common steps of PSO like updation of position, velocity and fitness terms of the swarms will remain same in IPSO. Here, a new parameter called adaptive inertia weight (λ) is added to the basic equation of PSO. With the increase number of generations and by setting the parameters for λ, the value of λ can be decreased in a gradual manner. As a result, during the search procedure of IPSO method, when the number of possible generations will increase, the search space will be decreased. Hence, during the iteration the weak particle in the current generation will make a replacement with the best particle of previous generation which will helpful to avoid the premature convergence. Also, each particle will share the information with other particles having only the global best (gbest) value in the search space. In the dominion of the improving properties of PSO, IPSO has been applied in various application domains. A few among them have been discussed relevant to the proposed work. Yang et al. [3] described an improved PSO algorithm for onboard embedded applications in power-efficient wireless sensor networks (WSNs) and WSN-based security systems for significant improvement on the performance of basic PSO. The effectiveness of Vehicle routing and scheduling Problems by using the improved PSO is being realized by Zhang and Lu [4]. Wu et al. [5] has successfully used the improved PSO for optimizing the body of gravity dam and sluice gate. By improving the basic PSO, Tang et al. [6] has designed a S Curve controller for Motion control of Underwater Vehicle using IPSO technique. To optimize the authority and threshold values of back propagation nerve network, Chen et al. [7] has used the IPSO to obtain fast convergence speed. Park et al. [8] have applied the IPSO technique with chaotic sequence for Nonconvex Economic Dispatch Problems. Ran et al. [9] explained an improved PSO based amphibious mouse robot for the purpose of path planning. Chew and Zarrabi [10] have made an effort on digital speckle correlation method to measure surface displacements and strains by assuming first order linear deformation using IPSO and compared the resulting performance with PSO and GA. Ishaque et al. [11] have implemented an enhanced
An Improved PSO Based Back Propagation …
335
maximum power point tracking (MPPT) technique for the photovoltaic (PV) system using an IPSO algorithm. Qais and Wahid [12] have developed a novel method called TriPSO for the improvisation of IPSO. Geng and Zhao [13] designed a method for UAV based track planning using the IPSO algorithm. Shakiba et al. [14] have introduced a humanoid soccer playing robot by using Ferguson splines and IPSO technique. Yanqiu et al. [15] have developed an IPSO based three-axis measuring system calibration problems for solving the optimal local parameters. Qu and Yue [16] have used the IPSO algorithm to solve the constraint optimization problems and have used a new mutation operator to improve the global search ability of PSO. Barani et al. [17] have implemented a novel IPSO based chaotic cellular automata to get the high exploration capability in the randomness nature of the algorithms. An efficient classification method based on PSO and GA based hybrid ANN has been proposed by Naik et al. [18] and it is found relatively better in performance as compared to other alternatives. In this paper, a MLP with IPSO based back propagation learning has been proposed for classification. The rest part of this paper is organized as follow: Preliminaries, Proposed Method, Experimental Setup and Result Analysis, Conclusion and References.
2 Preliminaries 2.1 Particle Swarm Optimization Particle swarm optimization (PSO) [1, 2] is a widely used stochastic based algorithm and it is able to search global optimized solution. Like other population based optimization methods, the particle swarm optimization starts with randomly initialized population for individuals and it works on the social behavior of particle to get the global best solution by adjusting each individual’s positions with respect to global best position of particle of the whole population (Society). Each individual is adjusting by changing the velocity according to its own experience and by observing the position of the other particles in search space by use of Eqs. 1 and 2. Equation 1 is for social and cognition behavior of particles respectively where c1 and c2 are the constants in between 0 and 2 and rand(1) is random function which produces random number between 0 and 1. Vi ðt þ 1Þ ¼ Vi ðt þ 1Þ þ c1 rand ð1Þ ðlbesti Xi Þ þ c2 rand ð1Þ ðgbesti Xi Þ ð1Þ Xi ðt þ 1Þ ¼ Xi ðtÞ þ Vi ðt þ 1Þ
ð2Þ
336
D.P. Kanungo et al.
Basic steps of PSO can be visualized as: Initialize the position of particles (population of particles) and velocity of each particle . Do Compute fitness of each particle in the population. Generate local best particles (LBest) by comparing fitness of particles in previous population with new population. Choose particle with higher fitness from local best population as global best particle (GBest). Compute new velocity by using eq.1. Generate new position of the particles by using eq.2. While (iteration
