International Symposium on Mathematical Sciences and Computing Research 2013 (iSMSC 2013) 6-7 December 2013, Perak, MALAYSIA. Paper ID.:CS_22
Correlation Study of Genetic Algorithm Operators: Crossover and Mutation Probabilities Haruna Chiroma* Sameem Abdulkareem* Adamu Abubakar** Akram Zeki** Abdulsalam Ya’u Gital **** Mohammed Joda Usman**** *University of Malaya, Department of Artificial Intelligence, Faculty of Computer Science and Information Technology, Kuala Lumpur, Malaysia.
[email protected] ** International Islamic University, Malaysia, Department of Computer Science, Kuala Lumpur, Malaysia. *** University of Technology Malaysia, Faculty of Computing, Department of Computer Science, Jahor, Malaysia ***
Abubakar Tafawa Balewa University, School of Science, Mathematical Science Programme, Bauchi, Nigeria.
****
Liaoning University of Technology, School of Electronic and Information Engineering, Jinzhou, China.
[email protected]
new passion in the 1980’s with full force by computational research community. This led to development of neural networks, machine learning and evolutionary computation like genetic algorithms [1]. The idea of genetic algorithms (formerly genetic plans) was conceived by Holland. This came to fruition while working in conjunction with colleagues and students at the University of Michigan in the 1970s in pursuit of optimum solution to problem based on principle of natural selection and natural genetics [2 – 5]. They hinge on Darwin’s theory as an inspirational guide and carefully learned principle of evolution and applied the tacit knowledge acquired to developed algorithms based on selection in biological genetic systems [6].There are several parameters that require settings of values when implementing genetic algorithms but the most critical parameters are population size, mutation probability, crossover probability and they interrelate among themself [6]. Since the mechanism for determining genetic algorithms initial parameters values were not in existence, majority of authors refer to previous literature for guide [7]. Others resort to trial and error, for instance in [8 – 9]. The most valuable way to determine initial parameters values of genetic algorithms was to refer back to literature for proper guide on the optimum parameters values to use [10 – 12]. The setting of parameters values can have significant impact on efficiency of genetic algorithms. Therefore, choosing the optimum combination of critical parameters values is a pre – requisite to successful implementation of genetic algorithms. Since there is no mechanism for determining values of genetic algorithm parameters but to refer to literature for guide. Thus, in this paper, we survey optimum values of critical parameters that were successfully use to implement genetic algorithms in previous literature in order to serve as a guide to future researchers in the area of genetic algorithms. We further investigate the relationship between mutation probability and crossover probability.
Abstract— Successful implementation of genetic algorithms depends on optimum values of several parameters but the most critical parameters are population size, crossover and mutation probability. Mechanism for determining the values of these critical parameters is not in existence. The best way to obtain the optimum values is to refer back to previous literature for guide. In this paper, we conduct a survey of optimum parameters values reported in literature. We further investigate the relationship between mutation and crossover probabilities. Several combinations of optimum parameter values are tabulated in this paper in order to serve as a guide for future researchers. It was revealed that crossover probability is positively associated with the use of mutation probability in the implementation of genetic algorithms but the correlation is not significant. Index Terms— Mutation probability, crossover probability, population size, genetic algorithms
I.
INTRODUCTION
The determination to generate computers with artificial intelligent and artificial life started from the early days of computer age. Alan Turing, John Von Neumann, Norbert Wiener among others were the pioneers in the field of computer science fortified with a dream of building intelligent (self- reproduction, ability to learn and have regulation of their environment) into computer programs. Apart from electronics, early computer scientists were also absorbed in biology and psychology in which natural system was their guide towards realizing the building of intelligent into computer programs. This accounts for why applications of computer programs were not only restricted to missile trajectories computation and deciphering military code but also extended to representation of biological brain, imitating human learning paradigm and mimicking biological evolution. The listed biological computational events have faded away over a period of years but it was resurrected with a
39
International Symposium on Mathematical Sciences and Computing Research 2013 (iSMSC 2013) 6-7 December 2013, Perak, MALAYSIA. Paper ID.:CS_22
The structure of this paper is organized as follows: section II provides an overview of genetic algorithm. Section III provides a description of the methods applied in this survey. Section IV discusses results before concluding remarks in Section V. II.
mutation probability) used must be reported in the article. Many articles were rejected because they not match criteria’s for selection. Table I reported the optimum values of population size, crossover and mutation probability obtained from previous literature. Relationship between crossover and mutation probability was investigated using Pearson correlation. The following hypothesis is formulated for the investigation:
OVERVIEW OF GENETIC ALGORITHMS
When a problem is given as an input, the fundamental idea of genetic algorithms is that the pool of genetics specifically contains population with potential solution or better solution to the problem. Genetic algorithms uses the principle of genetic and evolutionary to recurrently modify a population of artificial structures through the use of genetic algorithms operators including initialization, selection, crossover and mutation in order to obtained an optimum solution. Normally, genetic algorithms start with randomly generated initial population represented by chromosomes. Solutions derived from one population are taken and used to form the next generation population. This is done with the expectation that solutions in a new population are to be better than these in old population. The solution used to generate next solution are selected based on fitness value, solutions with higher fitness value have higher chances of been selected for reproduction while solution with lower fitness value have lower chances of been selected for reproduction. This evolution process is repeated severally until criteria set for termination is satisfied. For instance number of population or improvement of best solutions is satisfied [13]. Genetic algorithms have been used for training neural networks [14 – 17], feature selection [18 – 19], determining architectural configurations of neural networks [20], process optimization [21] and pattern recognition [22]. III.
H1: Crossover probability is positively associated with the use of mutation probability in the implementation of genetic algorithms. IV DISCUSSION The optimum efficiency of genetic algorithms is affected by population size, since smaller population generate poor results due to insufficient sample size. A large population size is required for effective search as well as prevention of early convergence. Over population result to more evaluation in each generation and this may cause significant slow rate of convergence. Thus, moderate value is required which is not too high or too low. The population size in Table I ranges from 12 to 4000. Higher value of population size require expensive computational experiments because the mass of solutions to be considered are equal to the population size but the search is more efficient than lower population size. The frequency which genetic algorithms operator is applied is controlled by crossover probability. Rate of crossover probability determine how fast new structures are introduce into the next generation population. Very high crossover value may ignore solution with higher fitness while very low crossover value may truncate the search due to low rate of exploration. Therefore, values between too high and very low have to be used for effective implementation of genetic algorithms. In Table I the rate of crossover probability ranges from 0.1 – 1.
METHODS EMPLOY IN LITERATURE SEARCH
For this research to ascertain articles which genetic algorithms technique was specifically applied in searching for optimum solution, a search of literature was extensively explored in two phases: Phase 1: At this stage, literature were retrieved from ACM digital library, IEEEXplore, science direct, Scopus, springer link, web of science, Google scholar, direct open access journals, Microsoft academic search, ProQuest and CiteSeerX. The search for literature was for a period of 1975 to 2012. This search was able to provide 256 articles within 36 years from 198 journals across the globe based on the following keywords: genetic algorithms and genetic programming. Each keyword was used for search in each of the aforementioned search engines. Phase 2: Each and every literature retrieved through the process described in phase 1, was scrutinized and properly reviewed to ensure criteria’s for selection are adequately matched before inclusion in this survey. For an article to be included it must present an empirical description of genetic algorithms implementation and optimum values of critical parameters (population size, crossover and
TABLE I VALUES OF CRITICAL PARAMETERS USED TO IMPLEMENT GENETIC ALGORITHMS IN PREVIOUS LITERATURE References
40
PS
CP
MP
[23]
30
0.95
0.01
[23]
80
0.45
0.01
[24]
100
0.9
1
[25]
100
0.8
0.005
[26]
50
0.9
0.03
[26]
50
0.9
0.03
[26]
50
0.9
0.03
[26]
100
0.9
0.03
[26]
50
0.9
0.03
[26]
50
0.9
0.03
[26]
50
1
0.03
International Symposium on Mathematical Sciences and Computing Research 2013 (iSMSC 2013) 6-7 December 2013, Perak, MALAYSIA. Paper ID.:CS_22
The standard set of combination was widely accepted by evolutionary community as standard settings for implementing genetic algorithms. It works well in solving several optimization problems in different problem domain. Other critical parameters setting are reported in Table I. The values of critical parameters reported in Table I obtained from separate studies are very close to each other. Probably the literature refers back to previous study for guide. [26] applied the same combination of population size; crossover and mutation probability values in different problems and results from the experiments were satisfactory. Thus, researchers may use combination of values reported in Table I in solving different optimization problems.
[26]
50
1
0.03
[26]
50
0.9
0.03
[27]
50
0.8
0.01
[28]
15
0.7
0.05
[29]
100
1
0.003
[30]
30
0.8
0.07
[31]
100
0.8
0.01
[32]
500
0.8
0.2
[20]
40
0.6
0.1
[33]
100
0.8
0.3
[8]
100
0.6
0.02
[34]
30
0.6
0.05
[35]
76
0.8
0.05
[24]
4000
0.5
0.001
[36]
30
0.6
0.001
[37]
384
0.8
0.05
[38]
50
0.8
0.2
[39]
50
0.9
0.1
products
[40]
20
0.6
0.15
Covariance
[41]
30
0.9
0.1
N
[42]
20
0.8
0.02
[43]
100
0.75
0.01
[9]
12
0.25
0.01
TABLE II CORRELATION RESULTS
Pearson Correlation
CP
MP
1
.049
Sig. (2-tailed) CP
MP
Sum of Squares and Cross-
.753 2.129
.080
.050
.002
44
44
Pearson Correlation
.049
1
Sig. (2-tailed)
.753
Sum of Squares and Cross-
[19]
100
0.75
0.01
[44]
20
0.2
0.05
Covariance
[45]
50
0.9
0.01
N
[46]
20
0.9
0.3
[47]
330
0.5
0.5
[48]
300
0.3
0.2
[5]
100
0.8
0.1
[49]
20
0.1
0.01
[50]
50
0.95
0.09
[51]
100
0.5
0.01
products
.080
1.270
.002
.030
44
44
The values of crossover probability and mutation probability were subjected to correlation analysis in order to test the hypothesis in section 3. From Table II, results of correlation indicated that H1 is accepted but the correlation is not significant as the value is 0.049. This results was expected because the values of mutation probability are always much lower than crossover probability as reported in Table I.
Population size (PS), crossover probability (CP) and mutation probability (MP).
Mutation probability increases population variability. Mutation with high values normally led to random search while lower value of mutation probability protect any given allele position from convergence to single value in the whole population. Moderate value of mutation is required for genetic algorithms implementation; the value shall not be too high or too low. Range of mutation probability in previous literature as indicated in Table I is from 0.001 – 1. It is observed that the rate of mutation probability is always lower than crossover probability. [23] reveals that, standard genetic algorithms settings of population size is 50, crossover probability is 0.6 and mutation probability is 0.001 which were proposed by De Jong in 1975 after conducting an extensive experiments.
V
CONCLUSIONS
Optimum combinations of population size, crossover probability and mutation probability values that are successfully implemented in previous studies are reported in this survey paper in order to serve as a look up table to future researchers. Crossover probability and mutation probability are positively related in implementation of genetic algorithms but the correlation value is not significant. FURTHER RESEARCH Further review will be conducted to include application domain, type of neural networks being optimized by 41
International Symposium on Mathematical Sciences and Computing Research 2013 (iSMSC 2013) 6-7 December 2013, Perak, MALAYSIA. Paper ID.:CS_22
[18]
genetic algorithm, results obtain and design issues in each paper selected for the future review.
[19]
ACKNOWLEDGEMENT We wish to acknowledged TETFUND Nigeria for funding this research work.
[20]
REFERENCES [1] [2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[21]
M. Mitchell, An introduction to genetic algorithms. Massachusetts:MIT Press, 1998, ch. 1, Pp. 1 – 6. M. Sakawa, Genetic algorithms and fuzzy multiobjective optimization. Dordrecht: Kluwer Academic Publishers, 2002, ch. 3, pp. 234 – 244. D. A. Coley, An introduction to genetic algorithms for scientist and engineers, Singapore: World Scientific Publishing Co. Pte. Ltd, 1999, ch. 2, pp. 211 – 223. D. E.Goldberg, Genetic algorithms in search, optimization, machine learning. Reading :Addison Wesley,1989, ch. 3, pp. 298 – 322. F. Guo, and M. Qi, “Application of Elman Neural Network based on Improved Niche Adaptive Genetic Algorithm,‖ in proc. 2nd IEEE International Conf. on Intelligent Control and Information Processing, 2011, pp.660 – 664. D. Season, Non – linear PLS using genetic programming. PhD thesis submitted to school ofchemicalengineering and advance materials, University of Nwecastle, 2005. J. Zhang, Y. Zhang, and R. Gao, ―Genetic Algorithms for Optimal Design of Vehicle Suspensions,‖ in proc. Engineering of Intelligent Systems, IEEE International Conf. on, 2006, pp. 1 – 6. J. Liu, ―Application of fuzzy neural networks based on Genetic Algorithms in Integrated Navigation system,‖ in proc. IEEE second conf. on Intelligent Computation Technology and Automation, 2009, pp. 656 – 659. D. Zhang, and L. Zhou, ―Discovering Golden Nuggets: Data Mining in Financial Application,‖ in proc. IEEE Trans. on Systems, Man, and Cybernetics—part c: Applications and Reviews, vol. 34, no. 4, pp. 513 – 522, 2004. D. J. Montana, and L. D. Davis, ―Training feedforward networks using genetic algorithms,‖ in Proc. of the International Joint Conference on Artificial Intelligence. Morgan Kaufmann, 1989, pp. 762 – 767. A. Ganatra, Y. P. Kosta, Panchal, G and Gajjar, C. ―Initial classification through backpropagation in a neural network following optimization through GA to evaluate the fitness of an algorthms‖. Inter. J comput sci inf Tech, vol. 3,no. 1, 98 – 116, 2011. R. Z. Abu, ―Genetic – neural approach versus classical approach for Arabic character recognition freeman chain feature extraction,‖ The International Arab J Inf. Tech, vol.2,no. 4, pp. 291 – 300, 2005 V. R Saishanmuga, and S. P. Rajagopalan, ―A neuro – genetic system for face recognition,‖ Int. J comput. sci. Issues, vol. 9, no.3, pp. 263 – 267, 2012. R. Tehrani , F. Khodayar, ―A hybrid optimization artificial intelligent model to forecast crude oil using genetic algorithms‖. Afr J Bus Manag, vol. 5, no.34, pp.13130 – 13135, 2011. P. Yacci,. ―Feature selection of microarray data using genetic algorithms and artificial neural networks‖ Msc thesis submitted to department of biological science, Rochester Institute of Technology, 2009. E. Kalyvas, Using neural networks and genetic algorithms to predict stock market returns. Msc thesis submitted to faculty of science and engineering, University of Manchester, 2001. S.D Ahmed, F. Bendimerad, L. Merad, and S. D. Mohammed, ―Genetic algorithms based synthesis of three dimensional microstrip arrays‖, Inter Arab J Inf. Tech. vol.1,no. 1, pp. 80 – 84, 2004
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
42
H. Ali, and H. Al-Salami, ―Timing Attack Prospect for RSA Cryptanalysts Using Genetic Algorithm Technique‖. Inter Arab J Inf. Tech. vol. 2,no. 3, pp. 182 – 190, 2005. J.J. Grefenstette, J. J. ―Optimization of control parameters for genetic algorithms‖. IEEE Trans Syst. Man, and Cybernetics, vol. 16, no. 1, pp. 122–128, 1986. K. Krawiec, ―Generative learning of visual concept using multi objective genetic programming‖ Pattern Recognition Letters. Vol. 28, pp. 2385 – 2400, 2007. I. De Falco, D. R. Balio, D. A. Cioppa, E Tarantino,―Breeder genetic algorithms for airfoil design optimization,‖ Evolutionary Computation, in Proc. of IEEE International Conference on, 1996, pp. 71 – 75. M. W. Bartschi, ―A genetic algorithms for resource – constrained scheduling,‖ PhD thesis submitted to department of mechanical engineering, Massachusetts Institute of Technology, 1996. Y. Huang, and K. Shi, ―Genetic algorithms in the identification of fuzzy compensation system,‖ System, man, and cybernetics, IEEE international conference on. 1996, vol 2, pp. 1090 – 1095. J. Renders, S. P. Flasse, ―Hybrid methods using genetic algorithms for global optimization,‖ IEEE Tran. Syst. Man, cybernetics – part B: cybernetics, 26 (2): 243 – 258, 1996. F. Vavak, and T. C. Fogarty, Comparisonof steady state and generational genetic algorithms for use in nonstationaryenvironments,‖ Evolutionary Computation, in Proc. of IEEE International Conference on,1996, pp. 192 – 195. N. Chaiyaratana, and A. M. S. Zalzala,―Hybridisation of neural networks and genetic algorithms for time-optimal control,‖ Evolutionary Computation, CEC 99, in Proc. of the Congress on, vol 1, doi: 10.1109/CEC.1999.781951, 1999, vol. 1. F. K. Man, S. K. Tang, and S. Kwong, Genetic algorithms: Concepts and Designs, Springer-Verlang, Heidelberg, 1999. H. H. Ammar, and Y. Tao, ―Fingerprint registration using genetic algorithms,‖ Application-Specific Systems and Software Engineering Technology. Proceedings. 3rd IEEE Symposium on, pp. 148 – 154, 2000. C. Jareanpon, W. Pensuwon, R. J. Frank, and N. Davey, ―An adaptive RBF network optimised using a geneticalgorith m applied to rainfall forecasting,‖ Communications and Information Technology, 2004. ISCIT 2004. IEEE International Symposium on, 2004, vol 2, pp. 1005 – 1010. X. Yu, and B. Meng, . ―Research On Dynamics in Group Decision SupportSyste ms Based On Multi-Objective GeneticAlgorithms,‖ Service Systems and Service Management, IEEE International Conference on, 2006, vol 2, pp. 895 – 900. E. Alexandre, L. Cuadra, and M. Rosa, ―Feature selection for sound classification in hearing aids through restricted search driven by genetic algorithms,‖ in proc. IEEE Tran. Audio Speech and Language processing, 2007, vol. 15, no.8, 2249 – 2256. X. Meng, B. Song, ―Fast genetic algorithms used for PID parameter optimization,‖ IEEE International Conference on automation and logistics, 2007, pp. 2144 – 2148. M. Hamdan, ‖A heterogeneous framework for the global parallelization of genetic algorithms‖. Int. Arab J Inf. Tech., vol. 5, no. 2, 192 – 199, 2008. Y. W. Ka, and L.Y. Chi, ―Positioning Weather Systems from Remote Sensing Data Using Genetic Algorithms,‖ Computational Intelligence for Remote Sensing, vol.133, pp. 217 – 243, 2008. A. Sorsa, R. Peltokangas, and K. Leiviska, ―Real – Coded genetic algorithms and nonlinear parameter identification,‖in proc. 4th International IEEE Conf. ―Intelligent System”, 2009, pp. 10 – 42- 10 – 47. J. Ai – ping, and H. Feng-wen, ―Methods for optimizing weights of wavelet neural network based on adaptive annealing genetic algorithms,‖ Industrial Engineering and
International Symposium on Mathematical Sciences and Computing Research 2013 (iSMSC 2013) 6-7 December 2013, Perak, MALAYSIA. Paper ID.:CS_22
[37]
[38]
[39]
[40]
[41]
[42]
[43]
[44]
Engineering management, in proc. IE & EM ’09. IEEE 16th International Conf. on,2009, pp. 1744 – 1748. Y. Fan, Q. Liang, Y. A. Wei, ―Generalized pattern matching approach for multi – step prediction of crude oil price,‖ Energy Economics, vol. 30, pp. 889 – 904, 2008. P. Kromer, J. Platos, V. Snasel, Snasel.―Modeling permutation for genetic algorithms,‖ in proc. IEEE International Conf. on soft computing pattern recognition, 2009, pp.100 – 105. C. Lin, ―An adaptive genetic algorithms based on population diversity strategy,‖ in proc. IEEE 3rd International Conference on genetic and evolutionary computing, 2009, pp. 93 – 96. S. Y. Chin, V. Ganapathy, and L. M. Ooi, ―Improved genetic algorithms based optimization path planning for mobile robot,‖ IEEE Intertional Conference on Control, Automation, Robotics and Vision, 2010, pp.1565 – 1570. Y. Lizhe, X. Bo, and W. Xiangjie, ―BP network model optimized by adaptive genetic algorithms and the application on quality evaluation for class teaching,‖ in proc. Future Computer and Communication (ICFCC), 2nd International Conference on, 2010, vol. 3, pp. v3 – 273 – v3 – 276. R. Belevičius, S. Ivanikovas, D. Šešok, S. Valentinavičius, and J. Ţilinskas, ―Optimal placement of piles in real grillages: experimental comparison of optimization algorithms,‖ Information Technology and Control, 2011, vol. 40, no..2, pp. 123 – 132. D. Capko, A. Erdeljan, S. Vukmirovic, and I. Lendak, ―A hybrid Genetic Algorithms for partitioning of data model in distribution management system,‖ Information Technology and Control, 2011, vol. 40, no. 4, pp. 316 – 321. I. Saenko, I. Kotenko, ―Genetic algorithms for role mining problem,‖ in proc. IEEE 19th Int. Euromicro Confefernce Parallel, distributed and network based processing, 2011, pp. 646 – 650.
[45]
[46]
[47]
C. T. Capraro, I. Bradaric, G.T. Capraro, L.T.Kong, ―Using genetic algorithms for radar waveform selection,‖ Radar Conference, 2008. RADAR '08. IEEE, 2008, pp. 1 – 6.
[48]
P. Guo, X. Wang, Y. Han, ―The enhanced genetic algorithms for the optimization design,‖ in proc. IEEE Int. Conf. Biomedical Eng Inf. pp. 2990 – 2994, 2010.
[49]
R. Leardi, A.G. Lupianez, ―Genetic algorithms applied to feature selection in PLS regression: how and when to use them. Chemometrics and Intelligence laboratory syst., vol. 41, pp. 195 – 207, 1998. T. Murata ,H. Ishibuchi, ―MOGA: multiobjective genetic algorithms,‖ Evolutionary Computation, IEEE International Conferenceo n, doi:10.1109/ICEC.1995.489161,1995. M.H. Rafiul, B. Nath, and K. Michael, (2007). ―A fussion of HMM, ANN and GA for stock market forecasting‖. Expert system with applications, 33: 171 – 180, 2007.
[50]
[51]
43
L. Zhang, and X. Zhang, ―Measurement of the optical properties using genetic algorithms optimized neural networks,‖ Photonics and Optoelectronics (SOPO), Symposium on,2011, pp. 1 – 4. Z. Laboudi, S. Chikhi, ―Comparison of genetic algorithms and quantum genetic algorithms,‖ Int. Arab J Inf. Tech. vol. 9, no.3, pp. 243 – 249, 2012.
