Enhanced Genetic Algorithm Applied for Global Optimization ...

Enhanced Genetic Algorithm Applied for Global Optimization ( ) Fadzil Ahmad1,2 ✉ , Nor Ashidi Mat Isa2, Zakaria Hussain1, 1 Saiful Zaimy Yahaya , Rozan Boudville1, Mohamad Faizal Abdul Rahman1, Aini Hafiza Mohd Saod1, and Zuraidi Saad1

1

Faculty of Electrical Engineering, Universiti Teknologi MARA (UiTM), Pulau Pinang, 13500 Permatang Pauh, Penang, Malaysia 2 Imaging and Intelligent Systems Research Team (ISRT), School of Electrical and Electronic Engineering, Universiti Sains Malaysia (USM), 14300 Nibong Tebal, Penang, Malaysia [email protected]

Abstract. Conventional genetic algorithm (GA) has several drawbacks such as premature convergence and incapable of fine tuning around potential region. Thus, new enhanced GA that focuses on new search, crossover and elitism strategy is proposed in this study. It involves solution enhancement phase by performing search among high quality chromosomes via new crossover operator. A modified elitism operation is devised to ensure that the performance of enhanced GA not getting worse than the standard GA in case of solution enhance phase fails to find better chromosomes. In modified elitism, best chromosomes resulted from the enhancement phase and normal population will have to compete among each other to survive in next generation. The enhanced GA has been applied for solving global optimization of benchmark test functions and compared with standard GA. Based on the occurrences of the algorithms produce the best result across different test functions and elitism size; it is proven that the proposed method outperforms standard GA. Keywords: Genetic algorithm · Global optimization · Crossover · Elitism

1

Introduction

Global optimization is a process of obtaining the best solution of a given problem that would have either minimized or maximized the problem objective and to satisfy all the given constrains. This concept is widely used by mankind of modern civilization for solving various problems in many research areas such as in finance [1], engineering design [2] and biomedical [3]. Genetic Algorithm (GA) is a suitable candidate in solving global optimization problem. As a family of global search technique, it performs a global multi-directional search and capable of quickly locating near optimal solution even in complicated search space. It has been successfully applied for global optimization of neural network [4], © Springer International Publishing Switzerland 2015 S. Arik et al. (Eds.): ICONIP 2015, Part II, LNCS 9490, pp. 198–205, 2015. DOI: 10.1007/978-3-319-26535-3_23

Enhanced Genetic Algorithm Applied for Global Optimization

199

aerodynamic shapes and the composite material of a car [5], microarray technology [6] and structural and operational design of skyscrapers, mega cities, highways, factories and sophisticated machines [7]. Unfortunately, GA is unable to fine tune the solution around the potential region due to the incapability of local search. Besides, the pre-mature convergence is another oftenobserved problem in GA [8]. It is a phenomenon in which the quality of the generated offspring is always being inferior to their parent. Therefore in certain optimization problem where the search space is too complex, conventional GA may not able to produce promising result. In order to attain better optimization performances, modifi‐ cation to conventional GA is needed. Furthermore, it is broadly acknowledged that the optimization performances of a GA can be further improved either by integrating new search strategy into the GA or modifying the standard genetic operator such as selection, reproduction, crossover, mutation and elitism scheme. In a standard GA procedure, high quality chromosomes will be destroyed if they are not selected as parent and not involved in elitism process. In this work, new enhanced GA that focuses search around high potential search region related with high quality chromosomes is proposed. It is also based on the theory that the most highly fit parents (both male and female) are most likely to produce healthiest offspring. The new search procedure involves genes exchange among high quality chromosomes via newly proposed crossover operator. It differs from conventional crossover in a way that the destruction of near optimal genetic information in the chromosome is avoided during the crossover operation. It is not guaranteed that the GA enhancement phase via new crossover operator will always produce a better quality chromosome. This is due to the probabilistic nature of the GA approach. If low quality chromosomes are allowed to survive, the search capa‐ bility of the GA will deteriorate. Therefore a modified elitism strategy is introduced whereby the chromosomes before and after the enhancement phase will have to compete among each other to survive in next generation. By implementing the modified elitism, in the case of the enhancement phase fail to produce better chromosomes the perform‐ ance of enhanced GA will not get worse than the standard GA.

2

Methodology

The general idea of the new GA enhancement phase and modified elitism is visualized in Fig. 1. It can be observed that the chromosomes are ranked based on their fitness value and put in a sorted list with the highest rank chromosome be on top of the list. Then, the enhanced GA exploits a group of top ranked chromosomes of size k through selection and modified crossover. It is can be clearly observed that the modified elitism involves two groups before (normal population) and after enhancement (enhanced population) phase. The implementation of the algorithm is summarized as follows: (1) Initial population: The initial population of enhanced GA is randomly generated. Each chromosome in the population consists of 2 components or gene segments; each of them varies within a pre-specified range. Based on the quantified interval, each gene segment is coded using binary value.

200

F. Ahmad et al.

(2) Fitness calculation: The gene segments for each chromosome in binary are decoded to obtain their equivalent decimal values and substituted into the test func‐ tion equation that we intend to obtain its global minimum. Chromosome that returns the smallest value represents the best solution. (3) Selection: Based on the fitness value, the selection method is executed using Roulette rank-based selection [9]. The fittest chromosome occupies the largest area in the Roulette wheel. (4) Modified crossover: The crossover location is controlled so that it only occurs at location that will not split the gene segment. This is to avoid the destruction of near optimal information contain in gene segment. Top quality chromosomes (size k) are obtained from GA population. First group of first gene segments (size l) is created via rank based Roulette selection using the ranking of their respective chro‐ mosome. Second group of second gene segments is created in the same way of first gene segment group is created. The two groups are combined to form a new group of enhanced chromosomes. Selection Modified crossover, size l Initial population 10011… 10110… 11101… Calculate fitness, rank and sort

After enhancement

Calculate fitness, rank and sort

Modified Elitism Before enhancement

Top ranked chromosomes, size k

Pairing

Mutation and crossover

Max gen?

Return best solution End

Normal parent Selection ith population

(i+1)th population

Fig. 1. The architecture of the enhanced GA.

(5) Modified elitism: The best chromosomes before and after enhancement phase will have to compete each other to survive in next generation. E chromosomes are selected from both normal and improved population. The two groups are combined and reordered based on fitness value so that the best chromosome being on the top of the list. The top E chromosomes are then selected and moved to next generation.

Enhanced Genetic Algorithm Applied for Global Optimization

201

(6) Standard crossover and mutation: The parents are selected (Roulette ranked selection) from normal population and paired for the crossover operation. It is followed by mutation operation. These are based on standard crossover and muta‐ tion operation as described in [10]. (7) New generation: The offspring obtain from Step 6 is then combined with the elitist chromosomes (the chromosomes that survive the modified elitism) to form a new generation. All Step 2 until Step 7 are repeated until maximum number of generation is reached (Fig. 2). Select E top ranked chromosomes from both before and after enhancement. Fittest

+ Before enhancement, size E

After enhancement, size E

Only best E chromosomes are selected to survive in next generation Next generation

Combine and reorder according to fitness

Fig. 2. The operation of modified elitism.

2.1 The Test Functions The proposed algorithm is employed for optimizing highly complicated and multimodal benchmarking test functions – Ackley, Rastrigin and Schaffer2 and Shubert. The solu‐ tion domain of these test functions consist of many local optima solutions. Only the algorithms that have reached a certain level of robustness are capable to avoid the local optimal and obtain the global optimal solution. The formulation and search surface of all the test functions can be found in [11]. Their characteristics are tabulated in Table 1. Table 1. The characteristics of the test functions the evaluated range.

Function name

Global minima values

Global minima locations

Evaluated range

Ackley

0

0,0

[−32.768; 32.768]

Schaffer N.2

0

0,0

[−100; 100]

Rastrigin

0

0,0

[−5.12; 5.12]

Eggholder

−959.6407

512,404.2319

[−512; 512]

202

F. Ahmad et al.

2.2 Experimental Setup In order to evaluate the performance of the enhanced GA and to compare with the standard GA, two set of experiments for finding global minimum is conducted by varying the elitism size, E. First set uses standard GA and second set uses enhanced GA. Standard GA does not have the enhancement phase and modified elitism. As a member of stochastic compu‐ tational techniques, GA does not guarantee to produce consistent result every time it is executed. Therefore, the performance of both the standard and improved GA is evaluated based on the average and best result produced from 100 repetitions of the algorithm. All the parameters used for all the experiments are shown in Table 2. All the fixed parameters are determined based on trial and error experiment for good performance. For a fair comparison, all the common parameters between the standard and improved GA are set to the same values. Table 2. The parameter of the GA.

Common GA parameters

Size/value

Population size

50

Generation size

100

Chromosome length

32 bits

Elitism size, E

2, 4 and 6

Selection scheme

Rank based roulette selection

Crossover probability

0.8

Mutation probability

0.05

Improved GA parameters

3

Top ranked chromosome size, k

20

Enhanced crossover chromosomes size, l

20

Results and Discussion

Table 3 tabulates the results of the best global optima solution found from the 100 repe‐ titions of the algorithms for all the test functions and elitism size, E. The algorithms aim to find the minimum value in the solution space of the test functions and the smallest values is the best. At E = 2, it is obvious that, for all the test functions the enhanced GA outperforms the standard GA. At E = 4, enhanced GA produces better results on Ackley and Rastrigin, however on Schaffer2 standard GA outperforms enhanced GA. At E = 6, both enhanced and standard GA score better results on 2 out of 4 test functions. From Table 3, number of occurrences the enhanced and standard produces better results is 8 and 3 respectively. From the results it is proven that the enhanced GA outperforms standard GA in obtaining best global minimum location.

Enhanced Genetic Algorithm Applied for Global Optimization

203

Table 3. The best performance over 100 repetitions of the algorithm. The best results are high‐ lighted in bold font.

Elitism Size, E

2

4

6

Standard GA

2.24E-05

1.05E-05

1.75E-06

Enhanced GA

7.54E-06

5.96E-06

1.60E-06

Standard GA

2.55E-09

2.04E-11

2.25E-12

Enhanced GA

1.38E-09

4.84E-12

1.41E-11

Standard GA

3.15E-11

9.98E-13

2.60E-14

Enhanced GA

1.41E-11

2.30E-12

3.55E-14

Ackley

Rastrigin

Schaffer2

Eggholder Standard GA

−959.6385 −959.6404 −959.6406

Enhanced GA

−959.6404 −959.6404 −959.6407

Table 4. The average performance over 100 repetitions of the algorithm

Elitism Size, E

2

4

6

Standard GA

2.1E-03

4.3E-04

2.3E-04

Enhanced GA

2.0E-03

3.4E-04

1.7E-04

Standard GA

4.5E-05

2.1E-06

1.0E-02

Enhanced GA

8.6E-06

1.9E-06

1.0E-02

Standard GA

1.4E-03

4.0E-04

2.3E-04

Enhanced GA

6.3E-04

2.5E-04

5.2E-04

Standard GA

−933.01

−931.00

−931.44

Enhanced GA

−941.36

−939.67

−938.26

Ackley

Rastrigin

Schaffer2

Eggholder

204

F. Ahmad et al.

Table 4 compares the average performance of the algorithms for all the test functions and elitism size. It is demonstrated that, at E = 2 and 4 the enhanced GA produces better results for all the test function. At E = 6, the results are mixed. The occurrences enhanced and standard GA produces better results is 10 and 1 respectively. To determine the best algorithm considering both the best and the best averaged global optima found, the occurrences for Tables 3 and 4 are total up. Enhanced GA produces 18 occurrences, meanwhile standard GA only scores 4 occurrences. At E = 2, for the best and best averaged performance, the enhanced GA produces better result for all the test functions. This result proves that at low elitism size the search capability of enhanced GA is better than standard GA.

4

Conclusion

This paper introduces enhanced GA that involves solution enhancement phase by performing search among high quality chromosomes via new crossover operator. A modified elitism operation is devised to ensure that the performance of enhanced GA not getting worse that the standard GA in case of solution enhance phase fails to find better results. In modified elitism, best chromosomes resulted from the enhancement phase and normal population will have to compete among each other to survive in next generation. The enhanced GA is applied for solving global optimization problem of highly completed and multimodal test function. The results prove that the capability of enhanced GA in finding global minimum location is better than standard GA.

References 1. Pan, W.-T.: A new fruit fly optimization algorithm: taking the financial distress model as an example. Knowl.-Based Syst. 26, 69–74 (2012) 2. Grossmann, I.E.: Global Optimization in Engineering Design, vol. 9. Springer-Science +Business Media, Dordrecht (2013) 3. Liu, X., Yang, Q., Qing, H., Long, N.: A new global optimization strategy for medical image elastic registration. In: 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), pp. 337–340. IEEE (2012) 4. Ahmad, F., Isa, N.A.M., Hussain, Z., Osman, M.K., Sulaiman, S.N.: A GA-based feature selection and parameter optimization of an ANN in diagnosing breast cancer. In: Pattern Analysis and Applications, pp. 1–10 (2014) 5. Takako, S., Takemura, Y., Schmitt, L.M.: Minimizing wind resistance of vehicles with a parallel genetic algorithm. In: Madaan, A., Kikuchi, S., Bhalla, S. (eds.) DNIS 2014. LNCS, vol. 8381, pp. 214–231. Springer, Heidelberg (2014) 6. Ooi, C.H., Tan, P.: Genetic algorithms applied to multi-class prediction for the analysis of gene expression data. Bioinformatics 19, 37–44 (2003) 7. Renner, G.B., Ekart, A.: Genetic algorithms in computer aided design. Comput.-Aided Des. 35, 709–726 (2003) 8. Rocha, M., Neves, J.: Preventing premature convergence to local optima in genetic algorithms via random offspring generation. In: Imam, I., Kodratoff, Y., El-Dessouki, A., Ali, M. (eds.) IEA/AIE 1999. LNCS (LNAI), vol. 1611, pp. 127–136. Springer, Heidelberg (1999)

Enhanced Genetic Algorithm Applied for Global Optimization

205

9. Lin, G., Huang, C., Zhan, S., Lu, X., Lu, Y.: Ranking based selection genetic algorithm for capacity flow assignments. In: Cai, Z., Tong, H., Kang, Z., Liu, Y. (eds.) ISICA 2010. CCIS, vol. 107, pp. 97–107. Springer, Heidelberg (2010) 10. Goldberg, D.: Genetic Algorithms in Search and Optimization. Addison-Wesley, Boston (1989) 11. Surjanovic, S., Bingham, D.: Virtual Library of Simulation Experiments: Test Functions and Datasets. Simon Fraser University. http://www.sfu.ca/~ssurjano/index.html. Accessed 30 Mar 2014