Journal of Computational Science and Technology
Vol. 5, No. 1, 2011
Technique for Reducing the Time Required for Local Optimum Searching in a Hybrid Genetic Algorithm* Supakit NOOTYASKOOL** and Boontee KRUATRACHUE** ** Department of Computer Engineering, Faculty of Engineering, King Mongkut's Institute of Technology Ladkrabang, Bangkok Thailand. E-mail:
[email protected]
Abstract Many applications use a combination of a local optimum search (LOS) and a genetic algorithm (GA), called a hybrid genetic algorithm (HGA), to solve problems. This hybrid can improve the performance of finding optimum solutions, but the HGA may produce redundancy when applying an LOS to inappropriate populations. This redundancy is the cause of high computation time, and it generates premature convergence and decreases HGA performance. This research therefore aims to reduce redundant LOS in HGAs. We propose a new technique called diversity selection (DS) and measure redundancy when applying an LOS in HGA. In this work, the DS selects appropriate populations to which to apply an LOS. The experiment then compares DS with other HGAs on numerical optimization problems. In addition, the HGAs were tested on two LOSs, a Nelder-Mead method and a quasi-Newton method to compare the speed of finding the optimum point in an LOS. The experimental results show that DS was able to quickly find the optimum point and give fewer redundant LOSs than other HGAs. Key words: Hybrid Genetic Algorithm, Diversity Selection, Nelder-Mead Method, Quasi-Newton Method, Optimization Technique
1. Introduction
*Received 27 Oct., 2010 (No. 10-0491) [DOI: 10.1299/jcst.5.38]
Copyright © 2011 by JSME
Genetic algorithms (GAs) are generally based on the principle of natural selection. The first GA was proposed by Holland (1), and the first detailed implementation of a GA is found in the literature of Goldberg (2). The GA represents a solution problem in form chromosomes in a population, which encode candidate solution to an optimization problem by evolves toward better solution. In each generation, populations in GA generate a new population with inspiring natural evolution, such as selection, crossover and mutation. GAs are excellent in global optimization properties, but problems with large search-space or many local optima lead to a lack of good convergence criteria and exploitation capabilities, as shown in the work of Rander and Flasse (3). Increased efficiency of GAs, however, can be realized in many ways. One available way is combining a GA with a local optimum search (LOS) to create a hybrid genetic algorithm (HGA). In the past, many researchers have expressed various styles of HGA to improve performance of finding optimum solution Yen et al. (4) classified the types of HGA architecture into four categories: pipelining hybrids, asynchronous hybrids, hierarchical hybrids, and added operators. Gudla (5) then reclassified the pipelining hybrid technique and changed the model names. Based on this work, Gudla listed three basic attributes of HGAs: 1) Pre-hybridization (PreGA) designs to apply the GA after applying the LOS, 2) post-hybridization (PosGA) designs to apply the
38
Journal of Computational Science and Technology
Vol. 5, No. 1, 2011
GA before applying the LOS, and 3) organic hybridization designs to integrate the LOS as an operator in the GA (Fig. 1). However, an LOS applied in organic hybridization can perform many styles, which have different criteria for selecting a population to input into the LOS. For example, Yun (6) designed an LOS application in two parts: high fitness populations and populations measured by the similarity co-efficient method. Kwong et al. (7) designed an LOS for the offspring in every generation and an LOS for parents in every ten generations. Additionally, Deep et al. (8) and Guo et al. (9) designed a HGA that selected all populations for application to an LOS. Oh et al. (10) designed a HGA that selected only the offspring to apply to an LOS. Their HGA models are different in applying LOSs. An LOS is the iteration process with many loops to find a local optimum point. Many LOSs use high computation time, so selecting a population to apply to an LOS should be done carefully, selecting only appropriate populations to reduce computation time. We divide the many HGAs in organic hybridization into three models. Age selection (AS) utilizes the population ages for applying an LOS to a group of parents or offspring. Fitness selection (FS) utilizes the fitness value of a population as a criterion for applying an LOS. Diversity selection (DS) utilizes the degree of similarity of a population as a criterion for applying an LOS. In this paper, however, we focus on designing DS in organic hybridization. Before giving the details on this developed DS, we extend our studies to each of the theoretical HGAs.
Fig. 1
A group of hybrid genetic algorithms.
For PosGA, Gudla (5) solved the multimodal optimization problem by combining a GA with a conjugate gradient algorithm. Their model was interested in the control-stopping of the GA cycle with a Fibonacci value. Chelouah and Siarry (11) solved the multimodal optimization problem by combining the GA with a Nelder-Mead method (NMM). The designed HGA used a measuring distance between the best population and the new populations to control the termination of the GA cycle, and the NMM was then applied to the best population. Specific to the PosGA, a GA runs a global search to ensure the results approach a global optimum, and the LOS is then applied to the best population to move directly to an optimum point. However, very large spaces are a problem for this method, as it has many local optimum points or unknown locations of the optimum points, making the termination of the GA cycle difficult to determine. For PreGA, Chan (12) used a heuristic LOS method applied to initial populations of GA to move initial populations toward their local optimum points. These populations are then used in a GA process to find the optimum solution. Chan advised that applying an LOS to the initial populations is done to prevent an exhaustive search and to ensure that the initial populations stay at the best point. However, his experimental results compared PreGA with a pure heuristic method and a pure GA, but not with PosGA. Lee et al. (13) used HGA to solve a shape-based block layout problem. Lee’s model seemed to be PreGA, but the model had a few differences. From this work, Lee suggested that the number of LOS loops should not exceed 100 iterations to prevent premature convergence. Organic hybridization can be divided into three models: AS, FS, and DS. These models have been used in various applications and combinations with different LOS. For example, Deep (8) experimented with HGA using a quadratic approximation to solve numerical optimization problems. Deep designed similar AS by dividing populations to
39
Journal of Computational Science and Technology
Vol. 5, No. 1, 2011
apply to an LOS into two groups: parents and offspring. Deep’s design had a few differences in the AS in that the population in the first generation was not applied an LOS. Kwong (7) and Kruatrachue et al. (16) independently designed AS to adjust the parameters of a Hidden Markov Model (HMM). Kwong’s model was interesting in that the number of iteration loops in an LOS, which applying iteration loops to offspring is higher than parents have at three times. This higher number of iteration loops for the offspring gave a chance to search more than a standard exploitation search. The offspring usually represents the exploration search and the parent represents the exploitation search (21). The HGA designed by Yun (6) used FS and DS by dividing into two parts. The first part was FS that used the average of the fitness values of the populations as a criterion for applying an LOS. The second part, DS, used similarity coefficients of the populations as a criterion for selecting an LOS. Organic hybridizations divide the applied LOS populations by two methods: 1) selection by measuring from phenotype, such as AS and FS, and 2) selection by measuring from genotype, such as DS. The fitness value from the phenotype and the similarly coefficient between populations are then used as criteria for applying LOS. The phenotype is a fitness value obtained from calculating an objective function, while the genotype is a value found in each gene of a population. PosGA and PreGA are faced with several problems, including 1) termination before reaching the best point – a problem where an unknown optimum point determines the position for terminating the GA cycle or number of iteration loops; 2) acting redundancy, which is the loss in computation time by applying an LOS to similar populations or populations located at the same position; and 3) premature convergence (14) (15), which occurs when the different levels of fitness of a populations cause the GA to escape from a local optimum. Organic hybridization, however, can compare PreGA and PosGA and is superior in flexible running of GA and LOS. During the computation, organic hybridization can select appropriate populations to apply to an LOS and control the number of iteration loops in an LOS given to a population. The flexibility of organic hybridization reduces the problems of PosGA and PreGA mentioned above. In this paper, we aim to 1) reduce the computation time of HGA by selecting appropriate populations to apply to an LOS, and 2) measure the loss that occurs in applying LOS when HGA selects inappropriate populations to apply to that LOS. To achieve this, we created a new HGA technique in a group of DS. This new DS is different in that it uses the genotype and phenotype of populations to select appropriate populations to apply to an LOS. For more details, both the new HGA technique and the measuring loss observed in applying an LOS will be explained in section 2. In the experiments, all HGAs will be carried out on numerical optimization problems in order to find an optimum solution of ten mathematic functions. Performance of the HGAs responded to each of the LOS methods differently, so we tested on two different LOS methods: 1) a non-gradient method using the Nelder-Mead method (NMM), and 2) a gradient method using the quasi-Newton method (QNM). Section 2 describes the measuring of redundant LOS in HGA and a new DS. In section 3, the experimental setup, which consisted of specific mathematical functions, and all the parameters of the HGA are described in detail. Section 4 presents the experiment results and the conclusions are contained in section 5.
2. Hybrid genetic algorithm techniques In this section, we detail how to measure redundant LOS in HGA, analyze the cause of redundant LOS in organic hybridization models, and propose a new DS. 2.1. Measuring redundant LOS in HGA Applying an LOS to HGAs can achieve an optimum solution quickly, as has been
40
Journal of Computational Science and Technology
Vol. 5, No. 1, 2011
proved by many researchers already. However, their research measured performance based on the resulting optimum points or the computation time used. Measuring a redundant LOS in HGA, however, uses the counting number of the appropriate populations and the number of inappropriate populations that were applied to an LOS. As such, the best HGA should select only appropriate populations to apply to an LOS. Therefore, this technique used only the number of inappropriate populations applied to an LOS. The specifics of the inappropriate population that creates the redundant LOS are divided into two cases: 1) the redundant LOS in case one, LossC1, occurs when the HGA selects similar populations to apply to LOS in the current generation. 2) The redundant LOS in case two, LossC2, occurs when the HGA selects populations by using similarity as a criterion, but the selected population then has a level of fitness not sufficient for selection as a parent in the next generation. Start LossC1 = 0 LossC2 = 0 gen = 1
gen
