An Adaptive Genetic Algorithm for Permutation Based Optimization Problems
1
1
Koh Sue-Yi , Leow Soo Kar , Loke Kar Seng
1
1
Monash University Malaysia, No.2 Jalan Sunway, 46150 PJ, Malaysia .
[email protected] , {leow.soo.kar, loke.kar.seng}@infotech.monash.edu.au http://www.monash.edu.my
Abstract. Genetic Algorithms (GA) are a robust heuristic search technique capable of taking on a broad range of optimization problems. In most GAs, components and parameters are predetermined and remain static throughout its run. Much research has indicated that improvements to its performance can be sought by varying some of these components and parameters during its application on a problem. In this paper, a brief survey of the techniques used to vary the GA’s components and parameters is presented. We hypothesize that a GA’s performance and robustness can be enhanced through the ‘online’ adaptation of the operators. An operator based adaptive genetic algorithm (AGA) based on these concepts is designed and implemented. A number of permutation based problems were selected to evaluate the performance of AGA.
1 Introduction Genetic algorithms have been known to be a robust technique in finding optimal or near solutions to many large and complex optimization problems. However, its parameter settings are required to be tuned for the specific problem at hand in order for it to perform efficiently and effectively during its search for an acceptable solution in a reasonable amount of time. While there exist a number of methods that can and have been used to predetermine such settings before running it on a problem, the problem specific nature of some of these settings coupled with the mutual dependence of these settings on one another still render the tuning of a GA to be a tedious process. One suggested improvement would be to facilitate the ease of the application of the GA on different problems by perhaps reducing the amount of
retuning required. This would be quite beneficial, since GAs are applied across a wide spectrum of optimization problems and are required to be retuned each time. Among the settings to be tuned, are those that are related to its operators and consist of the crossover and mutation operators and their rates. These operator-related settings typically receive considerable attention during the tuning process, since they are considered to be some of the main conduits by which exploration and exploitation of the search space is achieved by the GA. Some research has indicated that the implementation and use of more than the usual one crossover and one mutation operator during the run of the GA can be beneficial. However, the increase in the number of operators in a GA results in more parameters being needed to be tuned. Where classic GAs employ the use of settings that remain static throughout its application run, some existing work was found that indicated that by varying these settings appropriately during the GA’s application, improvements can be made to the overall performance of the GA in terms of both quality of the solution found and the amount of time required finding it. Generally, the approaches to settings variation can be divided into two categories. The first consists of variation that is predetermined before the run of the GA. However, with the difficulty involved in just finding good static settings for a GA, predetermining varying settings would be considerably more difficult [1]. Moreover, since such settings are problem specific, the predetermined varying settings would not work as well for different problems. The second approach adaptively determines and varies the settings in question during the application of the GA itself. Hence, this approach, not only the changing requirements of a GA throughout its application be met, but also possibly the different requirements associated with being applied on different problems may be able to be met automatically without the need for having to retune these settings. This work presented here attempts to use this approach to develop a more efficient GA. This paper is organized as follows: Section 2 presents some techniques that have been known to be used to tune GA settings. In section 3, the design of the proposed variation of the GA called Adaptive Genetic Algorithm (AGA), is described in detail. In Section 4, the implementation of the experiments to evaluate the performances of the AGA is described and the results are presented and analyzed. Finally, Section 5 provides a summary of the research.
2 Related Work One requisite of the classic GA for it to be efficient is that it has to be configured with settings suitable to the problem at hand. These settings come in the form of components and parameters such as crossover and mutation operators, crossover and mutation rates, chromosome representation, population sizes and replacement techniques, with some settings being more sensitive to the problem than others. With the
large number of choices of such settings, GA users have been known to use some methods to predetermine some of these settings before the run of the GA. The most commonly used method is empirically through hand tuning. This is where the GA’s settings are determined by first experimenting with a selection of combinations of settings and then selecting the combination that gives the best results. In an attempt to reduce this burden of choice, DeJong [2] used this technique to find some recommended settings that were ‘good’ for a set of numerical function optimisation problems, known as the DeJong test suite. Using a traditional algorithm with a fixed population size, DeJong worked out ‘good’ parameter values for single point crossover and bit mutation [3]. Another method of parameter tuning involves the use of another GA to evolve parameter settings for the GA in question. Using this method, Grefenstette [4] managed to find settings for the traditional GA with a fixed population size that were able outperform DeJong’s on the DeJong test suite. Similar to finding settings empirically through hand tuning, but with more aggression and thoroughness is a technique known as brute force search. Davis [3] and David et al. [5] used this technique on the DeJong suite and some additional problems. These techniques mainly focused on optimal tuning the operator rates. Despite the presence of such techniques, the number of components and the mutual dependence of these components on one another still render the finding of appropriate settings a tedious task in itself. The crossover and mutation operator are considered to be two of the main conduits by which to control exploration and exploitation of the search space for a GA. This is partly why most of the work performed on finding good configurations for a GA consists of keeping the other settings constant and just looking for good operator rates. Much research has advocated the use of more than the usual one mutation operator and one crossover operator in a GA, stating that this can be beneficial to the GA’s performance [6, 7, 8, 9,10, 11] Selecting the appropriate operators for a GA can be difficult. Previously, a crossover operator with a small number of crossover points was the recommended operator, partly because of the lesser amount of disruption caused to schemata. However, later research indicated that there were situations where a crossover with a higher number of crossover points proved to be more beneficial [11]. Many factors such as the disruption analysis of an operator on a problem space and population size have to be taken into account in the decision to decide on which crossover is the overall better choice for a run of the GA on a particular problem. However, because theory appears to be inadequate for the predetermination of the best overall operators to be used, an adaptive mechanism is suggested to enable the GA to select the optimal crossover operator as it runs. In one of the earliest works of variation of settings in a GA, Davis [6] suggests the division of operators into two categories based on the roles they play. In one category are those operators that directly improve a chromosome, while in the other, operators that set the stage for such improvements. For more generic GAs, where the goal is to be able to find good solutions for a range of problems in a reasonable amount of time, having just one crossover and
mutation operator is insufficient. The abilities of the crossover and mutation operators are also influenced by solution spaces from different problems. This is because different forms of schemata exist in solutions for different problems [12, 13] and different operators, crossover or mutation, are better at exploiting and exploring different forms of schemata. Where a crossover operator successfully exploits wanted traits from a particular solution space, for a solution space of a different problem, it may be that it is able to exploit the wanted traits, but at a different rate of efficiency (thus explaining the need for different operator rates) or unable to do so at all. Thus, part of the criterion for a classic type GA to be able to perform well for a specific problem, is to have a crossover operator that is able to exploit schema types found in the problem solution space and a mutation operator which enables the crossover operator access to all the possible variations of the types of schemata it exploits. Through operator adaptation, the GA will be able to find the most suitable operators for the particular search space of the problem it is currently dealing with. There are two categories of approaches by which researchers have attempted to meet the changing requirements of exploration and exploitation during the run of the GA. The first category consists of predetermined variation based on time and the second, variation through adaptation. Approaches in this category rely on a set of predetermined time varying settings such that the settings of the GA are varied with time or the number of generations, while disregarding what the actual state of the population may be. Typically, much study, theoretical or empirical, is done beforehand to first determine how the settings in question are to be varied, followed by the determination of better or optimal variation steps and the frequency of such variations. Considering the difficulty involved in finding appropriate static settings for a classic GA where a thorough search of the space of settings may be more difficult than the problem to be solved [1], finding appropriate settings for different points of the run of the GA would prove to be a bigger headache. Some approaches in this area include a Simulated Annealing like strategy where the mutation probability is reduced from 0.5 to 0 with the change in generations [14] and in [15] where several time varying schedules for the mutation rate were studied in a GA that was to optimize the settings for a furnace and it was empirically found that a mutation rate that decreases exponentially over the course of the run of the GA was one of the better time varying schedules. The adaptive variation approach, also known as an online approach [11] involves the utilization of information gained about the state of the GA during the run of the GA, which may consist of information about the population as a whole, individuals in the population or components of the GA, to adaptively vary its settings there and then. Based on how the adaptive mechanism is employed, the approaches in this category can further be split into three subcategories, namely, the rule based approach, [16], the co-evolutionary approach generations [11, 17] and the evolutionary approach.[18].
3 The Adaptive Genetic Algorithm (AGA) The proposed operator based GA employs the evolutionary approach to adaptive variation and thus, utilizes two genetic algorithms, an operator level (OL) GA and a problem level (PL) GA. The problem level GA is used to search the problem space for good or optimal solutions to the problem at hand, while the purpose of the operator level GA is to synchronously adapt the operators applied in the problem level GA according to the status and requirements of its search. A list of 17 operators, after a survey of literature is selected for the OL GA to use [19]. To evaluate the operator level population, the encoded operators in each chromosome are utilized on one generation of the PL GA with the same PL population. After which, feedback on the performance of each chromosome in the form of a resulting PL population is returned to the OL GA and each OL chromosome is assigned fitness values based on its relative performance to the other OL chromosomes. Among the resulting PL populations, the one that resulted from the OL chromosome with the highest fitness value is selected to be the next PL GA population for the next generation of the OL GA. In the next two subsections, the main components of the operator level GA and the problem level GA are described in detail. 3.1 Operator Level Genetic Algorithm (OL GA) The operator level genetic algorithm can be considered as the main component of the AGA. The application of AGA on a problem starts and ends in OL GA. It basically consists of the same components and structure as a classic GA, except that the implementation style or algorithm is slightly different. Its pseudo code is shown in Figure 1, where it should be noted that for this pseudo code as well as the other pseudo code in this section, the words in italics refer to variables names, while the words or phrases starting with a capital letter refer to function calls or actions to be performed. For a more detail description of the pseudo code of OL GA the reader is referred to [19].
OL_GA (end_condition) BEGIN Generate population Generate plPopulation PLEvaluate plPopulation /* evaluate operator level GA’s population returns best resulting PL population */ plPopulation = Evaluate (population, plPopulation) WHILE end_condition has not been reached best_chromosome = Get_fittest_chromosome (population) Initialise children of size num_children_required num_children_required = crossover_rate * population_size num_parents_required = Get_num_parents_required (crossover_operator) num_children_produced = Get_num_children_produced (crossover_operator) num_children = 0 WHILE num_children < num_children_required parent_chromosomes = Select (population, num_parents_required ) child_chromosomes = Perform_crossover_operation (parent_chromosomes) /* ensure that children does not go out of bounds during Append */ Append child_chromosomes to children Increment num_children by num_children_produced ENDWHILE new_population = Perform_replacement (population, children) num_mutations_required = mutation_rate * population_size num_mutations = 0 WHILE num_mutations < num_mutations_required Randomly_select chromosome from new_population Perform_mutation_operation (chromosome) Increment num_mutations ENDWHILE Replace last chromosome in new_population with best_chromosome plPopulation = Evaluate (population, plPopulation) Evaluate_population (population) population = new_population ENDWHILE END
Fig. 1. Pseudo Code of the Operator Level GA In Figure 1, end_condition refers to the stopping condition of the run of the AGA. This stopping condition can be in terms of the number of generations run for, a time limit or the quality of the current best solution found for the problem at hand. To distinguish between the population of the OL GA and the population of the PL GA, the first is referred to as population, while the latter is referred to as plPopulation. The function Generate is used to randomly generate population of permutation encoded chromosomes and works by randomly producing n numbers of values between 0 and 1, where n is equivalent to the length of a chromosome, and using the rank of each number in the series to denote the value of each corresponding gene in
a chromosome. Similar to how the populations of both levels of GAs are distinguished, PL_Evaluate refers to the evaluation function of the problem level GA and the function Evaluate, shown in the pseudo code in bold refers to the evaluation function of the OL GA. Get_num_parents_required and Get_num_children_produced are functions used to retrieve the number of parents required by the specified operator and the number of children it produces. For example, the Edge Recombination Crossover operator requires two parents and produces one child, while the Cut Paste operator requires one parent and produces one child. Although the operators utilised by the OL GA are fixed throughout the application of the AGA, these functions are there to allow for flexible changes to the selection of these operators to be made should it be deemed necessary. Other functions to take note of are the Select, Perform_replacement and the Evaluate_population functions. Select refers to the parent selection technique, while Perform_replacement refers to the replacement strategy and are described in the latter sections. Evaluate_population should be distinguished from the PL_Evaluate and Evaluate functions and basically ranks the chromosomes in the population according to their fitness values and calculates the standard deviation, mean and total of the fitness values of the chromosomes in the population. It should be noted that just after mutation has been performed and right before the evaluation of the OL population, the last chromosome in new_population is replaced with the best_chromosome found in the previous population. This is to ensure that the previous best OL chromosome is handed down to the next OL generation’s population, just in case it was modified through mutation. 3.1.1 Chromosomal Representation The 17 operators chosen from [19] were used in the operator level GA. The search space of OL GA is the configuration space of the problem level GA and the candidate solutions consists of sets of operators to be applied on the PL GA during its run. In order to evolve this set of operators, the OL GA employs a permutation based encoding for its chromosomes. Based on this encoding, the genes are the operators and are represented by integer values in the chromosome. Each chromosome is comprised of all the available operators in the search space and it is the positions of the genes in the chromosome that determine which operators are passed down to the PL GA. A typical operator level chromosome is shown below: 8
3
1 5 1 1 6 1 9 1 7 2 1 4 5 6 7 2 Fig. 2. Example of an Operator Level Chromosome
1 1
1 3
The value of each gene denotes the operator being represented by it. The first n genes denote the operators that are to be passed to the PL GA, where n is a predetermined variable set before the run. Should n = 5, for example, the operators represented by values 8, 3, 14, 5 and 15 are passed to the PL GA to be utilized during its search.
4
10
The main purpose and advantage of having such a representation is that genes are ensured to never disappear from the population, meaning that all the possible traits are present in the population at all times. However, this means that the number of operators to be used in the PL GA must be specified beforehand. While this adds to the tuning requirements of the AGA, it should be noted that the need to tune other components of the AGA such as operators to be used and their rates have been eliminated, thus already eliminating much of the tuning needed. 3.1.2 Operators Following classic style genetic algorithms, the operator level GA of AGA uses a standard set of one crossover operator and one mutation operator, and are referred to as crossover_operator and mutation_operator respectively in Figure 1, where a call upon them to perform is denoted by Perform_crossover_operation and Perform_mutation_operation. Noting that only the first n encoded operators in the OL chromosome will be utilised during the evaluation of the OL chromosome, perhaps positional information can be said to be the primary information that needs to be captured. After this observation, it was decided that the Coin Toss operator [19] would play the role of the crossover operator, and Random Create Coin Toss operator would play the role of the mutation operator, where the use of this mutation operator ensures that each gene in each locus of the chromosome has a chance of being mutated. As for their rates, after some arbitrary testing, these are set to be 0.8 and 0.2 for the crossover and mutation operators respectively. 3.1.3
Evaluation Functions
In order for synchronous adaptation to take place in accordance with the change in status of the object to be adapted, frequent communication is required to occur between the adaptation mechanism and that object. In this case, the adaptation mechanism is the OL GA and the object to be adapted is the configuration of the PL GA, which consists of the set of operators to be used. For successful adaptation to occur, the OL GA must be able to accurately gauge the status and performance of the current and various configurations of the PL GA. This is done through the evaluation of the OL chromosomes and thus, in AGA, communications between the OL GA and the PL GA occur in the operator level evaluation function, known as Evaluate in Figure 1, and are in relation to the performance of the configuration of the PL GA. Because it is the settings of the PL GA that guide the search through the PL search space, and it is the evaluation function of the OL GA that guides the forming of the sets of operators used in the PL GA through the assignment of fitness values to these sets of operators, it can be said that the OL GA’s evaluation function is one of the main components whose performance is vital to the overall performance of the AGA on the problem it is applied on. Fig-
ure 3 below illustrates the interaction between the two levels of AGA and roughly what goes on in the fitness function of the OL GA.Operator Level GA
Feedback on Performance
PL Population Operators Problem Level GA
Fig. 3. Interaction between the Operator Level and Problem Level of the AGA For a more detailed illustration of what goes on during the evaluation of the OL population, the pseudo code of the Evaluate function is given below. In this pseudo code, for each chromosome in the OL population, the function PL_GA_set_operators is used to decode the selection of operators from it and configure the problem level GA with these operators, so that the PL GA may search through the specified PL population using these operators in PL_GA_Evolve. The resulting PL populations are then kept in temp_plPopulations as feedback for each of these chromosomes. After this is done, the fitness values of the OL chromosomes are calculated and the PL population resulting from the OL chromosome with the highest fitness values is deemed to be the best and kept for use for the next OL generation. For record keeping, the number of children produced by each operator in the best OL chromosome is stored.
Population Evaluate (population: Population, plPopulation: Population) BEGIN Initialise the array temp_plPopulations to be of size population_size FOR each chromosome in population operators = Decode_operators (chromosome, num_operators) PL_GA_set_operators (operators) pop = plPopulation /* num_generations in AGA is set to one */ FOR (num = 0; num < num_generations; num+1) pop = PL_GA_Evolve (pop) ENDFOR Append pop to temp_plPopulations ENDFOR /* Calculate_and_assign_fitnesses returns best problem level population index based on the fitness values assigned by the fitness function */ best_plPopulation_index = Calculate_and_assign_fitnesses (population, temp_plPopulations) best_plPopulation = temp_plPopulations[best_plPopulation] /* Updates the number of children produced by each of the operators in the problem level GA for during the evolution of best_plPopulation */ Update_PL_operator_children_produced (best_plPopulation_index) RETURN best_plPopulation END
Fig. 4. Pseudo Code of Evaluate in OL GA In Figure 4, the OL GA repeatedly passes the same PL population to the PL GA during the evaluation process of each OL chromosome, as seen in the line PL_GA_Evolve (pop). While this may first appear to be odd, it is really not so. In order to gauge how well a set of operators performs, it must be taken into account that the state of the population is an important factor on how operators perform the fitness value of a set of operators that resulted from being applied on the population at generation g may no longer be relevant with its application on the changed population at generation g + 1. Thus, to accurately gauge the performances of the various sets of operators at each point of time, these sets of operators must be applied on the same population that is repeatedly passed down from the OL GA. While this may result in an appearance that no adaptation can take place as a result of the constantly changing fitness landscapes faced by the OL GA, it is hypothesized that such changes between generations are gradual enough that OL GA is still able to adapt the configuration required at different points of time, hence explaining the setting of the number of generations to run a particular configuration for as one. Another reason for the OL GA to pass the PL population to the PL GA is that during the evaluation of the n OL chromosomes in the OL population, n PL populations are created as a result, and the OL GA is where the analysis of this feedback to gauge the performances of the various OL chromosomes in the form of fitness values assigned takes place, thereby resulting in the decision of which PL population is best to be retained for utilization in the next generation of the OL GA in the same manner.
Before the analysis of the performances of the various configurations encoded in the OL chromosomes can take place, a look what is desired in a performance is a must. This can take the form of measuring the performance in terms of viewing selected portions of the output produced, the overall resulting output, or a mixture of both. In the first category reside individualistic performance measures such as the number of children produced that are better than their parents and the fitness of the best chromosome found, while the second category consists of overall performance measures such as the mean and standard deviation of the fitness values of the chromosomes in the population. Specifically, 7 fitness functions have been formulated in an attempt to be able to properly gauge and capture the desired properties of the performances of the various configurations on the PL GA, or in other words, the fitness values of the chromosomes in the population of the OL GA. The candidate fitness functions are shown below. (1)
Fitness _ Function _ 1OL = R( f b )
(2)
Fitness _ Function _ 2 OL = R( BC )
Fitness _ Function _ 3OL = R( BC ) × Weight BC + R( f sd ) × Weight sd (3)
+ R( f m ) × Weight m (4)
(5)
(6)
(7)
Fitness _ Function _ 4 OL =
Fitness _ Function _ 5 OL =
Fitness _ Function _ 6 OL =
Fitness _ Function _ 7 OL =
{ {
{ {
R( f b )
f b > prev. f b
R( f sd ) + R( f m ) ,
f b ≤ prev. f b
R(BC )
f b > prev. f b
R( f sd ) + R( f m )
f b ≤ prev. f b
R( f b )
f b > prev. f b
R( f sd ) + R( BC )
f b ≤ prev. f b
R(BC )
f b > prev. f b
R( f sd ) + R( BC )
f b ≤ prev. f b
where R(x) is a function that returns the rank of the parameter x among the x values associated with the chromosomes in the OL population, such that the rank of 1 is considered the lowest rank, the rank of value equivalent to population_size is considered the highest rank, larger values of x are assigned higher ranks, and for similar values of x, they are assigned the same highest possible rank, that is, values of {3, 6, 2, 5, 5} for example, are assigned corresponding ranks of {2, 5, 1, 4, 4}; fb is the fitness of the best candidate solution found in the associated PL GA’s population; BC is an acronym for Better Children, and is used to denote the number of child chromosomes that have been produced during a particular configuration that have better fitness values than the parents involved in creating them; fsd is the standard deviation of the fitness values of the chromosomes in the associated PL population; fm is the mean of the fitness values of the chromosomes in the associated PL population; WeightBC, Weightm and Weightsd are the weights assigned to BC, fm and fsd respectively, where they are set to WeightBC = 1.0, Weightsd = 0.8 and Weightm = 0.8; and prev.fb refers to the fitness of the best PL chromosome found during the previous OL generation. The first two fitness functions, (1) and (2), utilize individualistic performance measurements. The idea behind equation (1) is to aid the convergence of the OL population, by attempting to exploit traits of OL chromosomes that result in the fittest PL chromosomes being produced, while (2) aids the convergence of the OL population by exploiting traits that result in a higher probability of children produced that are better than their parents. A difference between (1) and (2) is that where (1) will eventually turn OL GA into a random search algorithm should no improvements occur over the best PL chromosome produced from the previous generations, (2) will always maintain some form of control on the evolution of the OL population, while every now and then promoting some randomness when no improvements can be seen to be made by the existing encoded sets of operators. Because the fitness values for the solutions of different problems can be different in terms of how closely knit or far apart they are, the purpose of the function R here, is to ensure that the selection pressure of the selection process for parent chromosomes is solely dependent on the selected selection technique and is not specific to the problem at hand. Unlike the fitness functions described in (1) and (2), functions (3), (4), (5), (6) and (7) employ the use of a combination of individualistic performance measurements and overall performance measures. While for some properties such as fb where what is desired about this property is straightforward and simple like ‘the higher, the better’, for other properties like standard deviation, what is desired is more in the range of ‘not too high, and not too low’ and furthermore, with what may be the actual values varying with the different problems it is applied on, such that with a certain value it is unknown whether to classify it as high, low or average, makes what is desired even more difficult to specify. One way to go about this problem is to specify the wanted properties in relation to the properties found among the results of the various configurations. This can be done by ranking the properties in the way of R and using the rank as an indication of classification of that property. Moreover, another advantage of combining different properties based on their ranks
is that because different properties have different ranges of values, a direct combination of these properties may result in the property that has a higher range of values to be inevitably given a higher weightage. In (3), it is proposed that a population with the best individual found thus far, does not necessarily indicate that that resulting population or the configuration is the best in terms of potential to find better solutions and that diversity and quality of the individuals in the population are also indications of the quality of the population as a whole and should also be taken into account. Hence, standard deviation and mean are added into the equation as gauges of the diversity and quality of individuals respectively. By setting BC to have a higher weightage, convergence of the population is promoted, while at the same time, the lower but similar weights given to standard deviation and mean attempts to ensure that some amount of diversity and quality are promoted in the population. Where (1), (2) and (3) use an unchanging form of the fitness function throughout the run of AGA, functions (4), (5), (6) and (7) varies the fitness function according to the information gotten about the state of the population at hand in an attempt to match the differing requirements of different states of the population. In the latter set of functions, whether there is improvement of fb over prev.fb or not, is used as guiding factor. The idea behind this, is perhaps that the existence of improvements in the best fitness indicate that the settings encoded by the OL chromosomes are doing well and thus convergence or fine tuning of the OL population based on fb in (4) and (6), and BC in (5) and (7) is all that is currently needed. On the other hand, no improvements to the best fitness is perhaps an indication that the settings encoded in the OL population are no longer effective or that the PL population has converged to a local optima, and thus, more diversification in the PL population is required, thereby focusing the fitness function on the diversity or standard deviation, while at the same time, utilizing fm in (4) and (5), and BC in (6) and (7) to ensure the presence of some quality individuals in the PL population. 3.2 Problem Level Genetic Algorithm The primary purpose of the problem level genetic algorithm is to find good solutions to the problem at hand. At the same time, it is also used to evaluate the settings encoded in the OL GA’s chromosomes so that the OL GA may gain feedback about the performances of the settings encoded in these chromosomes as for the current state of the search and further evolve them to find suitable and better settings for a more effective and efficient PL search. In order to serve these purposes, during each generation of the OL GA, each OL chromosome is decoded and the PL GA is configured with these decoded operators that are then used in its search of the current search space in the form of a PL population. How this search is conducted is illustrated in the pseudo code of the PL_GA_Evolve function of the PL GA shown below in figure 5. The operators to be used for the current search, known as operators in Figure 4, are set just before this PL_GA_Evolve function is called in the Evaluate function of the OL GA.
Population PL_GA_Evolve (plPopulation) BEGIN Initialise children of size num_children_required FOR each operator in operators num_parents_required = Get_num_parents_needed (operator) num_children_produced = Get_num_children_produced (operator) number = Get_num_children_to_be_produced_by_operator (operator) num = 0 WHILE (num < number) parents = Select (plPopulation, num_parents_required) child = Perform_Operation (operator, parents) PLEvaluate (child) Append child to children Increment num by num_children_produced ENDWHILE ENDFOR new_population = Perform_replacement (plPopulation, children) evaluate_population (new_population) RETURN new_population END
Fig. 5. Pseudo Code of PL_GA_Evolve Get_num_children_to_be_produced_by_operator is a function used to calculate the allotted number of children that are to be produced by the specified operator. This is calculated by getting the number of children needed to be produced in total per PL generation and evenly dividing them among the operators, operators, to be used in that generation. Should there be leftover children from the division, these are then distributed to the first occurring operators from the OL chromosome. As for the number of children to be produced in each generation, this is defined by a predetermined rate expressed as a fraction of the PL population size and is set before the start of the application of the AGA. In the implementations of the AGAs, this parameter is arbitrarily set to a value of 0.9. 3.2.1
Operator Settings
The operators to be used in this GA are determined by chromosomes in the population of the Operator Level GA and may consist of any combination of the 17 operators mentioned section 3.1.1. Concerning the number of operators to be deciphered from each chromosome to be used, this has to be set and predetermined before the run of the GA. With this parameter set, the rate for each operator is also indirectly determined, since from the predefined set number of children to be produced each generation, each operator is assigned an equal number of children for it to produce. Hence, when compared to a classic GA that requires the crossover and mutation operators and their rates to be retuned for different problems in the same domain, the proposed AGA, which only requires the number of operators to be used
retuned, can be said to be more easily applied on different problems in the same domain. 3.2.2 Reproduction Strategy In classic GAs, the crossover operator creates child chromosomes, while the mutation operator modify the existing chromosomes present in the new generation’s population that may or may not have been produced by a crossover operator. However, because of the inability of distinguishing between crossover and mutation operators in a relatively problem independent GA such as this as well as for simplicity’s sake, it was decided to have all operators produce children, such that a modified chromosome is also considered a child chromosome. 3.2.3 Selection and Replacement Techniques Both the OL GA and PL GA use the same selection and replacement technique. The selection technique refers to the approach used to select chromosomes or parents for recombination. In Figures 1 and 5, the use of this technique is denoted by the term Select. During the search for a suitable selection technique, several candidates were initially selected. They were random selection, roulette wheel selection, linear ranking and tournament selection. Since the idea behind recombination is to combine building blocks and there is a higher probability for them to appear in individuals with higher fitness values, it was decided that this selection technique would take into account the fitness values of chromosomes, thus eliminating random selection. It was also decided that the roulette wheel selection technique would be eliminated from the list of candidates because selecting individuals directly based on fitness values risk the two extremes of either giving too much selective pressure to the individual with a much higher fitness value than the other individuals in the population or of having too little selective pressure in a population where the fitness values of individuals are close to one another [20]. This disadvantage is even more pronounced in a GA like AGA, since its application on different problems or even just different instances of the same problem would result in different ranges of differences in fitness values among the solutions. Between the linear ranking and tournament selection strategies, tournament selection was selected as it requires less computation, while behaving similarly to linear ranking when the parameters of tournament selection and linear ranking selection are set to 2 [21]. Hence the tournament selection technique with its parameter, the number of competitors set to 2 was selected as the selection technique. The replacement strategy of a GA refers to how individuals from the present population are selected to live on in the population used in the next generation. In figures 1 and 5, the use of this replacement strategy is referred to as Perform_replacement. Several replacement strategies were considered for this algorithm. Finally, it was the strategy, steady state with elitism that was selected. This is because while ensuring that good individuals in the old population that possess good
qualities, but that did not produce better individuals, are passed on to the next generation’s population [20], it also attempts to make sure that some amount of diversity is kept in the next generation’s population by allowing a mixture of good and not so good children to be passed onto the next generation. 4
Evaluation of the performance of AGA
To provide benchmarks against which the proposed variations of the AGA may be tested against, a classic GA, denoted by (RGA) and another variation of the AGA to act as a control experiment are designed and implemented Seven AGAs with fitness functions corresponding to those described in 3.1.3 are implemented. Henceforth, the AGA that utilizes Fitness_Function_1OL will be referred to as (1), the AGA that utilis=zes Fitness_Function_2OL will be referred to as (2) and so on. For all variations of the AGA, the OL population size is set to 20 and the PL population size is set to 50. During the application of the AGAs on different problems, other than the evaluation function and meaning of the encoding for each chromosome in the PL GA, the only other variable that is subject to changes is the predetermined number of operators to be passed to the PL GA during the evaluation of each OL chromosome. The tuning process for the AGAs consists of setting the number of operators to be used in the PL GA and a range of 1 to 9 operators is selected to be used for these purposes. 4.1 The Regular Genetic Algorithm
The regular genetic algorithm (RGA) implemented for the purpose of this research is basically a classic GA that uses most of the same components as the problem level GA of the AGA. These components include the chromosomal representations and evaluation functions used for each problem as described in chapter 3, as well as the selection technique and replacement strategy. The population size is also set to 50, same as the population size of PL GA. 4.2 The Control Adaptive Genetic Algorithm
To test the capability of the first level GA or in other words, to test whether any adaptation is taking place and if so, the extent of its influence on the results, a control adaptive genetic algorithm is implemented. This control adaptive genetic algorithm works in the same manner as the AGA except that no evolution takes place in the operator level GA. This means that no selection, crossover, mutation or replacement is performed in the OL GA. Instead, during each OL generation, a population of operator level chromosomes is randomly created and the encoded settings
are applied on problem level GA in the same manner as the AGA, except that no feedback is returned to the OL GA on the performance of the encoded settings of each OL chromosome and the next PL population to be used for the next OL generation is randomly selected. To ensure some form of performance, during each OL generation, the best PL chromosome found during that generation is kept and reinserted into the randomly selected PL population. Consequently, by putting this algorithm that randomly selects operators to be used in the PL GA and the proposed AGAs side by side, a gauge of the performance of the adaptation mechanisms may be then be derived. Similar to the parameter tuning done for the AGAs, parameter tuning for the control AGA consists of predetermining the number of operators to be used in the PL GA in the range of 1 to 9. 4.3 Experimental Results The results of the experiments are presented in the following manner; in the first column “RGA” refers to the regular GA, “Control” refers to the control adaptive genetic algorithm and (1), (2), (3), (4), (5), (6) and (7) represent the proposed adaptive genetic algorithms; the second column Opt shows the number of times the corresponding algorithm found optimal solutions; B.B.Fitness is an acronym for best best fitness and is the fitness of the overall best solution found among the set of tests; Mean.B.Fitness refers to mean best fitness and is the mean of the fitness values of the best solutions found for the set of tests of that particular problem; SD.B.Fitness is the standard deviation of the fitness values of the best solutions found during the set of tests for that problem; M.Time refers to the average amount of time taken to run the set of tests, where each algorithm is run on a particular instance of a problem until it finds the optimal or until it reaches the time limit; and under the column rank, states the rank of the algorithms according to their performance evaluation and in relation to one another, where a rank of 1 signifies the best performing algorithm and 9, the worst. Tables 1 and 2 show the results of the experiments on the Knb150 and kb150 problem sets [22]. From these two tables we can see that the AGAs outperform both the RGA and Control algorithms
Table 1. Summary of the Performances of the GAs on the Knb150 Problem Algorithm
Opt
RGA
0
Control
0
B.B.Fitness 0.07081966 0.07089325 8
M.B.Fitness 0.07082511 4 0.07094645 1
SD.B.Fitness 3.43E-06 2.64E-05
M.Time 120.002 5 120.040 5
Rank 7 9
(1)
0
(2)
7
(3)
10
(4)
1
(5)
1
(6)
0
(7)
7
0.07082536 1 0.07081880 5 0.07081880 5 0.07081880 5 0.07081880 5 0.07081887 7 0.07081880 5
0.07083825 5 0.07081886 1 0.07081880 5 0.07081896 5 0.07081915 0.07081931 5 0.07081882 1
8.97E-06 1.19E-07 0 1.97E-07 3.88E-07 4.22E-07 2.98E-08
120.065 7 107.736 1
8 3
95.5412 119.941 5 119.759 1
1
120.052 111.285 8
6
4 5
2
Table 2. Summary of the Performances of the GAs on the Kn150 Problem Algorithm
Opt
RGA
3
Control
0
(1)
2
(2)
10
(3)
10
(4)
10
(5)
10
(6)
5
(7)
8
B.B.Fitness 0.07081880 5 0.07081885 3 0.07081880 5 0.07081880 5 0.07081880 5 0.07081880 5 0.07081880 5 0.07081880 5 0.07081880 5
M.B.Fitness 0.07081887 9 0.07081926 4 0.07081885 3 0.07081880 5 0.07081880 5 0.07081880 5 0.07081880 5 0.07081883 1 0.07081880 7
SD.B.Fitness
Rank
3.77E-07
M.Time 104.317 1 120.053 8
6.55E-08
117.106
8
1.39E-17
64.9406
2
1.39E-17
58.1638
1
1.39E-17
91.1741
4
1.39E-17
90.8306
3
3.75E-08
113.87 102.513 1
6
5.09E-08
5.47E-09
Table 3 summarizes results for a 51 city travelling salesman problem (Eil51), taken from [23]. Out of the 7 AGAs, 3 of them (AGA 1, 4 and 6) managed to find optimal solutions to the problem whereas the RGA and control AGA failed to do so. Furthermore, the other 4 AGAs (2,3,5 and 7) performed better in terms of the consistency of the solutions found (lower standard deviations). Table 4 is a summary of the results of the Nugent problem of size 25 (Nug25) [24]. Again, the AGAs perform better than both RGA and Control.
7 9
5
Table 3.Summary of the Performances of the GAs on the Eil51 Problem Algorithm
M.Time
Rank
RGA
Opt 0
B.B.Fitness 431
M.B.Fitness 453.5
SD.B.Fitness 12.4036285
90.0037
9
Control
0
427
436.4
6.696267617
90.0193
8
(1)
1
426
435.2
4.643274706
82.5172
2
(2)
0
427
433.7
5.020956084
90.0123
4
(3)
0
427
435.4
4.92341345
90.0167
6
(4)
2
426
434
4.582575695
82.7868
1
(5)
0
427
433.7
5.86600375
90.0129
5
(6)
1
426
435.3
5.98414572
82.0365
3
(7)
0
427
435.8
3.841874542
90.0166
7
Table 4. Summary of the Performances of the GAs on the Nug25 Problem Algorithm
M.Time
Rank
RGA
Opt 0
B.B.Fitness 3754
M.B.Fitness 3805.4
SD.B.Fitness 38.29412488
90.0055
9
Control
0
3746
3810.2
62.96951643
90.01
8
(1)
1
3744
3803.2
42.22984726
84.8974
4
(2)
1
3744
3785.4
41.87648505
82.036
1
(3)
0
3746
3799
25.36533067
90.0088
6
(4)
1
3744
3801.8
43.27077536
89.1336
3
(5)
0
3746
3808.4
41.22911593
90.0075
7
(6)
1
3744
3796
54.80875842
81.4303
2
(7)
0
3746
3780.2
28.73951983
90.0135
5
Across all the test problems used in the experiments, an AGA was always found to be at the head in performance in terms of solution quality and speed of the find. From the results it can also be deduced that different AGAs are better suited to different problem types. Finally, it is argued that while Control was implemented to act as a benchmark for the adaptive mechanism used and is basically a GA that randomly varies the operators used throughout its application, it is considered a valid search algorithm, since there exist other successful search algorithms such as simulated annealing [1] that apply random search strategies.
5
Conclusion
A genetic algorithm utilizing the evolutionary based approach to adaptive variation of operators was designed and implemented. Seven operator level fitness functions were formulated to be tested since it was hypothesized that this component played a major role in determining the performance of the AGA. From the experimental results, it was shown that this hypothesis was true, with the fitness functions based on the individualistic performance of the operators as well as a couple of fitness functions that which varied based on the current state of the search being able to consistently outperform a regular GA throughout the various test problems, demonstrating the versatility of these particular adaptation mechanisms for the test problems. On the other hand, based on the different performances of the various AGAs in relation to one another on the different test problems, it was found that different adaptation mechanisms are better suited to different problems. These results also showed that the previously feared high overhead of such an approach was unfounded, since the proposed AGAs were able to find better solutions in a shorter time.
References 1. Tuson, A. L. (1995), “Adapting Operator Probabilities in Genetic Algorithms”, Unpublished M.Sc. thesis, Department of Artificial Intelligence, University of Edinburgh. 2. DeJong, K. A. (1975), “An Analysis of the Behavior of a Class of Genetic Adaptive Systems”, University of Michigan, Ph.D. Dissertation. 3. Davis, L. (1991), “Handbook of Genetic Algorithms”, Van Nostrand Reinhold. 4. Grefenstette, J. J. (1986), “Optimization of control parameters for genetic algorithms,” in IEEE Trans. On Systems, Man, and Cybernetics. 5. David, J., Caruana, R. A., Eshelman, L. J. and Das, R. (1989), “A study of control parameters affecting online performance of genetic algorithms for function optimisation”, in Proceedings of the Third International Conference on Genetic Algorithms, Morgan Kaufmann Publishers. 6. Davis, L. (1989), “Adapting operator probabilities in genetic algorithms,” in Proceedings rd of the 3 International Conference on Genetic Algorithms. 7. Hatta, K., Matsuda, K., Wakabayashi, S. and Koide, T. (1997), "On-the-fly crossover adaptation of genetic algorithms," in Proceedings of IEE/IEEE Genetic Algorithms in Engineering Systems: Innovations and Applications, pp.197-202. 8. Hong, I., Kahng, A. B. and Moon, B. (1995), “Exploiting Synergies of Multiple Crossover,” in IEEE International Conference on Evolutionary Computing. 9. Julstrom, B. A. (1997), “Adaptive operator probabilities in a genetic algorithm that applies three operators”, in SAC 1997. 10. Schaffer, J. and Morishima, A. (1987), “An adaptive crossover distribution mechanism for genetic algorithms”, in Proceedings of the Second International Conference on Genetic Algorithms. 11. Spears, W. M. (1995), "Adapting crossover in a genetic algorithm", in Proceedings of the 5th Conference on Evolutionary Programming, Cambridge MIT Press. 12. Starkweather, T., McDaniel, S., Mathis, K., Whitley, D. and Whitley, C. (1991), "A Comparison of genetic sequencing operators", in Proceedings of the Fourth International Conference in Genetic Algorithms, Morgan Kaufmann.
13. Whitley, D., Starkweather, T. and Fuquay, D (1989), “Scheduling Problems and Traveling Salesmen: The Genetic Edge Recombination Operator”, in ICGA 1989. 14. Davis, T. E. (1991), “Toward an extrapolation of the simulated annealing convergence theory onto a simple genetic algorithm”, Ph.D. Thesis, University of Florida 15. Fogarty, T. C. (1989), “Varying the probability of mutation in genetic algorithms”, in Proceedings of Third International Genetic Algorithms. 16. Herrera, F and Lozano, M. (1996), “Adaptation of Genetic Algorithm Parameters Based on Fuzzy Logic Controllers”, in Genetic Algorithms and Soft Computing, Physica-Verlag. 17. Tuson, A. L. and Ross, P. M. (1998), “Adapting Operator Settings in Genetic Algorithms. Evolutionary Computation”, 6(2). 18. Chen, S. and Liu, Y. (2001), “The application of multi-level genetic algorithms in assembly planning”, Journal of Industrial Technology, 17(4). 19. Koh S. (2004) “An Operator Based Adaptive Genetic Algorithm for Permutation Based Optimization Problems” MIT Thesis, Monash University Malaysia. 20. Pohlheim, H. (2004), “GEATbx Introduction – Evolutionary Algorithms: Overview, Methods and Operators”, Available: http://www.geatbx.com/download/ GEATbx_Intro_Algorithmen_v35a.pdf, 8/6/2004. 21. Blickle, T., and Thiele, L. (1995), “A Comparison of Selection Schemes used in Genetic Algorithms”, Technical Report, Computer Engineering and Communication Networks Lab, Swiss Federal Institute of Technology. 22. Leow, S. K. and Koh, S. (2004), “Using the Invariant Optimal Assignment of a k-out-ofth n: G system to Test the Effectiveness of Genetic Algorithms”, in 8 IEEE International Conference on Intelligent Engineering Systems. 23. TSPLIB (2000), “TSPLIB – Travelling Salesman Problem Library”, Available: http://www.iwr.uni-heidelberg.de/groups/comopt/software/TSPLIB95/, 24/5/2004. 2.4 Burkard, R. E., Çela, E., Karisch, S.E. and Rendl, F. (2002), “QAPLIB – A Quadratic Assignment Problem Library”, Available: http://www.seas.upenn.edu/qaplib/, 12/6/2004.
