Genetic algorithms for design of experiments on assembled products 1 ...

Genetic algorithms for design of experiments on assembled products

D.K.Anthony and A.J. Keane University of Southampton Technical Report No. 385

1

Introduction

The purpose of this technical report is to give supplementary details on the coding and investigation of parameters for genetic algorithms developed for experiments on assembled products, as given by Sexton et al (2004). These experiments required non-standard designs and hence the genetic algorithm described by Heredia-Langner et al (2003)would not suffice. Genetic algorithms are stochastic-based optimization algorithms commonly used in difficult optimization problems. These problems can be difficult either due to a multi- modal or highly combinatorial search space, or for discrete search spaces. They are commonly used for one particular class of problems, known as NP-hard. One such problem is the traveling salesman problem, in which the best round-trip route between a number of cities is sought to minimize the overall distance traveled. Each particular solution is represented as a permutation of cities. In D-optimal design, a similar problem is presented. The objective function used in the D-criterion is optimized by altering the permutation of component builds represented in the design via a label matrix, which defines the design matrix X. Each experiment design solution is represented by a chromosome made up of a string of concatenated permutations, either in an explicit or coded form as explained below. For a permutation of N , n values are taken to form an “n from N ” selection. This can be used to represent the selection of n components from a pool of N . Additionally, a “1 from N ” selection can be used to specify a manufacturing option. The construction of the design matrix from the chromosome is

1

not discussed further here. Genetic algorithms are based on an abstraction of biological evolution. The genetic algorithm was first reported by Holland (1975) but has been publicized mainly through the work of Goldberg (1989). The optimization process evolves from one generation of design solutions to the next by a process of natural selection. Each generation is formed from a population of a set of chromosomes that are themselves strings of numbers (normally binary) representing all of the variables whose values are set to optimize the objective function. The fitness of each particular chromosome is evaluated by an objective function (or fitness function). The value of this function is minimized (or maximized dependent on the specific problem) in order to achieve the optimized design, and thus the smaller the value of each evaluated chromosome the fitter the design solutions it represents. The genetic algorithm is initialized with a random pool of chromosomes. Each subsequent generation of this pool is then achieved by three key operations: Chromosomes are selected in such a way that those with greater fitness have a higher probability of selection. Some of these chromosomes are then mated in pairs; two mating chromosomes swap information and two offspring result. The new generation is made up of both existing chromosomes and chromosomes newly formed by the operators: selection and crossover. The last operation, mutation, involves randomly changing elements of each chromosome with a low probability. This provides random diversity in the evolution and helps to prevent premature convergence before too little evolutionary experience has been gained. Only the basic operations necessary to define a genetic algorithm have been described above. There are many additional operations that are applied to improve performance. These are not discussed here as they are not implemented in the genetic algorithm. An elitist strategy was used where the best solution so far is automatically admitted to the subsequent generation. More recently there has been much critical discussion regarding the early work of Goldberg (as for example reported in Mitchell, 1996). In particular, many caveats in his “schema” analysis have been shown, especially when considering that a finite population is used in practice. However this analysis still demonstrates the mechanism by which genetic algorithms achieve better solutions,

2

by (after Mitchell, 1996) discovering promising solutions, emphasising their importance in each population and recombining them to (possibly) produce even better solutions.

2

Background to using GAs and similar approaches for finding D-optimal designs

In the last 15 years methods other than exchange-type algorithms have been applied to finding Doptimal designs, using mostly so called Natural Algorithms. These are stochastic-based techniques commonly used to find good solutions for high combinatorial, NP-hard or ‘difficult’ problems. Bohachevsky et al (1986) used a search technique known as simulated annealing for artificial problems. Their use of this technique initially concentrated on the parameter selection and followed a case study on a D-optimal design which produced encouraging results. Haines (1987) used an annealing algorithm with a nested Metropolis algorithm to determine the exploration element. This technique was found to be superior to the Fedorov exchange algorithm in terms of efficiency for D-optimal problems for which solutions were already known and achievable using the exchange algorithm. The performances for other optimality measures (G-optimality, I-optimality) were also considered. Jansen et al (1992) used an annealing algorithm to govern the use of exchange an interchange steps on D-optimal problems where the global solution was known. The power of this technique was demonstrated although no comparison was performed with other algorithms. Jung and Yum (1996) used a Tabu search algorithm to solve D-optimal problems. This technique is a heuristic method guiding the search using a “tabu” list to alleviate the risk of becoming trapped in a local optimum. More recently, Montepiedra et al (1998) used a genetic algorithm (GA) to solve D-optimal problems: univariate polynomial regression, additive and non-additive response surfaces. The GA used a binary representation, and the features of a common GA were mixed with those of a so-called micro-GA. In general substantial improvements in relative efficiency and relative computing time were found (in one case by about 10 times) although there was no general improvement in the quality of the resulting solutions.

3

3

Coding permutations problems for GAs

In order to perform the optimization it is necessary to code the solution in a suitable way. A highcardinality alphabet (i.e., one having a large number of possible symbols to encode the information in the chromosome) was selected. While binary is by far the most commonly used alphabet, and most problems can be reduced to a binary representation, it is best to use the alphabet which enables the most natural representation; “The user should select the smallest alphabet that permits a natural expression of the problem” (Goldberg, 1989, p80). Permutations are ordered sets of consecutive integers. Encoding these in binary would produce a search space that is more complex and make the search less efficient. Two chromosome representations are used here to represent a set of permutations, which are decoded to form the design matrix. The first, step index permutation (SIP) uses a set of N − 1 indices to represent a permutation of N . It has the advantages that there is no redundancy in the chromosome representation and that the standard operations used for binary representations for site selection for the crossover operation and mutation operation can be used. The mutation operation is slightly more complex, however. The second, extended chromosome representation (ECR) uses an explicit representation of each permutation. The selection of the sites for crossover and mutation must be on a gene by gene basis, and special operator, already reported, are required. There is a redundancy in the gene representation as only N − 1 integers are required to define a permutation of N integers. The two coding schemes and their operators are now described in detail.

3.1

Step Index Permutation (SIP) chromosome coding

To represent a permutation of N integers (an ordering of the integers: 1 to N ), N − 1 indices are generated with ranges {1 : N, 1 : N − 1, . . . , 1 : 2}, consecutively. The first index is used to select one value on the integer range 1 : N , the second is used to select a second value, stepping over any previously selected value. After N − 1 such selections, the final value of the permutation is the one value remaining unselected. Valid permutations result for all indices in the ranges detailed above. 4

An option was also used as to whether each index started at the first unselected value (absolute indexing) or at the last selected value, counting using circular wrapping (relative indexing). When using a high cardinality alphabet, the mutation operator needs to allow the original value to be randomly changed to another feasible value, with equal probability. The chromosome mask is used to indicate the feasible range of values at each location in the chromosome.

3.2

Extended Chromosome Representation (ECR)

For the ECR, the crossover and mutation operations are performed on a gene-by-gene basis using special operators, to preserve valid combinations. Other operators and approaches are available which require repair of invalid permutations (for example see, Falkenauer 1998). However, not only is it more efficient to have operators that guarantee that valid permutations result, but if a significant amount of repair is required then the benefit of the operators may be dubious. The crossover operators used are partially matched crossover (PMX) (Goldberg, 1989), order crossover (OX) (Goldberg, 1989), cycle crossover (CX) (Goldberg, 1989) and a simple gene swap crossover (SX). These are not described in detail here and the reader is referred to the references given. Variations on each exist and the implementation used here is as cited. Both the PMX and OX operators used randomly selected crossing sites, PMX retains the absolute position within the permutations, while OX retains the relative positions. CX uses the information within each crossed permutation to determine the crossover sites and information is swapped between corresponding locus positions. It is thus most similar to the crossover between multiple selection sites used in the standard binary alphabet chromosomes. It is also noted that the operation is reversible and when performed twice on the result of two crossed permutations the original permutations are restored. Finally the SX was also used. This is simply a swap of entire genes elected for crossover. This only changes the combination of permutations in each chromosome, and only new permutations can be introduced into the population via mutation. The two mutation operators used for the ECR are allele swap mutation (ASM) and scramble sublist mutation (SLM), and operate with a gene, when it has been elected for mutation. Both

5

are described in (Davis, 1991). For ASM two distinct alleles are selected and are simply swapped. For SLM two sites are selected within the gene and the alleles are randomly scrambled. As the ECR coding crossover and mutation operators work on a gene-by-gene basis, if the chromosome is composed of gene permutations which greatly vary in size then it is more likely that much more selection of sites and genes will take place around the smaller genes. In order to help address this select site weighting can be used to alter the probability of selecting a gene in proportion to its length. For the mutation operator the average probability of the mutation per chromosome remains the same.

3.3

Probabilistic single-point / 2-point crossover operator

A single-point crossover suffers from the disadvantage that the end portion of the chromosome beyond the crossover point will always be crossed over, and thus it is not possible to crossover a middle section independently to at least one end. Very high order point crossover operators, however, are more likely to destroy chromosomes of high fitness. In the GA implementation used here, both single- and 2-point crossover operations are used. Firstly the decision to apply the crossover operation to a chromosome pair is governed by a probability, Pxover . Subsequently, determining whether the crossover operation is to be extended to a 2-point crossover is governed by a second probability, P2x . Thus, with P2x set to 0 or to 1, the crossover operation will be fixed at single-point or 2-point crossover respectively. This goes some way to escaping the disadvantage of only using single-point crossover, in which the success can depend upon the specified chromosome ordering, but avoids more complex approaches, for example using techniques such as chromosome inversion.

4

Applying GA with good operational parameter values

It is well known that in applying GAs the success can be strongly dependent on the GA parameters. Unless there is previous experience in using a GA with a similar type of problem, the parameter selection can take some time and become a significant part of the entire search for a 6

good solution. Previous experience of problems of a similar type can aid the selection for further such problems. Reported here is the entire evaluation of the GA parameters used, including the initial investigations. In determining the GA parameters to be used trial experiments were run for one-tenth of the total number of evaluations for the full-length GAs, and the results used as a basis to decide those to be used for the full-length GAs. (The GA parameters are detailed in Table 3.) Where there is a simple choice of operator, for example XOVER TY, then each of the factor options is investigated. Also with continuous parameters like, for example, P2x , the two extreme values and a mid-point is used. Other parameters are more open-ended. However, in utilizing the characteristic operation of a GA it has been suggested that Pxover should be at least a value of 0.6, and three values were chosen up to 0.9 (Mitchell, 1996). Also, pmut should be less than 0.5 or the GA becomes little more than a random search. The number of generations decided upon for the comparison was 100,000, and three combinations were chosen to see whether better performance was achieved using more small generations, or a smaller number of larger generations. The combination chosen were (generation size / number of generations): 50/2000, 100/1000, 200/500. 50 was chosen as the smallest generation size as this was considered to be a reasonably small number with which there was still sufficient pool of individuals to usefully exchange genetic information using the crossover operation. Finally Pmut was chosen as follows. The mutation operator is design to add diversity to the search, but if the effect is too dominant then it may interfere with the crossover operator. It is argued that if on average the mutation operator affects each chromosome, then on average no crossover result will remain un-mutated which may decrease the benefit of this operator. Thus a maximum average value of 0.6 mutations per chromosome was assumed to be sufficient, affecting on average more than half of the chromosomes, but still allowing a large proportion to be unaffected. Three other smaller values, 0.1 and 0.3 mutations per chromosome were also elected. For each problem the chromosomes differ in length both in number of alleles and number of genes, and as for the SIP and the ECR coding the operations are performed on an allele-by-allele or a gene-

7

by-gene basis. The length of the chromosomes for each problem and coding type is shown in Table 1, and the actual mutation operator value Pmut required to achieve the above number of average mutations per chromosome in Table 2. Table 3 shows the candidate factor values for each GA parameter used to find the best combination of parameters. It is noted that some factors only apply to the SIP or ECR coding. The table also provides further information on the parameters, especially where not discussed in the text explicitly. Fractional factorial experimental designs were used to efficiently evaluate the “best” combinations of parameters. For the SIP coding there are three 2-level factors and four 3-level factors, and this was achieved by crossing the fractional factorial experimental designs OA8(23 ) and OA9(32 ). For the ECR coding there five 2-level factors, four 3-level factors and one 4-level factor, which was achieved by crossing the fractional factorial experimental designs OA16(24 ) and OA9(32 ). The four level factor was achieved by the combination of two columns in the 2-factor array and leaving one other related column idle. A brief fully-factorial investigation on a smaller problem showed that there was likely to be a small, but significant, interaction in between SEL TYPE and SEL SC and the relevant column in the array was let idle to prevent this from affecting the results. For experimental investigations the GA was also run with four different random number start seeds for each parameter combination, thus in total there were 288 for each investigation with the SIP coding and 576 investigations with the ECR coding. Although these were only run for one-tenth of the number evaluations used for the final GAs, the overhead in this investigation is significant. However, with no previous knowledge of parameters, it is an efficient and worthwhile investment as the GA performance and thus the results are very dependent upon sensible parameter choices for a particular problem, or a class of similar problems. Plots of the main effects of the value of the D-criterion were examined and the suggested best parameter combinations for each problem considered. The best combination for each coding type is shown in Table 4. In some cases there was little to distinguish between two of the factor values and any close second-best choice is shown in parenthesis. Using the best combinations of values

8

for each chromosome coding and problem, the GA were then run for the full 100,000 evaluations using ten different random number start seeds. GAs were also run for these problems using any close second-best values apparent from the main effect plots. The distribution of the performance against factor value was also investigated using box-plots of the performance against each factor value. These show the distribution of the median and lower and upper quartiles. It is possible, for example, that for one factor value, although the performance has a better mean it might have a large spread, and another factor value with a slightly higher mean may have a smaller spread and be more suitable, or at least worthy of investigation. More full-length experiments were run for ten different random number seeds, where information from the box-plots showed it was prudent to investigate. Further to this, GAs with larger values of Pmut were also run, as the effects plots showed a steady improvement with a higher value. Details of the final best parameter values are shown in Table 5. Values that are different to those suggested by the effects plots are marked with an asterisk. It is seen that in all cases the parameter settings suggested by the effects plots provided the best average results except for values of Pmut and, in one case, P2x . Pmut was increased in integer multiples of the largest value used in the experimental analysis and in most cases an average of 1.2 mutations per chromosome was found to be the best, except for Example 2 where 2.4 performed better. Further increases in Pmut were found to not increase the average as the variance of the individual results increased, normally resulting in a worse average. Although in some case these may provide the best individual results, the higher variance indicates that there is less certainty in obtaining a good result with this value of Pmut . It is noted that the new probability values are in rough agreement with the suggestion of a good static value of Pmut of 1/L suggested in Eiben (1999), except for the Example 2 problem where the best value for Pmut was about 2/L.

4.1

Application of good parameter values to similar type problems

While it is always possible to continue the investigation to find better parameter settings, this may be academic in relation to the effort expended and the improvement in results. The results

9

here have been achieved after an analysis of experimental results to find good parameter settings, which were then used, with some variations, in order to arrive at good results. It would not be necessary to repeat such a detailed analysis for further cases of these types of problems for all parameters. It is clear that, for example, that CX and PMX perform well, whilst both SX and OX do not. In this case such conclusions can also be extended to the selection method, the generation size and the crossover probability, for example.

5

Application of the genetic algorithm to searching for designs for conventional factorial experiments

Before considering the three non-standard problems, the performance of the genetic algorithm in searching for designs where all of the factors are fully controllable was assessed. Two examples were used, the first of which, Example A, has seven factors, three of which have three values and four of which have two values. The second example, Example B, has seven factors, all at three values. In each case a second order model was assumed. The performance of the genetic algorithm was compared with the Modified Fedorov algorithm. Ten runs of each algorithm were made from different random starting designs and each time the determinant value was evaluated, the best value found so far was recorded. These values were recorded for 100,000 evaluations for each of the ten runs of each algorithm. The average of 10 at each evaluation number was calculated. The results are illustrated in Figures 1 and 2. Figure 1 shows that the genetic algorithm converges fastest, but after approximately 30,000 evaluations there is little to choose between the two algorithms. The results of Example B, however, (see Figure 2) show a greater difference between the convergence rates of the two algorithms, with the genetic algorithm making little improvement after 20,000 evaluations and having the smaller determinant, on average, at the 100,000th evaluation. However, at the 100,000th evaluation the difference between the average determinants from the two algorithms is sufficiently small that the designs obtained by either algorithm would be acceptable.

10

2.8 2.6 2.4 2.2 2.0 1.8 1.4

1.6

Average smallest determinant

Exchange algorithm Genetic algorithm

0

20000

40000

60000

80000

100000

Number of D evaluations up to l

Figure 1: Comparison of genetic algorithm and the Modified Fedorov algorithm for a design for four two-level factors and three three-level factors (Example A)

11

4.5 4.0 3.5 3.0 2.0

2.5

Average smallest determinant

Exchange algorithm Genetic algorithm

0

20000

40000

60000

80000

100000

Number of D evaluations up to l

Figure 2: Comparison of the genetic algorithm and the Modified Fedorov algorithm for a design for seven three-level factors (Example B)

12

6

Encoding of the problem

In order to perform the optimisation it is necessary to code the solution suitably. A high- cardinality alphabet was used. Whilst binary is by far the most commonly used alphabet, and most problems can be reduced to a binary representation, it is best to use the alphabet which enables the most natural representation; ”The user should select the smallest alphabet that permits a natural expression of the problem” (Goldberg, 1989, p80). Two chromosome representations are used to represent a set of permutations, which are transformed into the design matrix. The first, step index permutation (SIP) uses a set of N − 1 indices to represent a permutation of N . It has the advantages that there is no redundancy in the chromosome representation and that the standard operations used for binary representations for site selection (on an allele-by-allele basis) for the crossover operation and mutation operation (with slight modification). The second, extended chromosome representation (ECR) uses an explicit representation of each permutation. The selection of the sites for crossover and mutation is on a gene-by-gene basis and special operators, already reported in the literature, are required. There is a small redundancy in the gene representation as only N − 1 integers are required to define an ordering of N integers.

6.1

Step Index Permutation (SIP) Encoding

To represent a permutation of N integers, a set of N − 1 indices are generated with ranges {1 : N, 1 : (N −1), ...., 1 : 2}, consecutively. The first index is used to select one value on the integer range 1 : N , the second is used to select a second value, stepping over any previously selected value. After N − 1 such selections, the final value of the permutation is the one value remaining unselected. Figure 3 shows by example how the permutations are decoded from the chromosome. An option also exists as to whether each index started at the first remaining unselected value (absolute indexing, as shown in the figure) or at the last selected value, counting using circular wrapping (relative indexing). The mutation operator needs to randomly change the original to any other feasible value with 13

a)

6 5 4 3 2 4 3 2 2 2 a

b

c

d

XSOME MASK

e

4 3 4 2 2 3 3 1 1 2

EXAMPLE XSOME

b)

{4,3,6,5,2,1}

{3,4,1,2} {1,2}{2,1}

DECODED PERMS

e

c)

a c

3

4

EXPLICIT DECODING OF LEFT PERMUTATION

b

d

5

2

start

1

6

unselected

Figure 3: SIP1 equal probability. The chromosome mask is used to indicate the feasible range of values at each location in the chromosome. The chromosome mask for the example chromosome is also shown in Figure 3.

6.2

Extended chromosome representation (ECR) encoding

For the ECR encoding, the crossover and mutation operations are performed gene-by- gene basis using special operators to preserve valid permutations. Other operators and approaches have been reported which require repair of invalid permutations (for example see, Falkenauer, 1998). However, not only is it more efficient to have operators that guarantee that valid permutations result, but the repair element interrupts the natural operation of the crossover, and can also prejudice 14

results. As the name suggests, the permutations are simply extracted from the chromosome, and no decoding is required, as illustrated in Figure 4.

4 3 6 5 2 1 3 4 1 2 1 2 2 1 {4,3,6,5,2,1}

{3,4,1,2}

EXAMPLE XSOME

{1,2} {2,1} PERMUTATIONS

Figure 4: ECR1

The use of four crossover operators was investigated, the first three are partially matched crossover (PMX), order crossover (OX) and cycle crossover (CX). These are not described in detail here and the reader is referred to Goldberg (1989). Variations on each exist and the implementation used here is as cited. Both the PMX and OX operators used randomly selected crossing sites, PMX retains the absolute position within the permutations, while OX retains the relative positions. CX uses the information within each crossed gene to determine the internal crossover sites and information is swapped between corresponding locus positions. It is also noted that the operation is reversible and when performed twice on the result of two crossed permutations the original permutations are restored. The fourth operator is a simple gene swap crossover (SX), which is simply the interchange of entire genes between chromosomes. This only changes the combination of the permutations in each chromosome, and only new individual permutations can be introduced into the population via the mutation operator. The two mutation operators used for the ECR are allele swap mutation (ASM) and scramble sublist mutation (SLM), and operate with a gene, when it has been elected for mutation. Both are described in Davis (1991). For ASM two distinct alleles are selected and are simply swapped. For SLM two sites are selected within the gene and the alleles are randomly scrambled. As the ECR encoding the crossover and mutation operators works on a gene-by-gene basis, if there is a large variation in the size of the genes it would more likely that the operation of crossover and mutation may be influenced by the specific order of the genes. In order to alleviate this, select site weighting was used (as an option) to weight the probability of selecting a gene in 15

proportion to its length whilst the average probability of the mutation per chromosome remains the same.

6.3

Probabilistic one-point/2-point crossover operator

A single-point crossover suffers from the disadvantage that the one end portion of the chromosome will always be exchanged and it is not possible to exchange a middle section independently. Results may be dependent on upon the specified chromosome ordering. Higher order point crossover operators or techniques such as chromosome inversion require more complexity. In the GA implementation used here, single- and 2- point crossover operations are used. First, the decision to apply the crossover operation to a chromosome pair is governed by a probability, pxover . Then, determining whether the crossover operation is to be extended to a 2-point crossover is governed by a second probability, p2x . Thus, with p2x set to 0 or to 1, the crossover operation will be fixed at single-point or 2-point crossover respectively. This goes some way to escaping the disadvantage of only using single-point crossover, but avoids more complex approaches.

7

Decoding the chromosome to the candidate experiment plan

The experiment design is specified by the information matrix, X, as described in the main text. This is constructed by a simple transformation of the label matrix whereby the component labels are replaced with the actual parameter values, and a left-hand column of 1’s is added. Each column of the label matrix is one permutation of component indices, or one or more part-permutations of component indices, which define across the rows the component builds and manufacturing options for each product in the experiment. In Figure 5 column 1 of the design matrix is made up of the ordering of all the available components in the pool, column 2 is made up of the partpermutation of n components from a pool of N , and to achieve this the permutation specified in the chromosome is truncated. Column 3 is made up of multiple part-permutations, and thus this

16

column is defined by many genes. Where the permutations are truncated to specify one value, a 1 from N selection is achieved, and this can be used to specify pre-set factors. Typical examples might be a manufacturing option, or component orientation. 1

2

3

LABEL MATRIX

GENE TYPE A



















 

 

 

 

 













GENE TYPE B

4 X GENE TYPE C 







XSOME COLUMN 1

COLUMN 2

COLUMN 3

Figure 5: MTRX1

It is the relative alignment of the permutations in the label matrix columns that define the experiment, so reordering the matrix rows does not define a different experiment. This means that there is a many-to-one mapping between the chromosome and the label matrix; if there are l rows in the matrix, there are l! chromosomes defining the same experiment. In practice as only part-permutations are used, the number of chromosome defining identical experiments is greater. Despite this, the GA was found to perform well. This redundancy can be avoided for the case for the case when all of the parts in a component pool are utilised as one factor in the experiment (one entire permutation fits exactly into one column of the label matrix, as for column 1 in Figure 5). In this case this column could be defined as fixed and not represented in the chromosome. The remaining columns would then be specified relative to that column. This case does not occur in the examples studied.

7.1

Applying the GA with good parameter values

For the successful applications of GA, good values of GA parameters should be used. This can be a daunting task due to the numerous parameters that specify the operation of even a simple GA, and is problem-specific. Most applications of GA reported in the literature simple report results 17

obtained using a set of quoted parameters, often with little or no justification. While for most it is certain that at least some empirical investigation has taken place, it corrupts the true story of applying GA to a problem from scratch. It is possible that the initial investigation may have a higher computational cost than for the best results reported. Here a small initial investigation was conducted for each problem using fractional-factorial experiments in order to see values of GA parameters which appeared to perform best over one-tenth of the number of evaluations to be used. It is similar, but simpler, to investigations reported by Pongcharoen et al (2002) and Lee (2002), in the sense that factorial experiments were used to find good GA parameters. In determining good GA parameters trial experiments were run for a one-tenth of the total number of evaluations to be used for the GA, and the results used as a basis to decide those to be used for the full GA runs. Where there is a simple discrete choice of parameter, as for an operator, for example XOVER TY, then each of the values is investigated. Also with continuous parameters as, for example, p2x , the two extreme values and a mid-point is used. Other parameters are more open-ended, however in utilizing the characteristic operation of a GA it has been suggested that pxover should be at least a value of 0.6, and three values were chosen up to 0.9. The value of pmut used were chosen as follows. The mutation operator is design to add diversity to the search, but if the effect is too dominant then it may interfere with the crossover operator. It is argued that if on average the mutation operator affects each chromosome, then on average no crossover result will remain un-mutated which may decrease the benefit of this operator. Thus, a maximum average value of 0.6 mutations per chromosome was thought to be sufficient, affecting on average more than half of the chromosomes, but still allowing a large proportion of a generation to be unaffected. Three other smaller values, 0.1 and 0.3 mutations per chromosome were also elected. In order to have these three values probability of mutation per chromosome for all of the problems considered, which have chromosomes of different lengths and that fact that for the SIP or ECR coding the mutation operation is applied on either an allele-by-allele or a gene-by-gene basis, the actual probability pmut (the probability of a mutation occurring each time the mutation operation is applied) is different for each problem and coding. Table 1 show the number of alleles

18

PROBLEM

PERTINENT GA XSOME LENGTHS NO ALLELES (SIP)

NO GENES (ECR)

Example 1

50

14

Example 2

481

179

Example 3

610

242

Example A

400

280

Example B

630

315

Table 1: Chromosome size for each problem pertinent to the operation of the GA for each chromosome coding type.

or genes per chromosome for each problem, pertinent to the SIP and ECR coding respectively. The values of pmut required to achieve the above number of average mutations per chromosome for each problem and coding is shown in Table 2. The number of evaluations used was 100,000, and three combinations were chosen to compare the performance for using a larger number of smaller generations, or a smaller number of larger generations. The values used were (generation size / number of generations): 50/2000, 100/1000, 200/500. Fifty was chosen as the smallest generation size as this was considered to be a reasonably small number with which there was still sufficient pool of individuals to usefully exchange genetic information using the crossover operation. Table 3 shows the candidate factor values for each GA parameter for the investigation to find good GA parameter values. It is noted that some factors only apply to the SIP or ECR coding. Fractional factorial experimental designs were used to efficiently evaluate the best available from the combinations of parameters used. For the SIP coding there are three 2-level factors and four 3-level factors, and investigation was achieved by crossing the fractional factorial experimental designs OA8 and OA9. For the ECR coding there five 2-level factor, four 3-level factors and one 4-level factor, and the crossing of the fractional factorial experimental designs OA16 and OA9 was used. The four level factor was achieved by the combination of two columns in the 2- factor array 19

20 0.0055 0.011

0.3

0.6

0.043

0.021

0.0071

ECR

0.00120

0.00062

0.00021

SIP

0.00340

0.00170

0.00056

ECR

Example 2

0.00100

0.00050

0.00016

SIP

0.00250

0.00120

0.00041

ECR

Example 3

0.00150

0.00075

0.00025

SIP

0.00210

0.000110

0.000360

ECR

Example A

0.00100

0.00050

0.00016

SIP

0.00190

0.00095

0.00032

ECR

Example B

chromosome size and coding type

Table 2: The conversion of the probability of mutation occurring in each chromosome to the value of the mutation operator Pmut , depending on

0.0018

SIP

Example 1

0.1

Pmut XSOME

Pmut

and leaving a pertinent column idle to prevent aliasing effects. A brief fully- factorial investigation on a smaller problem showed that there was likely to be a small, but significant, interaction in between the parameters SEL TYPE and SEL SC and the relevant column in the array was let in anticipation of this interaction. For each parameter combination, four instances were run using different random number start seeds, and in total there were 288 runs for each problem with the SIP coding and 576 runs for the ECR coding. Although these were only run for one-tenth of the number evaluations used for the final GAs, the overhead in this investigation is significant. For problems using the SIP coding the investigation is about 12 times more expensive than for one full GA run, and about 23 times more expensive with problems using the ECR coding. However, with no previous knowledge of parameters, it is an efficient and worthwhile investment as the GA performance, and thus the results, are very dependent upon good parameter choices for a particular problem, or a class of similar problems. An analysis of the means of the value of the D-criterion was performed The suggested best parameter combinations for each problem considered, and for each coding type is shown in Table 4. In some cases there was little to distinguish between two of the factor values and any close second-best choice is shown in parenthesis. Then, using the best combinations of values for each chromosome coding and problem, the GA were run for the full 100,000 evaluations using ten different random number start seeds. The distribution of the performance against factor values for each parameter was also studied using box-plots as mean values can hide large values of variance. It is possible, for example that for one factor value the performance has a better mean, it might have a large spread, and that another factor value with a slightly higher mean may have a smaller distribution and be more suitable, or at least worthy of investigation. The GA were also run for these problems using any close second-best values for parameters on an empirical basis, entailing about five more sets of runs for each problem. In this, different values of p2x , and XOVER TY and MUT TY (where applicable) were tried. Further to this GA with larger value of pmut were also run, as the analysis of means showed a steady improvement with a higher value. After these further GA runs, all were run for ten different random number seeds as before,

21

GA PARAMETER CODE SEL TYPE SEL SC GENPOP PXOVER P2X PMUT XSOME

SID TY

X OVERTY GXOVER TY S SEL W MUT TY

DEFINITION chromosome selection using proportional (roulette wheel) selection, or tournament selection whether fitness of population is scaled using Sigma selection before selection, or not combinations of population size and number of generations involving constant number of evaluations (pop size / number gen) probability of applying crossover operator probability that crossover operator will be 2-point crossover (in place of 1-point) probability of mutation occurring in chromosome. (Note Pmut is indirectly defined from this according to GA coding type and chromosome size) whether SIP representation is decoded in an absolute or relative scheme whether crossover sites represent individual genes or boundaries between which all genes are crossed type of crossover operator used between crossed genes (see text) Off: selection of each gene equal probability ON: selection of each gene weighted according to length mutation operator used (see text)

XSOME CODING TYPE SIP ECR {PROP, TOUR } {NONE, SIGMA } {50/2000, 100/1000, 200/500} {0.60, 1.0, 1.90} {0.0,0.5, 1.0} 0.1,0.3,0.6

{ ABS, REL } {INDIV, RANGE } { SX, PMX, OX,CX, } { OFF,ON } {ASM, SLM }

Table 3: Definition of the GA parameters investigated and the parameter values used during initial investigation into good parameter values

22

23

SIGMA

SEL SC

ASM

MUT TY

T

0.6

0.5

0.90

50/2000

NONE

PROP

SIP

ASM

T

CX(PMX)

RANGE

0.6

1.0

0.90

50/2000

NONE

PROP

ECR

Example 2

T

0.6

0.5

0.90

50/2000

SIGMA

PROP

SIP

ASM

T

CX(PMX)

RANGE

0.6

1.0

0.90

50/2000

NONE

PROP

ECR

Example 3

T=F

0.6(0.3)

0.5

0.90

50/2000

(SIGMA)

NONE

PROP

SIP

SLM

F

CX

RANGE

0.6

0.5

0.90

50/2000

NONE

PROP

ECR

Example A

T=F

0.6

0.5

0.90

50/2000

SIGMA

PROP

SIP

SLM

F

CX

RANGE

0.6

0.5 (1.0)

0.90

50/2000

NONE

PROP

ECR

Example B

Table 4: The best values of the GA parameters, evaluated by the fractional factorial experimental investigation, for each problem and coding type.

T=F

S SEL W

T

SID TY

0.6

PMX

0.3

PMUT XSOME

0.5

GXOVER TY

0.5

P2X

0.90

INDIV

0.90

PXOVER

50/2000

NONE

PROP

ECR

X OVERTY

100/1000

GENPOP

(NONE)

PROP

SIP

Example 1

SEL TYPE

GA PARAMETER

details of the parameter settings and GA coding yielding the best result are given in Table 5. Values that are different to the good values suggested by from the analysis of means of the results of the investigation are marked with an asterisk. It is seen that in all cases the parameter settings suggested by the analysis of means provided the best average results except for values of pmut and, in one case, p2x . pmut was increased in integer multiples of the largest value used in the experimental analysis and in most cases an average of 1.2 mutations per chromosome was found to be the best, except for SS3 where a value of 2.4 performed better. Further increases in pmut were found to not increase the average as the variance of the individual results increased, normally resulting in a worse average. Although in some cases these may provide the best individual results, the higher variance indicates that there is less certainty in obtaining a good results with this value of pmut . It is interesting to note that the best final probability values are in rough agreement with the suggestion of Eiben (1999) that a good value of pmut is the reciprocal of the chromosome length, except in the case for the SS3 problem. While it would have been possible to continue the investigation to find even better parameter settings, this would be academic in relation to the effort expended and the improvement in results. What was sought were good values to ensure effective application of the GA. It would not be necessary to repeat such an detailed analysis for further analyses for these types of problem for all parameters. It is clear that, for this type of problem, the crossover operators CX and PMX perform well using a high value of pxover , that proportional selection is better than tournament selection, and it is better to distribute the number of evaluations with a larger number of smaller-sized generations.

Acknowledgements This work was supported by ESPRC (GR/N16754) together with Jaguar Cars, Hosiden Besson and Goodrich Engine Control Systems.

24

References Bohachevsky, I. O., Johnson, M. E. and Stein, M. L. (1986) Generalized Simulated Annealing for Function Optimization. Technometrics, 28, 209-217. Davis, L., ed. (1991) Handbook of Genetic Algorithms. Van Nostrand Reinhold. Eiben, A. E., Hinterding, R. and Michalewicz, Z. (1999) Parameter control in Evolutionary Algorithms. IEEE Trans Evol Comp, 3, 124-141. Falkenauer, E. (1998) Genetic Algorithms and Grouping Problems. Chichester UK, Wiley. Goldberg, D. E. (1989) Genetic Algorithms in Search, Optimisation and Machine Learning. Cambridge MA: Addison-Wesley. Haines, L. M. (1987) The Application of the Annealing Algorithm to the Construction of Exact Optimal Designs for Linear Regression Models. Technometrics, 29, 439-447. Heredia-Langner, A., Carlyle, W. M., Montgomery, D. C., Borror, C. M. and Runger G. C. (2003) Genetic Algorithms for the Construction of D-Optimal Designs. Journal of Quality Technology, 35, 28-46. Holland, J. H. (1975) Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. Ann Arbour MI: The University of Michigan Press. Jansen, J., Douven, R. C. M. H., and Vanberkum, E. E. M. (1992) An Annealing Algorithm for Searching Optimal Block Designs. Biom. J., 34, 529-538. Jung, J. S. and Yum, B. J. (1996) Constructions of exact D-optimal designs by tabu search. Computational Statistics & Data Analysis, 21, 181-191. Lee, L. H. and Fan Y. L. (2002). An adaptive real-coded genetic algorithm. Applied Artificial Intelligence 16, pp. 457-486.

25

Mitchell, M. (1996) An Introduction to Genetic Algorithms. USA, CIP. Montepiedra, G., Myers, D. and Yeh, A. B. (1998) Application of genetic algorithms to the construction of exact D-optimal designs. Journal of Applied Statistics, 25, 817-826. Pongcharoen, P. Hicks, C., Braiden P. M. and Stewardson D. J. (2002). Determining optimum Genetic Algorithm parameters for scheduling the manufacturing and assembly of complex products. International Journal of Production Economics 78, pp. 311-322. Sexton, C. J., Anthony, D.K., Lewis, S. M., Please, C. P. and Keane, A.J. (2004) Design of experiment algorithms for assembled products. Submitted.

26

27

2.4∗

NONE 50/2000 0.9 0.5 1.2∗

SEL SC

GENPOP

PXOVER

P2X

PMUT XSOME

CX

CX∗ F ASM

GXOVER TY

S SEL W

MUT TY

T

SLM

F

CX

RANGE

1.2∗

0.5

1.0∗ 1.2∗

0.90

50/2000

NONE

PROP

ECR

Example A ecr0004

0.90

50/2000

SIGMA

PROP

SIP

Example 3 sip0006

SLM

F

CX

RANGE

1.2∗

0.5

0.90

50/2000

NONE

PROP

ECR

Example B ecr0004

where the value differs from that resulting after the fractional factorial parameter investigation.

Table 5: The final best GA parameter values for each problem after further experimentation on the GA parameter values. An asterisk indicates

ASM

T

RANGE

INDIV

0.90

50/2000

NONE

PROP

X OVERTY

SID TY

1.0

PROP

SEL TYPE

ECR

ECR

GA CODING TYPE

Example 2 ecr0006

Example 1 ecr0005

GA PARAMETER