A genetic algorithm framework using Haskell

A genetic algorithm framework using Haskell Deryck F. Brown

A. Beatriz Garmendia-Doval

John A.W. McCall

School of Computer and Math Sciences, The Robert Gordon University, St Andrew Street, Aberdeen AB25 1HG, Scotland. fdb,bgd,[email protected]

Abstract

We present a generic functional framework for the construction of GAs using Haskell. The framework consists of a general structure for evolving populations, a library of standard functions and operators, and two user-de ned modules used to implement speci c GAs. Four very dierent GAs, taken from the literature, are implemented in this framework demonstrating its exibility. The framework also allows precise control of the stochastic processes underlying the GA, allowing experimentation, as demonstrated by the Inverse Pendulum Problem. The framework provides a bridge between emerging theories and practical implementation of GAs. Keywords: genetic algorithm, functional programming.

1 Introduction Genetic Algorithms (GAs) [1] are a heuristic search technique inspired by the Darwinian survival of the ttest concept. GAs have been successfully applied to an impressive range of diÆcult problems, but the typically ad-hoc use of these techniques has hindered the development of a theoretical understanding of their operation. Without such an understanding, further progress is unlikely. Research aimed at understanding and improving GA performance has traditionally explored various methods of GA con guration, GA parameter selection, and hybridization with other techniques. However, the results obtained from such work tend to be tied to special features of speci c problems, and it is hard to extract general principles. On the other hand, traditional theory in the area, while stimulating interest, has delivered little in terms of implementable results. More recent approaches have attempted to construct theoretical models of GAs that capture their essential properties [2]. To be of any interest, such models must also provide assistance in developing GAs. This article is an attempt to move in this direction. Section 2 describes a generic functional framework for the construction of genetic algorithms. In Section 3, we demonstrate that the framework encapsulates the essential properties of GAs by specializing it to reproduce a range of published applications. Section 4 illustrates how the framework can be used to analyze a speci c GA under varying con guration and parameter values. Finally, Section 5 concludes and suggests how the framework might be used to support a theory of genetic algorithms.

2

The functional framework

A genetic algorithm may be described loosely as having the following structure: 1. Create an initial population by randomly generating individuals. 2. Repeat until best individual created or maximum number of generations: 1

(a) (b) (c) (d)

Assign a tness value to each member of the current population, using the evaluation function. Select a pool of individuals that will act as parents, using tness as a selection criterion. Mate a group of parents to create ospring. Combine the ospring and the current population to create a new population.

This structure naturally leads to a functional approach to de ning a GA. Each action \select", \mate" and \combine" becomes a function to be de ned, in addition to the evaluation function. The operation of the GA itself is a function which has as input a starting population and some random seeds, and outputs a set of successive populations. In what follows, the short sections of code given are in the functional language Haskell [3]. The diÆculty in setting up this structure is in choosing the right level of genericity. Too generic and the framework becomes trivial, lacking suÆcient complexity to support a meaningful theory. Too speci c and the properties of the framework will not be those of a suÆciently wide class of GAs. The framework is split into three levels: population handling, the mechanics of selection, and the evaluation of individuals. At the population level, movement from one population to the next is controlled by a function breed de ned as follows: breed :: Population -> Prob Population breed pop = selectAll pop >>= \pool -> (applyMate (mate) pool) >>= \offspring -> combine pop offspring

[Here Prob is a monad used to give precise control over random numbers used in the GA. We will address Prob in more detail in Section 4.] First selectAll pop selects groups of parents from the current population to form the mating pool. The result is an in nite list of such groups. This list is then given to the second step where applyMate (mate) pool applies the mating function mate to each element of the list in turn. The mating function combines the parents using the genetic operators to produce the ospring. The overall result of this step is, therefore, an in nite list of new ospring. This list is then given to the nal step where combine pop offspring combines the original population and the ospring to produce the new population. The mechanics of selection is handled by a library of generic operators where most standard operators are implemented:  Crossover: 1-Point, order, etc.  Mutate: bit wise, one position, etc.  Selection: roulette-wheel, tournament, linear ranking, etc.  Combine: generational, steady state, etc.

For example, the module for 1-point crossover is as follows: onePointCross :: [Chromosome] -> Prob [Chromosome] onePointCross [p1, p2] = posProb 1 >>= \[pos] -> let (v1', v1'') = splitAt pos (alleles p1) (v2', v2'') = splitAt pos (alleles p2) bs1 = v1' ++ v2'' bs2 = v2' ++ v1'' in

2

newChrom bs1 >>= \c1 -> newChrom bs2 >>= \c2 -> return ([c1, c2])

onePointCross takes a pair of chromosomes, p1 and p2, and a random number from Prob used to generate the position pos at which crossover is to take place. Each parent chromosome is split at pos into segments v' and v''. The segments are recombined and returned as child chromosomes c1 and c2. This allows a wide spectrum of GAs to be implemented within the framework, while at the same time standardising their con guration. This makes possible widely applicable abstract reasoning. Another practical advantage is that GAs can be implemented with minimal writing of new code. One of the many bene ts of using Haskell is the ability to specify functions that operate on (potentially) in nite data structures. A program will evaluate only as much of an in nite structure as required to compute its result. This liberates our program from considerations such as the number of ospring needed | we can generate the in nite list of ospring and rely on the combine function to use only as many ospring as necessary to construct the new population. For example, a generational GA will typically require as many ospring as the size of the population, whereas a steady state GA will require only a single pair of ospring. The actual number of ospring required will feed backwards to force the mate function to mate suÆcient sets of parents, and the select function to select the required number of parents from the original population. Speci cation of the framework to a particular problem now boils down to the creation of two userde ned modules. The rst, ChromUser, contains the de nition of a chromosome, the tness function and GA parameters. The second, User, de nes selectAll c mate c, and combine c. selectAll c de nes how chromosomes are selected for mating from the current population; mate c de nes how a set of chromosomes is mated to produce ospring; and combine c de nes how ospring are combined with the current population to create the next population. In most cases, these routines will simply call on functions already in the library.

3 Problems implemented As tests of the genericity and usefulness of our framework, this section presents several problems from the literature that we have implemented using it. In each case, we were able to reproduce the results given in the original references. We now brie y describe the problems, indicating the amount of work needed in each case to adapt the framework. 3.1

Simple function optimization

This problem was taken from [4, pages 18{22] and is to nd x in the range [-1, 2] which maximizes the following function f : f (x) = x sin(10x) + 1

Each chromosome is a binary vector of length 22, representing a number x in the given range. The evaluation function is equal to the value of f (x). Crossover is applied with probability 0.25 and the resulting ospring are mutated with probability 0.01. Population size is equal to 50. All of these are de ned in ChromUser as follows: module ChromUser where import Prob type Probability = Double type RandomNumber = Integer maxRandom = 10000 :: Integer

3

The following are the default types for the Chromosome. The user will have to modify this part if his chromosomes are not bit strings. type Allele = Bool type Fitness = Float type Position = Int newtype Chromosome = Chromosome (Fitness, [Allele])

Next we set the chromosome length, population size, mutation rate and operator rate. chromLength = 22 popSize popSize = getProb snd mutate_p = 0.01 operate_p = 0.25

:: Int :: Prob Int :: Probability :: Probability

bits2int, bits2double, and evaluate together de ne the evaluation function. The rst two are

generic for bitstring chromosomes.

bits2int :: [Allele] -> Int bits2int bs = sum [p | (p, b) Double bits2double bs = -1.0 + (x' * 3) / (2 ^^ 22 - 1) where x' = fromIntegral (bits2int bs) evaluate :: [Allele] -> Double evaluate bs = x * sin(10 * pi * x) + 1.0 where x = bits2double bs newChrom sets up a chromosome as a list of alleles paired with the associated tness value. newChrom :: [Allele] -> Chromosome newChrom bs = Chromosome (evaluate bs, bs)

The genetic operators used are one-point crossover and bit-wise mutation. Selection is roulette wheel selection. All of these are de ned in the library, and the User module is as follows: module User (module GenericOper, combine_c, selectall_c, mate_c) where import GenericOper {GenericOper is the library of pre-defined operators} bitwisemut is the pre-de ned bit-wise mutation operator. It has as input a function that mutates one

allele.

mutate_c :: Chromosome -> Prob Chromosome mutate_c = bitwisemut (\a -> return(not a)) onePointCross is a pre-de ned crossover operator. operate_c :: [Chromosome] -> Prob [Chromosome] operate_c = onePointCross

4

operateThenMutate is a pre-de ned mating operation combining a crossover-type operation followed by

a mutation

mate_c :: [Chromosome] -> Prob [Chromosome] mate_c = operateThenMutate operate_c mutate_c

Roulette wheel selection is also pre-de ned. select_c :: Population -> Prob Chromosome select_c = rouletteselect replaceSelectAll uses select c and pop to select (popsize=2) groups of 2 parents each. selectAll_c :: population -> Prob [[Chromosome]] selectAll_c pop = popsize >>= \ps -> replaceSelectAll select_c pop (ps 'div' 2) 2 generelitism is a pre-de ned repopulation strategy generational with elitism (preserving the best mem-

ber of the current population).

combine_c :: Population -> [Chromosome] -> Prob Population combine_c = generelitism 1 3.2

Traveling salesman problem

The Traveling Salesman Problem (TSP) is taken from [5]. A chromosome represents a tour of the cities. Each city is represented by a number, so that each allele is a number between 0 and the number of cities. The length of the chromosome is equal to the number of cities. The initial population is generated by creating random permutations of the cities. The genetic operator used is the edge recombination operator. This operator is not considered standard, and it was therefore not included in the library. In this case, a new module, EdgeOp, was created. It contains the operator edgerecombination. The edge recombination operator has to make random repairs to ensure that valid paths result. For this reason, no mutation operator is used. The User module changes to re ect this as follows: mate_c :: [Chromosome] -> Prob [Chromosome] mate_c = edgerecombination

The selection operator used is linear selection. The combination used is steady state. 3.3

Inverted pendulum

This problem was taken from [6]. The problem is to balance an inverted pendulum that sits over a cart. This is achieved by pushing the cart to the right or to the left. The cart can only move in one dimension on a nite track. A representation of the system can be seen in Figure 1. The genetic algorithm is used in this case to train the weights for a neural network with ve input nodes, ve hidden nodes and one output node controlling the inverted pendulum. Each chromosome is implemented as a list of 36 real numbers: the values of the weights, plus a number that represents the probability of crossover. The initial population is created at random. The evaluation function counts the number of pushes the system gives to the cart before the cart reaches the end of the track or the pole reaches an unstable position. If the neural network manages to balance the pendulum for 120,000 time steps, it is considered to have learned how to balance the pendulum. Therefore, in this case the evaluation function requires the implementation of the neural network in Haskell. The initial position of the system can be chosen to be either balanced (perpendicualr to the oor), or random and dierent for each evaluation. 5

Genes Present None Any one D and E A and B and C All present Figure 1: The Inverted Pendulum

Score 1 5 50 100 1000

Figure 2: Fitness function for Introns

Two individuals are selected according to their relative tness using linear-bias selection. Crossover is applied with probability determined by the crossover probability allele of the string selected as parent 1; otherwise mutation is performed on parent 1. The ospring always inherits the crossover probability of parent 1. If parent 1 has a higher tness than the ospring, the ospring's crossover probability is incremented by a factor of 0.10 (to maximum 0.95); otherwise it is decreased by a factor of 0.10 (to minimum 0.05). The new ospring are evaluated and inserted in the population according to tness. Iteration continues until error is acceptable or MAXITERATIONS = True. Mutation involves mutating all weights on the rst selected individual by adding a random value with range plus/minus 10.0. This is not in the library and must be de ned by the user. Crossover is not performed if the parents dier by two or fewer alleles. Otherwise, recombination of the strings is carried out using one-point crossover between the rst and last positions at which the parents have dierent weight values. Here some of the one point crossover function can be re-used, but there is still an element of user de nition. 3.4

Introns

This problem is taken from [7], and is designed to explore the evolution of a ying creature. Various combinations of genes, labeled A to F, are rewarded cumulatively, the highest tness being awarded to a unique combination equated to the power of ight. Each gene is represented as six positions in the bit string. When all six positions are equal to 1, the individual is said to possess the corresponding gene. All possible scores are added (see g. 2), to get the value of the tness function. Therefore, the maximum value the tness function can take is 1156. The length of the chromosome is 58. Each gene is represented by six positions, so thirty bits are used to represent the ve genes. The remaining bits are irrelevant to the calculation of tness but are there to emulate introns in DNA. The population sizes used are 16, 64, 256, and 1024. The genetic operators used are bitwise mutation (0.003 probability) and one-point crossover (0.5 probability). The selection operator is roulette wheel. Combine is steady state. These functions were already de ned in the library, so most of the work for this problem, de ning the chromosome and the tness function, was done in ChromUser.

4

Analysis of the Inverted Pendulum Problem

A major goal in constructing our framework is to allow the analysis of GA performance. One feature of this is the random number monad Prob mentioned earlier. Monads allow the incorporation of imperative features into a purely functional language [8]. Prob maintains several in nite lists of random numbers 6

associated to each stochastic process, while keeping the lists hidden from the user. This allows very ne control of factors aecting the outcome of a GA run. We conducted experiments into how the inverted pendulum GA fared as we varied its operating conditions. As an example of what is possible, Figures 3(a) and 3(b) each show the results of ve GA runs (mean and best tness at each generation). The seeds for the random numbers controlling the generation of the initial population are the same for all runs, but the remaining seeds are dierent. Therefore, all runs start with the same initial population but vary afterwards. The dierences between the runs show the eect of a balanced or a random starting position on the performance of the inverted pendulum GA. Figures 3(c) and 3(d) show runs using the same seeds but with the crossover probability set to zero, i.e., using mutation only. The results suggest that mutation acting alone is a more successful con guration than using crossover and mutation together.

5 Conclusion A framework has been designed for the construction of genetic algorithms. Four very dierent GAs have been implemented with a high degree of standard module re-use. The framework also allows ne control of the stochastic processes underlying the GA. This allows the eects of varying dierent parameters to be explored in isolation. One aspect of functional programs remains unmentioned | the facility with which one can reason logically about their properties. The next step in developing our approach will be to build a theory of GAs by reasoning about the generic framework. The framework will therefore provide a bridge between sound theoretical foundations and practical implementation.

References [1] Goldberg, D. E. (1989).

Genetic algorithms in search, optimization and machine learning.

[2] Vose, M. D. The Simple Genetic Massachusetts, USA, (1999).

Algorithm: Foundations and Theory.

[3] Peyton Jones, S. and Hughes, J., editors. Haskell 98: http://www.haskell.org/onlinereport. [4] Michalewicz, Z. edition, (1996).

Addison-Wesley,

The MIT Press, Cambridge,

a non-strict, purely-functional language, (1999).

Genetic Algorithms + Data Structures = Evolution Programs.

Springer-Verlag, 3rd

[5] Whitley, D. The genitor algorithm and selection pressure: why rank-based allocation of reproductive trials is best. In 3rd International Conference on Genetic Algorithms, Schaer, D. J., editor, 116{121. Morgan Kaufmann, USA, (1989). [6] Whitley, D., Dominic, S., Das, R., and Anderson, C. W. Genetic reinforcement learning for neurocontrol problems. Machine Learning (13), 259{284 (1993). [7] Levenick, R. Inserting introns improves genetic algorithm success rate: taking a cue from biology. In 4th International Conference on Genetic Algorithms, Belew, R. K. and Booker, L. B., editors. Morgan Kaufmann, USA, (1991). [8] Hill, J. M. D. and Clarke, K. An introduction to category theory, category theory monads, and their relationship to functional programming. Technical Report 681, Queen Mary and West eld College, England, August (1994).

7

10000 100000

1000 Mean fitness (log)

Best fitness (log)

10000

1000

100

run 1 run 2 run 3 run 4 run 5

run 1 run 2 run 3 run 4 run 5

100

10 0

5

10

15

20 Generation

25

30

35

40

0

5

10

15

20 Generation

25

30

35

40

(a) best and mean for balanced initial position

100000

1000

Mean fitness (log)

Best fitness (log)

10000

1000

100

run 1 run 2 run 3 run 4 run 5

run 1 run 2 run 3 run 4 run 5

100

10 0

10

20

30

40 Generation

50

60

70

80

0

10

20

30

40 Generation

50

60

70

80

(b) best and mean for random initial position

10000 100000

1000 Mean fitness (log)

Best fitness (log)

10000

1000

100

run 1 run 2 run 3 run 4 run 5

run 1 run 2 run 3 run 4 run 5

100

10 0

5

10 Generation

15

20

0

5

10 Generation

15

20

(c) best and mean for balanced initial position using only mutation

100000

1000

Mean fitness (log)

Best fitness (log)

10000

1000

100

run 1 run 2 run 3 run 4 run 5

run 1 run 2 run 3 run 4 run 5

100

10 0

5

10

15 Generation

20

25

30

0

5

10

15 Generation

20

25

30

(d) best and mean for random initial position using only mutation

Figure 3: Five runs with the same initial population, and dierent seeds for the genetic operators.

8