A General Model of Co-evolution for Genetic Algorithms - CiteSeerX

A General Model of Co-evolution for Genetic Algorithms Jason Morrison and Franz Oppacher

fmorrison,[email protected] Intelligent Systems Lab School of Computer Science Carleton University Ottawa, ON K1S 5B6

Abstract. Compared with natural systems, Genetic Algorithms have a limited adaptive capacity, i.e. they get quite frequently trapped at local optima and they are poor at tracking moving optima in dynamic environments. This paper describes a general, formal model of co-evolution, the Linear Model of Symbiosis, that allows for the concise, uni ed expression of all types of coevolutionary relations studied in ecology. Experiments on several dicult problems support our assumption that the addition of the Linear Model of Symbiosis to a canonical Genetic Algorithm can remedy the above shortcomings.

1 Introduction Although Genetic Algorithms (GAs) have demonstrated their robustness and eciency as search and learning techniques in many application domains, they often suer from the problem of premature convergence: as the individuals in an evolving population approach a local optimum, the resulting loss of genetic diversity may prevent the GA from nding the global optimum1 . A related problem arises in dynamic environments even if the landscape is unimodal. The loss of diversity in the population resulting from successfully converging to an optimum renders the GA incapable of tracking the optimum if it should drift to another point in gene space. Currently there are a limited number of mechanisms available, short of a restart, that allow the GA to nd a new optimum (see [2, 4]). It is the hypothesis of this paper that the limited adaptive capacity of the GA - which has no counterpart in natural systems and which is acknowledged by the widespread interest in the problem of premature convergence - can be overcome by enhancing the canonical GA with a 'plug-in', tunable co-evolution module. Section 2 describes related work in coevolutionary computing, Section 3 gives the de nitions neces1

e.g., [16, 12].

sary to express the coevolutionary relations studied in ecology, Section 4 presents our Linear Model of Symbiosis, Section 5 describes the results of our experiments with two classical GA test functions, and Section 6 concludes with a discussion of the properties and capabilities of our model.

2 Previous Work in Co-evolutionary Computing In a classical GA each individual in the (single) population is considered a potential solution to the problem the GA attempts to solve, and an individual's tness, which determines its probabilities of surviving and/or reproducing, measures how well it solves the problem. In co-evolutionary systems, by contrast, the tness of an individual depends not only on how well it solves a problem but also on other individuals. Since the introduction of co-evolution into evolutionary computing in [7, 8] many variants of coevolution have been implemented. These variants fall into two broad classes. The rst and more popular class comprises systems that rely on a tournament approach to calculating `Competitive Fitness Functions' (CFFs) (see e.g. [1, 6{8, 13, 17, 19]). To calculate a CFF requires two (or more) individuals to compete in a game and their tness values are then based on relative performance in the game. The key idea is that given dierent pairs (or groups) of individuals the tness value of a speci c individual varies. Hence an individual's tness is aected by other individuals in its population and therefore GAs that use CFFs are co-evolutionary. The second class contains systems using multiple populations (see e.g. [15, 9]). Typically a problem is broken into components and each component is assigned a dierent GA. The tness value of an individual is calculated by randomly choosing the necessary individual(s) from the other population(s) and then evaluating how well this group of individuals

solves the problem. The individual being evaluated is then assigned its tness according to how well it assisted in the solution or according to how well the entire group solved the problem. Since the tness of an individual depends on the other individuals chosen, this type of system is also co-evolutionary. Several authors, e.g. [7, 8, 17, 13, 14], claim signi cant successes for their co-evolutionary systems - they can nd solutions that regular GAs missed or they can achieve more accurate solutions after fewer tness evaluations. Unfortunately, it is dicult to generalize previous work on co-evolutionary computation because it is plagued by inconsistent terminology and con icting classi cation schemes2 and because the various architectures are designed to deal with speci c problems. We attempt to remedy these drawbacks by presenting a simple set of de nitions that are in agreement with an in uential portion of the ecological literature and by proposing a single model of coevolution that can express all types of co-evolutionary relations studied in the ecological literature and that can be easily added to a regular GA.

3 A De nitional Framework for Co-evolutionary Relations We follow [11] and [18] in adopting a wide notion of symbiosis which encompasses, e.g., parasitism and contrasts with interactions with the abiotic environment3 . De nition 1. Symbiosis is a relationship between two (or more) individuals such that the tness of one individual directly aects the tness of the other individual(s). While it is in principle possible to study symbiotic relationships between arbitrary and varying numbers of individuals, we con ne ourselves here to look only at pairs of individuals. De nition 2. A symbiotic connection, A * B (A aects B ) between two individuals A and B exists if and only if the tness of A has a direct eect on the tness of B . Most forms of symbiosis can be expressed in terms of two specializations of this broad notion of a symbiotic connection. e.g., Hillis' use of `parasitism' to describe a model involving competition. 3 Note also that we interpret co-evolutionary relations to obtain primarily between individuals, not species. 2

+ B (A protagonizes B ) if and De nition 3. A *

only if there exists a connection A * B such that as the tness of A increases, the tness of B increases, and as the tness of A decreases, the tness of B decreases. ? B (A antagonizes B ) if and De nition 4. A *

only if there exists a connection A * B such that as the tness of A increases, the tness of B decreases, and as the tness of A decreases, the tness of B increases.

Since we wish to take the eects of individuals on the tness of other individuals into account, we need to distinguish two notions of tness. Absolute tness coincides with the usual notion of tness, i.e., it measures how well an individual solves the problem the GA is working on, and expressed tness includes the eects of other individuals. De nition 5. The absolute tness of individual xi at time t, fia (t), depends only on the genotype and phenotype of xi at t, and excludes the eects of other individuals. De nition 6. The expressed tness of individual

xi at time t, fie (t),depends on the genotype and phenotype of xi at t, and on all individuals xj such that xj * xi . With these de nitions in place it is possible to describe the various forms of symbiosis that exist in nature4 , including complex varieties of indirect mutualism5 and indirect competition between more than two species or individuals. However, the following de nitions are con ned to the symbiotic relationships between pairs of individuals. De nition 7. Amensalism occurs between two in? dividuals, Host and Amensal, if and only if Host * Amensal and :(Amensal * Host). De nition 8. Commensalism occurs between two individuals, Host and Commensal, if and only if + Commensal and :(Commensal * Host). Host *

We use [3, 11, 18] as base classi cation schemes and include all cases of pairwise relationships. 5 See [10], who analyzes symbiotic relationships among more than two species. As an example of such a relation that we could handle in our framework, consider the following case: although species A competes with B , and B competes with C , the net interaction between A and C is mutualistic.

4

De nition 9. Competition occurs between two in-

dividuals, CompetitorA and CompetitorB , if and ? CompetitorB and Comonly if CompetitorA * ? petitorB * CompetitorA. De nition 10. Predation occurs between two indi-

viduals, Predator and Prey, if and only if Pre? Prey and Prey * + Predator. dator * De nition 11. Mutualism occurs between two in-

dividuals, SymbiontA and SymbiontB , if and only + SymbiontB and SymbiontB * + if SymbiontA * SymbiontA.

The next relation is not classi ed in the ecological literature but can be usefully incorporated into evolutionary algorithms. It characterizes the case where an individual's current expressed tness is affected by previous changes in its expressed tness (as, e.g., in an individual's immune system). De nition 12. Adaptism occurs if and only if In+ IndividualA or IndividualA * ? IndividualA *

dividualA.

4 The Linear Model of Symbiosis We begin the description of our model with Equation 1 which calculates expressed tness at time t or generation t. In words, the expressed tness of individual xi at time t, fie (t), equals the expressed tness of individual xi at time t ? 1, fie (t ? 1), plus the total change, Ci (t), in the tness of xi at time t.

fie (t) = fie (t ? 1) + Ci (t) (1) This enigmatic \total change", Ci (t), can be bro-

ken down into two components: i) changes due to connections to individuals and ii) a change in absolute tness of the individual. Connections, are assumed to be independent of one another so that the rst component of change, i.e., the total eect of all connections, is the sum of all eects due to individual connections. We represent the eect of connection xj * xi at time t by cij (t). The second component of total change can eectively be ignored for most GAs because absolute tness (i.e. how well an individual solves the problem) is static. To determine initial conditions, assume that a new individual xi is born at time t. Since xi has not been aected by other individuals, its initial expressed tness is its absolute tness, fie (t) = fia (t).

Similarly, since there was no previous tness the total change is equal to the initial tness Ci (t) = fia (t). Using this idea of birth, and assuming that all individuals are born at time t = 0, the initial conditions of expressed tness can be stated in Equation 2.

fie (0) = fia (0) Ci (0) = fia (0)

(2) Equation 3 represents the full formulation for Ci (t).

Ci (t) =

S X j =1

cij (t) + (fia (t) ? fia (t ? 1))

(3)

This leads to Equation 4 which is the de ning equation for our model of symbiosis.

fie (t) = fie (t ? 1)+

S X j =1

cij (t)+(fia (t) ? fia (t ? 1)) (4)

Using the simplest function for a connection leads to the linear connection. Equation 5 shows that the linear connection cij is a protagonist connection if ij > 0 and an antagonist connection if ij < 0. Throughout the remainder of this paper the values ij will be referred to as connection strengths or weights.

cij (t) = ij Cj (t ? 1)

(5) The Linear Symbiosis Model of Co-evolution models co-evolution using only linear connections. Given the form of the linear connections it is possible to restate the basic equation of the symbiosis model (Equation 4). Thus let F a (t) be the vector of absolute tnesses for all S individuals in all populations [f1a (t); : : : ; fia(t); : : : ; fSa(t)]. Similarly let F e (t) be the vector [f1e (t); : : : ; fSe (t)] and C (t) be the vector [C1 (t); : : : ; CS (t)]. Finally let A be the matrix with components ij . Using these de nitions with Equations 5 and 4 produces the de ning equation of the linear model given in Equation 6.

F e (t) = F e (t ? 1) + AC (t ? 1) +F a (t) ? F a (t ? 1) where C (t) = F e (t) ? F e (t ? 1)

(6)

Thus a GA with our linear model of co-evolution uses the vector F e (t) as the tnesses of all individuals instead of F a (t).

Table 1. The values in the column time t ? 1 are assumed to have been taken from midpoint in the individuals' lives.

ij = 0:4 and ji = 0:5. It is important to note the change in Cj (t) to 0 once the death has occurred. The bracketed value is the value that would have been appropriate had the death not occurred. Quantity t?1 t t+1 t+2 t+3 fia(t) 10 10 10 10 10 fie (t) = fie (t ? 1) + cij (t) + fia (t) ? fia (t ? 1) 19.6 20.2 20.2 21.4 21.4 cij (t) = ij Cj (t ? 1) 1.6 0.6 0 1.2 0 Ci(t) = fie (t) ? fie (t ? 1) 1.6 0.6 0 1.2 0 fja(t) 5 5 11 11 11 fje (t) = fje (t ? 1) + cji (t) + fja (t) ? fja (t ? 1) 13.5 14.3 11.3 11.3 11.9 cji (t) = ji Ci (t ? 1) 2.5 0.8 0.3 0 0.6 Cj (t) = fje (t) ? fje (t ? 1) 2.5 (0.8) 0 3 0 0.6 Event Death Birth of of new

xj

5 Implementation and Experiments In order to implement the expressed tness of an individual, several questions concerning births and deaths must be answered. These questions are: How does the death of an individual in symbiosis aect its partners? How does a newborn aect expressed tnesses? How does the system change as a whole because of births and deaths? Suppose a child xj is born at time t + 1. At the time of birth the expressed tness of xj should be equal to the absolute tness of xj plus any eects felt at time t. However, since the child was not alive in the previous time then Cj (t) = 0. The remainder of the equations apply. In the model explored here each child is assigned its connections at birth and they remain constant throughout its life. When an individual dies the new child that is created to take its place inherits the dead individual's connections. This is possible because the population size is kept constant in the implemented GA. Death and its eects are also critical to the implementation of the model. Consider again Equation 4. Suppose at time t an individual xj is alive and affects individual xi , and at time t +1 xj is dead. This means that the change Cj (t) in xj between times t?1 and t will not aect individual xi at time t + 1 (i.e., Cj (t) = 0, regardless of fje (t) and fje (t ? 1)). Suppose that an individual xi is in symbiosis with xj and that xj lives at time t and is dead at time t + 1. Further suppose that at time t + 1 a new individual is born and takes xj 's place. Since the death and birth both indicate that Cj (t) = 0 there is nothing further to model. If xi and xj are connected with the s given in the caption, then their tnesses would be calculated as shown in Table 1.

xj

In this paper we present tests of a few basic types of symbiosis patterns that include feedback between individuals, i.e., Mutualism, Predation, Competition and Adaptism. In the case of Adaptism each individual is simply given a connection to itself and no other connections are made. The other cases involve pairing individuals xi and xi+1 were i = 0 (mod 2). Each pair is then given connections appropriate to the type of symbiosis being tested. Thus in a population of S individuals there are S2 symbioses of the same type. Furthermore all connection weights are assigned a single magnitude (i.e. jij j = ). This allows a direct comparison of the eect of varying connection strengths between otherwise equivalent experiments. Our GA implementation uses standard parameter settings to avoid unfairly favouring the coevolutionary GA6 . Due to space limitations, we describe only two problems, i.e., Rosenbrock's and Griewangk's Functions (Equations 7 and 8 respectively). Both of these classical Operations Research functions are dicult to solve with normal line search techniques due to non-linearity and non-separability. For both of these problems an individual xi is represented by a binary string of 32 bits which is mapped to two numbers yi1 and yi2 . For Rosenbrock's function, Max(yi1 ,yi2 ) is (2.048, 2.048), Min(yi1 ,yi2 ) is (-2.048, -2.048), maximum tness occurs are (1; 1) and equals 0. 6

Our GA uses rank selection and is elitist with a 90% replacement rate in a population of 100 individuals. Mutation and uniform crossover rates are 0.001 and 0.60 respectively.

0

16

-0.005

14

Regular GA Alpha = 0.0125 Alpha = 0.8000

-0.01 12 -0.015 10

-0.02 -0.025

8 Regular GA Alpha = 0.0125 Alpha = 0.8000

-0.03

6

-0.035 4 -0.04 2

-0.045 -0.05

0 0

20

40 60 Generation

80

(a) Best absolute tness acheived

100

0

20

40 60 Generation

80

100

(b) Average Hamming distance of survivors to the current best survivor

Fig. 1. Adaptism tested on Rosenbrock's Function. After the co-evolutionary GA has converged for For Griewank's function, Max(yi1 ,yi2 ) is (511; 511), Min(yi1 ,yi2 ) is (?512; ?512), maximum tness oc- a while, the amount of change introduced by the new individuals is lower, i.e., the average Ci (t) = curs are (0; 0) and equals 0. fie (t) ? fie (t ? 1) is smaller. This implies that new individuals survive less frequently into the next genfia (t) = ?100(yi21 ? yi2 )2 ? (1 ? yi1 )2 (7) eration. At some point the value of Ci (t) is too small to allow new individuals to survive. It is at this point that the co-evolutionary GA changes to its   y  2 y2 X second stage of behaviour. From this point onward ij ij (8) the fia (t) = ?1 ? 4000 + j2=1 cos p surviving population freezes, i.e., changes bei j =1 come more and more infrequent and eventually stop. Several measures of performance are used to de- This means that after this point the genetic diverscribe the experiments: the best tness (in the sense sity remains constant. Improvement in the best ever of \absolute tness") of every individual (best ever tness, as compared to a standard GA, is achieved tness) and the best tness of current individuals because of the constant random recombination of (current best tness) express the system's potential the existing genetic diversity into new individuals. for optimization, and the average Hamming distance Thus the second stage of behaviour is equivalent to of the survivors to the best current survivor mea- a random search. The beginning of the change between the two sures the genetic diversity. Before describing the speci c results for the dif- stages and the length of time during which the change ferent problems a description of behaviour common occurs depend on the form of co-evolution, the streto all problems is warranted. Since the eects on ngth of the connections and the problem structure. expressed tness are cumulative, an individual's ex- The results show that co-evolutionary GAs can outpressed tness may become higher than the global perform the regular GA. That is, the best individual optimum of the absolute tness. This leads to two found (in absolute tness) by the co-evolutionary GA is on average better than that found by the regpossible stages in a co-evolutionary GA. In the rst stage, during the early generations, ular GA. the co-evolutionary GA behaves similarly to a regAdaptism, Mutualism and Predation show very ular GA. The exact behaviour seems dependent on similar behaviour to one another. Each of these symthe form of symbiosis used as well as the problem biosis patterns preserves genetic diversity and can itself. However, the interesting stage occurs in the outperform the regular GA. Since in Competition later generations. the eects of connections balance one another, Com-

0

16

-0.05

14

-0.1

12

-0.15

10 Regular GA Alpha = 0.0125 Alpha = 0.0750 Alpha = 0.8000

-0.2 -0.25

Regular GA Alpha = 0.0125 Alpha = 0.0750 Alpha = 0.8000

8 6

-0.3

4

-0.35

2

-0.4

0 0

20

40 60 Generation

80

100

(a) Best absolute tness achieved

0

20

40 60 Generation

80

100

(b) Average Hamming distance from survivors to the current best survivor

Fig. 2. Predation tested on Griewangk's Function. petition is not signi cantly dierent from the regular GA. The results given here have been selected because they display typical behaviour. A more detailed presentation of results and analysis will be available in a forthcoming paper. For each combination of connection strength (), symbiosis type and problem, the results show the average of 200 experiments. Figure 1a) shows the best absolute tness achieved by Adaptism while optimizing Rosenbrock's Function. Clearly both extremes of  perform better than the regular GA. It is important to note that the co-evolutionary GAs continue to improve while the regular GA stagnates at a sub-optimal point. Figure 1b) shows the average Hamming distance from the survivors to the current best survivor achieved by Adaptism while optimizing Rosenbrock's Function. For this problem even the weakest connection weight causes a dramatic increase in genetic diversity. Also important is the fact that genetic diversity plateaus very early (7th to 20th generation) and never decreases further. Figure 2a) shows the best absolute tness achieved by Predation while optimizing Griewangk's Function. The curve for  = 0:8000 shows that the survivors freeze too early. Despite a lot of diversity the co-evolutionary GA is unable to outperform the regular GA. The curve for  = 0:0125 shows that the population freezes too late. Despite converging to a high tness level the lack of genetic diversity once the survivors do freeze restricts the overall performance of the co-evolutionary GA. Finally  =

0:0750 is the compromise that outperforms all others tested. Figure 2b) shows the average Hamming distance from survivors to the current best survivor achieved by Predation while optimizing Griewangk's Function. For this problem there is a wide range of increase in genetic diversity with varying . Also important is the fact that once genetic diversity plateaus in the co-evolutionary GA it never decreases further.

6 Conclusions We have described a co-evolutionary model that successfully maintains genetic diversity in a population. This is not the rst attempt to use co-evolution (see Section 2); however, our model seems to be the rst systematic approach to co-evolution. Using the Linear Model of Symbiosis, it is possible to add coevolution to any GA. More importantly, this co-evolutionary \add-on" is a tunable system, such that an increase in the strengths of connections results in an increase in the sustained genetic diversity. This relationship holds across a wide variety of possible absolute tness functions. Fortunately the same solution that solves the local optima problem also greatly improves the ability of the GA to gain back diversity quickly to allow it to nd a new global optimum, and should allow for quick response time to enable even a converged GA to track a moving optimum.

References 1. P. J. Angeline and J. B. Pollack. Competitive Environments Evolve Better Solutions for Complex Tasks. In Fifth International Conference on Genetic Algorithms, pages 264{270, 1993. 2. T. Back. Self-adaptation. In T. Back, D. Fogel, and Z. Michalewicz, editors, Handbook of Evolutionary Computation. Oxford University Press, 1997. 3. D. Boucher, editor. The Biology of Mutualism: Ecology and Evolution. Croom Helm, 1985. 4. H. Cobb and J. Grefenstette. Genetic Algorithms for Tracking Changing Environments. In S. Forrest, editor, International Conference on Genetic Algorithms: ICGA'93, pages 523{530, 1993. 5. L. Eshelman, editor. Proceedings of the Sixth International Conference on Genetic Algorithms, San Mateo, California, 1995. Morgan Kaufmann. 6. G. Hartvigsen and W. Starmer. Plant-herbivore coevolution in a spatially and genetically explicit mode. Arti cial Life, 2(2):129{156, 1995. 7. W. Hillis. Co-evolving parasites improve simulated evolution as an optimization procedure. In C. Langton, C. Taylor, J. Farmer, and S. Rasmussen, editors, Arti cial Life II, SFI Studies in the Sciences of Complexity, volume 10, pages 313{323. AddisonWesley Publishing Co., 1991. 8. W. Hillis. Co-evolving parasites improve simulated evolution as an optimization procedure. In C. Langton, C.Taylor, J. Farmer, and S. Rasmussen, editors, Arti cial Life II, pages 264{270. Massachusetts: Addison-Wesley, 1992. 9. P. Husbands and F. Mill. Simulated Co-Evolution as The Mechanism for Emergent Planning and Scheduling. In Fourth International Conference on Genetic Algorithms, pages 264{270, 1991. 10. L. Lawlor. Direct and indirect eects of n-species competition. Oecologica, 43:355{364, 1979. 11. D. Lewis. Symbiosis and mutualism. In Boucher [3], pages 29{39. 12. J. Maresky, Y. Davidor, D. Gitler, and G. Aharoni. Selectively destructive re-start. In Eshelman [5], pages 144{150. 13. J. Paredis. Co-evolutionary Constraint Satisfaction. In H. S. Y. Davidor and R. Manner, editors, Parallel Problem Solving from Nature - PPSN III, pages 46{ 55, Berlin, Oct 1994. Springer-Verlag. 14. J. Paredis. The symbiotic evolution of solutions and their representations. In Eshelman [5], pages 359{ 365. 15. M. A. Potter and K. A. DeJong. A Cooperative Coevolutionary Approach to Function Optimization. In H. S. Y. Davidor and R. Manner, editors, Parallel Problem Solving from Nature - PPSN III, pages 249{257, Berlin, Oct 1994. Springer-Verlag. 16. H. Sakanashi, H. Suzuki, and Y. Kakazu. Controlling dynamics of ga through ltered evaluation function. In H. S. Y. Davidor and R. Manner, editors,

Parallel Problem Solving from Nature - PPSN III, pages 239{248, Berlin, Oct 1994. Springer-Verlag. 17. K. Sims. Evolving 3D Morphology and Behavior by Competition. In R. Brooks and P. Maes, editors, Arti cial Life IV, pages 28{39, 1994. 18. M. Starr. A generalized scheme for classifying organismic associations. Symposia of the Society for Experimental Biology, 29:1{20, 1975. 19. X. Yao and P. J. Darwen. Evolving Robust Strategies for Iterated Prisoner's Dilemma. In X. Yao, editor, Progress in Evolutionary Computation, pages 276{292, 1994.