A Cellular Genetic Algorithm with Disturbances: Optimisation Using ...

Journal of Heuristics, 8: 321–342, 2002 c 2002 Kluwer Academic Publishers. Manufactured in The Netherlands. 

A Cellular Genetic Algorithm with Disturbances: Optimisation Using Dynamic Spatial Interactions MICHAEL KIRLEY School of Environmental and Information Sciences, Charles Sturt University, P.O. Box 789 Albury, NSW, 2640, Australia email: [email protected]

Abstract This paper describes a novel evolutionary algorithm inspired by the nature of spatial interactions in ecological systems. The Cellular Genetic Algorithm with Disturbances (CGAD) can be seen as a hybrid between a fine-grained and a coarse-grained parallel genetic algorithm. The introduction of a “disturbance-colonisation” cycle provides a mechanism for maintaining flexible subpopulation sizes and self-adaptive controls on migration. Experiments conducted, using a range of stationary and non-stationary optimisation problems, show how changes in the structure of the environment can lead to changes in selective pressure, population diversity and subsequently solution quality. The significance of the disturbance events lies in the new “ecological” patterns that arise during the recovery phase. Key Words: cellular genetic algorithm, disturbances, spatial interactions, optimisation

1.

Introduction

Evolutionary Algorithms (EA) are computer-based problem solving procedures that find their origin and inspiration in the biological world. Individuals (or solutions to a given problem) in an initially random population, compete with each other for the opportunity to survive and reproduce. The population adapts by slowly accumulating genetic changes with natural selection “weeding out” unfit individuals. Despite powerful similarities between EAs and natural evolution there are many differences between them. For example, EAs typically ignore the complex interactions between individuals (local populations) in spatially structured environments and speciation events. Selection and the subsequent replacement of individuals in natural ecosystems are dynamic processes, occurring in local populations that are connected to varying degrees. Genetic Algorithms (GA) (Holland, 1975; Goldberg, 1989) are perhaps the most well known EA for finding “good” solutions for many search and optimisation problems. GAs can be considered to be a “generate-and-test” heuristic, in which crossover and mutation are the sources of variation. If selection pressure is too intense in GAs the evolving population may converge prematurely to a suboptimal solution. A number of techniques have been proposed to help maintain diversity in the evolving population, and subsequently avoiding the problem of premature convergence, including crowding schemes (De Jong, 1975) and fitness sharing (Goldberg and Richardson, 1987). Significant disadvantages of such techniques are the artificial constraints imposed on the search process and the added time complexity. Adaptive

322

KIRLEY

genetic operators (Spears, 1994; Tuson and Ross, 1996) have also been implemented with varying degrees of success. More recently, there has been increased interest in spatially structured populations (Belding, 1995; Cantu-Paz, 1998). It is now well established in the ecological literature, that natural disturbances (eg. fire, floods or climate changes) are an important factor in influencing biodiversity systems (Connell, 1978; Sousa, 1984; Reice, 1994; Turner and Dale, 1998). Small differences in initial conditions, perturbations and spatial subdivision all contribute to the complexity of the community, and the viability of behaviour (Kauffman, 1993; Levins, 1998). Disturbances may shape the landscape and directly influence the local ecosystems for significant periods of time. A disturbance could possibly result in the extinction of a species, which in turn provides an opportunity for other species to flourish. In this study, the dynamical consequences of local spatial interactions and the emergence of spatio-temporal patterns, generated by simulated disturbance events were examined. To my knowledge, no single approach has previously focussed on diversity issues and population dynamics when investigating the behaviour of GAs. The primary objective of this study was to investigate the performance and utility of alternate disturbance schemes in parallel GAs. Specifically, the response of the evolving population to the process of fragmentation and disturbance was examined. To attain this objective, a new hybrid parallel GA, the Cellular Genetic Algorithm with Disturbances (CGAD) was developed. In the CGAD the evolving population is mapped on to a 2-dimensional toroidal lattice or “pseudo landscape”. The introduction of disturbances (disasters), which cull individuals in specific regions, provides a mechanism for controlling interactions across the lattice. A disturbance is defined as being a discrete event in time that disrupts the population structure. Any empty sites in the lattice may be colonised by individuals from the local neighbourhood. The effects of the “disturbance-colonisation” cycle are examined using benchmark stationary and non-stationary optimisation problems. Here, I suggest that the significance of the disturbance events may lie in the new “ecological” patterns that arise during the recovery phase. In addition, a new multi-objective optimisation algorithm is introduced that exploits the heterogenous spatial patterns of the CGAD. The remainder of the paper is organised as follows: In Section 2, background material relating to evolution in a landscape and disturbance events are presented. An overview of parallel GAs follows in Section 3. In Section 4, the CGAD is described in detail. In Section 5, experiments and results are presented. Section 6 discusses the results and examines the relationship and interplay between environmental complexity, diversity and solution quality. The paper concludes with the implications of the findings and future research directions. 2.

Evolution in a landscape

The complexity of ecological systems can be described in terms of the intricate interactions among species and changing patterns of observed abundance. No ecological system is purely deterministic. There are always some unexpected events, which have a strong effect on the dynamics of the population. Consider the scenario where an environment undergoing slow,

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES

323

steady change, experiences a number of environmental disturbances that clear large tracts of land, resulting in a fragmented population (isolated local populations or subpopulations). Fragmentation produces barriers to movement in certain directions, forcing the dispersing individuals to concentrate movement in “restricted zones”. This in turn will affect the genetic make-up of the population. Some subpopulations may even experience local extinction before recolonising. Natural disturbances occur with different frequencies, intensities and spatial distributions. Two different time scales are of interest when discussing the impact of disturbances: 1. Microevolutionary scale—A disturbance causes the environment to change, individuals (populations) become extinct, other individuals establish a foothold, and the system “recovers”, possibly converging to an equilibrium state. In the Intermediate Disturbance Hypothesis (Connell, 1978) it is claimed that high levels of ecosystem diversity are only sustainable at some intermediate regime of disturbance. However, the ecological effects of large, infrequent disturbances are not well understood (Turner and Dale, 1998). 2. Macroevolutionary scale—The fossil record details a number of extinction events that have occurred at different intensities and at different times throughout history. Interleaved with many small events (background extinctions) are a few larger events (mass extinctions) that have caused a large number of species to become extinct (Raup and Sepkoski, 1982). There is a great deal of debate in the scientific community about the cause of the “big five” mass extinction events. Non-biotic causes are perhaps the best known explanation. For example, the impact from meteors (Alvarez et al., 1980). Alternately biotic explanations, which include food web interaction and species stress, have also been proposed (Sol´e et al., 1998). Whatever the cause, biological and physical “natural disturbances” perturb all ecosystems. They serve as a catalyst for sudden and unexpected changes, destroying niches and providing an opportunity for alternate species to flourish in the new community structure. Large-scale population dynamics can be extremely sensitive to variability among individuals, and this response depends on the spatial structure of the interacting populations. Kareiva (1990) has shown that diversity among local populations increases as the number of local populations increases. In related work, Ellstrand and Elam (1993) have demonstrated that small populations maintain lower levels of genetic variation than larger populations. Green (1993, 1994) has shown that the structural changes reflected in a cellular automaton simulation play a critical role in many ecological changes and species evolution. In particular landscape “connectivity” is an important factor. Connectivity means processes that affect genetic “communication” within a population. If connectivity falls below a critical level, then regional populations effectively break up into isolated subpopulations. A crucial implication of this study is that simply by changing the proportion of active cells within a spatial grid (perhaps by introducing disturbances), it is possible to induce phase changes in the genetic communication within a population. An important research question, therefore, is whether we can exploit our understanding of natural disturbance events, landscape ecology and population dynamics to enhance the

324

KIRLEY

efficacy of EAs. Before describing the CGAD model, which has been developed to address this question, I provide a brief critique of parallel GAs in the next section. 3.

An overview of parallel genetic algorithms

The original motivation for parallel GAs was the processing speeds available on massively parallel computers. However, there are also performance advantages from a parallel implementation, which utilise spatially distributed populations. Cantu-Paz (1998) classifies parallel GAs into four categories: 1. Global procedures—one population is used, but evaluation and genetic operators are executed in parallel. 2. Coarse-grained—subpopulations work independently with only occasional exchanges of individuals. 3. Fine-grained—defines a spatial distribution for the population and selection and crossover are restricted within the individual’s neighborhood. 4. Hybrid model—some combination of the other techniques. In GAs the evaluation of the fitness function is typically the most time consuming process. Global parallel GAs distribute the computational effort among a number of processors to improve efficiency. However, the behaviour of the algorithm is essentially the same as a serial GA. Coarse-grained parallel GAs (also known as distributed or “island” models) rely on spatial separation of the evolving populations (Cohoon et al., 1987; Tanese, 1989; Belding, 1995). Local populations must be (a) small enough for random genetic drift to occur, (b) large enough for selection pressure to produce highly fit individuals, and (c) sufficiently isolated with only occasional migration. Cohoon et al. (1987) reports that in the time between migration there was relatively little change in the local populations. However, new solutions were found shortly after migration. This result provides support for the biological evolution theory of Punctuated Equilibria (Eldrege and Gould, 1972). According to this theory, species tend to remain stable for long periods of time. The equilibrium is punctuated by abrupt changes in which existing species are suddenly replaced. Vose and Liepins (1991) have also reported similar results when investigating the interaction between selection and recombination in GAs. In fine-grained parallel GAs (or cellular GAs) individuals are usually placed on a large toroidal two-dimensional grid, one individual per grid location (Manderick and Spiessens, 1989; Muhlenbein, 1989; Whitley, 1993). Fitness evaluation is done simultaneously for all individuals, selection and reproduction take place within a local neighbourhood (deme). In this model, the topology directly influences the behaviour of the algorithm since it controls the rate of “diffusion”. Clusters or semi-isolated niches of like individuals emerge across the grid as evolution progresses. Lin, Goodman, and Punch (1997) completed an interesting study using a hybrid parallel GA—a coarse-grained parallel GA with spatially structured (toroidal) subpopulations. Different configurations were investigated using a job-shop scheduling problem as a

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES

325

benchmark. Experimental results indicated that (a) a fine-grained model did not perform as well as the hybrid approach and (b) increasing the number of subpopulations in the coarse-grained model improved performance more than increasing the population size. Nowostawski and Poli (1999) introduced a “dynamic deme” parallel GA based on a combination of distributed fitness evaluations and static subpopulation models. Here, the subpopulations (or demes) are constantly reorganised to avoid “bottlenecks” due to variations in computational effort associated with fitness evaluation. In general, the behaviour of an evolving population is governed by the complicated interplay of genetic and population forces—selection, mutation, crossover, genetic drift, and migration. In multiple population parallel GAs there are numerous artificial constraints that can impede performance. Cantu-Paz (1998, 1999) has shown that the number and size of populations, communication topology and the migration rate affect the convergence speed of the algorithm. Typically, a lot of “fine-tuning” is required to establish the right parameters for a given problem. The introduction of disturbances into a parallel GA, provides a mechanism for generating dynamically sized subpopulations, and the possibility of changing the evolutionary trajectory of the evolving population. Mimicking landscape evolutionary processes in this way, will lead to the emergence of more robust algorithms. In the next section, I describe the CGAD and mechanisms employed to control the spatial interactions in the model. 4.

The cellular genetic algorithm with disturbances

The CGAD is a hybrid parallel GA with certain biologically inspired modifications. The underlying basis for the model is that living systems have evolved effective solutions to many complex problems. Previously we have argued that by developing algorithms designed to more closely reflect evolution (in a landscape), new robust optimisation techniques will emerge (Green and Kirley, 2000; Kirley, Li, and Green, 1998). Typically, in GAs the population size and the way individuals in the population interact with each other are fixed. The CGAD approach is unique in that changes in landscape connectivity are exploited, to make populations and gene pools highly adaptive. 4.1.

The model

The CGAD combines the “diffusion” properties of fine-grained models and the “island” properties of the coarse-grained model, creating variably sized, spatially distributed local populations. Individuals in the evolving population occupy sites on a l × l toroidal lattice or pseudo landscape (figure 1). All lattice sites are updated simultaneously according to the following rules: 1. Selection and crossover is confined to sites within the same local neighbourhood (9 cell deme or Moore neighbourhood). 2. Sites in the lattice are classified as: 2.1. Active sites—contain a member of the population (an individual solution). 2.2. Empty sites—no member of the population occupies them in the current generation.

326

KIRLEY

Figure 1. The 2-dimensional lattice or “pseudo landscape” of the CGAD. Individuals (shaded) occupy sites in the lattice. Selection is restricted to the local neighbourhood—8 adjacent cells or a Moore neighbourhood.

2.3. Inactive sites—never contain members of the population of solutions, and take no part in crossover and other processes. 3. Intermittent disturbances (disasters) clear sites in the lattice, leaving them temporarily empty. Disasters are characterized by two parameters: 3.1. Size—the maximum radius of the disaster zone (number of sites cleared) drawn from a Uniform distribution [0, Size]. 3.2. Frequency—likelihood of a disaster striking a particular generation. 4. Disasters always strike in random locations across the lattice. 5. A randomly selected neighbouring site is selected to “seed” or colonise an empty site. As the number of neighbouring occupied sites increases, so too does the probability of colonisation. Sites in the CGAD lattice are “connected” if one belongs to the neighbourhood of the other. Here, the overlapping demes provide the necessary communication path for genetic material to diffuse across the lattice. The initial impact of disturbance events is to cull individuals, creating empty sites. Generally, disasters that strike each generation are small and the population remains fully connected. Sometimes, however, a disaster wipes out large numbers of sites. Compounded disasters lead to a mosaic lattice (reduced connectivity), which creates barriers in certain directions, restricting genetic communication to isolated patches or subpopulations (figure 2). The resulting “patchy subpopulations” provide an opportunity for previously suppressed individuals to survive and reproduce. Connectivity gradually builds up again when individuals from active sites within the local neighbourhood, spread out and “colonise” empty sites. The disturbance-colonisation cycle may be thought of as a dynamic “re-wiring” of the communication topology of the lattice. That is, the communication links within local neighbourhoods are continually being dissolved and reestablished, creating dynamic selection pressure. In this instance, colonisation events may be thought of as a form of adaptive migration.

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES

327

Figure 2. Patchy subpopulations of a CGAD model. Active sites are shaded, while empty sites are represented by white cells on the lattice. The shaded area shows the extent of “connectivity” across the lattice after repeated disturbance and colonisation events.

5.

Experiments

The testing and development of the CGAD was divided into three phases. In the initial phase, benchmark static optimisation problems were used to establish how the introduction of disturbances affected population diversity and subsequent solution quality. In the second phase, non-stationary environments were examined. Finally, in the third phase, the spatial properties of CGAD provided a framework for the development of a new multi-objective optimisation algorithm. 5.1.

Static optimisation using the CGAD

The first question to be addressed was to determine how solution quality changes when the population structure was disturbed. 5.1.1. Test problem. Three benchmark static optimisation problems, which incorporate a range of features known to cause problems for EAs, have been used in the initial investigation. The global minimum for each problem is 0. Schaffer F6   sin2 x12 + x22 − 0.5 f 1 (x1 , x2 ) = 0.5 +   2 1 + 0.001 x12 + x22 where: −100 ≤ xi ≤ 100

(1)

328

KIRLEY

Generalised Rastrigin f 2 ( x ) = nA +

n 

xi2 − A cos(2π xi )

A = 3, n = 20

(2)

i=1

where: −5.12 < xi < 5.12 Griewank   10 10   1 xi − 100 2 f 3 ( x) = (xi − 100) − cos +1 · √ 4000 i=1 i i=1

(3)

where: −600 ≤ xi ≤ 600 5.1.2. Parameters. The size of the lattice was set to 50 × 50 cells for all simulation trials. The large lattice was required to create a more realistic pseudo landscape model. Solutions were encoded using an n bit binary chromosome-problem dependent length. A “fitness nearest neighbour” strategy was used for mate selection (Kirley, Li, and Green, 1998). In this strategy, the fitness values of each of the eight adjacent cells (deme) are sampled. In a form of tournament selection, the fittest individual from the local neighbourhood was selected as the mating partner. Two-point binary crossover was used. The crossover and mutation rates were set to 0.8 and 0.05 respectively. The values for the disaster parameters were systematically varied: the maximum size parameter was varied from s = 0 cells up to s = 20 cells, in steps of 5; the frequency parameter was varied from f = 0.0 to 1.0 in steps of 0.25. 5.1.3. Results. Table 1 compares the performance of each CGAD configuration examined on each of the test functions. The mean best fitness value and standard error over 20 trial runs, after 500 generations are recorded. The best result for each function is shown in bold. For function f 1 the best result was found when s = 10 and f = 0.50. For function f 2 the best result was found when s = 10 and f = 0.25. For function f 3 the diasters parameters were s = 5 and f = 0.25. Table 1 reveals that solution quality is affected when the spatially distributed population is disturbed. In fact, the results suggest that improved adaptation (performance) may occur when the level of disturbances is at some intermediate level. That is, the compounded affect of disasters is such that a number of local subpopulations have emerged, however, the majority of the lattice is still connected. The next stage of analysis involved an examination of the level and distribution of population diversity. In figures 3–5 the problem used was f 2 —Rastrigin function. The lattice size = 50 × 50, disaster size = 15 cells and disaster frequency = 0.5. In figure 3, a time-series performance comparison between the CGAD with disasters and the CGAD operating in standard fine-grain mode is presented. Figure 4 details how the percentage of active sites varied over the course of the run. Figure 3 shows that the CGAD has extended periods of stasis, with burst of improved performance. There was a strong tendency for the best fitness in the population to increase in jumps following a disaster (as the percentage of active sites increases—for example, approximately generations 100, 190, 260,

329

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES Table 1.

CGAD performance comparisons. f1

s

f

f2

f3

Mean

Std

Mean

Std

Mean

Std

0

0

1.17E-04

2.35E-04

5.15E-01

1.78E-01

2.53E-02

1.08E-02

5

0.25

1.73E-04

2.18E-04

5.55E-01

2.53E-01

2.01E-02

1.03E-02

5

0.50

1.10E-04

9.37E-05

4.18E-01

1.58E-01

2.36E-02

5.15E-03

5

0.75

1.20E-04

1.30E-04

4.61E-01

2.13E-01

2.54E-02

8.02E-03

5

1.00

8.80E-05

8.78E-05

4.43E-01

1.81E-01

2.49E-02

1.59E-02

10

0.25

6.10E-05

1.72E-04

3.72E-01

1.33E-01

2.66E-02

6.91E-03

10

0.50

3.70E-05

4.97E-05

3.89E-01

9.80E-02

3.12E-02

1.05E-02

10

0.75

2.06E-04

4.08E-04

6.10E-01

1.99E-01

3.07E-02

1.31E-02

10

1.00

4.82E-04

5.98E-04

7.03E-01

2.70E-01

2.86E-02

9.32E-03

15

0.25

6.59E-04

1.43E-03

6.09E-01

1.81E-01

3.35E-02

1.14E-02

15

0.50

1.76E-04

4.05E-04

6.25E-01

1.91E-01

2.97E-02

1.13E-02

15

0.75

5.10E-05

7.02E-05

6.60E-01

3.24E-01

2.90E-02

8.99E-03

15

1.00

9.77E-04

1.43E-03

7.87E-01

3.18E-01

3.31E-02

1.28E-02

20

0.25

1.48E-04

2.22E-04

5.27E-01

2.55E-01

2.64E-02

9.78E-03

20

0.50

1.27E-04

2.10E-04

5.79E-01

2.91E-01

3.46E-02

1.27E-02

20

0.75

1.65E-04

1.61E-04

8.14E-01

1.95E-01

3.59E-02

9.23E-03

20

1.00

1.01E-03

1.81E-03

8.01E-01

3.00E-01

4.48E-02

1.50E-02

Figure 3. CGAD performance comparison. The best individual fitness values vs the generation number for typical trial runs for the CGAD with disaster size = 15, frequency = 0.5 and the CGA without disasters for the Rastrigin function.

330

Figure 4.

KIRLEY

The percentage of active (alive) sites in the lattice vs generation for the trial runs illustrated in figure 3.

Figure 5. Population diversity in the CGAD model. Screen-shots of the normalised average hamming distance are displayed for (a) Standard cellular fine-grained model (disasters turned off), and (b) Cellular fine-grained model with disturbances. Three separate generations: g = 50, g = 200, and g = 300 are plotted to illustrate the evolutionary progress. Note the compounded affect of disasters across the lattice in (b).

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES

331

360). There are similarities between the improved performance observed, corresponding to repeated disturbance-colonisation cycles, and the improved performance in island models after migration (Cohoon et al., 1987). The time-series plots in figure 3 represent phenotypic fitness values. In GAs, population diversity is often defined as variability in the fitness values. The stochastic nature of disturbances radically alters the selection pressure. How does this “dynamic selection pressure” affect the genotypic diversity in the population? To answer this question, it was necessary to examine the genotype of individuals in the evolving populations. Genotypic diversity is represented here as the normalised average Hamming distance in the local neighbourhood. The Hamming distance dab between any two solutions a and b is defined as: dab =

l 

ai ⊗ bi

(4)

i=1

where l denotes the length of the bit strings, and ⊗ denotes the logical XOR operator. The normalised Hamming distance Dab is defined as: Dab =

dab l

(5)

Identical bit strings have a normalised Hamming distance of Dab = 0.0, whereas maximally different bit strings have a distance of Dab = 1.0. For the cellular model, the average normalised Hamming distance in the local neighbourhood is used: n Dlocal =

b=1

n

Dab

(6)

where a represents the centre site in the local neighbourhood, b represents an adjacent active site, and n is the number of active sites in the local neighbourhood. Figure 5 presents a series of screen-shots of genetic diversity in the CGAD. In figure 5(a) genetic material slowly diffuses across the lattice, leading to “corridors” of like individuals. The plots in figure 5(b) show how the accumulation of disasters alters the connectivity in the lattice. The density of occupied sites is continually changing at each generation. An interesting observation is that the fragmentation of the population results in “clumps” of like individuals, resulting in greater diversity across the population as a whole. In summary, in this section the performance of the CGAD on three benchmark static optimisation problems of varying complexity has been investigated. The introduction of the disasters into the lattice clears tracts of “land”, leading to dynamically-sized local populations. Some local populations are large enough for selection pressure to produce highly fit individuals, while others are small enough for random genetic drift to be a factor too. Repeated cycles of this process helps to maintain high levels of diversity across the population as a whole. Subsequent colonisation of vacant sites leads to hybridisation of local populations, often resulting in jumps in the best fitness value in the population. In Section 6, I discuss the implications of these results and the balance between exploration-exploitation of the search space.

332 5.2.

KIRLEY

Non-stationary optimisation problems

In natural ecosystems individuals interact in constantly changing environments. Living organisms usually seek “adequate” solutions in order to improve their survival chances and reproductive capabilities. Likewise, finding an adequate solution suffices for many nonstationary optimisation problems. For these problems, the goal is to track the progression of good solutions across the fluctuating fitness landscape, rather than to find the best possible solution for a static instance of the problem. Current adaptations are important, but genetic diversity is also critical if a population is to maintain the ability to respond to changing environmental conditions. In Section 5.1 it was established that the dynamic spatial topology used in the CGAD helps to maintain diversity in the evolving population and consequently solution quality. An important question to be answered is whether the corresponding trade-off between exploration and exploitation of the search space, brought about by disturbances, helps in dynamic environments. 5.2.1. Test problems. The non-stationary optimisation problems used in this study are based on the model proposed by Grefenstette (1999). This artificial landscape is made up of number of peaks of varying heights, which can be changed independently using runtime parameters. This problem is ideal because the rate of change of the environment can be controlled, the landscape is rugged in the sense that fitness is defined as the maximum contribution of all peaks and, finally, the problem is scalable. The fitness landscape is specified as a set {gi }. Each gi is a component in the landscape consisting of a single time varying n dimensional Gaussian peak (figure 6). The objective to be optimized is defined as follows: f (x) = max gi (x)

(7)

Figure 6. Fitness Landscape. A 3D plot of a static view of the dynamic landscape. The maximum peak amplitude = 100 units. When the environment changes, all peaks move in a random direction based on the run-time parameters.

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES

The fitness contribution to point x from gi is defined as:

   gi (x) = Ai (t)exp −d(x, ci (t))2 2σi2 (t)

333

(8)

where Ai (t) Amplitude is the fitness contribution obtained by an individual located at the center of the peak. ci (t) Centre specifies the coordinates associated with the maximum value of the peak at time t. σi (t) Width specifies how the fitness contribution from this peak decreases as a function of the distance from the center of the peak d(x, ci (t)) Euclidean distance between x and the peaks center. The initial landscape consists of 256 peaks uniformly distributed over the 2 dimensional region bounded by (0,0) and (100,100). All peak amplitudes were initialized uniformly from the interval [10,50]. One randomly selected peak is then assigned the optimal amplitude of 100. All peak widths were set to 4, i.e. the peak’s fitness contribution drops to about 50% of its amplitude at a distance of 4 from its centre. The dynamic landscape is implemented by moving all peaks in randomly selected directions over time. It is important to note that the amplitudes are kept constant in all environments. The motion of individual peaks was controlled by two run-time parameters: drift rate— the distance each peak moves (number of cells) in a random direction, drawn from the uniform interval [0,50], and the punctuation rate—how often the peaks move (the number of generations before the next change). The performance of the CGAD is examined using three different non-stationary environments, based on the classification scheme proposed by De Jong (1999): 1. Drifting—the landscape changes slowly over time. That is, all peaks move a specified distance (drift rate) each generation. The punctuation rate = 1. 2. Abrupt—the landscape undergoes major changes in a non-deterministic manner. The changes represent “cataclysmic events”. The magnitude of the change was equivalent to 50 generations of gradual change associated with the drifting environment. The punctuation rate = 20. 3. Oscillating—the landscape cycles between a small number of states (created using the abrupt method described above). Once again the punctuation rate = 20. 5.2.2. Parameters. The following parameter settings were used for each trial: a 40 bit binary chromosome (2 × 20 bits strings, one for each dimension) was used to encode solutions. The lattice size was set to 15 × 15 cells. Once again, a fitness nearest neighbour strategy was used for mate selection and the crossover and mutation rates were set to 0.6 and 0.05 respectively. An “elite migration scheme” is also used. At each generation the population is scanned and elite individuals are copied (added) to an external migration pool. When a change in environment is detected, the elite migrants are randomly reintroduced into the “disaster zones” distributed across the lattice. A run-time parameter is used to control the rate of migration.

334

KIRLEY

Four different EA configurations were investigated for each of the problems as per Kirley and Green (2000): 1. 2. 3. 4.

Standard GA. Fine Grain—standard fine-grained CGA—disasters turned off Disaster—CGAD with disasters: size = 5 (1/3 grid size), frequency = 0.3. Reload—same as 3 above, plus elite migration. Any remaining empty sites in a “disaster zone” were re-seeded with a randomly generated individuals.

5.2.3. Results. Figure 7 provides a summary of the CGAD performance in each of the dynamic landscapes (Note: this is a maximisation problem). For each of the problems the standard GA was not able to track the optimum value effectively. In fact, the currentbest fitness value fluctuates dramatically. The performance of the CGAD fine-grained and disaster models were very similar in all cases. The spatial population structure maintains diversity, which helps to track the moving optima and thus solution quality. Not suprisingly, the performance of the “reload” option is dependent upon the rate-ofchange encountered. For the oscillating environment, when the elite migrants were reintroduced into the population, individuals that had performed well in the past were able to track the global optimum. In contrast, in the drifting environment the population is unable to adapt fully to the new problem before the environment changes again. The influx of elite migrants each time the landscape changes, has the negative effect of steering the search into areas already examined. For the given population size, after generation 100, more than half the population in each generation is made of up individuals reloaded from memory. In the abrupt environment there appears to be slightly less variation in the current-best fitness values. However, the difference is not significant. 5.3.

Multi-objective optimization using the CGAD

In multiple criteria or multi-objective optimization problems (MOP), there are typically several conflicting objectives and as a consequence it is very difficult to identify the “best solution”. In fact there is no universally accepted definition of “optimum” as is the case in single objective optimization problems. Often there is no single optimal solution, but rather a set of alternate solutions to a particular problem. The goal is to find the set of optimal trade-off solutions—the Pareto-optimal set. An evolved solution belongs to the Pareto-optimal set if it cannot be improved in any objective without causing degradation in at least one other objective measure. Since the mid 1980s GAs have been applied to a range of MOPs with varying degrees of success (eg. Fonesca and Flemming, 1995; Zitzler, Deb, and Thiele, 1999). A comprehensive review of EA for MOPs can be found in Van Veldhuizen and Lamont (2000). Formally a MOP can be defined as follows: F( x ) = ( f 1 ( x ), f 2 ( x ), . . . , f n ( x )) gi ( x ) ≤ 0 i = 1, 2, . . . , m, x ∈

(9) (10)

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES

Figure 7.

Current-best fitness vs generation for the each of the dynamic environments examined.

335

336

KIRLEY

A MOP solution minimizes the components of a vector F( x ) where x is an n dimensional vector x = (x1 , x2 , . . . , xn ) from some universe . The concept of Pareto-optimally can be defined in terms of a dominance relationship. A vector u = (u 1 , u 2 , . . . , u n ) is said to dominate v = (v1 , v2 , . . . , vn ) denoted by u ≺ v, iff u is partially less than v. Maintaining diversity in the evolving population is a critical success factor when applying EAs to MOPs. The dynamic connectivity used in the CGAD helps to meet this goal. The spatial population also provides a framework to include, not just a single population, but several cooperating “species” that might evolve alternative kinds of solutions to a MOP. Here “species” means genetically distinct populations that compete for space but do not interbreed. Each site in the lattice includes several different species (or partial solutions) which cooperate with each other to produce a solution for the given problem. This coevolutionary model is relatively simple in the sense that it is defined with respect to a fixed and finite number of interacting species (one for each objective). An aggregation procedure similar to the weighted average rank method (Bentley and Wakefield, 1997) is used for fitness assignment and selection. The fitness values of each partial solution in the local neighbourhood are ranked from highest-to-lowest, for the each objective in turn. The overall fitness value at a particular site is the sum of the weighted ranks of each objective. In this case, each objective has an equal weight. An indirect cooperation mechanism (mutualism) guides the evolving populations (De Jong and Potter, 1995; Potter and De Jong, 2000). However, there is still evolutionary pressure for the partial solutions (for a particular objective) to compete for survival. If we consider EAs to be cooperative learning systems, where the fitness function is a mechanism used to determine which solutions survive, there is evolutionary pressure for the partial solutions to cooperate—to produce a composite trade-off solution for the MOP—rather than compete. 5.3.1. Test problems. Two MOP benchmark problems are used to evaluate the effectiveness of the CGAD. MOP1 is Fonesca’s f 1 problem, a scalable unconstrained two-objective problem (Fonesca and Fleming, 1995). minimise where

F(x) = ( f 1 ( x ), f 2 ( x ))   2 n   1 f 1 ( xi − √ x ) = 1 − exp − n i=1   2 n   1 xi + √ f 2 ( x ) = 1 − exp − n i=1

(11)

Any number of decision variables may be added to increase the search space. In this instance n = 2. A single Pareto-optimal range exists along the line (−1,1) and (1,−1). There are two best compromise solutions at the optima of each function, (−1,1) and (1,−1) (figure 8). The second problem used, MOP2, is T4 from Zitzler, Deb, and Thiele (1999). T4 is a multimodal 2-dimensional problem with a large number of Pareto fronts. T4 is defined as follows: minimise subject to

T (x) = ( f 1 (x1 ), f 2 (x2 )) f 2 (x) = g(x2 , . . . , xm )h( f 1 (x1 ), g(x2 , . . . , xm ))

(12)

337

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES

Figure 8.

Test function MOP1 (Fonesca and Fleming, 1995).

where:

f 1 (x) = x1 g(x) = 1 + 10(n − 1) +



xi2 − 10 cos(4π xi )

i=2

h( f 1 , g) = 1 −

m  

fi g

and n = 50, x1 ∈ [0,1], x2 , . . . , xn ∈ [−5,5] 5.3.2. Parameters. For each problem, a binary chromosome was used to encode each objective (32 bits per variable) The lattice size was set to 20 × 20, with disaster size = 5 and frequency = 0.5. The crossover and mutation rates were set to 0.8 and 0.05 respectively. The maximum number of generations was set to 200 for both configurations. 5.3.3. Results. Figure 9 plots the non-dominated solutions on a f 1 – f 2 plot for MOP1. The CGAD was able to find a wide range of trade-off solutions for the problem. MOP2 is a more challenging problem. Figure 10 shows the non-dominated (trade-off) solutions on a f 1 – f 2 plot for MOP2. Also plotted are results from two recently developed MOP EAs that have been shown to perform very well when compared with other algorithms: the Strength Pareto Evolutionary Algorithm (SPEA)1 proposed by Zitzler, Deb, and Thiele (1999), and the Pareto Archived Evolution Strategy (PAES)2 proposed by Knowles and Corne (1999). The plots were created by filtering the non-dominated solutions found in ten independent trials of each algorithm. CGAD clearly outperforms PAES for this particular problem. The range of Pareto optimal solutions found by CGAD are competitive with SPEA—although not significantly different. 6.

Discussion

The use of distributed populations in parallel GAs is based on the idea that isolation of populations leads to greater genetic differentiation (Wright, 1943). In fact, spatial features of the environment have long been identified as crucial in characterising environmental

338

Figure 9.

Figure 10.

KIRLEY

The trade-off solutions for MOP1 shown on a f 1 – f 2 plot.

The trade-off solutions for MOP2 shown on a f 1 – f 2 plot. Also plotted are SPEA and PAES results.

complexity for both natural and artificial systems (Kareiva, 1990; Green 1993; Holland, 1995). In parallel GAs, local selection and reproduction allows the population to evolve independently. Occasional migration between the isolated populations enhances diversity. Unfortunately, a common problem with both island and cellular parallel models are the artificial constraints imposed by the selected population sizes, topology and migration rates.

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES

339

In this study, insights from landscape ecology and population dynamics have been utilised, to create a robust cellular parallel GA. In the CGAD model, I pay particular attention to the consequences of local spatial interactions and random changes in population structure. A key idea was to control the number, and size of disasters during the simulation. The disturbance-colonisation cycle represents fluctuations in a non-equilibrium dynamical system. The CGAD provides a flexible population structure, resulting in dynamic selective pressures, population diversity and subsequently improved solution quality. A systematic analyses of the effects of the disturbance-colonisation cycle have been presented. The simulation trials conducted show how changes in the environment structure can affect the “evolutionary outcomes”. Table 1 compares the performance of the CGAD when disasters were introduced into the lattice. For each problem examined, intermediate levels of disturbance often leads to improved performance. As expected, if the magnitude and frequency of disturbance events are too high, the variability of the population is destroyed. There was a strong tendency for the best fitness in the population to increase in jumps following a disturbance (figures 3 and 4). This result supports the findings of Green (1993, 1994) in biological systems. In that study, disasters introduced into a landscape lead to an explosion of small populations that were previously suppressed by their competitors. In the CGAD, the disturbance-colonisation cycle creates a template upon which interaction among individuals occur. This dynamic spatial structure generates population responses very different from neo-Darwinian dynamics of standard GAs. Most of the time the system rests in a “connected phase” where selection predominates. The accumulation of disturbances across the lattice leads to a “disconnected phase”, wherein variation dominates. Disturbance events do not create diversity as such—they create opportunities for previously suppressed individuals to survive and reproduce. Disasters remove individuals from randomly selected zones in the lattice. The resulting fragmented population structure restricts genetic communication to local patches or subpopulations (figure 5). Different subpopulations potentially move into “different basins of attraction”, and thus potentially explore different regions of the search space. In this instance, “evolutionary novelty” has a much better chance of dominating a small, variably sized local population than a large one. During the recovery phase—a small window in time after a disturbance—empty sites are colonised by individuals from active sites in the local neighbourhood. This migratory process leads to the hybridisation of the patches, often leading to improved solutions. In natural environments the manner in which organisms interact is constantly changing. Lessons from ecology teach us that perfect solutions are not always desirable. Many organisms require adaptation for variance in order to increase their survival and subsequent reproductive opportunities. This also applies for complex optimisation problems such as non-stationary and multi-objective problems. Previous theoretical biology models have suggested that species redundancy is important for the functioning and reliability of ecosystems (Naeem, 1998). The CGAD exploits this notion, and consequently is more amenable to cover optimisation, that is, where ranges of solutions are represented in the population. In the case of non-stationary optimisation problems, the heterogenous spatial population helps to maintain the necessary diversity to track the moving optimum (figure 6). For multi-objective optimisation problems the goal is always to identify a range

340

KIRLEY

of solutions distributed across the non-dominated front. The results show that the CGAD is able to find multiple good solutions (figure 9). The CGAD performs favourable when compared to other specialised EAs for a given multi-objective problem (figure 10). Experimental results obtained, using different classes of optimisation problems, have confirmed the idea that the disturbance-colonisation cycle impacts on the quality of solution found. The balance between exploration-exploitation of the search space can be altered when the evolving population experiences repeated disturbances. The No Free Lunch Theorems (Wolpert and Macready, 1997) clearly establish that no single method of optimisation is best for all classes of problems. It is not possible to claim than any given algorithm will perform better than any other, over all possible problems. Here, it has been shown that the heterogenous spatial distribution employed in the CGAD allows the algorithm to perform favourably in complex optimisation environments. In a similar, but unrelated study, Marin and Sole (1999) introduced a Macroevolutionary Algorithm (MA), which simulates the dynamics of species extinction and diversification for large time scales. In their model, a network ecosystem is used where the dynamics is based only on the relationship between species. A species becomes extinct if the majority of the neighbouring species have a better fitness value. Simulation results indicated that the MA reaches better fitness values and has a higher success probability than standard GAs for benchmark static optimisation problems. Although there are significant differences between the MA and CGAD models, spatial interactions between species (individuals) in the evolving population coupled with local extinction events are common features which appear to beneficial. A major contribution of this study has been to provide a first step towards understanding the ways in which changes in population characteristics (eg. connectivity changes brought about via the disturbance-colonisation cycle) can affect the performance of GAs. The CGAD model is an abstraction of some of the fundamental effects of diversity in ecosystems. The evolving population can be viewed as a distributed network of connected “organisms”, many of which interact in highly nonlinear relationships. The model has incorporated a number of simplifying assumption. In future work, the ecological metaphor will be extended to examine evolutionary patterns and coevolution (eg. predator-prey scenarios) in complex search and optimisation problems.

Acknowledgments I would like to thank David G. Green for his invaluable comments and suggestions relating to the model. I would also like to thank my colleagues in the SEIS and the anonymous reviewers and editors whose comments and discussion helped to improve the contents of this paper considerably.

Notes 1. SPEA results are available at http://www.tik.ee.ethz.ch/∼Zitzler/testdata.html 2. PAES results are available at: http://www.reading.ac.uk/∼ssr97jdk/multi/

CELLULAR GENETIC ALGORITHM WITH DISTURBANCES

341

References Alvarez, L.W., W. Alvarez, F. Asaro, and H.V. Michel. (1980). “Extraterrestrial Causes for the Cretaceous-Tertiary Extinction.” Science 208, 1095–1108. Belding, T.C. (1995). “The Distributed Genetic Algorithm Revisited.” In L. Eschelman (ed.), Proceedings of the Sixth International Conference on Genetic Algorithms, pp. 114–121. Bentley, P.J. and J.P. Wakefield. (1997). “Finding Acceptable Solutions in the Pareto-Optimal Range using Multiobjective Genetic Algorithms.” In P. Chawdhry (ed.), Soft Computing in Engineering Design and Manufacturing. Berlin: Springer, Part 5, pp. 231–240. Cantu-Paz, E. (1998). “A Survey of Parallel Genetic Algorithms.” Calculateurs Parallels Reseaux et Systems 10(2), 141–171. Cantu-Paz, E. (1999). “Topologies, Migration Rates and Multi-Population Parallel Genetic Algorithms.” In W. Banzhaf et al. (eds.), Proceedings of the 1999 Genetic and Evolutionary Computation Conference. San Mateo, CA: Morgan Kaufmann, pp. 91–98. Cohoon, J.P., S.U. Hegde, W.N. Martin, and D. Richards. (1987). “Punctuated Equilibria: A Parallel Genetic Algorithm.” In J. Grefenstette (ed.), Proceedings of the Second International Conference on Genetic Algorithm. Lawrence Erlbaum Associates, pp. 148–154. Connell, J.H. (1978). “Diversity in Tropical Rainforests and Coral Reefs.” Science 199, 1302–1310. De Jong, K. (1975). An Analysis of the Behaviour of a Class of Genetic Adaptive Systems. PhD Thesis. Univeristy of Michigan. De Jong, K. (1999). “Evolving in a Changing World.” In Z. Ras and A. Skowron (eds.), Foundation of Intelligent Systems. Lecture Notes in Artificial Intelligence, Vol. 1609. Berlin: Springer, pp. 513–519. De Jong, K.A. and M. Potter. (1995). “Evolving Complex Structures via Cooperative Coevolution.” In J. R. McDonnell et al. (eds.), Evolutionary Programming IV, Proceedings of the Fourth Annual Conference on Evolutionary Programming. Cambridge: MIT Press, pp. 307–316. Eldredge, N. and S.J. Gould. (1972). “Punctuated Equilbria: An Alterantive to Phyletic Gradualism.” In T.J.M. Schopf (ed.), Models in Paleobiology. Freeman: San Francisco, pp. 82–115. Ellestrand, N.C. and D.R. Elam. (1993). “Population Genetic Consequences of Small Population Size: Implications for Plant Conservation.” Annual Review of Ecology and Systematics 24, 217–242. Fonesca, C.M. and P. Flemming. (1995). “An Overview of Evolutionary Algorithms in Multi-Objective Optimization.” Evolutionary Computation 3(1), 1–16. Goldberg, D.E. (1989). Genetic Algorithms in Search, Optimization and Machine Learning. Reading, MA: Addison-Wesley. Goldberg, D.E. and R.E. Richardson. (1987). “Genetic Algorithms for Multimodal Function Optimization.” In Proceedings of the Second International Conference on Genetic Algorithms. San Mateo, CA: Morgan Kauffman, pp. 42–50. Green, D.G. (1993). “Emergent Behaviour in Biological Systems.” In D.G. Green and T.R.J. Bossomaier (eds.), Complex Systems—From Biology to Computation. IOS Press, pp. 24–35. Green, D.G. (1994). “Connectivity and the Evolution of Biological Systems.” Journal of Biological Systems 2(1), 91–103. Green, D.G. and M.G. Kirley. (2000). “Adaptation, Diversity and Spatial Pattern.” International Journal of Knowledge-Based Intelligent Engineering Systems 4(3), 184–190. Grefenstette, J. (1999). “Evolvability in Dynamic Fitness Landscapes: A Genetic Algorithm Approach.” In Proceedings of the Congress on Evolutionary Computation (CEC’99). IEEE Press. Vol. 3, pp. 1875–1882. Holland, J.H. (1975). Adaptation in Natural and Artificial Systems. Ann Arbour: University of Michigan Press. Holland, J.H. (1995). Hidden Order: How Adaptation Builds Complexity. Reading MA: Addison-Wesley. Kareiva, P. (1990). “Population Dynamics in Spatially Complex Environments: Theory and Data.” Phils. Trans. R. Soc. Lond. 330, 175–190. Kauffman, S. (1993). Origins of Order: Self Organisation and Selection in Evolution. Oxford: Oxford University Press. Kirley, M. and D.G. Green. (2000). “An Empirical Investigation of Optimization in Dynamic Environments Using the Cellular Genetic Algorithm.” In D.Whitley et al. (eds.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000). San Mateo, CA: Morgan Kaufmann, pp. 11–18.

342

KIRLEY

Kirley, M., X. Li, and D.G. Green. (1998). “Investigation of a Cellular Genetic Algorithm that Mimics Landscape Ecology.” In R. MacKay et al. (eds.), Simulated Evolution and Learning—SEAL98, Volume 1585 Lecture Notes in Artificial Intelligence. Berlin: Springer, pp. 90–97. Knowles, J. and Corne, D. (1999). “Local Search, Multiobjective Optimization and the Pareto Archived Evoltuionary Strategy.” In B. McKay et al. (eds.), Proceedings of the Third Australian-Japan Joint Workshop on Intelligent and Evolutionary Systems. ADFA. Univeristy UNSW, pp. 209–216. Levin, S.A. (1998). “Ecosystems and the Biosphere as Complex Adaptive Systems.” Ecosystems 1, 431–436. Lin, S., E.D. Goodman, and W. Punch. (1997). “Investigating Parallel Genetic Algorithms on Job Shop Scheduling Problems.” In P.J. Angeline (eds.), The Proceedings of the Sixth International Conference on Evolutionary Programming VI. (EP97). Berlin: Springer, pp. 383–394. Manderick, B. and P. Spiessens. (1989). “Fine-Grained Parallel Genetic Algorithms.” In J.D. Schaffer (ed.), Proceeding of 3rd International Conference on Genetic Algorithms. San Mateo, CA: Morgan Kaufmannn, pp. 428–433. Marin, J. and R.V. Sole. (1999). “Macroevolutionary Algorithms: A New Optimization Method on Fitness Landscapes.” IEEE Transactions on Evolutionary Computation 3(4), 272–286. Muhlenbein, H. (1989). “Parallel Genetic Algorithms, Population Genetics, and Combinatorial Optimisation.” In J.D. Schaffer (ed.), Proceeding of Third International Conference on Genetic Algorithms. San Mateo, CA: Morgan Kaufmann, pp. 416–421. Naeem, S. (1998). “Species Redundancy and Ecosystem Reliability.” Conservation Biology 12, 39–45. Nowostawski, M. and R. Poli. (1999). “Dynamic Demes Parallel Genetic Algorithms.” In L.C. Jain (ed.), Proceedings of the Second Knowledge-Based Intelligent Information Engineering Systems Conference (KES99). IEEE Press, pp. 93–98. Potter, M. and K. De Jong. (2000). “Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents.” Evolutionary Computation 8(1), 1–29. Raup, D. and J. Sepkoski. (1982). “Mass Extinctions in the Marine Fossil Record.” Science 215, 1501–1503. Reice, S.R. (1994). “Nonequilibrium Determinants of Biological Community Structure.” American Scientist 82, 424–435. Spears, W.M. (1995). “Adapting Crossover in Evolutionary Algorithms.” In J.R. McDonnell et al. (eds.), Proceedings of the Fourth Annual Conference on Evolutionary Programming. Cambridge, MA: MIT Press, pp. 367–384. Sol´e, R.V., S.C. Manrubia, J. P´erez-Mercader, M. Benton, and P. Bak. (1998). “Long-Range Correlations in the Fossil Records and the Fractal Nature of Macroevolution.” Advances in Complex Systems 1, 255–266. Sousa, W.P. (1984). “The Role of Disturbances in Natural Communities.” Annual Review of Ecology and Systematics 15, 353–391. Tanese, R. (1989). “Distributed Genetic Algorithms.” In J.D. Schaffer (ed.), Proceedings of the Third International Conference on Genetic Algorithms. San Mateo, CA: Morgan Kaufmann, pp. 434–439. Turner, M.G. and V.H. Dale. (1998). “Comparing Large Infrequent Disturbances: What Have We Learned?” Ecosystems 1, 493–496. Tuson, A. and P. Ross. (1996). “Cost Based Operator Rate Adaptation: An Investigation.” In H.-M. Voigt et al. (eds.), Proceedings of the 4th Conference on Parallel Problem Solving from Nature. Volume 1141, Lecture Notes in Computer Science. Berlin: Springer-Verlag, pp. 461–469. Van Veldhuizen, S.A and G.B. Lamont. (2000). “Multi-Objective Evolutionary Algorithms: Analyzing the Stateof-the-Art.” Evolutionary Computation 8(2), 125–147. Vose, M.D. and G.E. Liepens. (1991). “Punctuated Equilibria in Genetic Search.” Complex Systems 5(1), 31–44. Whitley, D. (1993). “Cellular Genetic Algorithms.” In S. Forest (ed.), Proceedings of the Fifth International Conference on Genetic Algorithms. San Mateo, CA: Morgan Kaufmann, pp. 658–662. Wolpert, D.H. and W.G. Macready. (1997). “No Free Lunch Theorems for Optimization.” IEEE Transactions on Evolutionary Computation 1(1), 67–82. Wright, S. (1943). “Isolation by Distance.” Genetics 28, 114–138. Zitzler, E., K. Deb, and L. Thiele. (1999). “Comparison of Multi-Objective Evolutionary Algorithms: Empirical Results.” Evolutionary Computation 8(2), 173–196.