Spatially-Structured Evolutionary Algorithms and Sharing: Do They Mix?

Spatially-Structured Evolutionary Algorithms and Sharing: Do They Mix? Grant Dick and Peter A. Whigham Department of Information Science, University of Otago, Dunedin, New Zealand [email protected]

Abstract. Spatially-structured populations are one approach to increasing genetic diversity in an evolutionary algorithm (EA). However, they are susceptible to convergence to a single peak in a multimodal fitness landscape. Niching methods, such as fitness sharing, allow an EA to maintain multiple solutions in a single population, however they have rarely been used in conjunction with spatially-structured populations. This paper introduces local sharing, a method that applies sharing to the overlapping demes of a spatially-structured population. The combination of these two methods succeeds in maintaining multiple solutions in problems that have previously proved difficult for sharing alone (and vice-versa).

1

Introduction

Traditional evolutionary algorithms (EAs) have difficulty in discovering and maintaining multiple solutions to multimodal fitness landscapes. Through a combination of selection and genetic drift, an EA tends to converge its population onto a single point in a fitness landscape. Previous work has identified this problem and has looked to nature for to provide possible solutions [1]. One method, fitness sharing, has received particular attention [2]. This method uses the concept of finite resources to encourage elements of the population to explore different regions of the fitness landscape. Sharing has been shown to generally perform well, although it is known to have difficulty on at least one class of multimodal problems. Another method which claims to encourage population diversity is to impose a structure on the population and restrict selection and mating to geographically close individuals. Previous researchers have noted that, while spatially-structured evolutionary algorithms (SSEAs) do not prevent population convergence to a single phenotype, they tend to increase the time required for diversity loss. Previous work has explored the concept of merging fitness sharing with SSEAs [3]. The author noted that a sharing-like operator, combined with a spatially-structured population, increased an EA’s ability to maintain multiple peaks within a single population. However, the work was peculiar in that sharing was applied globally, while selection and mating was confined to demes. T.-D. Wang et al. (Eds.): SEAL 2006, LNCS 4247, pp. 457–464, 2006. c Springer-Verlag Berlin Heidelberg 2006 

458

G. Dick and P.A. Whigham

A method in which sharing acted locally was proposed, however it has never materialised in subsequent work. This paper introduces the concept of local sharing. The proposed method applies sharing to each deme prior to selection and offspring replacement. This method reduces the overall complexity of sharing, and is applicable to problems that sharing has difficulty handling. The remainder of this paper is structured as follows: §2.1 describes the concepts of sharing and SSEAs as used in this paper; the proposed local sharing method is described in §3 and a comparison between this method and existing SSEAs is given in §4. Finally, a brief discussion of the findings of this paper and a suggested path for future work is presented in §5.

2

Sharing Methods and Spatially-Structured EAs

The concept of sharing finite resources in an evolutionary algorithm was first proposed by Holland [4], but fitness sharing [2] was the first successful attempt at modelling resource contention within a simple EA. With sharing, each optima in the fitness landscape is allocated a finite number of resources in relation to its size. The fitness of a given optimum in space must then be shared by the number of individuals representing this solution. This introduces two benefits over a simple EA: 1. Rather than attempt to crowd around a single peak, individuals within the population will actively seek out less populated optima. This ensures the constant presence of a selection pressure and counteracts the effects of genetic drift. 2. The effective value of a peak is reduced as the number of individuals exploiting it increases. This in turn makes the lesser-valued optima in the fitness landscape more attractive and drives selection towards them. This reduces the likelihood of losing the lesser peaks from the population. In order for sharing to work, some assumptions need to be made; it is assumed that the optima in the problem occupy similar-sized areas and that they are evenly distributed throughout the fitness landscape. This is due to the mechanism sharing uses to determine which resources are being consumed by each individual. Sharing uses a radius measure, called the sharing radius, to determine the amount of similarity between individuals and hence the niche count of an individual, given by: α   d , if d < σshare 1 − σshare sh(d) = 0, otherwise. The value of σshare is the sharing radius and α is a value that alters the shape of the function (typically, this is set to 1). In the above function, d represents the distance between two individuals in either a genotypic or phenotypic space.

Spatially-Structured Evolutionary Algorithms and Sharing: Do They Mix?

459

The distance measure is preferably in the phenotype space, in other words, the distance metric should be applied as closely to the problem space as possible. Implementing fitness sharing is a simple extension to standard EAs. Its sole purpose is to dynamically alter the fitness of individuals after evaluation; in terms of implementation, it plays no direct part in the fitness function. The other operators in an EA, such as selection and recombination, are equally unaffected. 2.1

Spatially-Structured Evolutionary Algorithms

Fitness sharing is one method in which an EA can incorporate speciation. Another method is to impose a spatial structure on the population so that there is a concept of geographic distance between individuals. Geographically close individuals form localised subpopulations, or demes, within the global population. Mating is confined to demes and offspring are placed within the vicinity of their parents. Evolution potentially acts on demes in different ways so that they explore different regions of the fitness landscape. Given enough time, individuals within a deme will possess sufficiently different genotypes from those of other geographically distant demes that they could be considered a different species. A spatially-structured evolutionary algorithm (SSEA) [5] is essentially an implementation of the parapatric speciation concept; there are no complete boundaries within the population to restrict gene flow. Instead, SSEAs use the vast distances between some individuals to slow the rate of exchange of genetic material to a sufficiently low level so as to promote local divergence of genotypes [6, 7]. Though each SSEA implements a slightly different algorithm, the overall method for using a spatially-structured population within an EA is shown in Algorithm 1. Essentially, there are three basic steps to an SSEA; first, for a given location, construct a deme. After this, select the parents from this deme. Finally, the newly created offspring must be inserted into the deme. Algorithm 1. The general sequence for a spatially-structured evolutionary algorithm

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

input : A given problem output: A spatially-structured population of evolved candidate solutions to the problem. population ← {}; foreach location in space do population[location] ← initialIndividual(); end while not done do generation ← {}; foreach location in space do deme ← constructDeme(location); parents ← select(deme); of f spring ← breedOffspring(parents); evaluate(of f spring); generation[location] ← pickSurvivor(of f spring, deme); end population ← generation; end return population;

460

3

G. Dick and P.A. Whigham

Local Sharing Methods for SSEAs

Spears, as part of his simple subpopulation schemes concept, used a spatiallystructured population in conjunction with a sharing method operating of tag bits [3]. The sharing component operated once per generation and considered the population as a whole. After this step, selection and reproduction continued as for a typical SSEA. Spears noted that this scheme was more effective at maintaining multiple peaks in a fitness landscape than a simple subpopulation scheme that did not impose a population structure. It is not entirely clear that the increase in performance was down to the combination of an SSEA and the sharing method, or if the improvement was purely through the adoption of spatial constraints in the population. The issue is further complicated by the fact that the sharing component of the algorithm and the local mating did not interact; the population topology played no part in determining the shared fitness of the population members. Spears himself noted this peculiarity and suggested a third subpopulation scheme which used the population topology within the sharing process [3]. However, this new method has not appeared in any subsequent work. This paper extends Spears’ idea to create a new niching method, local sharing. The concept is actually rather simple; at the start of a generation, each location calculates the shared fitness of its occupant by determining the individual’s niche count with respect to the other members of the deme. After this, selection and reproduction are performed as per a normal SSEA except that individuals are selected via their shared fitness. Upon completion of mating, the shared fitness of offspring are calculated. Offspring are then inserted into the population if their shared fitness exceeds that of the current occupant. One benefit of the local sharing method is a reduced complexity compared to that of panmictic fitness sharing. Typically, fitness sharing is of O(N 2 ) complexity, as all pairs of individuals must be considered. With local sharing, only individuals that share a common deme are compared, therefore the complexity of local sharing given a deme size of d is O(2N d). For small deme sizes, this represents a considerable saving in the number of distance comparisons required. Another benefit of local sharing is the number of sharing instances that are performed per generation. Local sharing performs multiple, smaller instances of sharing in parallel per generation. The isolation of demes allows each sharing instance to concentrate of different parts of the fitness landscape, increasing the chance of discovering multiple optima. Finally, local sharing permits the use of elitism, which helps to preserve optima once they are discovered.

4

Empirical Analysis

The local sharing SSEA was compared with a traditional SSEA and a panmictic EA incorporating fitness sharing. Each algorithm was tested against two problems. The first problem, referred to as M6, is the Shekel’s foxhole problem as used in De Jong’s thesis on genetic algorithms [8]. This problem is frequently used in comparisons of niching methods [9] as it has 25 local optima in the fitness landscape. The peaks vary in value, so a typical SSEA will have difficulty

Spatially-Structured Evolutionary Algorithms and Sharing: Do They Mix?

461

maintaining all but the global optimum. The second problem, M7, is a massively multimodal, deceptive problem [10]. This problem has over five million optima in the fitness landscape, of which only 32 are global and of interest. Fitness sharing struggles with this problem as the deceptive optima lie within the sharing radius of the global peaks and hence compete for the same resources. Together, M6 and M7 form a good test suite for comparing the hybrid local sharing SSEA. The fitness landscapes of M6 and M7 are shown in Figure 1. 1

0.8

u(x)

0.6

0.4

0.2

0 0

(a) M6

1

2

3 x

4

5

6

(b) M7

Fig. 1. The fitness landscapes of the test problems used in this paper

Population size plays an important role in the performance of EAs. The population sizes for each of the test problems was taken from previous work. The population size for M6 was determined via a population sizing model developed for fitness sharing [11] and was 289 (a 17 × 17 torus for the SSEAs). The population size for M7 was taken from empirical studies [9] and was 676 (a 26 × 26 torus). Selection in all EAs was via fitness proportional selection. Selection in the sharing EA was via stochastic universal sampling [12] to remain consistent with previous work [9]. Roulette-wheel selection was used for the SSEAs, again for consistency with previous work [13]. Two parents were selected (without replacement) to create offspring. A single offspring was created via one-point crossover for the SSEAs, while the fitness sharing EA created two offspring for each reproduction instance. The parameters for each test were as follows: One-point crossover with probability 1.0; mutation was via bit-flipping applied with probability 0.002 per locus; the sharing radius σshare was 8 for M6 and 6 for M7; each SSEA used a torus population structure and a Von-Neumann neighbourhood was used to determine demes; elitist replacement of offspring was used in both SSEAs (via shared fitness for local sharing); each run of an EA lasted for 500 generations. 4.1

Results

The performance of each SSEA on the test problems was measured through two statistics used in previous niching studies [14]. The first measurement records the number of peaks discovered and maintained by the population over 500 generations. Higher values for the number of maintained niches indicates better

462

G. Dick and P.A. Whigham

25

Peaks Maintained (MAX 32)

Peaks Maintained (MAX 25)

algorithm performance. The second measure is the Chi-Square-Like performance which measures the deviation of the population from that of an “ideal” population in which all individuals reside on a peak in distributions relative to a peak’s value. Lower values for the Chi-Square-Like measure indicate better performance (a value of zero being ideal). These measurements were averaged over 100 runs and are reported below. The number of peaks maintained over time is shown in Figure 2. On M6, local sharing maintains approximately half of the 25 optima present in the fitness landscape after 500 generations. However, local sharing on M7 is far superior to that of either panmictic sharing of the traditional SSEA, with nearly all 32 optima present in the population after 500 generations. The Chi-Square-Like performance of the three EAs, as shown in Figure 3, tells a similar story to the number of peaks performance; local sharing distributes individuals among peaks more efficiently than the traditional SSEA on both M6 and M7 and is superior to panmictic sharing on the M7 problem.

20 15 10 5 0

30 25 20 15 10 5 0

0

100 Standard SSEA

200 300 Generations Sharing

400

500

0

Torus w/Sharing

100 Standard SSEA

(a) M6

200 300 Generations Sharing

400

500

Torus w/Sharing

(b) M7

90

Chi-Square-Like Performance

Chi-Square-Like Performance

Fig. 2. Number of peaks maintained by each algorithm on M6 and M7

80 70 60 50 40 30 20 10 0 0

100 Standard SSEA

200 300 Generations Sharing

(a) M6

400 Torus w/Sharing

500

40 35 30 25 20 15 10 5 0 0

100 Standard SSEA

200 300 Generations Sharing

400

500

Torus w/Sharing

(b) M7

Fig. 3. Chi-Square-Like performance of the tested EAs on M6 and M7

4.2

Discussion

The results presented here indicate that local sharing is able to discover and maintain more optima than a traditional SSEA. Local sharing is also able to support multiple optima on M7, a problem that has traditionally been difficult to handle using sharing methods. However, the performance of local sharing on M6 does not match that of panmictic sharing. This may be in part due to the elitist replacement of local sharing; too much emphasis is placed on the global optimum by elitism at the expense of other niches. Elitism plays an important role in local sharing’s performance on M7, so elitist replacement cannot be simply removed

Spatially-Structured Evolutionary Algorithms and Sharing: Do They Mix?

463

from the local sharing method. Instead, we propose a change in elitism from a strict, “always better than” approach to a probabilistic method. In the proposed strategy, offspring enter a tournament with the current occupant to determine who is passed into the next generation. The probability of an offspring o winning the tournament over the current occupant i is: p (o) =

fsh (o) fsh (o) + fsh (i)

25 20 15 10 5 0 0

100

200 300 Generations

400

500

Chi-Square-Like Performance

Peaks Maintained (MAX 25)

where fsh is the shared fitness of an individual. Experiments on M6 were repeated using this alternative elitism method and the results are shown in Figure 4. The probabilistic elitism method allows local sharing to support more optima within the population, and the distribution of individuals among those peaks is much closer to that of panmictic sharing (as indicated by the Chi-Square-Like performance). 35 30 25 20 15 10 5 0 0

Sharing Torus w/Sharing (Strict Elitism) Torus w/Sharing (Prob. Elitism)

(a) Peaks Maintained

100

200 300 Generations

400

500

Sharing Torus w/Sharing (Strict Elitism) Torus w/Sharing (Prob. Elitism)

(b) Chi-Square-Like Performance

Fig. 4. Comparison of elitism strategies for the local sharing method on M6

5

Conclusion and Future Work

Space often plays an important role in speciation of populations. Traditionally, EAs have used individual-based comparisons to perform niching in the absence of spatial population structure. This paper presents a new method for niching, local sharing, which takes the traditionally panmictic niching method of fitness sharing and applies it within the demes of a spatially-structured population. Initial results suggest that local sharing is applicable to problems that have proved difficult to solve by either sharing or SSEAs alone. Local sharing is clearly superior to panmictic sharing on the M7 problem. This is possibly in part due to local sharing’s elitist replacement policies. However, the same elitist strategy appears to prevent local sharing from maintaining all optima in M6. An alternative elitism strategy greatly improved the performance of local sharing on M6. Future work should investigate the elitism strategies of local sharing in greater detail. One possible direction might be to implement elitism as a Boltzmann tournament with annealing [15] so that the higher valued optima do not dominate elitism in earlier generations of a run.

464

G. Dick and P.A. Whigham

References 1. Mahfoud, S.W.: Niching methods for genetic algorithms. PhD thesis, University of Illinois at Urbana-Champaign, Urbana, IL, USA (1995) IlliGAL Report 95001. 2. Goldberg, D.E., Richardson, J.: Genetic algorithms with sharing for multi-modal function optimisation. In: Proc of the 2nd Int. Conf. on Genetic Algorithms and Their Applications. (1987) 41–49 3. Spears, W.M.: Simple subpopulation schemes. In Sebald, A.V., Fogel, L.J., eds.: Evolutionary Programming: Proc. of the Third Annual Conf., Singapore, World Scientific Press (1994) 296–307 4. Holland, J.H.: Adaptation in Natural and Artificial Systems. 2 edn. The MIT Press, Cambridge, Massachusetts (1992) 5. Tomassini, M.: Spatially structured evolutionary algorithms. Springer (2005) 6. Wright, S.: Isolation by distance. Genetics 28(2) (1943) 114–138 7. Mayr, E.: Populations, species and evolution; an abridgment of Animal species and evolution. Harvard University Press (1970) 8. De Jong, K.A.: An Analysis of the Behavior of a Class of Genetic Adaptive Systems. PhD thesis, University of Michigan, Ann Arbor, MI (1975) Dissertation Abstracts International 36(10), 5140B, University Microfilms Number 76-9381. 9. Mahfoud, S.W.: A comparison of parallel and sequential niching methods. In Eshelman, L., ed.: Proceedings of the Sixth International Conference on Genetic Algorithms, San Francisco, CA, Morgan Kaufmann (1995) 136–143 10. Goldberg, D.E., Deb, K., Horn, J.: Massive multimodality, deception, and genetic algorithms. In R. M¨ anner, Manderick, B., eds.: Parallel Problem Solving from Nature, 2, Amsterdam, Elsevier Science Publishers, B. V. (1992) 37–46 11. Mahfoud, S.W.: Population size and genetic drift in fitness sharing. In Whitley, L.D., Vose, M.D., eds.: Foundations of genetic algorithms 3, San Francisco, Morgan Kaufmann (1995) 185–224 12. Baker, J.E.: Reducing bias and inefficiency in the selection algorithm. In Grefenstette, J.J., ed.: Genetic Algorithms and their Applications (ICGA’87), Hillsdale, New Jersey, Lawrence Erlbaum Associates (1987) 14–21 13. Sarma, J.: An Analysis of Decentralized and Spatially Distributed Genetic Algorithms. PhD thesis, George Mason University, Fairfax VA, USA (1998) double sided. 14. Deb, K., Goldberg, D.E.: An investigation of niche and species formation in genetic function optimization. In Schaffer, J.D., ed.: Proc. of the Third Int. Conf. on Genetic Algorithms, San Mateo, CA, Morgan Kaufmann (1989) 42–50 15. Goldberg, D.E.: A note on Boltzmann tournament selection for genetic algorithms and population–oriented simulated annealing. Complex Systems 4(4) (1990) 445–460