Genetic Algorithm involving Coevolution Mechanism to Search for E ...

Genetic Algorithm involving Coevolution Mechanism to Search for Eective Genetic Information 3

Hisashi Handa , Tetsuo

Norio Baba

3 Sawaragi

* Department of Precision Engineering, Kyoto University Sakyo-ku, Kyoto, 606-01, JAPAN fhanda, katai, sawaragi, [email protected]

+ , Osamu Katai3 ,

and Tadashi Horiuchi

3

+ Department of Arts and Science,

Osaka Educational University Asahiga-Oka 4-698-1, Kashihara City, Osaka, 582, JAPAN [email protected]

Abstract |A new genetic algorithm which exploits an idea of \coevolution" is proposed. The proposed method consists of two GAs: Host GA and Parasite GA. The Host GA searches for the solutions, and these two GAs are closely related to each other. The Parasite GA plays an important role in searching for useful schemata in the Host GA. Furthermore, two methods of tness evaluation of Parasite GA are examined: dierentiating method and averaging method. The dierentiating method will yield the search for schemata that are not yet discovered by the Host GA. The averaging method will yield the search for schemata that have high average of tness. Various computer simulations con rm the eectiveness of the proposed methods. I. Introduction

In recent years, Evolutionary Computation [Fogel 95] has been studied by many researchers. Particularly, Genetic Algorithm (GA) [Goldberg 89], a eld of Evolutionary Computation, has been applied successfully to various real world problems. In this paper, a new genetic algorithm which improves the search ability of GA is proposed. The proposed method involves two genetic algorithm models. The rst GA model searches for a solution in a given environment, and the second GA model searches for useful schemata in the rst GA model. The new genetic algorithm has a higher search ability due to the coevolution consisting of these two GA models. Furthermore, the evaluation of the schemata stored in the second GA is done by two methods, and their search abilities are examined by referring to experimental results. Next, related works are described as follows: There

are schemata oriented search methods such as Cultural Algorithm [Reynolds 96], Stochastic Schemata Exploiter [Aizawa 94], and so on. In Cultural Algorithm, the population consists of two subpopulations: one that is subject to Genetic Algorithm and the other in the belief space. The Genetic Algorithm partly exploits the belief space to direct the action of individual in the Genetic Algorithm, and the belief space is generated by the search process in the Genetic Algorithm. The Algorithm is similar to the our proposed method, but contrast to the Cultural Algorithm, our method searches for useful schemata information by using the second GA. In Stochastic Schemata Exploiter, the schemata information is generated by good individuals in current generation, and is used to generate osprings in the next generation. Furthermore, we can consider the eect by the second GA as the eect by a kind of sophisticated crossover, namely, useful schemata information is propagated between individuals through the second GA. Sebag et al. [Ravise 96], [Sebag 94] used crossover in addition to inductive learning. They adopted a criteria on \good" crossover operation, and the criteria is derived by inductive learning. New crossover operator (Bit-based Simulated Crossover) that employs global information to help evolution is devised by Syswerda [Syswerda 92]. The crossover uses probability density at each locus for deciding crossing site, and is shown to have good performance. These novel algorithms including ours can be summarized as being attempts to search for subspaces that should be searched in detail. In section II, the new genetic algorithm is intro-

Fitness

Fitness Value Superposition, Transcription

0011100

1100101

1101100

1110110

0011100

0010101

0101001

reproduction crossover mutation

1011100 1100101 0000011

n

th

generation

th

n+1

Environment Fitness Evaluation

generation

Chromosome Fig. 1.

Parasite GA reproduction crossover mutation

Host GA reproduction crossover mutation

Environment Fitness Evaluation

Fitness Value Chromosome Fig. 2.

Process of the proposed method

Process of traditional genetic algorithm

duced. Further, in section III, the evaluation methods and experimental results on computer simulations are examined. The last chapter concludes this paper. II. Genetic Algorithms Involving a Mechanism of Coevolution

Some schemata that represent substructures of the solution space may have useful information to search for solutions more eectively. However, in the usual GA, useful schemata (building blocks) are extracted as a consequence of GA-based searches for the solution re ected in the genetic information commonly shared in the resultant population. To overcome this \selfreferential" diculty, we will extend the traditional GA's to make better use of eective genetic information given by a direct search for useful schemata. That may make the search ability of GAs more ecient. Namely, the our method consists of two GA models interacting with each other. As depicted in Figure 2, the rst GA model is the traditional GA model, storing individuals that may be \transcripted" by the schema information (genetic information) which is discovered by the second GA model. And the second GA model searches for useful schemata on the rst GA model, and the tness value of each individual of the second GA model is calculated by referring to the population distribution of the individuals in the rst GA model. A. Outline of the Proposed method

First, each GA model used in our proposed method is described as follows: The rst GA model is the traditional GA model that searches for a good individual (solution) t to the given environment (problem).

Hereafter, this GA model and each individual member of the population are called H-GA and H-indiv., respectively, where \H" stands for \Host". The tness value of H-indiv. is evaluated according to the given environment, and the GA search based on this tness value is carried out. Next, to improve search ability of H-GA, we adopt the second GA model to inform H-GA the candidate subspaces to be search for. Namely, the second GA model searches for eective schemata in H-GA, whose individuals consist of 0's, 1's, and \3" representing schemata in H-GA model. (Hereafter, we presume that the genotypes of H-GA is given as bit sequences.) Furthermore, as depicted in Figure 2, there are two operators on the second GA model to the rst GA model, i.e. superposition and transcription, which will be described in more detail in later subsections. The superposition operator uses genetic information in the H-GA to calculate the tness values of the second GA individuals, and the transcription operator propagates eective schemata information in the second GA into H-GA population. Hereafter, the second GA model and each of the corresponding individuals are called P-GA and P-indiv., respectively, where \P" stands for \Parasite". As similar to H-GA, P-GA searches for good solutions based on the tness value, by having eective interactions with H-GA, which leads to a desirable coevolution of H-GA and P-GA. Furthermore, genetic information of a P-indiv. is copied into the H-GA population by \superposing" P-indiv.'s onto H-indiv.'s , with a given probability, as depicted in Figure 3. The tness values of these Pindiv.'s are evaluated by referring to the phenotypes of these superposed chromosomes. (cf. Figure 4)

H-Individual

* * * 00 ** * 1** 0

P-Individual

1 1 0 00 00 0 111 0 Superposed H-Individual

transcription

Superposition

1 1 0 01 00 0 011 1

Fig. 3. Mechanism of superposition operator

Fitness Fitnesses of averaging method Fitnesses of differentiating method P-Indiv. (a schema in H-GA) Superposed Indv. Superposition (Transcription) Operation H-Indiv. Fig. 4.

Genetic Space

Fitness evaluation of P-indiv. and superposition

(transcription) operator in genetic space of H-GA.

B. Fitness Evaluation of Parasite Individual P-GA searches for useful schemata in H-GA. Then, the useful schemata in H-GA may be de ned in two ways as follows: (1) undiscovered useful schemata and simply (2) useful schemata, i.e., those with high average tness values. Hence, the tness functions of P-GA may be calculated as dierentiating or averaging the functions and are described as follows: Dierentiating Method: !! The tness value Fj of P-indiv. j is described as follows1 . The superposing (superposition) operation of each P-indiv. onto H-indiv.'s is carried out n times. (1) First, it is randomly decided which H-indiv. is superposed by P-indiv. j . (2) H-indiv.'s that are superposed by P-indiv. j are denoted as i1 ; . . . ; in . And the resultant superposed H-indiv.'s are denoted as ~i1 ; . . . ; ~in . (3) To calculate the tness value of P-indiv. j , the tness evaluation of each of the superposition operations is de ned as the contribution of 1

In this paper, the tness function of H-indiv. i and P-indiv.

j are represented as fi and Fj , respectively.

the superposition operation to H-indiv. de ned by the following equation: ^

=

fk

max(0; f~ik

0

fik )

(k = 1; . . . ; n)

:

(1)

As depicted in Figure 4, a P-indiv. represents a subspace (schema) in the genetic space of the H-GA. An H-indiv. is projected onto that subspace by the superposition operator. Thick lines in Figure 4 denote the dierence between the tness of the original H-indiv.'s and those of the superposed H-indiv.'s, that is, positive \contribution" of this superposition operation. If the dierence is negative, then the contribution of this operation is regard to be 0. (4) Finally, the tness Fj of P-indiv. j is given by the following equation:

X n

Fj

=

f^k :

(2)

k =1

Averaging Method: !! In our proposed method, the P-GA evaluates a subspace in the genetic space of H-GA. Hence, steps (1), (2), and (4) of the dierentiating method also apply to the averaging method. However, step (3) is altered as follows: (3') Similar to (3), to calculate the tness value of P-indiv., the tness function for the superposition operation is de ned not by the \contribution" of the operation but by the \result" of the operation, i.e., we set (cf. Figure 4) ^ =f : ~ik

fk

0

(1 )

C. Transcripting the Superposed Chromosomes into H-GA Population A superposed chromosome is used by P-indiv. to calculate the tness value of P-indiv. At this point, the population of the H-GA is not aected by the superposition operation. However, we will adopt a method to propagate eective (useful) schemata information obtained by P-GA into H-GA population, by probabilistically replacing the original H-indiv. ik with the superposed individual ~ik . In this paper, this

Average Fitness

Foxholes

fitness

400

200 60.0 0 -60.0

500 450 400 350 300 250 200 150 100

Differentiating method, pop = 20 Averaging method, pop = 20 sGA, pop = 120 sGA, pop = 40 0

5

10

0.0

15

20 25 Generation

30

35

40

0.0 60.0 -60.0

Fig. 8. Comparison of our method with sGA on the Foxholes problem

Fig. 5. A tness function: Shekel's Foxholes Average Fitness

3.5

Bohachevsky #1

fitness

3.0 2.0 1.0

Differentiating method, pop = 20 Averaging method, pop = 20 sGA, pop = 120 sGA, pop = 40

2.5 2 1.5

1.0

0 -1.0

3

0

0.0 0.0

20

40 60 Generation

80

100

1.0 -1.0

Fig. 9. The proposed method compared with the sGA on the Bochevsky problem

Fig. 6. A tness function: Bohachevsky No.1

III. Experimental Results

operation of replacing individuals is called \transcripIn this section, several experiments on function option" operation. The probability of applying the trantimization problems are used to evaluate the eectivescription operation is de ned as ness of our methods. To compare the proposed meth( ods with sGA (simple GA) performance, the following f^ =(f 0 fmin ); f^k > 0 Pparasite = k max 0 otherwise; tness functions are used as benchmark problems. F1 : One Max Problem where fmax and fmin denote maximum and minimum 200 X tness values in the H-GA, respectively, and hence f1 (~x) = xi Pparasite is a value such that 0 < Pparasite < 1. i=1 F2 : Shekel's Foxholes (See Figure 5) 25 1 1 X 1 = + : P f2 (~x) K j =1 cj + 2i=1 (xi 0 aij )6

Average Fitness

200 Differentiating method, pop = 20 Averaging method, pop = 20 sGA, pop = 120 sGA, pop = 40

180 160

F3 : Bohachevsky's Function No.1 (See Figure 6)

140

f3 = x21 +2x22 0 0:3 cos(3x1) 0 0:4 cos(4x2 )+0:7 :

120 100 0

20

40 60 Generation

80

100

The population size of H-GA and that of P-GA are both set as 20. To compare our method with sGA, Fig. 7. Comparison of the proposed methods with sGA on we carried out two computer experiments: rst, sGA the One Max problem with a population size of 40, i.e., the same as the total

H-GA

P-GA H-GA P-GA

150

10 8 6

100 4 50

2

0 0

20

40 60 Generation

80

0 100

200 Average Fitness

Average Fitness

200

H-GA P-GA

150 100 50 0 0

10

20

30

40 50 60 Generation

70

80

90

100

Fig. 10. Temporal behavior of the average tness values in Fig. 11. Temporal behavior of the average tness values the H-GA and the P-GA with Dierentiating Method in the H-GA and the P-GA with Averaging Method applied to One Max Problem applied to One Max Problem)

population (H-GA plus P-GA); and second, sGA with a population size of 120, in this case the number of the times of tness evaluations2 is set as the same as that in our proposed method. In these experiments, the GA parameters such as crossover probability and mutation probability are set as Pc = 0:8 and Pm = 0:001, respectively, and the number of H-indiv.'s that are superposed by a P-indiv is set as n = 5. The number of runs used to evaluate one experimental setting is 10. The coding for the case of F2 and F3 is set to be binary coding with 20 bits. Figures 7, 8, and 9 describe the experimental results on each of the problems. These gures show that the proposed methods could search for the optimal solution more eectively than sGA. The dierentiating method searched for a good solution faster than the averaging method because the P-GA in the dierentiating method searches for schemata in H-GA, which has not yet been discovered by the H-GA. Furthermore, in the averaging method, the tness function is evaluated merely as the average of the tness in a subspace; the size of the subspace is not evaluated. Thus, it may give a high tness value to a P-indiv. even if its phenotype doesn't represent a good schemata, provided that the H-indiv. that is superposed by the P-indiv. has a high tness value. This problem necessitates further researches. Figures 10 and 11 describe temporal change of the average tness values in the the H-GA and PGA under the dierentiating method and the averag2

We consider that the computational processes of tness evaluations are the most time consuming. Hence, we have no estimation of the proposed method on the CPU time, but on the number of tness evaluations.

ing method, respectively. As depicted in Figure 10, P-GA searches eectively until the 10th generation. After that generation, however, its eectiveness gradually decreased. The reason may be as follows: Until 10th generation, if a piece of genetic information stored in a P-indiv. represents an eective and undiscovered schemata, then the P-indiv. can gain high tness value. However, after that generation, such schemata will hardly be discovered in the P-GA, because useful schemata information that is already discovered has been propagated to the H-GA population by the transcription operation. Furthermore, as depicted in Figure 12, we examined the temporal change of the number of \3" in the P-GA by Dierentiating Method or Averaging Method. Although intuitively both imagined that the number of \3" would decrease in both search processes, there are no drastic changes. The reason why such phenomena have occurred is considered as follows: Suppose that, at each loci, P-GA converged to an allele which will provide higher tness values until H-GA converges to the allele. Then P-GA will converge to the allele and the number of \3" will decrease. Otherwise, that is, if H-GA converges to the allele until P-GA converges to the allele, then P-GA converges to either the allele or \3" because of no contribution at the locus. Hence, converged allele at the locus is decided by genetic drift. IV. Conclusion

In this paper, a Genetic Algorithm involving two dierent mechanisms of coevolution is proposed. As shown in section III, the eectiveness of the proposed method is con rmed by several experimental results.

No. of Asterisk

80 differentiating averaging

75 70 65 60 55 0

20

40

60

80

100

Fig. 12. Temporal behavior of the number of \3" in P-GA with Dierentiating Method and Averaging Method applied to One Max Problem)

In the proposed methods, H-GA and P-GA coevolve with limited interaction with each other. And the proposed methods are expected to have eective search abilities and showed better performance than sGA's, even having a smaller population size than sGA's. To verify the eectiveness of the proposed methods, it is important to devise further detailed analyses by using the correlation between the genetic information in the H-GA and the schemata information discovered by the P-GA. Also, more detailed analysis of the eect of transcription in the genetic space in H-GA should desirably be done. It is also important to devise the schemata-speci c operators for P-GA, such as generalization, specialization, merge, and so on, such that P-GA makes better search of eective schemata. Further experiments with other tness functions with other GA parameters should also necessarily be examined for the detailed analyses of the proposed methods.

References [Aizawa 94] Akiko N. Aizawa: \Evolving SSE: A Stochastic Schemata Exploiter", Proc. of the First International Conference of Evolutionary Computation, pp.525-529, 1994. [Fogel 95] David B. Fogel: \Evolutionary Computation", IEEE Press, 1995. [Goldberg 89] David E. Goldberg: \Genetic Algorithm in Search, Optimization, and Machine Learning", AddisonWesley, 1989. [Grefenstette 89] John J. Grefenstette and James E. Baker: \How Genetic Algorithms Work: A Critical Look at Implicit Parallelism", Genetic Algorithm and Their Applications: Proceedings of the Third International Conference of Genetic Algorithm, pp.20-27, 1989. [Paredis 95] Jan Paredis: \The Symbiotic Evolution of Solutions and their Representations", Genetic Algorithm and Their Applications: Proceedings of the Sixth International Conference of Genetic Algorithm, pp.359-365, 1995.

[Potter 95] Mitchell A. Potter, Kenneth A. De Jong and John J. Grefenstette: \A Coevolutionary Approach to Learning Sequential Decision Rules", Genetic Algorithm and Their Applications: Proceedings of the Sixth International Conference of Genetic Algorithm, pp.366-372, 1995. [Ravise 96] Caroline Ravise and Michele Sebag: \An Advanced Evolution Should Not Repeat its Past Errors", Proc. of the Thirteenth International Conference of Machine Learning, pp.400-408, 1996. [Reynolds 96] Robert G. Reynolds and ChanJin Chung: \A Self-adaptive Approach to Representation Shifts in Cultural Algorithms", Proc. of the Third International Conference of Evolutionary Computation, pp.94-99, 1996. [Sebag 94] Michele Sebag and Marc Schoenauer: \Controlling Crossover through Inductive Learning", Proc. of Parallel Problem Solving from Nature - PPSN III, pp.209-218, 1994. [Syswerda 92] Gilbert Syswerda: \Simulated Crossover in Genetic Algorithms", Foundation of Genetic Algorithms 2, Morgan Kaufmann Publishers, pp.239-255, 1992.