Developmental Evaluation in Genetic Programming: A ... - CiteSeerX

FRONTIERS IN THE CONVERGENCE OF BIOSCIENCE AND INFORMATION TECHNOLOGIES, OCTOBER 2007

1

Developmental Evaluation in Genetic Programming: A Position Paper Tuan Hao Hoang, Student Member, IEEE, R I (Bob) McKay, Senior Member, IEEE, Daryl Essam, and Xuan Hoai Nguyen, Abstract—Standard genetic programming genotypes are generally highly disorganized and poorly structured, with little code replication. This is also true of existing developmental genetic programming systems [1], [2], which exploit regularity by using procedures, functional modules, or macros and parameters passing. By contrast, in biological developmental evolution, nature works through code duplication to generate modularity, regularity and hierarchy. Previous developmental approaches have only one level of evaluation for each individual - an approach which limits the advantages of modularity to the species rather than the individual, and hence inhibits selection of modularity. We argued in [3], [4] that evaluation during development is necessary for structural regularity to emerge. To confirm the benefits of developmental evaluation and the contribution of code duplication to nature, our new developmental process uses a new representation. Developmental Tree Adjoining Grammar Guided GP (DTAG3P) uses L-systems to encode tree adjoining grammar guided (TAG) derivation trees, and has been investigated in [4]. We have demonstrated scalable solutions to difficult families of problems, and have evidence that this performance is linked to the generation and exploitation of structural regularities in the solutions. Index Terms—Regularity, developmental evaluation, developmental genetic programming.

✦

1

I NTRODUCTION

G

ENETIC programming (GP) was developed by Koza [5] and others with the aim of getting computers to automatically solve problems without having to tell them how to do it. Based on observations of biological systems, GP uses an abstraction of Darwin’s natural selection mechanisms to evolve populations of solutions to problems. One of the biggest problems with GP, known as bloat, is its tendency to produce solutions with large amounts of unnecessary code, exponentially increasing over time and with no relationship to any improvement in the quality of the solutions. In the corresponding natural system, less than 5% of the 3.2 ∗ 109 amino acid codons that make up the human genome code for proteins[6]. In this sense, bloat in GP has been seen as simply analogous to non-functional code in biological • H. Hoang and D. Essam are with the School of IT and EE, University of New South Wales@ADFA, Canberra, Australia. B. McKay and X. Nguyen are with the School of CS and E, Seoul National University, Korea.

genotypes. However there are important differences. Most obviously, bloat in natural systems is not exponential. Simple prokaryotes, separated from humans by around 2 ∗ 109 years, typically have 106 codons, while the puffer fish, whose lineage diverged perhaps 4 ∗ 108 years ago, has 2.8 ∗ 108 . Interestingly, the puffer fish has roughly the same number of functional genes as humans, so that the proportion of noncoding DNA in its genotype is far lower. The picture has been further complicated recently by the increasing realisation that the non-gene portion of the DNA of higher animals has important regulatory functions, and that the functional portion of human DNA may be much higher than previously thought. While accurate estimates of the proportion which has some function are not yet available, it is clear that the 5% of coding DNA represents a serious underestimate of the functional proportion of the genome. Thus human DNA may posses far less ’junk’ than previously thought. Moreover, unlike biological systems, genetic


programming relatively poor at finding scalable structured solutions, rarely finding hierarchical or structural regularity, and exhibiting little repetition of units, self-similarity, or reuse. By contrast, the natural evolutionary systems on which it is based are able to evolve structural regularity (e.g. closely related genes from the homeobox family control the segmentation of the vertebral column of vertebrates, one gene per segment, with further related homeobox genes controlling the development of segmented limbs budded off from the vertebral column [7]). This problem, of generating structured solutions in GP, has received wide attention. Koza introduced Automatically Define Functions (ADF) [1], where parts of the program are modularized and encapsulated; this approach also reduced coat bloat. Koza showed that ADFs allows genetic programming systems to better exploit regularities of problem domains, improving system performance. Spector [2] investigated Automatic Defined Macros (ADMs), similar to ADFs, but structured more as programming language macros than function calls. He observed that ADMs, like subroutines, can assist in the modularization of complex programs and in the exploitation of domain regularities. It was suggested that ADMs are likely to be useful for evolving intelligent action systems for complex environments. However, these approaches bear little relationship to biological processes. Natural evolution exploits regularities and modules by copying or duplicating. As examples, we may cite the Moghadam et al. [8] demonstration of homeobox gene duplication in rainbow trout, where many duplicated genes, sharing a high level of amino-acid similarity, appear in the genome. In his classic 1970 book Evolution by Gene Duplication citeOhno Ohno emphasised the importance of gene replication and subsequent modification,while Zhang in 2003 [10] demonstrated that the creation of new structures and behaviors of a biological organism begins with gene duplication. Without gene duplication, the flexibility of genomes or species in adapting to changing environments would be severely limited. In other words, nature does not use procedures or macros and parameters

2

passing, but instead works through code duplication to generate modularity, regularity and hierarchy. We don’t always need to do the same as nature, but we do need to understand why nature can work in a particular way and we can’t – in this case, why nature can generate structural regularity that GP cannot Over the years, a number of authors have built genetic programming systems incorporating developmental processes. Rosca investigated an Adaptive Representation [11], based on the discovery of useful building blocks of code. This approach greatly improved search efficiency on the problems considered. Jacob [12] proposed Genetic L-System Programming, which used context-free L-systems (OL) with a stack capability, and an evolutionary algorithm, to learn L-systems for the creation and development of artificial flowers. Haddow el al. used Lindenmeyer systems for digital circuit design [13]. Later, Developmental Cartesian Genetic Programming was investigated by Miller et. al.; it aimed to evolve a cell that can construct a larger program by iteration of the cell’s program in its environment [14]. Nevertheless, structural regularity, which is the correlation of patterns within an individual (e.g., the repetition of units, symmetries, selfsimilarities, smoothness), has not been clearly demonstrated in existing developmental GP systems. A developmental process added to GP allows regularity but it does not guarantee it: there has to be an evolutionary benefit if it is to emerge. But unless structural regularity is implicitly encoded into the fitness function, it has no direct benefit to the individual, whatever benefit it may offer the species as a whole (in adaptability) – hence it cannot be selected for. So how can regularity arise in natural systems? We argue that the benefit arises from repeated evaluation during development – that the process of development through a series of stages, with evaluation during each stage, provides the necessary fitness advantage to regularly structured individuals. We argue that, as a result, structural regularity does not emerge in these earlier GP systems because an individual is usually expected to solve problems immediately, without the benefit of a developmental phase.


Even where various morphological systems have been used in some previous developmental genetic programming systems to allow program individuals to ”grow” from simple to more complex forms, the individuals have still typically been evaluated only once, at maturity, providing no selective pressure toward regular structure. Thus Whitley and Gruau in [15] proposed the technique of a cellular developmental process encoded by a grammar tree to generate the families of Boolean neural networks to solve parity problems. Spector and et al. [16] implemented Ontogenetic Programming in the Ontogenetic HiGP systemm, which includes program self-modification functions in the genetic programming function sets, thereby allowing evolved program individuals to modify themselves during the course of the run. But where these approaches are used with a single evaluation, as in most artificial developmental systems, it only has adaptive advantages to entire species, not to particular individuals, and hence cannot be selected for by evolution. In developmental biological systems, on the other hand, evaluation is continuous throughout development – if the individual is insufficiently fit to survive at a particular stage of development, the fitness it would exhibit at later stages is immaterial. In other words, if an individual is uncompetitive at any stage of development, it is unlikely to survive to reproduce. In simpler organisms, the organism faces similar challenges at each stage of development, but in more complex organisms, the challenges increase in complexity as the individual develops. Moreover, once the genes controlling a particular stage of development have succeeded at a particular stage of development, they tend to become fixed, and to change only slowly. We aim to simulate these processes. There are a number of components to this simulation: 1) Developmental process governed by ’genes’: for this, we use a Deterministic L-system, in which the grammar simulates the genes, and the L-system development process the growth of the organism 2) Developmental evaluation: we require a phenotype which we can always guarantee to evaluate, at every stage of develop-

3

Fig. 1. TAG-Based L-Systems Example, Left: representations of individual rules, Right: stages of development ment. We use a tree adjoining grammar guided genetic programming (TAG) representation which has this property [17]. 3) Evolutionary process evolving the L-system 4) Adaptive variation rates, allowing genes used early in development to become fixed in later stages of evolution

2

A TAG BASED DOL SYSTEM

Briefly, TAG representation consists of an α tree (’The cat sat on the mat’) and instructions for adjoining β trees (’black’ or ’which it liked’) to form a more complex whole (’The black cat sat on the mat which it liked’). In our representation, the DOL triple G = (V, ω, P ) is mapped to TAG representation by defining ω to consist of an α tree together with a predecessor from P , and each letter {L1 , L2 , L3 , . . . ...} ∈ V to be either a predecessor from P or a β tree. Thus the DOL-rewriting of ω corresponds to adjunction of successive β trees into the initial α tree. For example, assume an L-system G′ = (V ′ , ω ′, P ′ ) with V ′ = {L1 , L2 , L3 , L4 }, ω ′ = (α1 , L1 ), and P ′ being P1′ : L1 → β1 β2 β3 L4 β1 L2 , P2′ : L2 → β2 β1 β4 β3 L2 L3 , P3′ : L3 → β5 β6 L4 β7 β8 L1 , P4′ : L4 → β1 L2 β4 β6 β7 L3 . Figure 1 shows tree representations of these productions, together with three stages of the expansion of this system into a TAG derivation tree.The expansion starts with the TAG initial tree α1 together with the predecessor L1 . In the stage 1 expansion, L1 is replaced by its successor in the production rule P1′ . This successor has two predecessors


L2 and L4 . In the stage 2 expansion, these are replaced by their successors using the corresponding production rules P2′ and P4′ . This leaves us with four available predecessors – two occurrences each of L2 and L3 . This process continues until predefined limits on the number of stages are reached.

3 D EVELOPMENTAL (DTAG3P)

TAG

4

TABLE 1 Percentage Success Rate on Symbolic Problems (100 runs) GP

TAG

DEVTAG

DTAG3PF9all

DTAG3P

0%

8%

13%

0%

73%

GP

DTAG3P uses TAG-based DOL-systems to encode Tree Adjoining Grammars.Each individual is expanded through its developmental stages, and at each stage is evaluated against a series of increasingly difficult problem (corresponding to the increasingly complex environments faced by growing higher organisms). DTAG3P uses a developmental form of tournament selection which only requires a fitness ordering of individuals. We first compare the individuals on stage 1 fitness f it(I1 ). The fittest individuals are carried to stage 2. This is repeated as necessary for I2 , . . . , Imaxlif e , (maxlif e being the fixed maximum number of developmental stages, corresponding to the size of the problem family). Two individuals I, J are considered equal if |f it(Is )−f it(Js ) ≤ δ| (see [3] for details of the evaluation mechanism; for more detail of DTAG3P, see [4], [18]).

Fig. 2. Cumulative success of DTAG3P, DEVTAG, TAG vs. No. function evaluations(on Symbolic Regression) 4.2 Results

Table 1 summarises the success rates for the symbolic regression problem family (cumulative success is plotted against the number of function evaluations in Figure 2, for the treatments which actually managed to achieve successes). The DTAG3P system dramatically out-performed the others. Figure 3 shows the median best fitness plotted against the number of node evaluations (AND,OR,XOR,NOT 4 E XPERIMENTAL WORK operators) for GP, TAG and DTAG3P+ on 8, 10 and 12 parity problems. Table 2 shows the 4.1 Experimental settings relative success rates at a fixed number of node The first experiments used a family of symbolic evaluations, 1.25 ∗ 108 , confirming the excellent regression problems due to Koza [1]: Fn (X) = performance of a developmental evaluation apX +X 2 +X 3 +...+X n , for n = 1, . . . , 9. We com- proach. To illustrate the scalability, figure 4 pared traditional GP, TAG-based GP (TAG3P), shows its cumulative probability of success of DEVTAG (a developmental system with a triv- DTAG3P+ vs number of generations for 6, 8, ial developmental process), DTAG3P (without 10, and 12 odd-parity problems. adaptive variation rates), and DTAG3PF9ALL (DTAG3P with evaluation only on F9 ). Details 4.3 Discussion of the experiments are given in [3], [18]. The second set used the family of odd-k- Even with a trivial developmental process (corparity problems k = 2 . . . 12. We compared the responding to the undifferentiated developadaptive variation system DTAG3P+ with the ment of simple multi-cellular organisms such original TAG3P and GP systems over parity as sponges), the DEVTAG system significantly problems of sizes 8, 10 and 12. The experiments outperforms the original TAG3P and GP sysare described in fulll detail in in [18]. tems. With a flexible developmental method


5

parity problems. In fact, although the results are not detailed here, the original DTAG3P performed relatively poorly; it was only when we incorporated adaptive variation rates, allowing the genes corresponding to early stages of development to become progressively fixed, that we saw substantial improvements with developmental methods. We believe this arises because in DTAG3P, good building blocks for early developmental stages might be lost in Fig. 3. Median of the best fitness of GP, TAG, later stages of evolution. The adaptive variaDTAG3P+ vs. No. node evaluations tion rate in DTAG3P+ preserves good building blocks for use in later stages of the evolutionary process; it allows DTAG3P+ not only to solve the k-parity problems reliably, but also to cope well with scalaing in problem complexity. In our current version, this fixing proceeds on a pre-determined schedule; future work will investigate mechanisms which might allow this adaptive variation to emerge during evolution. So developmental evaluation leads to effective solutions of problems, but does it promote regularity? In [20], [21], we used treeFig. 4. Cumulative success of DTAG3P+ on compression methods to directly measure the family of parity problems vs. No. generations level of repetition within genotypes. The highest compression ratios (and hence the highest TABLE 2 levels of repetition) were found with DTAG3P. Percentage Success Rate on 8, 10, 12 parity Interestingly, DTAG3PF9ALL showed the same after 1.25 ∗ 108 Node evaluation (30 runs) high level of repetition as DTAG3P at the start of the evolutionary process, but it rapidly deGP TAG DTAG3P+ cayed, whereas DTAG3P (i.e. developmental 8 Parity 23% 23% 100% evaluation) was able to maintain this regular10 Parity 0% 0% 100% ity throughout the evolutionary process; this 12 Parity 0% 0% 100% effect was particularly marked when only the effective code (i.e. the non-bloat part) was measured. (DOL systems), DTAG3P finds solutions even more reliably. Building blocks used in lower levels are copied to higher levels through the 5 C ONCLUSION rule rewriting of the developmental process. In At this point, we have observed good perother analyses, we were able to confirm that formance and scalability of the developmental the developmental system solved the problems evaluation approach on two families of stanincrementally, always finding the solutions to dard benchmark problems. A third family, fitlower level problems before it solved higher ting Fourier approximations of functions, has level problems. Importantly, development on also shown similar good performance. Howits own was not enough: without developmen- ever all these problems are artificial. The expertal evaluation, DTAG3PF9ALL never solved iments are subject to the criticism, that benchthe F9 problem. mark problems are chosen for their simplicity Figure 3 and table 2 show the effect of and ease of analysis; hence good solutions developmental evaluation on solving difficult might well exhibit levels of regularity that 1

12−DTAG3P+ 12−TAG3P 12−GP

0.9

Median of the best fitness

0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

10

20

30

40

50

60

70

80

90

100

x 106

Number of node evaluations

100

6−parity 8−parity 10−parity 12−parity

90

Cumulative frequency of success

80 70 60 50 40 30 20 10

0

0

10

20

30

40

50 60 Generations

70

80

90

100


might not be typical of real-World problems. Hence we are now applying DTAG3P to a more real-World problem, solving high-dimensional Markov decision problems. We hope to report our results in the near future. So far, our compression-based repetition analysis has only been applied to one family of problems, using polynomial symbolic regression. We are in the process of extending our compression-based techniques to other problem domains, and hope to confirm our compression analyses on other domains in forthcoming work.

ACKNOWLEDGMENTS This research was partially supported by the BK21 program of the Korean Scientific and Engineering Research Foundation.

R EFERENCES J. R. Koza, Genetic Programming II: Automatic Discovery of Reusable Programs. Cambridge Massachusetts: MIT Press, may 1994. [2] L. Spector, “Simultaneous evolution of programs and their control structures,” in Advances in Genetic Programming 2, P. J. Angeline and K. E. Kinnear, Jr., Eds. Cambridge, MA, USA: MIT Press, 1996, ch. 7, pp. 137–154. [3] R. I. McKay, T. H. Hoang, D. L. Essam, and X. H. Nguyen, “Developmental evaluation in genetic programming: the preliminary results,” in Proceedings of the 9th European Conference on Genetic Programming, ser. Lecture Notes in Computer Science, P. Collet, M. Tomassini, M. Ebner, S. Gustafson, and A. Ekárt, Eds., vol. 3905. Budapest, Hungary: Springer, 10 - 12 Apr. 2006, pp. 280–289. [4] H. T. Hao, D. Essam, R. I. McKay, and X. H. Nguyen, “Developmental evaluation in genetic programming: A TAG-based framework,” in Proceedings of the Third AsianPacific workshop on Genetic Programming, T. L. Pham, H. K. Le, and X. H. Nguyen, Eds., Military Technical Academy, Hanoi, VietNam, 2006, pp. 86–97. [5] J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection. Cambridge, MA, USA: MIT Press, 1992. [6] H. Gee, “Unexpected bits and pieces, special report: the ethics of genetics,” feb 2001. [7] G. Schlosser and G. P. Wagner, Eds., Modularity in Development and Evolution. The University of Chicago Press, 2004. [8] H. Moghadam, R. Danzmann, and M. Ferguson, “Organization of hox clusters in rainbow trout (oncorhynchus mykiss): a tetraploid model species,” Journal of Molecular Evolution, vol. 61, no. 6, pp. 804–818, Dec 2005. [9] S. Ohno, Ed., Evolution by Gene Duplication. SpringerVerlag, 1970. [10] J. Zhang, “Evolution by gene duplication: an update,” Trends in Ecology and Evolution, vol. 18, pp. 292–298, Jun 2003. [1]

6

[11] J. P. Rosca and D. H. Ballard, “Hierarchical selforganization in genetic programming,” in Proceedings of the Eleventh International Conference on Machine Learning. Morgan Kaufmann, 1994. [12] C. Jacob, “Genetic L-system programming,” in Parallel Problem Solving from Nature III, ser. LNCS, Y. Davidor, H.P. Schwefel, and R. Männer, Eds., vol. 866. Jerusalem: Springer-Verlag, 9-14 Oct. 1994, pp. 334–343. [13] P. Haddow, T. G., and van Remortel P., “Shrinking the genotype: L-systems for evolvable hardware,” in Evolvable Systems: From Biology to Hardware, 4th International Conference, ICES 2001, ser. Lecture Notes in Computer Science, Y. Liu, K. Tanaka, M. Iwata, T. Higuchi, and M. Yasunaga, Eds., vol. 2210. Springer-Verlag, 2001, pp. 128–139. [14] J. F. Miller and P. Thomson, “A developmental method for growing graphs and circuits,” in Evolvable Systems: From Biology to Hardware, Fifth International Conference, ICES 2003, ser. LNCS, A. M. Tyrrell, P. C. Haddow, and J. Torresen, Eds., vol. 2606. Trondheim, Norway: Springer-Verlag, 17-20 Mar. 2003, pp. 93–104. [15] F. Gruau and D. Whitley, “Adding learning to the cellular development process: a comparative study,” Evolutionary Computation, vol. 1, no. 3, pp. 213–233, 1993. [16] L. Spector and K. Stoffel, “Ontogenetic programming,” in Genetic Programming 1996: Proceedings of the First Annual Conference, J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, Eds. Stanford University, CA, USA: MIT Press, 28–31 Jul. 1996, pp. 394–399. [17] N. X. Hoai, R. I. B. McKay, and D. Essam, “Representation and structural difficulty in genetic programming,” IEEE Transactions on Evolutionary Computation, vol. 10, no. 2, pp. 157–166, Apr. 2006. [18] T.-H. Hoang, D. Essam, R. McKay, and H. X.N, “Building on success in genetic programming: Adaptive variation and developmental evaluation,” in The Second International Symposium on Intelligence Computation and Applications, ser. Lecture Notes in Computer Science, L. Kang, Y. Liu, and S. Zeng, Eds., vol. 4683. Springer-Verlag, 2007, pp. 137–146. [19] T.-H. Hoang, D. Essam, R. I. B. McKay, and X. H. Nguyen, “Solving symbolic regression problems using incremental evaluation in genetic programming,” in Proceedings of the 2006 IEEE Congress on Evolutionary Computation. Vancouver: IEEE Press, 6-21 Jul. 2006, pp. 7487–7494. [20] J. Shin, M. Kang, B. McKay, X. Nguyen, T.-H. Hoang, N. Mori, and D. Essam, “Analysing the regularity of genomes using compression and expression simplification,” in Proceedings of the 10th European Conference on Genetic Programming, ser. Lecture Notes in Computer Science, M. Ebner, M. O’Neill, A. Ekárt, L. Vanneschi, and A. I. Esparcia-Alcázar, Eds., vol. 4445. Valencia, Spain: Springer, 11 - 13 Apr. 2007, pp. 251–260. [21] R. I. McKay, J. Shin, T.-H. Hoang, and H. X. Nguyen, “Using compression to understand the distribution of building blocks in genetic programming populations,” in The Proceedings of IEEE Congress on Evolutionary Computation. IEEE Press,(to appear), 2007.