Code Size and Depth Flows in Genetic Programming Terence Soule
James A. Foster
Laboratory for Applied Logic Laboratory for Applied Logic Computer Science Dept. Computer Science Dept. University of Idaho University of Idaho Moscow, Idaho 83844-1010 Moscow, Idaho 83844-1010
[email protected] [email protected]
Abstract
This paper examines the process of genetic programming using a search space which measures not only program tness, but also program size and depth. We then explore two prominent
ows through this space. We nd support for a Replication Accuracy Force [McPhee and Miller, 1995] which produces a ow towards larger programs to protect against destructive crossover events. There is a related ow towards deeper programs. This ow is relatively stronger than that towards larger programs, resulting in progressively less dense programs as long as growth continues.
1 Introduction In genetic programming (GP) a population of programs can be viewed as a distribution of points representing particular solutions in the overall solution space. GP uses crossover to produce new solutions in the neighborhood of existing solutions and selection to preferentially keep those programs which have moved into regions of higher tness (or, as is more commonly the case, keeps those programs which have not moved into regions of lower tness). The combined action of these two operations is meant to create a general ow towards regions of higher tness and more successful programs. Program tness is the most important metric in GP so most research has focused on improving the movement of programs through the solution space towards more This work supported by funding from the URO Seed Grant program.
t solution. However, other program features, such as size and depth, can also be signi cant. Thus, it is reasonable to expand the typical solution space to include these other program metrics and to examine the ows in this expanded space. The most obvious such ow is the movement of programs towards regions representing larger programs, corresponding to code growth, or bloating. This presents a problem in the use of GP because populations with excessive growth use many more resources than other populations. A better understanding of the causes and rate of this ow would be helpful in reducing it. Also of importance is the depth of the programs as compared to their size. The size-to-depth ratio of a program determines the fullness of its syntax tree. For large, full program trees most crossover operations occur near the branch tips [Rosca and Ballard, 1995]. Such crossovers result in relatively small structural changes to the programs and, depending upon the current state of the population, may or may not be preferred. However, large, full trees will occur only if the
ow towards larger trees and the ow towards deeper trees are balanced. In addition, the shape of a program tree partially determines the nature of the solution. A short, full tree will represent a very di erent program structure than a tall, sparse tree. Thus, the ow towards or away from trees of a particular depth/size ratio can have an impact on both the evolutionary process and the resulting solution. In this paper we begin to empirically map some of the ows through a space including tness, total code size, operative code size, and program depth. Our rst experiment used a crossover operation in which each o spring is kept only if its tness surpasses that of its parent. This approach was rst suggested by Altenberg as means of improving the rate of evolution, improving the ow towards more t programs [Altenberg, 1994]. However, here it is applied to test the theory that code growth occurs to protect against destructive crossovers. With only constructive
Objective Terminal set Function set Predicates Fitness Selection Restrictions Population size Initial population Parameters Termination
To navigate a simple maze forward, back, left, right, no op if then else, while, prog2 wall ahead, no wall ahead distance traveled from left most wall (0 to 18) stochastic remainder programs were only allowed 3000 operations 500 random 66.6% crossover, no mutation, results are the average of fty trials 75 or 125 generations Table 1: GP Parameters
crossovers preserved code growth is reduced to only the growth necessary for improving program tness. We also measure the ow towards deeper programs. This appears to be relatively stronger than that towards larger programs, leading to progressively deeper and sparser trees as long as code growth continues.
2 Code Size and Code Depth In roughly equivalent theories, Nordin and Banzhaf [Nordin and Banzhaf, 1995] and McPhee and Miller [McPhee and Miller, 1995] have argued that code growth occurs to protect programs against the potentially destructive e ects of operations other than selection. Nordin and Banzhaf focus on the crossover operation, showing that its e ects are usually destructive and, thus, that programs with more inoperative code (code which doesn't contribute to program tness on the training set) and less operative code (code which does contribute to tness on the training set) are more likely to survive crossover unharmed. They predicted exponential growth in total code size. In addition, the selective pressure in favor of large total code size and small operative code size is directly related to the probability that crossover will be destructive. For example, in a less homogeneous population crossover is slightly more likely to be destructive which is predicted to cause a slightly higher rate of growth. McPhee and Miller argue more generally that evolution favors programs which replicate with semantic accuracy, in other words that there is a Replication Accuracy Force (RAF) acting on the population. Although this is a general force, in standard GP the only cause of replication inaccuracies is crossover. Thus, the programs evolve such that they avoid having their semantics changed during crossover, which entails maximizing inoperative code while minimizing operative code. Again the rate of code growth will be related to the probability of the code being damaged. These theories imply the existence of two separate
ows through program space, one towards programs with more inoperative code and one towards programs with less operative code. Thus, although there is overall code growth the actual operative code becomes more parsimonious. The relative velocity of these ows should be related to the probability that the code is damaged. Rosca and Ballard show that for full trees the average crossover point occurs near the branch tips [Rosca and Ballard, 1995]. However, in the case of minimal trees, trees with the minimal number of nodes for their depth, the depth of the average crossover branch is P Nd d (1) N where Nd is the number of nodes leading to a crossover branch of depth d and N is the total number of nodes. For a minimal, binary tree of depth D the total number of nodes is 2D +1. Also there are D +1 leaf nodes which all lead to crossover branches of depth 0 and there is one internal node for each of the other potential branch depths. So Equation 2 becomes
P
(D + 1) 0 + Dd=1 1 d (2) 2D + 1 which reduces to D D2 + D D?! !1 (3) 4D + 2 4 In the limit of large depth, crossover branches are roughly 1=4 as deep as the parent tree. Therefore, depending upon the original structure of the parent tree it can be expected to lose a small, constant sized branch (in the case of full trees) or a relatively large fraction of the entire tree (in the case of minimal trees). Because the the size of the crossover branch is strongly correlated with the amount of structural change in the programs, whether the programs tend to ow towards full trees or sparse trees is signi cant to the overall evolutionary process.
18 16 14
Average Fitness
12 10
Control No Destructive Crossovers
8 6 4 2 0 0
20
40
60 80 Generation
100
120
140
Figure 1: Average tness showing the e ects of destructive crossover. The relative fullness of a program tree can be measured as its density, de ned as its number of nodes divided by the maximum number of nodes possible in a tree with its depth. A tree with branching factor B , N . A depth D, and N nodes has a density of BD+1 ?1 full tree is a tree with a density of 1. The e ects of crossover are partially determined by the sizes of the crossed branches which depends upon the parent trees' densities. The density changes during evolution depend upon the ows of the points in the size and depth dimensions of the expanded solution space.
3 Experiment We ran several experiments to test and quantify these predicted ows empirically. Our test problem was a robot guidance problem. We used GP to develop a program for navigating a simple maze. The basic parameters are supplied in Table 1, further details have been provided previously [Soule et al., 1996]. Although this is not a particularly dicult problem for GP (a fully successful program evolves roughly sev-
enty percent of the time) it does produce the typical exploration-exploitation pattern of a GP search and so works well for testing general hypothesis regarding GP behavior. The one unusual function is the while function which sequences two instructions (as does the prog2 function) as well as testing its predicate. Thus, the function set produces strictly binary trees. The most signi cant change from the standard GP paradigm was that crossover points were chosen uniformly from all possible points, without the typical preference for internal nodes. Although this probably degrades the GP's performance slightly, we are not strictly interested in maximizing performance and such a selection presents one less confounding factor in the selection of the crossover point. No limits were placed on the depth of the programs.
4 Results The experiments were designed with two goals in mind, to test the theories that destructive crossover events are the motivating force behind code growth and to
Crossover Type
Average Fitness Normal Crossover 16.1 No Destructive Crossovers 15.3
Code Size 592.7 87.4
Operative Code Size 31.0 46.5
Fraction of code which is operative 0.05 0.53
Code Depth 57.8 19.7
Table 2: Results after 75 generations with and without destructive crossover events 600
Average Code Size (# of Nodes)
500
400
Control No Destructive Crossovers
300
200
100
0 0
20
40
60 80 Generation
100
120
140
Figure 2: Code growth over time, showing the e ects of destructive crossover. compare changes in code size to changes in code depth. 4.1
Code Growth
The rst experiment was designed to test the causes of code growth. A modi ed crossover operation was used to eliminate the destructive crossover operations. After each crossover operation we compared the tness of the o spring to the tness of their respective parents. An o spring is incorporated into the new population only if its tness exceeds that of its parent, otherwise the parent is kept. This is a similar operation to the crossover hill climbing used by O'Reilly and Oppacher [O'Reilly and Oppacher, 1995]. However, in this case only one attempt is made to produce more
successful o spring and if this attempt fails the parent is kept instead of being replaced by a randomly created individual, as was done in O'Reilly and Oppacher's modi ed crossover operation. The average tness with and without destructive crossovers is shown in Figure 1. For comparisons, the exact values at generation 75, along with the values for operative code size and code depth, are summarized in Table 2. There are two obvious features of this tness data. First, the average tness rises much more rapidly in the absence of destructive crossovers. This di erence is signi cant at each generation from generation 2 to generation 34 (student t test p < 0.01). Two factors contribute to this e ect. Without destructive crossovers more individuals remain much closer to the optimal,
55 50
Operative Code Size (# of Nodes)
45 40 35 30 25 Control No Destructive Crossovers
20 15 10 5 0 0
20
40
60 80 Generation
100
120
140
Figure 3: Operative code growth over time, showing the e ects of destructive crossover. thus, raising the average tness directly. In contrast it takes much longer for the population to become homogeneous in the presence of destructive crossover events and the average tness remains lower. In addition, with more near optimal individuals the probability of a bene cial crossover increases and so improvements occur sooner. Thus, the maximum tness without destructive crossovers also rises more rapidly. The second noteworthy feature of these data is the eventual superiority of the trial with destructive crossovers. However, in this case there is not a signi cant di erence between the values. At most the data show that normal crossover produces results statistically indistinguishable from those produced without destructive crossover. This is similar to the results of other researchers who have shown that eliminating destructive crossovers can be bene cial [O'Reilly and Oppacher, 1995]. Figure 2 shows the average code size for these trials. Without destructive crossover growth is much more limited. This di erent is signi cant for all generations beyond 16 (student t test p < 0.01). Additionally, com-
paring Figure 2 to Figure 1 shows that without destructive crossovers the rate of code growth is more closely tied to the rate of tness improvement. Even in the nal generations, Figure 2 shows a slight ongoing increase in the average tnesses which explains the growth seen in the trial without destructive crossovers. In a separate trial (data not presented) the average tness of all individuals eventually converged to the same tness as the best individual and all growth in both tness and size halted. Thus, in the absence of destructive crossover, code growth appears to be limited to what occurs while improving tness. Figure 3 shows the change in operative code size for the same trials. Any instructions which were executed were considered operative. This is not a perfect measure as it may have included some combinations of code that had no net e ect (such as a left followed by a right). However, a comparison of Figures 2 and 3 shows that a signi cant fraction of the code is classi ed as inoperative and, thus, we believe that this is a reasonable measure of inoperative code for this problem.
0.7 Control. No Destructive Crossovers
Operative Code Size/Total Code Size
0.6
0.5
0.4
0.3
0.2
0.1
0 0
20
40
60 80 Program Generations
100
120
140
Figure 4: Operative code as a fraction of the total code, showing the e ects of destructive crossovers. A comparison of the control trial and the trial without destructive crossovers agrees with the theoretical predictions: without destructive crossovers the operative code grew more rapidly than when destructive crossovers were allowed to occur, even though the overall growth was much smaller without the destructive crossovers. However, because the operative code was subject to wide variations our test did not show significance for this di erence. Figure 4 shows the ratio of operative code to total code for the two test cases. Here the di erence in total code size is taken into account and the suppression of operative code in the presence of destructive crossovers is quite dramatic. In this case the di erence is signi cant for all generations beyond 4 (student t test p< 0.01). Thus, destructive crossover events produce a general movement of the programs towards regions representing larger overall code size, but relatively less operative code. In these experiments the code growth was directly related to the occurrence of destructive crossover events, which supports both the work of Nordin and Banzhaf
and the more general RAF theory proposed by McPhee and Miller. It would be interesting to determine if other types of primarily destructive operators also cause code growth. This would imply that a general RAF applies to genetic programming. For example, a mutation operator which was primarily destructive in nature could have the same e ects on both operative and inoperative code sizes. So far experiments to test this theory have been inconclusive, possibly because our mutation operator was not suciently destructive to produce signi cant changes in code size. 4.2
Code Size vs. Code Depth
Next we compared the overall code growth to the growth in program depth. These two measurements determine the overall shape of the program trees. Figure 5 shows the ow of the programs through the code size and program depth subspace of the full program space. The movement over successive generations is up and to the right. The straight line, labeled \full trees" is a reference line showing where full trees would fall in this subspace. Similarly the lowest curve, labeled \sparse
1024
Program Size (# of nodes)
512
256
128
64
Full Trees Sparse Trees Control. No Destructive Crossover
32
16
8 0
10
20
30 Program Depth
40
50
60
Figure 5: This gure shows control and no destructive crossover trials plotted in the code size (in log2 ) and the tree depth subspace of the full program space. The straight line shows where full trees would fall in this graph. The lowest curved line shows where sparse trees would fall. The data points progress up and to the right showing that the programs are becoming larger and less dense as they evolve. trees", shows where minimally sized trees would fall in the subspace. The initial, random programs are relatively close to full trees, but as evolution proceeds they curve away from the full trees, becoming progressively less dense. Not surprisingly these results show that the increase in program depth is directly related to the increase in program size. As the overall growth slows so does the growth in program depth. What is surprising is that the trees are becoming progressively less dense thus the increase in depth must be occurring relatively faster than the increase in size. Preliminary results with the even parity problem have shown similar ows through the depth-size subspace, suggesting that this result is not a feature of this particular problem. It is unclear why this trend towards sparser trees occurs. One possible explanation is that there is a xed contour in the program space which represents an equilibrium between size and depth and the programs are
tending towards this line. Alternatively it may simply be the nature of the crossover selection process to increase the depth of trees relatively faster than their size. Of course, the common practice of limiting code growth by setting a xed limit on code depth will almost completely obscure the ow towards sparser programs. In our trials the average program reached a depth of seventeen (a common cut-o value) by generation twenty-two. This implies that with a depth limit the density of the programs trees should begin to increase somewhat before this point, which would produce an upswing in the data shown in Figure 5. Thus, using a depth limit creates denser trees specifically in the later generations. This is potentially bene cial because it means the structural changes caused by crossover will also be smaller in the later generations, exactly when the programs are approaching a solution and are likely only to need small changes.
5 Conclusions and Future Work Our results strongly support the theory that code growth occurs in response to destructive crossover events, without destructive crossover events code growth was much more limited. Destructive crossovers also create a second ow towards smaller operative code sizes. Thus, in one sense destructive crossovers lead to more parsimonious solutions, although those solutions are buried in inoperative code. In the standard GP paradigm the only operation which creates this ow is crossover. However, if a general RAF applies to GP then other primarily destructive operations can have the same e ect. This suggests the possibility of operations which cause the general motion of programs to ow in di erent directions, possibly including ow towards smaller program sizes, just as standard crossover creates a ow towards smaller operative code sizes. In addition to the ow towards larger program sizes, there is also motion through the program space towards deeper programs. This ow continues as long as code growth occurs. Overall the programs become less dense during evolution, implying that the ow towards deeper programs occurs more rapidly than the ow towards larger programs. As shown by Equation 3 less dense programs imply that relatively larger sections of code will be involved in the crossover operation, with potentially large repercussions for the evolutionary process. Clearly these results need to be tested for additional problems. Preliminary results with the even parity problem appear to support these conclusions, but tests of several more problems need to be performed. Additionally, it seems likely that an analytical approach would help to explain the reasons for the tendency towards sparse program trees which we observed. Furthermore, it would be interesting to look at the e ects of other operators. Whether other, primarily destructive, operators also lead to code growth, and whether other crossover-like operators produce similar changes in program tree density are two questions of particular relevance.
References [Altenberg, 1994] Altenberg, L. (1994). The evolution of evolvability in genetic programming. In Kenneth E. Kinnear, J., editor, Advances in Genetic Programming, pages 47 { 74. Cambridge, Mass: The MIT Press. [McPhee and Miller, 1995] McPhee, N. F. and Miller, J. D. (1995). Accurate replication in genetic programming. In Eshelman, L. J., editor, Proceedings of the Sixth International Conference on Genetic Algo-
rithms, pages 303{309. San Francisco, CA: Morgan
Kaufmann. [Nordin and Banzhaf, 1995] Nordin, P. and Banzhaf, W. (1995). Complexity compression and evolution. In Eshelman, L. J., editor, Proceedings of the Sixth International Conference on Genetic Algorithms, pages 310{317. San Francisco, CA: Morgan
Kaufmann. [O'Reilly and Oppacher, 1995] O'Reilly, U.-M. and Oppacher, F. (1995). Hybridized crossover-based search techniques for program discovery. Technical Report 95-02-007, Santa Fe Institute, Santa Fe, New Mexico. [Rosca and Ballard, 1995] Rosca, J. P. and Ballard, D. H. (1995). Causality in genetic programming. In Eshelman, L. J., editor, Proceedings of the Sixth International Conference on Genetic Algorithms, pages 256{263. San Francisco, CA: Morgan Kaufmann. [Soule et al., 1996] Soule, T., Foster, J. A., and Dickinson, J. (1996). Code growth in genetic programming. In Koza, J. R., Goldberg, D. E., Fogel, D. B., and Riolo, R. R., editors, Genetic Programming 1996, pages 215{223. MIT Press.
