Digital Circuit Evolution: The Ruggedness and Neutrality of Two-Bit Multiplier Landscapes Vesselin K. Vassilev, Julian F. Miller, Terence C. Fogarty School of Computing, Napier University, 219 Colinton Road, Edinburgh, EH14 1DJ, UK
[email protected] Abstract The two-bit multiplier is a simple electronic circuit, small enough to be feasible for evolutionary design, and practically useful as a fundamental building block used in the synthesis of many digital systems. To attain understanding of the evolvability of this digital circuit, we consider its evolutionary design as a search on a fitness landscape. We study the structure of two-bit multiplier landscapes in terms of their ruggedness and neutrality. The motivation behind this research is to attain better understanding of how these characteristics are related to the feasibility of evolving digital circuits.
1 Introduction Evolving a fully functional two-bit multiplier in a configuration of logic cells is a simple case of digital circuit evolution, recently studied in the nascent field of evolvable hardware (Miller et al., 1997; Sipper et al., 1997; Thompson, 1998). Digital circuits have been evolved by Miller et al. (1997, 1998b, 1998a), who suggested that the design of electronic circuits on a gate array in general can be implemented in terms of the functionality and routing of the array. These refer to the logic functions utilised in the array's cells, and the connections between the inputs, cells and outputs, respectively, which are then encoded in a genotype. The genotype representation, together with the fitness function and the evolutionary operators, defines fitness landscapes whose structure strongly relates to the effectiveness of the evolutionary search (Kauffman, 1989; Manderick et al., 1991), and hence, the feasibility of evolving digital circuits. We study the evolutionary synthesis of a two-bit multiplier in terms of the structure of its fitness landscapes. The motivation behind this research is to attain better understanding of how these characteristics are related to the feasibility of evolving digital circuits. In (Vassilev et al. 1999a, 1999b) we introduced a model for studying the structure of circuit evolution landscapes, based on the idea that a landscape might be decomposed to suitable for the statistics subspaces (Hordijk, 1997). Further, we investigated the ruggedness of these landscapes in general, by measuring their autocorrelations and the corresponding amplitude spectra derived from the landscapes' Fourier transformations. Here, we employ the model in order to study the structure of circuit evolution landscapes in terms of their ruggedness and neutrality. The ruggedness is related to the fitness differences between neighbouring points whereas the neutrality refers to the flat landscape areas. To attain understanding of how the circuit evolution relates to these landscape characteristics we concentrate on landscapes associated with a particular digital circuit – a two-bit multiplier – which is evolved by a genetic algorithm. To explore the interplay of the landscape ruggedness and neutrality we employ an information measure of landscapes, introduced by Vassilev (1997). We show that the two-bit multiplier landscapes are characterised by vast and sharply differentiated landscape plateaus. The paper is organised as follows. The next section introduces the concept of landscapes and their information characteristics. Section 3 defines the two-bit mutliplier landscapes. The results of the study are given in section 4. Finally, a brief discussion is proposed.
2 Fitness Landscapes The notion of a fitness landscape, introduced by Wright (1932), has become an important concept in evolutionary computation. The metaphor is taken from biology and it expresses the idea that evolution can be considered as a population flow on a surface in which the altitude of a point quantifies how well the corresponding organism is adapted to an environment. In evolutionary computation the fitness landscapes are simply search spaces defined over elements called phenotypes which are represented by their genotypes (sequences of elements taken from a finite alphabet). A fitness value is assigned to each genotype and the evolutionary algorithm refers to these values when deciding which phenotypes are to survive and reproduce. The fitness value of a genotype is evaluated by a fitness function, f , which measures how good the encoded phenotype is. The population of genotypes flow on a landscape guided by an evolutionary operator. In other words, the connections between the genotypes are determined by an operator that is employed to search in the landscape. Following Stadler (1995b), Stadler and Wagner (1997), we consider that a fitness landscape, L, is defined on a graph, Gf = (V; E ), whose vertices are genotypes labeled with fitness values, and the connections are defined by the evolutionary operator which agrees with the concept “one operator, one landscape” (Jones, 1995). The genotype representation, the neighbourhood relation, and the fitness function define the structure of a fitness landscape. The structure can be specified in terms of the ruggedness and the neutrality of the landscapes. According to Stadler (1995a), Reidys and Stadler (1998) the ruggedness and neutrality of a landscape refer to the fitness differences between neighbouring points, and the flat landscape areas, respectively.
To investigate the ruggedness and neutrality of a fitness landscape we measure its information content and stability, estimated on time series that is obtained by sampling the values fi along a random walk on the landscape (Vassilev, 1997). Consider time series fft gnt=0 , real numbers from an interval, I , obtained by a random walk on a landscape, L. To estimate the information content of fft gnt=0 , we operate on a string, S (") = s1 s2 s3 :::s n , whose elements, si , accept values from the set f1; 0; 1g. The 1; if fi ? fi?1 < ?" elements of the string are si = ft (i; "), where ft (i; ") = 0; if jfi ? fi?1 j " for any fixed ". The parameter " is a real 1; if fi ? fi?1 > " number from the interval [0; lI ], where lI is the length of the interval I . The string S (") contains information about the structure of the landscape (Stadler, 1995b). Note that the function ft associates each edge of the path, that unites the fitness values fftgnt=0 , with an element from the set f1; 0; 1g. Consequently, each vertex of the path is presentedPby a string, si si+1 , which is a sub-block of S ("). We define the information content of the time series for any " as H (") = ? p6=q P[pq] log6 P[pq] ; where P[pq] are the probabilities of the possible blocks pq of elements from set f1; 0; 1g. The accuracy of the estimation is tuned by the parameter ". We use this parameter in order to assess the information function H ("), and to estimate the information stability of the landscape. For us the information stability, "? , is the smallest value of " for which S ("? ) is a string of 0s. For further details of the analysis we refer to (Vassilev and Fogarty, 1998).
3 The Two-Bit Multiplier Landscapes We consider a genotype representation of an idealised field-programmable gate array (n rows, m columns, nI inputs, nO outputs) based on that proposed in (Miller and Thomson, 1998a). The genotype is a composition of three chromosomes (partitions) which are responsible for first, the gates functionality, second, the array internal connectivity, and third, the array outputs. The three chromosomes have different lengths and they are defined over two completely different alphabets. The “gate functionality” chromosomes are strings over alphabet with length the number of gates. The “internal connectivity” and “array outputs” chromosomes are defined over alphabet , and they are strings with length the number of gates and the number of array outputs, respectively. The alphabet is a set of integers which represents the allowed logic functions. Therefore, the alphabet size, l , is the number of logic functions used in the circuit design. The alphabet is related to the size of the neighbourhood of the cells and array outputs, which is dependent upon a levels-back L. The alphabet is a set of integers – reference numbers n nL;parameter, if L m of the cell's neighbours. Hence, the size of is l = nm + nI ; otherwise : Since each genotype consists of three chromosomes we assume that the original landscape for a given genetic operator is a 3nm superposition of three configuration spaces defined over alphabets and . Consider the Hamming hypercubes Glnm , Gl , and Gln O , which represent the configuration spaces of the chromosomes responsible for functionality, connectivity, and output connections, respectively. If is a genetic operator, the digital circuit evolution landscapes are
nm nO
nm+nO
nm 3nm
fGfi ; fi ; gli =0 ?1 ; fGgi ; gi ; gil =0 l ?1 ; fGhi ; hi ; gil =0 l ?1 : (1) nO 3nm The graphs Gfi , Ggi , and Ghi are obtained by assigning each vertex from Glnm , Gl , and Gl , respectively, with a fitness value. The fitness values are provided by fitness functions ffi g8i , fgi g8i and fhi g8i which are defined over the corresponding 3
configuration spaces (Vassilev et al., 1999a). Cell Allele(Gene)
4 3(0) 0(1) 3(2) 15(3)
5 0(4) 3(5) 1(6) 6(7)
6 2(8) 0(9) 2(10) 6(11)
7 2(12) 4(13) 1(14) 9(15)
8 1(16) 2(17) 2(18) 6(19)
9 5(20) 6(21) 5(22) 15(23)
10 4(24) 1(25) 7(26) 14(27)
11 8(28) 4(29) 2(30) 7(31)
12 3(32) 1(33) 7(34) 6(35)
13 4(36) 8(37) 8(38) 11(39)
14 6(40) 11(41) 12(42) 7(43)
15 0(44) 12(45) 9(46) 15(47)
Outputs
11(48) 14(49) 13(50) 12(51)
Table 1: The genotype of the two-bit multiplier evolved on 3 4 gate array. The row “Cell” gives the label of the cell whose genetic information is listed below at rows “Allele(Gene)”. Input 1
Input 2
Output 1
Output 2
Input 3 Output 3
Input 4
Output 4
Figure 1: The schematic of the two-bit multiplier, obtained by artificial evolution. The landscapes, studied in the paper, originate from a two-bit multiplier evolved on 34 array of logic cells. The levels-back
parameter is set 4, and the allowed two-input logic functions are 10 (Vassilev et al., 1999a). The genotype is shown in Table 1 and the corresponding phenotype is depicted in Figure 1. The most significant bits are inputs 1 and 3, and output 1.
4 Results The information characteristics of the two-bit multiplier landscapes for onepoint mutation and uniform crossover are estimated on time series obtained by random walks1 of 100; 000 steps, performed with respect to the genotype partitions. 1
1 (1) (2) (3)
0.8
0.8
0.6
0.6
H(epsilon)
H(epsilon)
(1) (2) (3)
0.4
0.2
0.4
0.2
0
0 0
0.1
0.2
0.3
0.4
0.5 0.6 epsilon
(a)
0.7
0.8
0.9
1
0
0.1
0.2
0.3
0.4
0.5 0.6 epsilon
0.7
0.8
0.9
1
(b)
Figure 2: The information functions H (") of (1) functionality, (2) internal connectivity, and (3) output connectivity subspaces of the two-bit multiplier landscape, obtained by (a) onepoint mutation and (b) uniform crossover operators. Figure 2a represents the information functions H (") of the (1) functionality, (2) internal connectivity, and (3) output connectivity subspaces of the onepoint mutation landscape. The subspaces are characterised with flat landscape areas, since the information content H (0) of each subspace is significantly higher than log6 2. The plots of H (") also imply to subspaces with significantly different profiles. For instance, the information functions of the functionality and internal connectivity subspaces decrease as " increases, while H (") of the output connectivity subspace increases as " increases from 0 to approximately 0:047. Consequently, the plateau forms in the functionality and internal connectivity subspaces prevail in the corresponding ensembles of information objects, which is not valid for the output connectivity subspace. Figure 2b represents the information functions H (") of the (1) functionality, (2) internal connectivity, and (3) output connectivity subspaces of the uniform crossover landscape. Again, the information content H (0) of each subspace is significantly higher than log6 2 which implies to existence of plateaus on the landscape. The depicted information functions, H ("), reveal that these subspaces have similar profiles. The subspaces are characterised with multi-peaks configurations which prevail over the flat landscape areas. Consequently, the uniform crossover landscape has higher modality than the onepoint mutation one, which implies a landscape with higher ruggedness. It agrees with Miller et al. (1997) who suggested that the uniform crossover operator might be dropped in order to improve the search. The plots in Figure 2 suggest a certain similarity between the mutation and crossover subspaces. Since the plots of H (") represent step functions we surmise that the landscapes consist of a small number of altitude levels which together with the flat landscape areas imply vast and sharply differentiated landscape plateaus. It explains why a genetic algorithm with simple elitism and population size 2 performs a successful search, locating a genotype with fitness values 1:0 (Miller, 1999). It is obvious that such type of landscapes are hardly searchable by an adaptive population. That is why, Miller (1999) has suggested that probabilistic hillclimbing algorithms might be much more effective for such type of problems.
5 Conclusion In this paper we studied the subspaces of onepoint mutation and uniform crossover landscapes, defined over a particular digital circuit. The circuit is a two-bit multiplier and it is evolved by a genetic algorithm. We showed that the two-bit multiplier fitness landscapes are characterised with vast neutral areas, hardly searchable by an adaptive population. It has been also shown that the functionality and internal connectivity subspaces of the onepoint mutation landscapes differs significantly from the other in terms of the landscape neutrality. This in turn poses the following question: is it efficient to implement digital circuit evolution applying the same evolutionary technique to all landscape subspaces? 1 A random walk on a mutation landscape is implemented as follows: start from a genotype (in our case, it is the evolved two-bit multiplier) generate all
neighbours of the current point by mutation and evaluate their fitness values, choose randomly one neighbour and record its fitness, generate all neighbours of the new point, which becomes “current”, and so on (Weinberger, 1990). A random walk on a crossover landscape is implemented by an algorithm which can be found in (Wagner and Stadler, 1998) (see also (Stadler and Wagner, 1997)). In short, it can be described as follows: start with a pair of genotypes, generate a set of offsprings by applying the crossover operator and evaluate their fitness values, from those choose randomly one (record its fitness) and mate it with a randomly chosen genotype, etc. until the termination conditions are satisfied.
References Hordijk, W. (1997). Correlation analysis of the synchronising-ca landscape. Physica D 107, 255–264. Jones, T. (1995). Evolutionary Algorithms, Fitness Landscapes and Search. Ph.D. thesis, University of New Mexico, Albuquergue, NM. Kauffman, S. (1989). Adaptation on rugged fitness landscapes. In: Stein, D. (ed.), Lectures in the Sciences of Complexity, SFI Studies in the Sciences of Complexity, pp. 527–618. Reading, MA: Addison-Wesley. Manderick, B., de Weger, M. and Spiessens, P. (1991). The genetic algorithm and the structure of the fitness landscape. In: Belew, R. K. and Booker, L. B. (eds.), Proceedings of the 4th International Conference on Genetic Algorithms, pp. 143–150. San Mateo, CA: Morgan Kaufmann. Miller, J. F. (1999). An empirical study of the efficiency of learning boolean functions using a cartesian genetic programming approach. In: Banzhaf, W., Daida, J., Eiben, A. E., Garzon, M. H., Honavar, V., Jakiela, M. and Smith, R. E. (eds.), Proceedings of the Genetic and Evolutionary Computation Conference. San Francisco, CA: Morgan Kaufmann. To appear (available via http://www.dcs.napier.ac.uk/julian/GP-411.zip). Miller, J. F. and Thomson, P. (1998a). Aspects of digital evolution: Evolvability and architecture. In: Eiben, A. E., B¨ack, T., Schoenauer, M. and Schwefel, H.-P. (eds.), Parallel Problem Solving from Nature V, pp. 927–936. Berlin: Springer. Miller, J. F. and Thomson, P. (1998b). Aspects of digital evolution: Geometry and learning. In: Sipper, M., Mange, D. and P´erez-Uribe, A. (eds.), Proceedings of the 2nd International Conference on Evolvable Systems: From Biology to Hardware, pp. 25–35. Heidelberg: Springer-Verlag. Miller, J. F., Thomson, P. and Fogarty, T. (1997). Designing electronic circuits using evolutionary algorithms. arithmetic circuits: A case study. In: Quagliarella, D., Periaux, J., Poloni, C. and Winter, G. (eds.), Genetic Algorithms and Evolution Strategies in Engineering and Computer Science, pp. 105–131. Chechester, UK: Wiley. Reidys, C. M. and Stadler, P. F. (1998). Neutrality in Fitness Landscapes. Tech. Rep. 98-10-089, Santa Fe Institute. Sipper, M., Sanchez, E., Mange, D., Tomassini, M., P´erez-Uribe, A. and Stauffer, A. (1997). A phylogenetic, ontogenetic, and epigenetic view of bio-inspired hardware systems. IEEE Transactions on Evolutionary Computation 1 (1), 83–97. Stadler, P. F. (1995a). Landscapes and Their Correlation Functions. Tech. Rep. 95-07-067, Santa Fe Institute. Stadler, P. F. (1995b). Towards theory of landscapes. In: Lop´ez-Pena, R., Capovilla, R., Garc´ia-Pelayo, R., Waelbroeck, H. and Zertuche, F. (eds.), Complex Systems and Binary Networks, pp. 77–163. Berlin: Springer-Verlag. Stadler, P. F. and Wagner, G. P. (1997). Algebraic theory of recombination spaces. Evolutionary Computation 5 (3), 241–275. Thompson, A. (1998). Hardware Evolution: Automatic Design of Electronic Circuits in Reconfigurable Hardware by Artificial Evolution. London: Springer-Verlag. Vassilev, V. K. (1997). An information measure of landscapes. In: B¨ack, T. (ed.), Proceedings of the 7th International Conference on Genetic Algorithms, pp. 49–56. San Francisco, CA: Morgan Kaufmann. Vassilev, V. K. and Fogarty, T. C. (1998). Information Characteristics and the Structure of Landscapes. Tech. Rep., Napier University. Available upon request. Vassilev, V. K., Miller, J. F. and Fogarty, T. C. (1999a). Digital circuit evolution and fitness landscapes. In: Proceedings of the Congress on Evolutionary Computation. Piscataway, NJ: IEEE Press. To appear (available via http://www.dcs.napier.ac.uk/vesselin/papers/cec99.ps.gz). Vassilev, V. K., Miller, J. F. and Fogarty, T. C. (1999b). On the nature of two-bit multiplier landscapes. In: Stoica, A., Keymeulen, D. and Lohn, J. (eds.), Proceedings of the 1st NASA/DoD Workshop on Evolvable Hardware. Piscataway, NJ: IEEE Computer Society Press. To appear (available via http://www.dcs.napier.ac.uk/vesselin/papers/eh99-2.ps.gz). Wagner, G. P. and Stadler, P. F. (1998). Complex adaptations and the structure of recombination spaces. Tech. Rep. 97-03-029, Santa Fe Institute. Weinberger, E. D. (1990). Correlated and uncorrelated fitness landscapes and how to tell the difference. Biological Cybernetics 63, 325–336. Wright, S. (1932). The roles of mutation, inbreeding, crossbreeding and selection in evolution. In: Jones, D. F. (ed.), Proceedings of the 6th International Conference on Genetics, vol. 1, pp. 356–366.
