Highlights on the Evolutionary Engineering Approach: the EE-Method A. Lehireche, A. Rahmoun, A.Gafour Evolutionary Engineering and Distributed Information Systems Laboratory, EEDIS Computer Science Department University Djilali Liabts of Sidi Bel-Abbts, Algeria TeVFax :(213)-48-577750 E-mail: elhir @ yahoo.com,
[email protected],
[email protected] EE is defined to be “the art of using evolutionary algorithms approach such as genetic algorithms [lo] to build complex systems”. Essentially this discipline aims to solve the problem of building complex systems without going through any design process. By imitating nature, the Evolutionary Engineering scientists describe an elementary structure of the system and then evolve this structure toward the desired system. Genetic Algorithms are used for evolving such huge systems. One may keep in mind that EE design might start from scratch, only prior knowledge and learning techniques and a powerful physical (or logical) machine support and devices are necessary to make grow a particular application. A relevant question then arises: ”how do Z proceed in EE ?” EE-Method intends to guide EE designer along the design process to achieve and implement a particular application [11,[21,[31,PI, [ 121. In section 2 and section 3, we describe some fundamental concepts of EE. In section 4,we describe the EE design methodology, in section 5 , we validate the EE-Method and in section 6, we present extracts of a leading project of “Brain Building” using EE foundations: the ATR’s CAM Brain Machine project [ 11.
Abstract Recent signijkant advances in artijicial intelligence have led to the emergence of new concepts of soji computing genetic programming, evolutionary engineering, hybrid intelligent systems, and artijicial life. EE design relies on evolutionary algorithms to construct or evolve complex systems. This paper is intended to highlight the concept of evolutionaly engineering, the motivations behind the use of EE in building complex systems, and a step by step methodology for designing such systems. Finally, we emphasize on a typical example of “Brain Building ’’ design project using EE: the CAM Brain Machine.
1. Introduction Intelligent systems have been extensively investigated in the few last years, and represent a real chalenge for software system developers for industrial applications of AI. The fusion of emerging concepts such as neural networks, fuzzy systems, genetic algorithms, expert systems, case-based reasoning and even chaos theory has proven to be a useful way to develop real-world applications, including the areas of control systems, robotics, diagnosis systems, and industrial operations. Two approaches toward soft computing are presently of interest, namely Engineefing soft computing and scientific soft computing. The soft computing research, whose purpose is to imitate and utilize the human intelligence, is now rapidly expanding its application field. On the other hand, the aim of scientific soft computing is considered to clarify the principle of intelligence, which realises the adaptation and learning ability in the dynamically changing environments. Evolutionary Engineering @E) is a discipline of soft computing engineering.
0-7695-1165-1/01 $10.000 2001 IEEE
2.Evolutionary Engineering Motivations The essential motivations behind EE is how to design hyper complex systems. Hyper complex systems [5] are defined to be all non-mathematicallyanalyzable systems. Neural networks with millions of neurons are considered as hyper complex systems. This means that humans can not design, nor build these systems. Researchers in connexionism are very aware of such difficulty in building large scale neural networks, it is not humanly tractable or even feasible by the technology of the present. The figure 1 [5] shows that all model-based systems are a special case of evolvable systems.
5
-
Analyzable systems
electronic circuits, graphs, programs etc...
Choose System Genotype : Use the Genetic Programming (GP) techniques.to encode the model. This step produces the structure of a Chromosome. A chromosome must encode all the system. The chromosome is the genotype form of the system. ( the CBM use CoDi lbit model) [3].
Figure 1. Evolvable versus Analyzable Systems
3. Dealing with hyper complexity Genetic algorithms (GA) are powerful tools of optimization. GAS are model-free and implicit parallel systems. Such characteristics make them indifferent to the internal complexity ( of the dynamics andor the structure) of the system they are evolving. In Evolutionary Engineering we are concemed by the socalled Genetic programming (GP) techniques [5]. Having to build a complex system, when there are so many variables linked in complex non linear relationships, The primary concem of genetic programmer is, "How do I get such design out of the ground?". GP use genetic algorithms to build/evolve complex system, those systems which are too complex to be predictable(mathematical1y analyzable) andor humanly tractable. GP extends the spectra of the GA to the so-called EVOLVABLE SYSTEMS (ES). High scale neural network is ES [5],[6],[7],[8].
Determine The Adaptation Function Of The System : The adaptation function is a measure that points out to the system evolution degree during the evolving phase. Generally this represents the measure of the inputoutput association conformity (i.e. the fitness or objective function in GAS).
9 . Choose System Phenotype : To be able , during the evolving phase, to evaluate the adaptation function of an Occurrence of the system; we must proceed as follows :
1. Extract the real characteristics of the occurrence of the system by decoding the chromosome. 2. Implement the system model according to its characteristics. 3. Simulate the behaviour of the system. The real characteristics of an occurrence of the system are called system phenotype.
4. Evolutionary Engineering Method (EEM) The EEM [121 aspires to explicit and simplify the EE approach. It also tries to enlarge the scope of GP. Six steps make up the EEM. Each step is a vital phase. The step order ensures the coherence of the approach but it is not unique [12].
9 Use A Genetic Algorithm As Follow : A) Generate randomly a population of chromosomes (each chromosome is a system genotype). B) For each chromosome do 1. genotype ++phenotype +-+ system implementation, 2. start up the system, inject inputs, retrieve outputs, 3. evaluate the system adaptation function. C) Select genotypes of the most adapted occurrences of the system. D) Produce a new generation by applying the crossover and the mutation operators to the selected genotypes. E) repeat B), C) and D) until the desired system is reached.
Given an COMPLEX system :
9 Ensure The Availability Of : The Inputs (data, parameters, or values), The outputs, and The association between Inputs and outputs : this means that we are able to express any output in term of input. Of the desired system. 0
9 . Choose A Model : Choose a model With which the desired system should be implemented. Examples: - Neural Networks, - Automata, - Petrinets,
5. EE-Method validation The EEM as specified above is defined as to be a method. A method is a pseudo-algorithm (algorithm-like) 6
of many different combined genets to display a quite complex behavior. LIZZY can walk, run,turn away, detect a predator and flee,detect a prey and eat and detect a mate and mate.
that describes, “ The way of doing systematically something”. In our case, the EE-Method describes the way of building systematically complex system(ES). Formally, a method is “a heuristical approach that benefits of some theoretical support &is validated by significantrealizations”.
Evolving Hardware : known as Evolutionary Hardware (EH) , deal with the fact that “if a hardware might evolve it must be highly programmable”. In [16] and [17] Nicholas J.Macias use highly reconfigurable/self-reconfigurableelementary unit called a “cell”. The cells are organized in a matrix format: cell matrix. Each cell can perform only simple logic computation. The core of a cell is of 192 bits length ((4 input X 8 outputs)*l6) . Programming a cell is equivalent to change some bits of its core. The authors use in [ 161new typed of GA called Ringed GA to evolve a cell matrix towards a desired hardware.
5.1. EE-Method backgrounds EEM uses GA[Holland,Goldberg] as a search space explorer. GA are widely substantiated by Schema Theorem [Holland,1975] and the Building Block Hypothesis[Goldberg,19891. The schema theorem predicts that: ”schemata with higher than average payoffs will be allocated more trials over time, while those with lower average payoffs will be allocated fewer trials”. The Building Block Hypothesis, especially the Dynamic Building Block Hypothesis (DBBH), is an extension of the schema theorem . DBBH states that, “Given any short, low order schema, a genetic algorithm is expected to converge to the best dynamic average schema”. Building Blocks are viewed as elementary bricks of the overall solution. the Schema Theorem and the DBBH ensure that : the fitness of a chromosome (offspring), created using the best partial solutions of previous chromosomes ( parents), is expected to be better. This is the fundamental principle of the EVOLVABILITYconcept.
6.The CBM Project As EE Example The descriptions ( not the figures) given below are extracts of the CAM Brain project overview [ 13.
6.1.Presentation of the CBM project The basic aim of the project is to build an artificial brain containing a billion artificial neurons. CBM stands for CAM-Brain Machine. It is 3D cellular automatonbased machine. The CBM is composed of programmable logic device which grows and evolves cellular automaton based neural network modules at electronic speeds, and will updates the states of an artificial brain consisting of tens of thousands of such modules in real time by updating their 3D cellular automaton (CA) at very high rate (150 billiodsecond). An artificial brain will consist of a large RAM memory space, into which individual CA modules are downloaded once they have been built.
5.2. Significant Realizations Evolving programs : Pr John Koza lead this field, known as Genetic Programming (GP). W.B Langton in [ 191 relates many relevant GP realizations: -BEAGLE: a rule finder program (commercially available and widely applied in weather forecasting, finance and forensic science) [Richard Forsyth, 19811. -Prediction of the shape of the protein(a-helis) [Simon Handly,1993],[J.Koza,19941. -Optical Character Recognition (OCR) [David Andre, 19941. -Generating Plan (Map-Making) for simulated mobile robots [Simon Handly,1994]. -Environment Monitoring Process [M..Atkin,P.R.Cohen, 19941.
6.2. CBM Architecture The CBM consists of the following five major blocks, as (Figure 2) : 0 Cellular Automata Module 0 Genotypephenotype Memory Fitness Evaluation Unit Genetic Algorithm Unit Module Interconnection Memory Each of these blocks is discussed in detail below.
Evolving Neural Networks : Pr Hugo de Garis evolved neural networks using GP techniques. Genetically programmed neural networks are Called GENETS. In [5] many applications of genets are studied, but the significant realization is in the field of artificial life. H. de Garis proved that the idea of building artificial nervous system from genets is valid. He applies the idea to an artificial creature (biot) called LIZZY. The LIZZY artificial nervous system is a genet “circuit” consisting
6.2.1.Cellular Automata Module. The cellular automata module is the hardware core of the CBM (Figure 3). It is intended to acceleratethe speed of brain evolution through a highly parallel execution of cellular state updates. The CA module consists of an array of identical hardware logic circuits or cells arranged in two identical 3D structures. The CA module is implemented with 7
Xilinx FPGA devices XC6264 (Figure 4).These devices are fully and partially reconfigurable. Logic cells are interconnected with neighbors at several hierarchical levels, providing identical propagation delay for any length of connection. This feature is very well suited for a 3D CA space configuration.To implement the CA module, a 3D block of identical logic cells is configured inside each XC6264 device, with CoDi specified 1-bit signal buses interconnectingthe cells. To assemble two identical 4096-cell modules, 32 XC6264 chips are needed in a BGA560 package
.
for the whole duration of the signaling phase. At the end of the signaling phase, a final measure of the module's fitness is instantly available (figure 3).
6.2.4.Genetic Algorithm Unit. To evolve a module, a population of 100 modules is evaluated by computing every module's fitness measure, as described above. The ten best modules are then selected for further reproduction and stored in a list ("current best ten" list). This list is an array of the current ten best module numbers. Each number is an address of the module chromosome in the chromosome memory. After each generation of modules, the ten best are mated and mutated to produce 100 offspring modules. Mating and mutation is performed by a host computer software on the chromosome memory. This process can be performed in parallel with the module evaluation, and only takes place once in a generation (Figure 2).
6.2.2.Genotype and Phenotype Memory. There are two modes of CBM operation, namely evolution mode and run mode. In the evolution mode, memory space is used to store the chromosome bit strings of the evolving population of modules (20480 bits / module genotype of 4096 cells). For each module the genotype memory also stores information concerning a maximum of 128 neurons locations and orientations inside the module. This includes, X, Y, Z coordinates (12 bits), gating code (3 bits), input functions (excitatory/inhibitory) (5 bits), giving a total of 20 bits per neuron. Thus, the total chromosome memory requirement per module is 20480+20*128 = 2880 bytes. In run mode, memory is used as phenotype memory for the evolved modules. The phenotype data describes grown axon and dendrite trees and their respective neurons for each module. For phenotype storage there are 6 bits required per cell, i.e. for gating (3 bits), cell type (2 bits), and signal value (1 bit). Neuron cells additionally require 4 bits for the accumulator value stored. Phenotype data is loaded into the CA module to configure it according to the evolved function. Genotype/phenotype memory is connected to the hardware CA module and is used for rapid reconfiguration of the neural module by loading new chromosome -data (or phenotype data). The genotype/phenotype memory size for 10,000 modules is 32MBytes. This amount is sufficient to evolve 10 thousand modules at high speed, or to run a simulated brain with one million of neurons.
6.2.5.Module Interconnection Memory. In order to support the run mode of operation, which requires a large number of evolved modules to function as an artificial brain, a module interconnection memory is provided. It consists of an output vector array stack, similar to the input array and target stack, and a 64MByte memory for inter modular pathway storage. When each module of a large brain is configured in the CA hardware core (by loading its phenotype), an input stack is loaded with an array of input vectors. These vectors are previously stored output vectors recorded during the signaling phase of other modules, connected to this module. A large module interconnection memory will store the connection map and the current state of signals "travelling" between modules. In addition, an externally connected set of inputs and outputs is provided in hardware, which will allow the reception of signals from external sensors and the sending of signals to external effectors for robotic control.
7. Conclusion This paper is intended to give highlights on new concepts of artificial intelligence. Evolutionary engineering takes a prominent place in these emerging and revolutionary techniques. Evolutionary Engineering creates an elementary structure of the system and then evolves this structure toward the desired system. Evolutionary Algorithms are used for evolving such huge systems. Such process relies on observing and imitating natural systems. EE-Method intends to guide EE designer along the design process to achieve and implement a particular application. As an example of EE, we presented the CBM machine as a hardware tool for building/evolving a particular behavior in a real time design process. All ingredients of EE method exist in the CBM.
6.2.3.Fitness Evaluation Unit. When a useful module is being evolved, each instance of a module must be evaluated in terms of its fitness for a targeted task. During the signaling phase, each module generates vectors, which are compared with a target array in order to guide the evolutionary process. This comparison gives a measure of performance, or fitness, of the module. A hardware unit which consists of an input vector array stack, a target vector array stack, and a fitness comparator supports fitness evaluation. During each clock cycle an input vector is read from its stack. and fed into the module's inputs. The fitness comparator computes a Hamming distance between each output vector and a corresponding target vector, and accumulates the result 8
-
Software interface
75
Host Computer
Genetic Algorithm
-............ CBM
r----------I
I I I
CA
'
Module
CA
CA
Module
Module
-----l 1 --
I
I
I
-L I Module
I I I
CA
I I
CA
SCA
Module
I
I
7 I
4
I+-?-+ I
I
Module
--
-b
Module
I
t I
I I
CA
Module
--
CA '
Module
e . 1 . 1 . 1 . 1 . 1 .
1
I
7 -
Xilinx XC6264
Xilinx XC6264
........................................................ Figure 3. CA Module
Figure4. P G A
9
I
.I
[ l l ] T. Toffoli & N. Margolus, "CellularAutomataMachines", M F Press, Cambridge, MA, 1987; and "CellularAutomata Machines, in Lattice Gas Methodsfor Partial Differential Equations", SFISISOC, eds. Doolen et al, Addison-Wesley, 1990.
8. References [l] Hugo de Garis ,h4ichael Korkin , Felix Gers , Norbert0 Eiji Nawa , MichaelHough , "CAM-Brain, Atr's Artificial Brain Project An Overview", Brain Builder Group, Evolutionary Systems Department, ATR Human Information Processing Research Labs, September 1998.
[12] A. Lehireche, A. Rahmoun , "Evolutionary Engineering Approach", Lecture Notes , EEDIS,Dept of computer science,UDL Sidi Bel abbes,Algeria, April 2000.
[2] Hugo de Garis, "CAM-BRAIN : The Evolutionary Engineering of a Billion Neuron Artificial Brain by 2001 Which Grows/Evolves at Electronic Speeds Inside a Cellular Automata Machine (CAM)", in "TowardsEvolvable Hardware : The Evolutionary Engineering Approach, Eduardo Sanchez & Marc0 Tomassini (eds.), Springer, 1996.
[13] A. Lehireche, A. Rahmoun, "The Cam Brain Machine (CBM)" ,Lecture Notes, FEDIS,Dept of computer science,UDL Si& Bel abbes,Algeria, May 2000. [14] A. Rahmoun, M.Benmohammed, "On Deriving Optimal Fuzzy Expert Systems by Genetic Algorithms and Competitive Learning", IEE proceedings on Control Theory, Great Britain, NOV.1998,~ ~ 5 8 2 - 5 8 .
[3] Felix Gers.8~Hugo de Garis, "CAM-Brain : A New Model for.ATR's Cellular Automata Based Artificial Brain Project", ICES96 Conference Proceedings, Tsukuba, Japan, 1996. [4] Felix en & Hugo de Garis,"pofiing a Cellular Automata Based fificid ~~i~ to ~m~ cellular A~~~~~~ ~ ~ ~ CAM-S", SEAL96 Conference Proceedings, Korea, 1996.
A. B e " d M-, M.Elarbi B o u U , " New h i nGenetic ~ k d g 0 r i t h " e d Networks-Based Methodologies to Derive Optimal Fuzzy Systems", AJSE proceedings, Saudi Arabia, Vo1.25, NTB, pp. 105-122. 9
[5]Hugo de Garis , Genetic Programming: GenNets, Artificial Nervous Systems, Artificial Embryos, PHD thesis 1993.
[16] Nicholas J.Macias, "Ring Around the PIG: A Parallel GA
with only local interactions coupled with a Self-Reconfigurable
[6] Hugo de Garis, "Genetic Programming : Modular Evolution for Darwin Machines", UCNN-90- WASH-DC, (Int. Joint Conf. on Neural Networks), Januaryl990, Washington DC, USA
Hardware Platform to implement an O(1) Evolutionary Cycle For Evolvable Hardware", Proceeding of 1999 Congress on Evolutionary Computation, Copyright 1999 IEEE.
[7] Hugo de Garis, "Genetic Programming : Building Artificial Nervous Systems Using Genetically Programmed Neural Network Modules", in Proc. 7th. Int. Conf. on Machine Learning, pp 132-139, Porter B.W. & Mooney R.J. (eds.), Morgan Kaufmann,l990.
[17] Nicholas J.Macias, Lawrence B.Henry ITI, MuraLi Dandu Raju, "Self-Reconfigurable Parallel Processor Made From Regularly-Connected Self-Dual CodeData Processing Cells", US Patent #5,886,537. [18] A. Lehireche, A. Rahmoun, " The Step Machine", Lecture Notes, EEDIS,Dept of computer science,UDL Sidi Bel abbes,Algeria,October2000.
[8] Hugo de Garis, "Genetic Programming", (3.8 in book Neural and Intelligent Systems Integration, ed. Branko Soucek, Wiley,NY,1991.
[19] William B. Langton, Adil Qureshi, "Genetic Programming, Computers Using 'Natural Selection' To Generate Programms", Lecture Notes of a survey, Dept Of Computer
[9] A J F Van Rooij, L C Jain, R P Johnson, "Neural Network Training Using Genetic Algorithm", World Scientific,Series In - Machine Perception & Artificial Intelligence, Vo1.26.
Science, University College London.
[IO] D.E. Goldberg, "GeneticAlgorithms in Search, Optimization, and Machine Leaming", Addison -Wesley Publishing Company, 1989.
10
