Design of Highly Parallel Edge Detection Nodes

Design of Highly Parallel Edge Detection Nodes Using Evolutionary Techniques G.S.Hollingworth1, S.L.Smith and A.M.Tyrrell Department of Electronics, University of York, U.K. [gsh100,sls,amt]@ohm.york.ac.uk

Abstract This paper considers the application of bio-inspired systems in the design of a novel and highly parallel image processing tool to detect edges within conventional grey-scale images. The aim of the work is to implement a new image processing architecture through evolvable hardware that is able to adapt according to the particular images encountered. The simulation of such a system through the use of evolutionary algorithms and genetic programming is demonstrated for the conventional image processing operation of edge detection. Results are presented for this system and evaluated with respect to a conventional Sobel edge detector.

1. Introduction Bio-inspired systems have been present in the electronics and computer science communities for many years [Von Neumann66]. It is possible to classify bioinspired systems into three domains: phylogeny, ontogeny and epigenesis. Each of these is relatively well understood in the world of natural science. However, inspiration is required to bridge the gap between natural sciences and engineering. Ontogeny refers to the evolution of multicellular organisms during their embryonic phase. When reproduction takes place, the new individual is formed out of a single cell. During the weeks that follow the time of conception, the egg divides through a process known as mitosis, the result of which is two cells with identical genetic material. The new cells continuously repeat this process of self-division passing a copy of the DNA to each offspring. During this reproductive process, cells differentiate to shape the different tissues, organs and limbs that characterise a complete healthy individual of a particular species. Differentiation takes place according to "instructions" stored in the DNA; different parts of the DNA are interpreted depending on the position of the cell 1

with respect to its neighbours. Before differentiation, cells are able to take over any function within the body since each one possesses a copy of the DNA. The ontogeny process has found it harder to bridge the gap to engineering, but with the advent of programmable logic devices, such as FPGAs, the interest of some has now turned to that domain and the term embryonics (embryology and electronics) has been coined [Ortega98a, Ortega98b]. Epigenesis is the evolution of individuals’ ability to adapt through learning and responding to changes in the environment. After birth, every individual must adapt to the environment within which s/he is born. To achieve adaptation every individual is born with a set of systems defined by its genome (nervous systems, immune system and endocine system) which undergo modifications when interactions take place with the outside world. Without learning abilities, individuals would not be able to adapt to fast changes in their environment and certainly would die. Epigenesis has appeared in the engineering community under the guise of artificial neural networks, implemented mostly in software [Aleksander97, De Garis96]. Phylogeny embraces the evolution of species through the passing of genes from one generation to the next. Infrequent errors occurring during the copying of genes, known as mutations, originate new traits on the species. The survival of species depends upon these traits and allow the species to adapt better to changes in the environment. The environment is represented by coevolving populations and the resources needed for the survival of species. The ideas of phylogenetics have been applied for more than three decades with artificial systems. These are generally known as evolutionary algorithms or evolutionary computation, with specific examples being genetic algorithms, evolution strategies, evolutionary programming and genetic programming [Murakawa97, Sipper97, Thompson97]. The evolution of hardware systems can be either extrinsic or intrinsic. In the first case a software description of the electronic

Supported by the Engineering and Physical Sciences Research Council

circuit is evolved using computer simulation, and only the final elite chromosome is downloaded onto the programmable chip. Examples of extrinsic evolution include simple synchronous logic circuits. In the latter case the adaptation is done on-line in real-time. Evolutionary design techniques like Genetic Algorithms (GAs), Genetic Programming (GP) and Evolvable Hardware (EHW) have been applied to many simple design applications [Thompson97, Thompson96a, Murakawa96] and some advanced ones[Iba97, Iwata96, Thompson95]. It is believed that the application of these techniques can create systems that will react to their inputs in a method akin to the adaptation of natural biological beings to the environment. In the work reported here, high level image processing applications are evolved that have the ability to adapt to changes in the environment that would normally create errors in the system. This paper first describes the image processing problem of edge detection, followed by a brief introduction to evolutionary algorithms like the GP, and the application of GP to the Image Processing (IP) problem, resulting in a natural parallel architecture. Finally the actual edge detector evolved is discussed and evaluated in its abilities, in comparison to the Sobel edge detector.

number of picture elements or pixels, each of which has a value relating to the grey-scale intensity of the image at that point[Gonzalez93]. The gradient value at a pixel f(x,y) is therefore related to the two dimensional differential:

 df  2  df  2    +     dx   dy   The other edge detection operators listed above use highly non-linear methods. Template matching uses a cross-correlation method of finding the highest correlation between the image and a set of templates that detect edges in various orientations. Edge fitting uses a mathematical model of a step-edge with a search function to find the best fitting model at each point in the image. Finally, statistical detectors use statistical techniques to segment the image and indicate edges that exist between these segmented regions. Gradient operators are more commonly used in IP than the other techniques described, as they can be simple to implement (particularly on Digital Signal Processor (DSP) technology) and are particularly effective in many applications.

3. Evolutionary Design Techniques 2. Edge Detection Edge detection is an important and commonly used operation in Image Processing (IP) to establish the boundary between two regions in an image depending on certain criteria such as image intensities or grey-levels. Common applications of edge detection might include locating cell walls, the outline of aircraft and the preprocessing stage for character recognition. The particular type of edge detection operator used will, of course, depend on the criteria specified for the image under consideration and may differ depending upon whether the located edges are intended for human visual interpretation or further machine manipulation. An ideal edge detection operator would be capable of detecting all types of edges, including simple steps, gradients and changes of texture, regardless of orientation and the quality of the image, which can commonly be distorted due to noise, corruption and poor lighting. As might be expected, no such edge detection operator currently exists, but a number of different fundamental approaches have been developed which include gradientbased, template matching and edge fitting operators, and statistical detectors [Vernon91]. Gradient-based operators work on the principle that edges may be defined between areas of different image intensity. It is common to represent images digitally as a

Genetic Algorithms were first described by Holland [Holland75]; he showed how a system could be described as a simple binary string similar to the DNA strings which we all posses. In the human body, the DNA string describes how to build the human form through a sequence of complex chemical interactions within the cells. In artificial evolution the string describes the system that is intended for evolution, for example the string could be a binary number which tries to solve some equation. Finally, a measure of the performance or fitness of the individual is required. In nature this is done by building the form (the phenotype) from the DNA sequence (the genotype) and evaluating the individual within the environment. High performance individuals reach maturity and reproduce, low-performance individuals die somewhere along the way. In artificial evolution, the fitness is measured by converting the binary string (the genotype) into the system (the phenotype) and evaluating it. An example of this is to place the binary string into the equation (as a number) to find the best solution, the better the solution, the better the fitness. Holland showed mathematically that this works when the probability of reproduction is higher for ‘more fit’ individuals. Genetic Programming was a simple extension to this idea introduced by John Koza [Koza92]. Instead of describing the system in terms of a simple binary string, a

tree structure of functions is used. This structure creates a system, with inputs at the end of each branch and a single output at the top. An example tree structure is shown in Figure 5. A further extension to the Evolutionary design methodologies came after the creation of Field Programmable Gate Arrays (FPGAs) [Xilinx95], a circuit that can be programmed to perform a wide variety of digital electronic tasks. The main idea is to use GAs to program the FPGA and asses the design either in simulation (Extrinsic Evolvable HardWare (EHW)) or by programming the device (Intrinsic EHW)[Yao96]. This has prompted much research on EHW [Thompson95, Thompson97, Thompson96a, Murakawa96, Iwata96, Iba97] showing not only the successful evolution of electronic circuits, but also some desirable features, such as fault tolerance [Thompson96b]. The power of bio-inspired electronics is in its potential as an adaptive hardware which can change its behaviour and improve its performance while executing in a real physical environment (as opposed to simulation). Such on-line adaptation is more difficult to achieve but theoretically gives many advantages over extrinsic systems. At present, work has mostly been concerned with off-line adaptation. That is, the hardware is not used in an execution mode while evolving. Problems involved with on-line adaptation include, time to adapt and accuracy of adaptation. However, if these problems can be overcome, the power of bio-inspired electronics offers much to many.

4. A new Image Processing Architecture using EHW The main problem with applying EHW to IP is the sheer size of data that must be processed. Images are usually composed of thousands of individual pixels, each of many bits, creating a massive number of input and output bits. Since the genotype describes how to connect the inputs to the outputs through some function network, a correspondingly large genotype for evolution would be required. In general, the larger the genotype, the longer the time required for evolution. A method is required to reduce the number of inputs and outputs to the evolvable system, to reduce the size of the genotype and therefore, reduce the evolution time, also preferably one that is independent of the image size. This problem can be overcome by exploiting the parallelism of images. Figure 1 shows a diagram of the basic structure of the architecture. Each block in the network is pre-loaded with a single pixel of the image which is then output to the local neighbourhood. This architecture allows a rich and varied form of image

processing from edge detection to high and low pass filtering.

Neighbourhood communication

Common function processing blocks Figure 1: Evolvable Hardware platform for grayscale image processing

Genotype String

Logic Function Unit, as defined by genotype.

Input pixel

Output pixel

Pixel values to/from neighbours Figure 2: Pixel element processing block diagram

Figure 2 shows the common processing block for each pixel. Within each block a genotype string defines the operation of the logic function, this is simple to implement, using functional blocks such as multiplexors and look-up tables. The logic unit provides the function to convert between the input bits and the output pixel bits, giving a non-linear output; due to the non-linear nature of the processing functions (e.g., and-or-not.) It should be noted that the element is restricted to processing in the local region only, although this will be addressed in later versions of the processing system. Since the system is ‘programmed’ using a simple binary string, it is possible to apply a genetic algorithm to the string to evolve the edge detector. The proposed architecture will have a number of characteristics to help with implementation, these include: high regularity, which simplifies its implementation on silicon; modular in nature making the actual function of the processing element independent from the function of the remaining blocks within a cell; simple in terms of the processing elements used allowing built-in self test logic to provide self-diagnosis without excessively increasing the silicon area.

5. Simulation of the Image Processing Architecture The application of EHW in the design of a new architecture for image processing described above is entirely novel and unproven. It is therefore prudent to evaluate the system performance as far as possible before committing the design to hardware. This is achieved by simulating the entire system in software, paying special consideration to the following areas: 1.

2.

3.

4.

Desired Edge Detection Operation The type of edge detection operation to be achieved by the system. Fitness Evaluation The method adopted for assessing the performance of the simulated algorithm. Type of Evolutionary Algorithm The type of evolutionary algorithm to be used (i.e. Genetic Algorithm or Genetic Program). Genetic String Coding The method used to translate from a binary string, function tree to a logic network.

5.1 Desired Edge Detection Operation As described earlier a number of different approaches to edge detection are available for conventional IP work and that the most popular of these are gradient operators. It was therefore decided that a gradient operator should be used as the operation by which the EHW system should be

compared. The specific gradient operator chosen for evaluation of the simulation is that devised by Sobel [Sobel70]. The Sobel operator is a simple, but effective neighbourhood processing or mask operator that combines good edge detection with immunity to noise. Neighbourhood processing is achieved by considering the grey-scale values of the 8 pixels that surround the pixel under investigation. According to the weights specified in the mask, a new value is calculated for the central pixel. This process is repeated for every pixel in the image. The Sobel operator utilises two such 3 x 3 pixel masks which, are shown in Figure 3. The first calculates the gradient in the horizontal plane and therefore detects vertical edges while the second calculates the gradient in the vertical plane and detects horizontal edges. In both cases the gradient is calculated by multiplying each pixel by the respective weighting and summing the result. 1 2 1

0 0 0

-1 -2 -1

1 0 -1

2 0 -2

1 0 -1

Figure 3: Mask to determine vertical and horizontal edges The vertical and horizontal gradients can be combined, as follows, to give a measure of the magnitude of the gradient at each point i,j in the image.

im[i , j ] =

(Sobelh[i, j]) (Sobelv[i, j]) 2

+

Where Sobelh[i,j] is the horizontal gradient at position i,j, and Sobelv[i,j] the vertical gradient as position i,j. The above is commonly approximated, as follows, to reduce computation time whilst maintaining the desired operation:

im[i , j ] = Sobelh[i , j ] + Sobelv[i , j ] The Sobel operator is generally followed by a simple thresholding operation in which each pixel in the image is assigned a value representing either black or white depending on the magnitude of the gradient at that point. This operation is illustrated in Figure 4:

2

Figure 4: (a) Original Image (b) Sobel Output

5.2 Fitness Evaluation The fitness of the genetically derived edge detector needs to be evaluated with respect to the Sobel operator described above. There are a number of methods commonly used to compare the performance of edge detectors [Heath97, Haralick84], although in general such methods are mathematically complex and too computationally intensive to perform practically in simulation. An alternative method, described here, is a simplified minimisation of under- and over-detection of edge pixels. In essence, the results of the application of a Sobel operator and the genetically derived edge detector to the same original image, are compared on a pixel-bypixel basis. Two calculations are made based on those edges identified by the Sobel operator but not the genetically derived edge detector, and vice-versa. Underdetection (Pef) is the number of edge pixels not detected by the genetically derived edge detector divided by the total number of edge pixels detected by the Sobel operator. Overdetection (Pnf) is the number of non-edge pixels detected by the genetically derived edge deter divided by the total number of non-edge pixels detected by the Sobel operator. We need to minimise both of these values simultaneously and is achieved by maximising the following equation.

fitness =

1 1 + ( pef +

p

nf

)

(GP). The survival of species depends upon these traits producing ’fitter’ individuals, and thus allowing the species to adapt better to the environment. Phylogeny implies populations of interacting individuals. A number of tests have been conducted to find the most suitable algorithm for this application. The first was to evolve the convolution kernels (one horizontal and one vertical) to perform the stated tasks. This method used the Genetic Algorithm approach and showed itself to be successful, although, the function unit described above (within the standard processing block) would be very limited in its abilities, due to the fact that the only function which can be applied is one of linear convolution. The efficiency of Genetic Programming (GP) for these problems has been investigated. This method creates a tree of functions, each of which has a number of inputs. The GP then tests each function simply by applying the tree of functions to the terminal data, the local image neighbourhood. Figure 5 shows a simple tree structure, using the functions AND/XOR. Each terminal is shown in bold and these nodes are named by the position they take relative to the current pixel, i.e. North, South etc. When the tree is evaluated a fitness value is assigned to it, and the GP then breeds a new population. Selection of the parents is weighted using the fitness, leading to fitter individuals reproducing more often. This operation tends to create a new individual that is a better, more fit individual, than either of its parents.

5.4 Genetic String Coding The coding method used with GP is simple, using a selection of the node functions and the terminal values: {and, or, not, xor} This set of functions was chosen to mimic the set of functions available within an FPGA. This then gives an indication of how well the application would transport across to current devices. An example tree is shown in Figure 5 using the first set of functions: OR

5.3 Type of Evolutionary Algorithm Phylogeny embraces the evolution of species through the passing of genes from one generation to the next. The basis of this level of evolutionary development is that infrequent errors occurring during the copying of genes (mutations) originate new traits on the species. Two classes of algorithm that fit into this type of operation are Genetic Algorithms (GA) and Genetic Programming

XOR

SW

XOR

NE

NW

SE

Figure 5: A simple logic tree using And/XOr

The terminals, shown in bold in the figure, are actually only a single bit wide (due to the fact we are using logic gates). This means that with an 8 neighbourhood, 8 bits per pixel, 64 terminals would be required, making the evolution time long. For the purposes of this simulation the task has been simplified to use only 3 bits per pixel.

6. Results of Evolution The results of the test runs are described as a set of images corresponding to various stages through a single run of the GP. These are compared with the output achieved by processing the same original image using the Sobel edge detection operator described earlier.

After first generation.

After 1000 generations

Figure 6: Sobel Output Figure 7 shows the results of a single run of the GP using the logic functions on 3 bit image data. (The Sobel edge detection operator was applied to the full 8 bit greyscale data). The bottom right picture shows the GP’s best attempt at an edge detection operation by generation 2333; while the results are encouraging, close inspection reveals differences to the Sobel edge detection operator.

7. Discussion The images considered above, resulting from a simulation of the proposed EHW architecture for image processing, indicate that an edge detection operator is being evolved. The image resulting from 2333 generations is still not the same as the image resulting from application of a Sobel edge detector operator, adopted for evaluation of fitness. However, an effective edge detector does seem to have been derived and hence, suggests that the Genetic Programming method of evolving the image operator has achieved the desired result without using conventional design techniques. (in

After 2333 generations. Figure 7: Results of Genetic Programming using functions And, Or, Not and Xor on 3 bit image data, in comparison to the Sobel edge detector operator. the order of ten times faster if implemented in an ASIC) and uses very much less silicon space per pixel The GP evolved a combination of AND/OR/NOT/XOR gates in 70 functions in a tree structure of 10 levels deep. This, when compared to a DSP multiply-accumulate function, is very much faster operation. This demonstrates that to obtain a Sobel edge detector operator, we would require a much greater

number of nodes in the tree structure, which would eventually lead to even greater evolution times (currently about 24 hrs). However, these results are promising and work continues to refine and improve these methods.

8. Conclusion The method described in this paper is an image processing operation that can be achieved using evolutionary algorithms. It is believed that the results presented demonstrate that this has been accomplished through the simulation of a new architecture designed to exploit the inherent parallelism of images. Although some measure of success has been achieved in the detection of edges within images, the important conclusion from these studies is that basic image processing operations can be evolved. This leads to future work currently underway to evolve higher level image processing operations based on low level image processing routines. It is believed that the work reported here will give rise to the possibility of producing novel solutions to well understood problems. The characteristics of these novel solutions are still to be determined.

9.References [Aleksander97] Aleksander, I. (1997) ’Iconic Learning in Networks of Logical Neurons’, in Higuchi et al. (Eds), Proceedings of 1st International Conference on Evolvable Systems: From Biology to Hardware, LNCS 1259, Springer-Verlag, pp 3-16. [De Garis96] Garis de, H. (1996) ’CAM-BRAIN: The Evolutionary Engineering of a Billion Neuron Artifical Brain by 20001’, in Sanchez and Tomassini (Eds) Towards Evolvable Hardware: The Evolutionary Engineering Approach, LNCS 1062, Springer-Verlag, pp 76-98. [Gonzalez93] Gonzalez, R.C. and Woods, R.E. (1993) ‘Digital Image Processing’, Addison-Wesley. [Haralick84] Haralick, R.M., (1984) ‘Digital Step Edges from Zero Crossing of Second Directional Derivatives’ IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol PAMI6, pp 58-68 [Heath96] Heath, M., Sarkar, S., Sanocki, T. and Bowyer, K., (1996) ‘Comparision of Edge Detectors: a Methodology and Initial Study’, Computer Vision and Pattern Recognition, San Francisco. [Holland75] Holland, J., (1975) ‘Adaptation in natural and artificial systems: an introductory analysis with

applications to biology, control, and intelligence.’ University of Michigan Press.

artificial

[Iba97] Iba, H., Iwata, M. and Higuchi, T. (1997) ‘Gate Level Evolvable Hardware: Empirical Study and Application’, Evolutionary Algorithms in Engineering Applications, Springer-Verlag pp 259-279 [Iwata96] Iwata, M., Kajitani, I.,Yamada, H., Iba, H. and Higuchi, T. (1996) ‘A Pattern Recognition System using Evolvable Hardware’, International Conference on Evolutionary Computation: The 4th Conference on Parallel Problem Solving from Nature, pp 761-770 [Koza92] Koza, J., ‘Genetic programming: on the programming of computers by means of natural selection and genetics.’ The MIT Press, Cambridge, MA. [Murakawa96] Murakawa, M., Yoshizawa, S., Kajitani, I., Furuya, T., Iwata, M. and Higuchi, T. (1996) ‘Hardware Evolution at Function Level’, International conference on Evolutionary Computation: The 4th Conference on Parrallel Problem Solving from Nature, pp 62-71 [Murakawa97] Murakawa, M., Yoshizawa, S. and Higuchi, T. (1997) 'Adaptive Equalization of Digital Communication Channels using Evolvable Hardware', in Higuchi et al. (Eds), Proceedings of 1st International Conference on Evolvable Systems: From Biology to Hardware, LNCS 1259, Springer-Verlag, pp 379-389. [Ortega98a] Ortega, C. and Tyrrell, A.M. (1998) 'Design of a Basic Cell to Construct Embryonic Arrays' IEE Proceedings on Computers and Digital Techniques, May. [Ortega98b] Ortega, C. and Tyrrell, A.M. (1998) 'Biologically Inspired Real-Time Reconfiguration Technique for Processor Arrays', Proceedings of 5th IFAC Workshop on Algorithms and Architectures for RealTime Control, Mexico, Elsevier Science. [Sipper97] Sipper, M. (1997) 'Designing Evolware by Cellular Programming', in Higuchi et al. (Eds), Proceedings of 1st International Conference on Evolvable Systems: From Biology to Hardware, LNCS 1259, Springer-Verlag, pp 81-95. [Sobel70] Sobel, I.E, (1970) ‘Camera Models and Machine Perception’, PhD thesis, Stanford Univ.

[Thompson95] Thompson, A., (1995) ‘Evolving Electronic Robot Controllers that Exploit Hardware Resources’, Proc. of the 3rd European Conf. on Artificial Life (ECAL95), Springer-Verlag, pp 640-656 [Thompson96a] Thompson, A.(1996) ‘Silicon Evolution’, Proceedings of Genetic Programming (GP96), J.R. Koza et al. (Eds), pages 444-452, MIT Press 1996. [Thompson96b] Thompson, A., (1996) ‘Evolutionary Techniques for Fault Tolerance’, Proc. UKACC Int. Conf. on Control (CONTROL'96), pp 693-698. IEE Conference Publication No. 427 [Thompson97] Thompson, A. (1997) 'An evolved Circuit, Intrinsic in Silicon, Entwined with Physics', in Higuchi et al. (Eds), Proceedings of 1st International Conference on Evolvable Systems: From Biology to Hardware, LNCS 1259, Springer-Verlag, pp 390-405.

[Vernon91] Vernon, D., (1991) ‘Machine Vision: automated visual inspection and robot vision’, Prentice Hall [Von Neumann66] Von Neumann, J. (1966) 'Theory of Self-Reproducing Automata', University of Illinois Press, Illinois, Ed. AW Burks. [Xilinx95] Xilinx Inc., ‘XC6200 Field Programmable Gate Array Data Book’, ‘http://www.xilinx.com/partinfo/6200.pdf’ [Yao96] Yao, X. and Higuchi, T., (1996) ‘Promises and Challenges of Evolvable Hardware’, Proceedings of The First International Conference on Evolvable Systems: from Biology to Hardware (ICES96). T.Higuchi, M. Iwata and W. Liu (Editors), LCNS 1259, Springer-Verlag.

Web Page: http://www.amp.york.ac.uk/external/aseg/evolarch.html