Cellular neural networks - IEEE Xplore

Download PDF
25 downloads 0 Views 827KB Size Report
Comment
a true revolution in techniques used for .... ∂x2 +. ∂2. ∂y2 +. ∂2. ∂z2. (3) is the three-dimensional Laplacian operator. ... dance with living moving beings), in.
Cellular Neural Networks: f (xijw ) I

–I I

xijw

–I

(c) Vu ijw

Vx ijw

Vy ijw

(a) (b) Figure 1. The CNN scheme: (a) the spatial grid showing local connections; (b) the circuit of each cell; (c) the classical output nonlinear function for each cell.

A Paradigm for Nonlinear Spatio-Temporal Processing 1531-636X/12/$10.00©2001IEEE

6

by Luigi Fortuna, Paolo Arena, David Bálya, and Ákos Zarándy

A

bstract—The paradigm of Cellular Neural Networks is going to achieve a complete maturity. In fact, from a methodological point of view, important results on their digitally programmable analog dynamics have been gained, completed with thousands of application routines. This has encouraged the spreading of a great number of applications in the most different disciplines. Moreover, their structure, tailor made for VLSI realization, has led to the production of some chip prototypes that, embedded in a computational infrastructure, have produced the first analogic cellular computers. This completes the framework and makes it possible to realize complex spatio-temporal and filtering tasks on a time scale of microseconds. In this paper some sketches on the main aspects of Cellular Neural Networks, from the formal to the hardware prototype point of view, are presented together with some appealing applications to illustrate complex image, visual and spatio-temporal dynamics processing. Introduction Recently the most important microprocessor manufacturers realized that one of the main challenges for the

P. Arena and L. Fortuna are with the Dipartimento Elettrico Elettronico e Sistemistico, Università degli Studi di Catania, Viale A. Doria 6, 95125 CATANIA, ITALY. E-mail: [parena, lfortuna]@dees.unict.it. David Bálya and Ákos Zarándy are with Analogic Computers Ltd, BUDAPEST, HUNGARY, e-mail:[email protected].

near future is to build efficient processors and infrastructure for the real time handling of images and videos or for general time signals coming from space distributed sources. Because both of these tasks are strictly related to spatio-temporal computing, a great effort was then performed to devise supercomputers able to perform spatio-temporal calculations on nonlinear partial differential equations (PDEs). Moreover, from one point of view these operations are to be performed in real time, while from another one a 32 bit floating point accuracy is often not required. From this perspective the possibility to exploit the capabilities of analog computation on signal flows instead of traditional digital computation on bits arises. The paradigm of Cellular Neural/Nonlinear Networks (CNNs) fully realizes this concept introducing a new paradigm for analog/logic (analogic) array processing [1]. This super processing aspect, joined to a fundamental programmability part leads to the concept of the so-called CNN Universal Machine (CNN-UM) based analogic computer [2]. Based on the huge number of applications developed, CNNs can be considered as a paradigm for solving nonlinear spatio-temporal wave equations (a very difficult task for digital supercomputers) within a microsecond, and with an equivalent I/O accuracy of 7–8 bits. More that ten years ago, some seminal papers from Leon O. Chua introduced the Cellular Neural Network concept. From then on, an ever 7

increasing number of related concepts and applications led to considering CNNs as a paradigm forming architectures in a very broad set of fields; and therefore CNN became the acronym of Cellular Nonlinear Networks. In this new perspective, they can be defined as “2D or 3D arrays of mainly locally connected nonlinear dynamical systems called cells, whose dynamics are functionally determined by a small set of parameters which control the cell interconnection strength” [3]. These parameters determine the connection pattern, and are collected into the socalled cloning templates which, once determined, define the processing of the whole structure. The CNN-UM analogic computer can therefore be considered as an environment in which spatio-temporal algorithms are defined in terms of template flows among spatio-temporal input signals which can be images, videos or general control variables acting in the analog world. In this review the main aspects of CNNs are briefly presented, together with some of the most impor-

Luigi Fortuna is full professor of system theory at University of Catania since 1994. He has published more than 250 technical papers and is coauthor of six books, including Cellular Neural Networks (Springer 1999). He holds several USA Patents. His scientific interests include nonlinear science and complexity, chaos, and cellular neural networks with applications in bioengineering. He is an IEEE fellow and is chair of IEEE Technical Committee on Cellular Neural Networks and Array Computing. Paolo Arena received the degree in electronic engineering and the Ph.D in electrical engineering in 1990 and in 1994, respectively, from The University of Catania, Italy. He is currently associate professor of system theory. He is co-author of more that 120 technical papers, three books and several international patents. His research interests include adaptive nonlinear and learning systems, neural networks, cellular neural networks, collective behavior in living and artificial neural structures, bio-inspired locomotion control, bio-image analysis, and DNA microarrays. He is a senior member of the IEEE.

8

tant and impressive applications which emphasize the role of these structures in the parallel real time computing panorama. Cellular Neural Networks The idea of Chua was to use an array of essentially simple coupled nonlinear dynamic circuits, called cells, to process large amounts of information in real time. This concept was inspired by the Cellular Automata [4] and the Neural Network [5, 6] architectures. This new architecture was able to perform time-consuming tasks, such as image processing and PDE solution, being at the same time, suitable for VLSI implementation. The original CNN model was introduced in 1988 [1], and it is shown in Fig. 1, referring to a general three-dimensional locally connected lattice scheme (Fig. 1(a)). The cell was defined as the first order nonlinear circuit shown in Fig. 1(b), uijw, yijw and xijw being the input, the output and the state variable of the cell respectively. The output is related to the state by the nonlinear equation: yijw = f(xijw) = 0.5 . (| xijw + 1| – |xijw – 1|) (1) shown in Fig.1(c). The CNN can be defined as a array of M x N x P identical cells arranged in a spatial grid, as depicted in Fig. 1(a). Each cell mutually interacts with its nearest neighbors. If we use a CMOS circuit, the interaction is implemented by means of the voltage controlled current sources Ixy(i, j, w; k, l, z) = A(i, j, w; k, l, z)yklz and Ixu(i, j, w; k, l, z) = B(i, j, w; k, l, z)uklz. The constant coefficients A(i, j, w; k, l, z) and B(i, j, w; k, l, z) are known as the cloning templates: if they are equal for each cell, they are called space-invariant. The CNN is therefore described by the state equations of all cells:

Cxijw = –

+ +

1 xijw(t) Rx

∑

A(i, j, w; k, l, z)yklz

∑

B(i, j, w; k, l, z)uklz + I

C (k,l ,z )∈ N r (i, j ,w )

C (k,l ,z )∈ N r (i, j ,w )

(2) with 1 ≤ i ≤ M, 1 ≤ j ≤ N, 1 ≤ w ≤ P where Nr (i, j, w) = {C(k, l, z) | max( | k – i |, l – j |, | z – w | ) ≤ r} with 1 ≤ k ≤ M, 1 ≤ l ≤ N, 1 ≤ z ≤ P is the r-neighborhood and xijw(0) = xijw ; | xijw | ≤ 1; C > 0; Rx > 0. 0

0

This model is known as the Chua-Yang model or linear CNN. From the definition it derives that the template coefficients completely determine the behavior of the CNN for a given input and initial condition for each cell, as well as for a given set of boundary conditions. In the most common case when all the cells have equal parameters (space-invariant templates) a set of 2 . (2r + 1)nD + 1 parameters (where r represents the neighbor radius, and nD the CNN spatial dimension: 1D, 2D, and so-forth), completely define the evolution of an arbitrary, large n-Dimensional CNN array. The Chua-Yang model has been generalized in many different ways, in order to enhance the capabilities of CNNs and improve their efficiency. Some of the most interesting are [3, 7]: CNNs with nonlinear templates, where general functional dependencies are introduced in the templates; delay CNNs, where the cell dynamics depend on past values of the input/output variables of the neighboring cells; different nonlinearities, besides the classical PWL saturation-like output

function; nonuniform grid CNNs and multiple neighbor size CNNs, to reflect some characteristics found in living/ visual systems; and discrete-time CNNs. Moreover, several conditions on CNN stability in terms of the template coefficients have been derived [8]. If a CNN is used as a real-time analog processor for arrays of data according to a well defined template set plus local and global logic, which represent the instructions of the CNN Universal Machine (CNN-UM) [2] processor, spatio-temporal algorithms can be defined for this processor, where a given template set is allowed to operate in a given time window, giving way to another template set able to continue to process data produced by the previous templates. Because of properties of real time computing, very complicated array data processes can be performed in very limited time frames. David Bálya received the M.Sc degree in computer science from the Technical University of Budapest, at Budapest, Hungary, in 1999. Since 1999 he has been a Ph.D student at the Budapest University of Technology and Economics and the Analogical and Neural Computing Laboratory of the Computer and Automation Research Institute of the Hungarian Academy of Sciences in the Neuromorphic Information Technology Interdisciplinary Graduate Program. In 1999 and 2000 he was visiting scholar at the Vision Research Laboratory of the University of California at Berkeley, co-operating with neuro-biologists to develop a mammalian retina model based on a complex-cell cellular nonlinear network. His main research interests include biological sensory systems modeling, visual computing, and applied artificial intelligence. Ákos Zarándy received the Ph.D degree in electronic engineering and information technology from the Hungarian Academy of Sciences in 1997. He is presently a senior research engineer in the Analogic Computing Laboratory at the Computer and Automation Institute of the Hungarian Academy of Sciences. He has been dealing with Cellular Neural/Nonlinear Networks (CNNs) since 1990, where his work dealt with mathematical morphology, color image processing, and multi-modal image fusion. He built the first visual computers based on various CNN chips. His activity is currently devoted to exploring the application areas of the CNN technology.

9

Recent discoveries and new techniques are completely changing our knowledge of living structures and opening astonishing possibilities to explore and characterize genetic diseases. The introduction of the DNAchip, or DNA microarray constituted a true revolution in techniques used for gene processing and classification, since it allows compression into a little microscope glass of hundreds of thousands of different DNA nucleotide sequences, and permits all of this information to appear on a single image. Of course, the characteristics of local interaction among the cells make CNN structures tailor-made for physical implementation. This is indeed a great advantage over, for example, general neural network architectures. On the other hand, the bottleneck of a simple CNN lies in the fact that no efficient strategy exists to “learn” the suitable template values to perform a given task. The strategy mainly used is therefore to find, by trial and error techniques, the suitable templates, or to “design” the CNN dynamics. In this case the model of the single cell, as well as of the local connections among cells, has to be determined. Indeed, from this point of view, a great number of templates and template algorithms have been introduced and optimized to perform a great number of tasks [9]. To include learning and adaptivity an extended CNN-UM has been introduced [10]. Stimulating applications of CNNs have in fact been developed into a wide range of disci10

plines, ranging from classical and sophisticated image filtering, to biological signal processing solution of nonlinear PDEs, physical system and nonlinear phenomena modeling, generation of nonlinear and chaotic dynamics, associative memories, neurophysiology, robotics, and so on [11, 12]. In the following part of the paper some key applications are shown. The first one aims to outline the role of these structures in processing images in real time, including some complicated ones, such as fluorescence images arising from DNA microarray processing. The second application is the appealing theme to construct new spatio-temporal based processing strategies able to mimic processing in the retina, with the aim to constitute future candidates as prosthetic analogic devices. The last one aims to demonstrate how CNNs can constitute analog primitives to represent complex dynamics in spacetime, like those able to drive bio-inspired walking machines, endowed with a large number of joints. The last section briefly sketches a development system based on one of the latest CNN-UM based supercomputers: the 64 x 64 CNN chip. Some conclusions are then presented. Analogic CNN Algorithms for Real Time Microarray Image Processing In the last years, molecular genetics has become an interesting and huge field of research. Recent discoveries and new techniques are completely changing our knowledge of living structures and opening astonishing possibilities to explore and characterize genetic diseases. The introduction of the DNA-chip, or DNA microarray constituted a true revolution in techniques used for gene processing and classification, since it allows compression into a little microscope glass of

Figure 2. The CNN chip for DNA microarray processing.

hundreds of thousands of different DNA nucleotide sequences, and permits all of this information to appear on a single image. A great number of research laboratories in the world are currently performing their research by using these devices; the well-known journal “Nature Genetics” published a supplement entirely devoted to DNA Microarrays [13]. While the time required to obtain fluorescence images can be reduced by means of specific technologies, the main drawback lies in the time-consuming analysis of the fluorescence spots. This is in fact performed by using traditional computing systems and constitutes the true bottleneck of the whole bio-protocol. CNN algorithms and devices can really provide a breakthrough in DNA microarray processing to obtain in real time the gene expression profile. The most important characteristics to extract from such images are assess-

ments of the hybridization degree. The left hand side of Fig. 2 shows an example of fluorescence imaging which depicts concurrent hybridization of two target DNAs, one marked with a green fluorescent, and the second with a red one. The microscope reveals a fluorescent mosaic of spots, each one representing the matching degree of a pair of genetic strands: their colors indicate which bases have joined, while the intensities of the red, green and yellow represent the hybridization degree of one, the other, or both of the target DNAs. In the example reported in this paper, the analogic spatio-temporal algorithm was implemented on the analogic CNN Engine Board containing the 64 x 64 analog I/O CNN-UM chip (Fig. 2), sketched in the following sections. This means that the algorithm is applied to sub-images of 64 x 64 pixels of the original fluorescence image; the Engine Board allows a final “tiling” of the sub results. The color image resulting from a confocal microscope is split into the 11

Cellular Neural Networks: A Paradigm for Nonlinear Spatio-Temporal Processing three basic color images: red, green, and blue. Because, in the example reported here, two fluorescent materials were used, giving red and green fluorescences, it was necessary to split the original image into these two basic color components, obtaining two gray scale images representing the intensity level for each basic color. These initial images were processed by using an algorithm consisting of a series of “instructions” which in this case are represented by CNN templates. Therefore the templates are sequentially applied, each one processing the images resulting from the processing with the previous template. The templates used in the algorithm designed for the DNA microarray processing are standard, and can be found in the template library [9]. The algorithm essentially consists of the following steps. The first step is to clean the images from background noise. This is done by using a series of “threshold” and “diffusion” elementary operations. Moreover, the spots not located within given sites, delimited by a pre-specified grid, are detected and deleted. The next step aims to analyze the morphology of the spots by using morphological template based procedures. The result is that irregular spots, bigger or smaller than pre-specified sizes, are detected and deleted. Finally the surviving spots are classified according to their intensity, outlined by means of several “threshold” operations. The corresponding intensity level spots are detected for 12

each channel and isolated. In this case, three thresholds were fixed, obtaining high, medium and small hybridization levels, for each probe (color channel). Of course, the number and the threshold levels can be adapted to the particular task to be performed in the particular DNA chip protocol. The main objective for this procedure consists of the determination of the yellow spots, characterized by the combination of both basic colors. The yellow spot classification is simply obtained by applying a “LogAnd” operation between thresholded images for both of the basic components. The on-chip results for the high level red, green and yellow spots are depicted in Fig. 2. The on-chip time required to run the whole algorithm on a 64 x 64 input image was about 7ms[14]. The whole time required to process the input image of Fig. 2 was about 4.5 ms, which actually represents a crucial advantage with respect to currently available microarray technologies. Moreover, the time spent could also be further reduced, because the CNN Engine Board is currently a prototype and the infrastructure could be improved. A further remark regards the fact that the number and value of the thresholds can be modulated by the user to select the spots corresponding to the desired degree of hybridization. However, besides the numerical results which could even be improved, it has to be stressed that the real breakthrough lies in the new, parallel processing of DNA

Measurement

Simulation

BriskTrN

OffBriskL

LED

time

Bistratified

OnSluggish

OnBeta

space Figure 3. The Structural Mammalian CNN Retina model.

microarrays allowed by CNN algorithms with respect to the traditional sequential one. CNN to Model Mammalian Retina Processing Many sensory processing parts of the nervous system have been adequately modeled through the years. The first major work showed many directions and possibilities [16]. Very recently, following the discovery of the parallel channels in a mammalian retina [17], we have developed a multilayer CNN model for reproducing some simple effects, as well as to develop a simulation framework which is available through the internet [18]. A summary of this model is shown in Fig. 3. Some parallel channels of mammalian retinal processing are shown schematically on the right-hand side. The first column shows the measured [17] space-time pattern of one channel in the rabbit retina and the second column shows the corresponding simulated outputs. These are the spike fre-

quency (last row), excitatory current (first row) and inhibitory current (middle row) representations of the flashed square in a given channel. The time runs vertically down and the space is on the horizontal axes. White bars indicate the time and space marked regions of the stimulus. The model qualitatively reproduces the measured rabbit retinal patterns [17]. The different model structures of the mammalian retina channel are shown on the right-hand side. The different neuron types in the retina are organized into two-dimensional strata modeled with CNN layers, which are represented by the spheres. A neuron in a given layer affects another neuron in another layer through synapses while the arrows represent the connections. The layers have different time and space constants and the synapses produce non-linear transfer functions [2]. The general neuro-biological names of the channels are given and the positions of the terms indicate the morphological depths of the parallel representations. 13

at least two generalized variables standing for the chemical concentrations of the activator and inhibitor respectively, in a so-called activator-inhibitor mechanism suggested in [22], F(U) is a vector nonlinear function, describing the kinetics of the phenomena, D is the diffusion matrix, and Figure 4 (a). The bioinspired robot prototype: WORMBOT.

CNN: A Paradigm for Complex Dynamics The intrinsic space distributed topology makes the CNN able to produce real-time solutions of nonlinear PDEs [19] and to reproduce spatiotemporal phenomena, like biological locomotion. In this case the continuous time flow of the CNN variables corresponds to specific trajectories of the robot joints. Since it is not easy to specify the class and the characteristics of suitable signals to be generated by a CNN, a biologically inspired approach is followed and consequently the neurobiological paradigm of the Central Pattern Generator (CPG) is considered [20], where information derived from sensory inputs modifies the signals driving actuators so as to adapt locomotion to the environment. From a behavioral point of view, the whole locomotion system (CPG) appears to be a complex activator-inhibitor system characterized by a hierarchical organization in which a group of neurons, due to sensory or central excitations, activate other neural assemblies that generate the appropriate timing signals for each type of locomotion. Such phenomena can be efficiently described by particular solutions of nonlinear PDEs, the so-called Reaction-Diffusion equations [21] ∂U = F(U) + D∇2U ∂t where U is a state vector consisting of

14

∂2 ∂2 ∂2 ∇ = 2+ 2+ 2 ∂x ∂y ∂z 2

(3)

is the three-dimensional Laplacian operator. According to the parameters, the corresponding solutions of this equation show, among others, self-sustained oscillations as well as steadystate stable configurations over the 3D lattice, that can be represented under the form of geometric spatial patterns of chemical concentration. Both of these phenomena have been shown to arise in a CNN structure. Moreover, because the above mentioned dynamics could also be found in living neural structures, CNNs have been designed to act as locomotion pattern generators and controllers, in particular for moving mechatronic structures with a great number of joints. From this perspective, the CNN paradigm succeeds in controlling the spatio-temporal structure, represented by the ensemble of actuators moving in time and spatially organized (in strict accordance with living moving beings), in a very efficient way, in contrast with traditional control schemes, whose powerful methodological results, implemented with digital microcontrollers, barely succeed in maintaining efficiency, as the number of actuation joints increases. Once designed, the basic cell as well as the laplacian connections, able to show both of the phenomena described, the CPG was modeled by using CNNs and

deriving the corresponding templates [11]. After a design phase, a robustness analysis was also performed and the corresponding circuits were also realized. In fact the problem of locomotion control has been found to be realisable by employing a low number of cells; therefore the circuit realization is simple and affordable. Following this strategy a number of mechatronic structures were developed, whose locomotion was driven by CNN based circuits. Among the most interesting robots there are: WORMBOT: A Ring-WormLike Walking Robot In Nematodes, the key role of the generation of traveling muscle activation waves has been enhanced [23]. Therefore it is quite natural to model a simple CPG by using a CNN. The mechanical structure built is shown in Fig. 4(a). It depicts a mechatronic device emulating the structure of a ring worm with ten legs. The CNN generating autonomous waves is organized in a ring structure. It is allowed to drive some servomotors, each one moving a couple of legs, in a direction which goes from the back to the front part of the body. The propagation acts so as to push the body feedforwardly, onsetting the locomotion phenomenon, just as it happens, for a macroscopic and behavioral point of view, in nematodes. It is also possible to control the direction of locomotion by using the CNN cells’ input signals. They can be used to make the wave propagate asynchronously along the body, with respect to the legs belonging to the same robot side, making a change of direction during locomotion. The flexibility of the approach and the possibility to control the locomotion direction also in this simple example, make the approach particularly appealing.

REXABOT: The Hexapod Insect-Like Robot The autonomous six-legged insectlike robot prototype, shown in Fig. 4(b) is about 25 cm long, 20 cm tall and weights about 2.5 Kg. Each leg is moved by two servo motors: one drives the position of the leg foot, while the other one drives the rotation of the leg to realize the locomotion steps. The locomotion generation takes place in the circuits in the upper part of the robot. They are arranged to form a ring where a neural-like wave is generated. The wave front is used to drive a particular leg or a group of legs according to which locomotion pattern is selected. According to the paradigm of the CPG a particular group of neurons generates a given starting pattern, imposing a particular locomotion style, for example the alternating tripod, or fast gait. According to specific stimuli received from the environment, prespecified state variables of these neurons are forced to make the whole pattern change to another pattern, which can be associated with another motion

Figure 4 (b). The bioinspired robot prototype: REXABOT.

15

W

b ody e h t n ti hi

Figure 5. The LAMPBOT prototype.

style, for example slow gait or swim. Here a network of sensors arrives, and each sensor is able to force a particular state variable to move to a particular locomotion style. The most important concept is that all this strategy is implemented in an analog fashion, and the whole locomotion control can be therefore realized in real time. More details on the implementation of the control strategy can be found in [24]. The last version of REXABOT is a sixlegged robot with three joints per leg, where CNNs are used for an efficient and real time attitude control [25]. LAMPBOT: A Lamprey Inspired Swimming Machine The Lamprey is an eel-like fish that swims by rhythmic undulations of its body. It swims by progressively contracting its muscles via undulatory 16

motions from head to tail [26]. The swimming system as a whole represents a complex reaction-diffusion system, which has similar characteristics to a lot of biological cases, among which are the examples presented before. Considering a number of specifications the swimming model can be once again described by using a CNN array connected to form a ring to generate suitable signals to drive swimming in the lamprey robot prototype, shown in Fig. 5. It roughly consists of an aluminum backbone, made-up of four vertebrae connected with rods to realize the whole structure. Motion of the segment is realized by means of pneumatic valves which drive some pneumatic muscles (Fig. 5). In particular each segment is able to perform horizontal motion, since two muscles

work as a flexor-extensor couple. Each muscle is controlled directly by a pneumatic valve, whose driving signal comes from the state variable of a particular CNN cell, showing oscillatory dynamics, coupled by diffusion template in the ring already discussed. Each vertebra, which accommodates the valves, is also able to rotate, in order to allow spiral 3D motions that realize downward and upward swimming. In fact, while in the other robot prototypes the CNN structure was implemented on a 2D array, in this example 3D CNNs were used, where the basic cell dynamics are exactly the same as in the previous robots, but the diffusive laplacian couplings take place in the 3D space, in accordance with (3). Inside the Lampbot head the CNN cells for the swimming pattern generation are placed together with some other circuitry (Fig. 5) that drive the actuators, according to the CNN cell state variables dynamics. Finally the robot was covered with waterproof elastic material typically used by skin-divers. Undulatory motion along the lamprey body can be efficiently realized, as well as its speed control, by using the completely analog approach based on CNNs, as detailed in [27]. Analogic Cellular Computers— Topographic/Visual Microprocessors Analogic Computers combine analog spatial-temporal dynamics and logic. After the introduction of the CNN paradigm, CNN Technology got a boost when the analogic cellular computer architecture, the CNN-UM, had been invented [2]. The most successful chips [29, 30] embedded in a computational infrastructure [15] provided the framework for analogic cellular software development. The industrial applications now rely on the

The Lamprey is an eel-like fish that swims by rhythmic undulations of its body. . . .the lamprey robot prototype. . . consists of an aluminum backbone, made-up of four vertebrae connected with rods to realize the whole structure. Motion of the segment is realized by means of pneumatic valves which drive some pneumatic muscles . . .Each muscle is controlled directly by a pneumatic valve, whose driving signal comes from the state variable of a particular CNN cell. . .Each vertebra, which accommodates the valves, is also able to rotate, in order to allow spiral 3D motions that realize downward and upward swimming. available ALADDIN system [10, 28]. In this short summary we briefly show the computational infrastructure, a typical application, and the biology relevance. More than a thousand references can be found on the web site: http://lab.analogic.sztaki.hu, or http:// www.ieee-cas.org/~cnnactc. The Analogic CNN Engine (ACE) CNN paradigm [1, 2, 3] related research, among many other results, led to the development of a number of analogic array processor chips (e.g. [29, 30]), based on the CNN-UM concept already presented. Currently the most advanced CNN chip generation is represented by a fully programmable 128 x 128 analog array processor, recently designed in Seville [31], implemented by STMicroelectronics technology and currently in the testing phase. The previous chip has instead 17

Figure 6(a). The industrial version of the Aladdin Visual Computer.

Figure 6(b). The desktop version of the Aladdin Visual Computer.

been fully tested. It is the so-called ACE4k [30], which is constructed of an array of 64 x 64 analog processor cells. The 4096 cell processors work in parallel. The overall computational performance of the chip provides the possibility of processing 64 x 64 sized images up to 10,000 frames/second

18

including I/O. Each cell in the array processor corresponds to one pixel in an image. If the image is larger than 64 x 64, it is processed tile-by-tile and then merged together again. The most important feature of the analog-andlogic (analogic) processor array is that it is stored programmable. Its operation set is wide. It consists of dynamic analog array operations, and local logic operations to grayscale or binary images. Moreover it can store 8 pieces of 64 x 64 images. Its input-output speed reaches the minimal requirements of video processing. The instruction and I/O accuracy of the chip is 7–8 bits, which is enough for many applications. (Note that this is not equivalent to the digital accuracy, here there is no iteration in time!) These advantages of the ACE4k chip encouraged the building of the Aladdin Visual

Computer (Fig. 6(a)), which is the first System Architecture Overview high performance industrial quality image processing device utilizing celThe block diagram of the present lular neural networks. The Aladdin architecture of the Aladdin Visual Visual Computer is based on the Computer can be seen in Fig. 7. The ACE4k Visual Microprocessor and a hosting PC, running under Windows high performance Digital Signal Pro- NT or Windows 2000 systems, can be cessor (DSP). A DSP is required for either a desktop or an industrial PC. As two reasons. First, the ACE4k chip the diagram shows, the Aladdin Visual does not contain the digital Global Computer is connected to the Analogic Control Unit (GACU) of the motherboard of the host PC and the CNN Universal Machine; hence a DSP frame grabber via the PCI bus. The provides for this function. On the other PCI bus interface provides for fast hand, the DSP also plays another im- image data transfer. Images may come portant role, when a sophisticated image processing task Figure 7. The block diagram of the Aladdin Visual Computer. contains some digital camera global operations. In some other cases, after the Hosting industrial Aladdin Visual computationally or desktop PC Computer Stack heavy pre-processing phase ACE4K FrameDSP Pentium class (including nuplatform grabber module PC motherboard merous convolucard tion-type local image processing operations) is acPCI bus Platform bus complished by the ACE4k, the DSP will complete the remaining part of the task. In from the frame grabber or the hardthis way, the ACE4k picks up that 1% drive. The output images can be disof the image which is relevant; and the played on the monitor of the PC. HowDSP has to work only on this greatly ever, in many cases, (e.g. industrial imreduced data set. These two high per- age processing or in quality control apformance complementary processors plications) only decisions or a couple are integrated to accomplish an ex- of measured parameters constitute the tremely powerful vision system, which output of the system. makes possible real time image evaluSoftware ation in even extremely demanding applications. The Aladdin Visual ComThe Visual Computer can be apputer has two versions (Fig. 6(b)). It plied in two modes. The first applies can be plugged into either a desktop when a high-speed external camera is PC, or a PC-104 plus industrial PC. In connected to the system via a PCI a next version of the ACE chip, the frame grabber. In this mode, ultra high DSP-like GACU will be integrated on frame rate (up to 10, 000) can be a single chip CNN-UM. reached with low resolution (64 x 64– 19

Cellular Neural Networks: A Paradigm for Nonlinear Spatio-Temporal Processing 20

256 x 256) images. The system in this mode can be applied as a visual trigger or a high-speed visual event detector. Moreover, the system can testify extremely high-speed events, like flashes of a spark-plug, or can make shape analysis of a rapidly moving object, like a pill, or a grain. In the second application mode, the system processes video image flows, coming from a single camera or a number of different cameras, in real-time. Due to the high computational power of the system, it can process whole frames, not just a small region of interest (like the digital image processing systems do). This makes possible surface quality control even in complex surface patterns. Another typical application in this mode is multi-modal image fusion, which can be used in a number of security or traffic safety applications. Conclusions In this paper an overview of Cellular Neural Networks has been presented, together with some of the most appealing applications that enhance the role of these structures as an analogic computer paradigm for nonlinear image spatial filtering, image flow processing, spatio-temporal system modeling, and space-distributed structures control. Moreover, a high performance CNN based analogic computer has been presented, to implement in real time most of the CNN algorithms already designed and available in the literature. The actual hardware is configured in order to use the CNN chip as a co-processor, controlled by a DSP processor. A great effort is being made to improve the already complicated chip structure in order to make the forthcoming CNNUM chips active independent processors able to show also efficient control capabilities. This will increase the al-

ready wide field of applications where the real time aspect of spatio-temporal processing is required. Acknowledgements The authors wish to thank Prof. T. Roska, Faculty of Information Technology, Pázmány University, Budapest, for his helpful suggestions. This work was partially supported by the European Project ESPRIT IST 1999–19007, “Dynamic Image Computing Using Tera-speed Analogic Visual Microprocessors”, (DICTAM). References [1] L. O. Chua and L. Yang, “Cellular Neural Networks: Theory and Application”, IEEE Transactions on Circuits and Systems, vol. 35, pp. 1257–1290, 1988. [2] T. Roska and L. O. Chua, “The CNN Universal Machine: An Analogic Array Computer”, IEEE Transactions on Circuits and Systems—II, vol. 40, no. 3, pp. 163–173, March 1993. [3] L. O. Chua and T. Roska, “The CNN Paradigm”, IEEE Transactions on Circuits and Systems—I, vol. 40, no. 3, pp. 147–156, March 1993. [4] S. Wolfram, “Cellular Automata as Models of Complexity”, Nature, vol. 311, pp. 419–424, October 4, 1984. [5] J. J. Hopfield, “Neural Networks and Physical Systems with Emergent Computational Capabilities”, Proceedings of the National Academy of Sciences of the United States of America, vol. 79, pp.2554–2558, 1982. [6] C. Mead, Analog VLSI and Neural Systems. Reading, MA: Addison-Wesley, 1989. [7] L. O.Chua and T. Roska, Cellular Neural Networks and Visual Computing. Cambridge: Cambridge University Press, (in print: 2001). [8] P. P. Civalleri and M. Gilli, “On Stability of Cellular Neural Networks”, Journal of VLSI Signal Processing, vol. 23, pp. 429– 435, 1999. [9] “CNN Software Library, Ver1.1”, Analogical and Neural Computing Laboratory, Budapest, 2000. [10] T. Roska, “Computer-Sensors: SpatialTemporal Computers for Analog Array

Signals, Dynamically Integrated with Sensors”, Journal of VLSI Signal Processing, Special Issue: Spatiotemporal Signal Processing with Analogic CNN Visual Microprocessors, vol.23. pp. 221–238, Kluwer, 1999. [11] G. Manganaro, P. Arena, and L. Fortuna, Cellular Neural Networks: Chaos, Complexity and VLSI processing. SpringerVerlag, 1999. [12] T. Roska and A. Rodríguez-Vázquez (eds.), Toward the Visual MicroprocessorVLSI Design and the Use of Cellular Neural Network (CNN) Universal Machine Computers. London: J. Wiley, (in print: 2001). [13] “Supplement”, Nature Genetics, vol. 21, January 1999. [14] P. Arena, L. Fortuna, and L. Occhipinti, “A CNN Algorithm for Real Time Analysis of DNA Microarrays”, IEEE Transaction on Circuits and Systems, Part I: Fundamental Theory and Applications, in press. [15] T. Roska, Á. Zarándy, S. Zöld, P. Földesy, and P. Szolgay, “The Computational Infrastructure of Analogic CNN Computing—Part I: The CNN-UM Chip Prototyping System”, IEEE Transactions on Circuits and Systems—I: Special Issue on Bio-Inspired Processors and Cellular Neural Networks for Vision, vol. 46, no. 2, pp. 261–268, 1999. [16] T. Roska, J. Hámori, E. Lábos, K. Lotz, L. Orzo, J. Takács, P. Venetianer, Z. Vidnyánszky, and A.Zarándy, “The Use of CNN Models in the Subcortical Visual Pathway”, IEEE Transactions on Circuits and Systems, Part I: Fundamental Theory and Applications, vol. 40, no. 3, pp.182– 195, 1993. [17] B. Roska and F. S.Werblin, “Vertical Interactions across Ten Parallel, Stacked Representations in the Mammalian Retina”, Nature, vol. 410, pp. 583–587, March 29, 2001. [18] D. Bálya, B. Roska, T. Roska, and F. S. Werblin, “A CNN Framework for Modeling Parallel Processing in a Mammalian Retina”, International Journal on Circuit Theory and Applications, vol. 30, no. 2, 2002 (in press). [19] L. O. Chua, “CNN: a Paradigm for Complexity”, World Scientific, 1998. [20] G. K. Pearson, “Common Principles of Motor Control in Vertebrates and Invertebrates”, Annual Review of Neuro-

science, vol.16, pp. 265–97, 1993. [21] V. I. Krinsky, “Autowaves: Results, Problems, Outlooks”, in Self-Organization: Autowaves and Structures Far from Equilibrium. Berlin: Springer-Verlag, pp. 9– 18, 1984. [22] J. D. Murray, Mathematical Biology. Berlin: Springer-Verlag, 1989. [23] E. Niebur and P. Erdös, “Theory of the Locomotion of Nematodes: Control of the Somatic Motor Neurons by Interneurons”, Mathematical Biosciences, vol 118, no. 1, pp. 51–82, November 1993. [24] P. Arena, L. Fortuna, and M. Branciforte, “Realization of a Reaction-Diffusion CNN Algorithm for Locomotion Control in an Hexapode Robot”, Journal of VLSI Signal Processing, vol. 23, pp. 267–280, 1999. [25] P. Arena, L. Fortuna. and M. Frasca “Attitude Control in Walking Hexapod Robots: An Analogic Spatio-Temporal Approach”, International Journal on Circuit Theory and Applications, 2002 (in press). [26] S. Grillner, P. Wallen, L. Brodin, and A. Lansner, “Neuronal Networks Generating Locomotor Behavior in Lamprey”, Annual Review of Neuroscience, vol. 14, pp. 169–200, 1991. [27] P. Arena, “A Mechatronic Lamprey Controlled by Analog Circuits”, Proceedings 9th IEEE Mediterranean Conference on Control and Automation (MED’01), Dubrovnik, Croatia, June 27–29, 2001. [28] The ALADDIN System (http:// www.analogic-computers.com). [29] R. Dominguez-Castro, Servando Espejo, A. Rodríguez-Vázques, R. Carmona, P. Földesy, Á. Zarándy, P. Szolgay, T. Szirányi, and T. Roska, “A 0.8 mm CMOS Two-Dimensional Programmable Mixed-Signal Focal-Plane Array Processor with On-Chip Binary Imaging and Instructions Storage”, Vision Chip with Local Logic and Image Memory, IEEE Journal of Solid State Circuits, vol. 32, no. 7, pp. 1013–1026, 1997 [30] S. Espejo, R. Domínguez-Castro, G. Liñán, and Á. Rodriguez-Vázquez, “A 64 x 64 CNN Universal Chip with Analog and Digital I/O”, Proceedings 5th International Conference on Electronics, Circuits and Systems (ICECS-98), Lisbon, Portugal, pp. 203–206, 1998. [31] Instituto de Microelectrónica de Sevilla, SPAIN, (http://www.imse.cnm.es).

Luigi Fortuna

Paolo Arena

David Bálya

Ákos Zarándy

21