CUMULUS - A Scalable Systolic Array for Binary

CUMULUS A Scalable Systolic Array for Binary Pattern Recognition and Networks of Associative Memories LIX | Technical Report

Martin Neschen Laboratoire d'Informatique, Ecole polytechnique F-91128 Palaiseau Cedex, France phone : + 33 1 69 33 34 79 fax : + 33 1 69 33 30 14 email :

Abstract Classi cation of binary patterns can be done highly eciently in a bit-sequential way using asynchronous counters for the determination of the Hamming distance between two patterns. As each pattern has to be compared with a large number of database prototypes, this inherent SIMD parallelism can be easily exploited on an array of counters which are connected to an array of memory units containing the database. The basic element, an asynchronous counter cell, is very simple and compact, so that hundreds of them, working at about 100 MHz, can be integrated on a single chip which can deliver a performance equivalent to some hundred SPARC-10 processors for that application. In order to overcome the bottleneck of the data transfer between o-chip memory and the counter array, we propose a two-dimensional M  N -array of asynchronous counters being able to determine in parallel all distances between N incoming patterns and M database prototypes. This scalable \SIMDMD" architecture requires only M + N fast data connections for MN counter cells. The bit-sequential organization facilitates addressing of single bits and allows for the simulation of very large networks of associative memories. Further interesting applications exist, like the simulation of dynamical neural networks.

Introduction Binary pattern recognition and classi cation are highly relevant problems with many interesting applications like the recognition of handwritten digits and ngerprints. Methods which perform simple pattern comparisons by calculating the Hamming distance have been proposed as a viable alternative to heavy algorithms like back-propagation neural networks for generalization or Hop eld networks for recognition. Although such direct algorithms, like KNN, the \k-nearest-neighbor" search, do require a higher number of operations than feed-forward neural networks, the overall computing requirements may be lower because only simple operations like bit counting are needed instead of more powerful commands such as oating-point multiplications. On general-purpose computers, however, those single-bit operations are not well adapted to the rather wide data word. Therefore, the development of a special-purpose bit-accumulation component may become an interesting alternative for the implementation of such simplistic algorithms. In typical applications, many comparisons can be performed 1

2 simultaneously, which allows the development of a massively parallel bit-oriented hardware. The aim of this paper is to de ne a new, rather simple architecture, CUMULUS, which is especially designed for comparisons of binary patterns. This approach is currently being elaborated in our laboratory, both by designing a VLSI chip and by elaborating a software for the simulation of possible applications. Here, we want to give the fundamental informations which are essential for further discussions about possible applications, algorithms, implementations and architectural aspects. In section 1, we discuss the elementary cell, an asynchronous counter which is highly optimized both for speed and density. In section 2, we describe the structure of a chip integrating a two-dimensional array of these counting elements which also allows for the evaluation of minima in parallel for each column. Section 3 describes how this component can be employed in a system specialized for binary treatment. In section 4, we present the primary application, the recognition of binary patterns, and give results from previous simulations. In section 5, we present further applications, including the simulation of dynamical neural networks. The CUMULUS architecture is especially well suited for the simulation of very large networks of associative memories. Section 6, nally, presents the basic ideas of this approach and discusses the implementation on the CUMULUS array.

1 Asynchronous counting Due to the large data word size, general-purpose computers are not well adapted to the accumulation of many single bit values. This type of calculation is required, for example, for the evaluation of the Hamming distance  or the binary product  between two L-bit vectors si and ti, (i = 1; :::; L):

=

L X k=1

sk  t k ;

=

L X k=1

sk tk :

(1)

As an alternative, an asynchronous counter, which is essentially realized as W cascaded d-type ip ops, can be used as a standard hardware realization for a bit-sequential binary comparison. Although, for this approach, it may take a longer time than for synchronous devices before the result can be read out, the maximal counting frequency, fmax , is higher, as it is limited only by the \toggle" frequency of the D- ip ops, whereas in synchronous counters, additional logic for the propagation of carry bits is needed. Figure 1 shows a realization which allows for both operations of eq. ( 1 ). For the calculation of  , a pulse has to be generated for TCLK0 if tk = 1 and for TCLK1 if tk = 0. For the determination of  , a pulse has to be generated only for TCLK1 and only if tk = 1. The inputs DINS and DINS must hold a stable value of sk during these pulses.

Figure 1: Asynchronous counter element

The main advantages of this bit-sequential, asynchronous approach are:

 The vector length L is exible and may take any value between 1 and 2W ? 1 because the values    

are presented serially. The maximal frequency of the counter, fmax , is only limited by the toggle frequency of a D- ip op, because no additional logic like carry propagation is necessary. The regular and linear approach facilitates a very dense linear layout in a single row of standard cells. This structure makes a parallelization as an array easy, as commands can be routed in vertical channels over the cells and thus need no additional space. Full-custom developments are made easy due to the regular structure. Registers for higher positions may be realized with transistors of minimal size, as the frequencies are very low. Power consumption is considerably lower than in synchronous structures. Only the ip ops of the lowest positions are working at the highest speed, and each stage divides the frequency by 2. For random patterns, on the average, the total dissipation corresponds to an equivalent of one

ip op working at fmax . This issue becomes very important as the compact integration allows a very large number of fast counters on a small area.

In a bit-serial approach, one needs to serially read out the resulting W bit counter value. This may be done by multiplexing the W register values on an internal tri-state bus. Alternatively, one may use the counter ip ops as a \shift register" and thereby avoid fan-out problems for large W and reduce the counter size at the same time. Although, in an asynchronous counter, any clock input is connected to the output of the precedent stage, an unusual kind of shifting is possible using only the W independent reset signals RES,...,RES. A reset on position k will propagate the value of REG to REG if the latter was formerly reset to 0. One can therefore \bubble through" zeroes, as illustrated in g. 2 for a counter of width W = 4. T 0 1 2 3 4 5 6

REG

c0 c0 c0 c0 0 0 0

REG

c1 c1 c1 0 c0 0 0

REG

c2 c2 0 c1 0 c0 0

REG

c3 0 c2 0 c1 0 c0

Figure 2: Shifting out the result (c0 c1 c2 c3 ) in a counter of width W = 4 Although optimizations like this may seem negligible, any reduction of the size directly results in an increased performance due to an even higher parallelization on a given surface. Using the double-metal CMOS technology of ES2, one can route the counter in a single row of standard cells with only 3 horizontal routing channels. This line can already include an additional logic for the determination of a global minimum in collaboration with other counters. In g. 3, we present results for such a counter of size W = 12. Figure 4 shows a layout of such a counter cell. The maximal frequency, indicated in this table, is determined by the pipelined control

4 logic for the signals TCLK0, TCLK1 and RES and not by the counter cell itself which is limited only by the toggle frequency. When the counters are driven at fmax , the maximal supply current lies between about 0.6 mA for 1:2m and about 0.9 mA for 1:0m. technology fmax size counters/mm2 ECPD15, ES2, 1:5m 70 MHz 1400m  92:8m 7.7 ECPD12, ES2, 1:2m 70 MHz 1050m  69:6m 13.7 ECPD10, ES2, 1:0m  110 MHz 875m  58m 19.7 Figure 3: Size of a counter cell for W = 12 in dierent technologies aa aaaaaaa aa aa aa aaa aaaaaa a aa a aa aaa aaa aaaaaaaaaaaaaaa aaaaaaaaaaaaa aa aa aa aa aa aa aa a a aa aa aa aa aa aa aa aa aa aaa a aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a a aa aa aa aa aa aa aa aa aa aa aaa aa aa aaa aaa aaa aaaaaaaaaaaaaaa aa a aa aa aaaaaaaaaaaaaaa a a aa aa aaaaaaa a aa aa aa aa aa aaaaaaaaaa aa aa aa aa aa aa a aa aa aa aa aa aa aa aa aa a a aa aa aa aa aa a a a aa aa aa a aa a a aaa a a aa aa aaa aa aaa aa aaa aa a a aaa aaa a a aa a a a a a a a a aa a a a aaaa a aa a a a a aa aaa aa aaa aa aa aaaa aaa aa aa aa aa aa aa aa aa aa aa aaa aa aaa aa aa aaa aa aaa aa aaa aaa aa aa aa aaaa aaaaaaa a aa aa aa aa aa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aaa aa aa aa a aaa aa aa a aa aa aa aa a a a a a aaa a a a a a a a aa a a a a aa aaaaa aa a a aaaa a a a a a a a a a a aaa aaa aa aaa aa aaa aa aaa aa aaa aaa aaa aa aa aa aa aaa aa aa aa a a aa a aa a aa aaaaa a a aa aa a a aaaaaaaaaaaa aa a a a a a aa aa aa aa aa aa aa aa aa aa aaaa a aaaa aa aa aa aa aa a aa a a a a aa aa a a a a a a a aa a a aa aa a a aa aaa a a a a a a a a a a aa a a a aa aa aa a a a aaa aa aaaaa aa a a a a a a aa a a a aa a a a a a aa aa aa aa a a a a aa aa aa aa aa a a a a a aa aa a a a a a a a a a aa a aa a a aa a aa a a aa a aa a a a a a a aa a aa aa aa aa aa aa aa aa aa a a a a aa aa a a a a a a aa a a a a aa a a aa aa a a a a a a a a aa a aa a a aa a a a a a a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aaa aa a a aa aa a a a a a a a a aa aa aa a a aaa a a a aa a a a a aa aa aa a a aa aa aa a a a aa a a a a a aa aa aa aa aa aa aa aa aa aa a a a aa aa aa aa aa aa aa aa aa a a aa aa a a a a a a a a aa aa aa aa a a aa a a a a a a a a a a a a a a a a a a a a a aa a a aa a a a a a a a a a a a aa a a a a a aa a aa a aa a a a a a a aa aa aa aa aa aa a aa a a aa aa a a a a a a a a aa aa aa a a aa a a a a a a a a aa a aa a a a aa a a aa a a aa a a aa a a aa a a a a a a a a a aa aa a aa a aa a a a a aaa a aaa aa aa aa aa aa a a a a a a a a a a a a a a a a a a a aa a a a a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa a a aa aa a a a a a a a aaa aa aa aa aa aa a aa aa a a aa a a a aa aa aa a a aa a a a a aaa a aaa aa aa aa a a a a a aa a aa aa a a a aa a aa a aa aa aa a aa aa aa aa aa aa aa a aa a a aa aa a a a a a a a a aa aa a aa a a aa a a a aa a aa aa a a a a a a a aa a a a a a aaa a a a a a a a a a aa a a a a a aa a aa a a a a aa aa aa aa aa aa aa a aa a aa aa a a a a a a a a aa a aa aa a a aa a a a a a a a a aaa a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa a a aa aa a a a a a a a aaa aa aa aa aa aa a aa aa a aa a a a a aaa a aaa aa aa aa a a a a aa aa a a a aa a aa a aa aa aa a a aa aa aa aa aa aa aa a aa a a aa aa a a a a a a a a aa aa aa aa a a aa a a a a a a a a aa a a a a a a a a a a aa aaa aaa aa aaa aa aa a aa aa a a a a a a a a a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa a a aa aa a a a a a a a aa aa aa aa a aa a aa aa a a a a a a aa aa aa aa aa aa a a aa a a aa aa a a a a a a a aa aa aa aa a aa a a aa a a a a a a a a a a aa aa aa aa aa aa a a aa a aa aa a a a a a a a a a aa aa a aa a a a a a a a a a a a a a a a a aaa aa aa aa aa aa aa aa aa aa aa a a aa aa a a a a a a a aa aa aa aa a aa a a a a aa aa a a aa aa aa aa aa aa a a aa a a aa aa a a a a a a a aa aa aa aa a aa a a a a a a a a a aa a a aa aa aa aa aa aa a a aa a aa aa a a a a a a a a a aa aa a aa a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa a a aa aa a a a a a a a aa aa aa aa a aa a aa aa a a a a a a a a aa aa aa aa aa aa a aa a aa aa a

aa aa aaa aaa a a a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a a a a a a a aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a aa aa

a a a aa a a a a a aa aaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa a aaaa aa aa aaaaaaaaaaaaaaaaaaa a a a a aa a a aa a aa aa a a a a a aa aa aa aaaa aa a a aa a a a a a a a aa aa aa aa aa aa aa a a a a a a aa a a a a a a a a a a a a a a a a aa a a aa a aa aa aa a a a a a aa a a a a a a a a a a a a aa a aa aa a a a aa a a aa a aa aa a a a a a a a aa a a a a a a a a a a a a a a a a a a a a a a a aa a a aa a aa aa a a a a a aa a a a a a aaa aa a a aa a a a a a a a a a a aa a a a a a a a a a a aa a a a a

aa aa aaa aaa a a a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a a a a a a a aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a aa aa

a a a aa a a a a a aa aaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa aa a aaa aa aa aaaaaaaaaaaaaaaaaaa a a a aa a a a aa aa a a a a a aa aa aaaaaaa aaa a a a a a a a a a aa aa aa aa aa aa aa a a a a aa aa a a aa aa a a a a a a a a a a a a aa a a a aa aa aa a a a a a a a a a a a a a a a a a a aa aa a a a aa a a a aa aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a aa aa a a a a a a a a a a a aa aa a a aa a a a a a a a a a a aa a a a a a a a a a a aa a a a a

aa aa aa aa aa aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a a a a a a a aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a aa aa

a a a aa a a a a aa aaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa aa a aaa aa aaaaaaaaaaaaaaaaaaaa a a a a aa a a a aa aa a a a a a a a a a a aa aa aa aa aa aa aa aa a a a a a a a a aa a a a a a a a a a a a aa a aa aa aa aa aa aa a a a a aa a a a aa aa aa a a a a a a a a a a a a a a a a a a aa aa a a a aa a a a aa aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a aa aa a a a a a a a a a a a aa aa a a aa a a a a a a a a a a aa a a a a a a a a a a aa a a a a

aa aa aa aa aaa a a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a a a a a a a aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a aa aa

a aa a aa aa a a a a a a a aa aa aaaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa a aaa aa aaaaaaaaaaaaaaaaaaaa a a aa aa a a a aa aa a a a a aa a aaaa aa aaaaa aa a a a a a a a a a aa aa aa aa aa aa aa aa a a a a aa aa a a a a a a a a a a a a a a a aa aa a a a aa aa aa aa a a a a a a a a a aa a a a a a a a a a aa aa a a a aa aa a a a aa aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa a a a aa aa a a a a aa a a a a a a a aa aa a a a aa aa a a a aa a a a a a a a a a aa aa a a a aa a a a a a a a a a aa aa a a a aa

aa aa aaa aaa a a a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a a a a a a a aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a aa aa

a a a aa a a a a a aa aaaaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa a aaaa aa aaaaaaaaaaaaaaaaaaaa a a a aa a a aa a aa aa a a a a a aaaaa aa aaaa aa a a aa a a a a a a a aa aa aa aa aa aa aa aa a a a a aa a a a a aa a a a a a a a a a a a a aa a a aa a aa aa aa a a a a a aa a a a a a a a a a a a a aa a aa aa a a a aa a a aa a aa aa a a a a a a a aa a a a a a a a a a a a a a a a a a a a a a a a aa a a aa a aa aa a a a a a aa a a a a a aaa aa a a aa a a a a a a a a a a aa a a a a a a a a a a aa a a a a

aa aa aaa aaa a a a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a a a a a a a aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a aa aa

a a a aa a a a a aa aaaaaaaaaaaaaaaaaaaa a aa aa aa aa aa aa aa aa aa aa a aaa aa aaaaaaaaaaaaaaaaaaaa a a a aa a a a aa aa a a a a a a a a a a aa aa aa aa aa aa a a a a aa aa a a aa aa a a a a a a a a a a a a a aa a aa aa a a a a aa aa a a a a a aa a a a aa aa aa a a a a a a a a a a a a a a a a a a aa aa a a a aa a a a aa aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a aa aa a a a a a a a a a a a aa aa a a aa a a a a a a a a a a aa a a a a a a a a a a aa a a a a

aa aa aa aa aa a aaa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a a a a a a a aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a aa aa

a a a aa a a a a a aa aaaaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa aa a aaa aa aaaaaaaaaaaaaaaaaaaa a a aa a a a aa aa a a a a a aaaaaaa aa aa aa a a a a a a a a a aa aa aa aa aa aa aa aa aa a a a a a a a a aa a a a a a a a a a a aa a a a aa aa aa a a a a a a a a a a a a a a a a a a aa aa a a a aa a a a aa aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a aa aa a a a a a a a a a a a aa aa a a aa a a a a a a a a a a aa a a a a a a a a a a aa a a a a

aa aa aa aa aaa a a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a a a a a a a aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a aa aa

a aa a aa aa a a a a a a aa aa aaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa a a aaa aa aa aaaaaaaaaaaaaaaaaa a a a aa aa a a a aa aa a a a a a a a aaaa aa aa aa aa a a a a a a a a a a aa aa aa aa aa aa aa aa a a a a a a a a a a a a a a a a a a a a a aa aa a a a aa aa aa aa a a a a a a a a a aa a a a a a a a a a aa aa a a a aa aa a a a aa aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa a a a aa aa a a a a aa a a a a a a a aa aa a a a aa aa a a a aa a a a a a a a a a aa aa a a a aa a a a a a a a a a aa aa a a a aa

aa aa aaa aaa a a a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a a a a a a a aa aa a a a a aa aa a a a a a a a a a a a a a a a a a a a a aa aa a a a a a a aa aa

aa aa aa aaaaa aa aa aaaaa aa aa aaaa aa aa aaa aaa a aaa a a aa aaa a a a a a a a a aa aa aa aa a aa aa a a a a a aa aa aa a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a aa aa aa aa aa aa aa aa aa aa aa aa aa a aaa aaa aa aaa aaa aa aa aaa a aa aaa aaa aaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaa aa aa aa a a a aaaaaaaa aa aaaaaaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aaaaa aa aa aa aa a a a a a a a a a aa aa aa aa aa aa aa a aa aa aa aa aa aa aa aa a aa aa a a a aa aaaa aa aa aa a aaaaaaa aa aa aa aa aaaa aa aa aa aa aa aa aa aa aa aa aa aa aaaaaa aa aa aa aa aa aa aa aaa aaa aa aa aa aa aa aa a aa aa aa aa aa aa a a a a a a a a a a a aa aa aa aa aa aa a aa aa aa aa aa aa aa a aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aaa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a aa aa aa aa a a a a a a a a a aa a a a a aaaa aaaaaaa aa aa aaa aa aa aa aa aa aaaa a a a a aa a a a aa aa a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aaaaa a a a a a a a a a a aa aaa aa aaa aaa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a a aa aa aaa aaa aa aa aaa aaaaaa aa a a a a aa a aaaaaaa a aa aa aa aaa aa aaaa aaa a a a a a aaaaaaaaaaaa aaaaaaaaa aaaaaaaaaaaaa aaaaaaaa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aaaaaa aaa aaa aaa aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa aa a a a a a aa a a a aa a aa a a a a a a a a a a a a aa a a a aa a a a a aa a a a a a a a a a a a a a a aa a a a a a a a a aa a aa a a aa a a a a a a a a a a a a a a a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a a a a a a a a a aa a a a a a a a aa aa a a a a a a a a a a aa aa a a a a a a a a a a a a a a a a a a aa a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa aa aa a aa aaa a a a a a aa aa a aa a a aa aa a a a aa a a a a aa aa aa a a aa a a a a a a a a a a a a a a aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a a a a aa a a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a a aa aa a a a a aa aa a a a aa a a a a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa aa a a a a aa a aa a a a a a a a a a a a a a a a aa a a a a a a a a aa a aa a a a aa a a a a a a a a a aa aa aa a aa aa aa a a a aa aa aa a a a a a aa aa a a aa aa aa a a a a aa a a a a a a a a a a a a a a a aa a a a a a a a a a a a a a a a aa a aa a a a a a a a a a a a a a a aaaaa a aa a a a a a aa a a a a a a a aa aa aa aa aa a aa aa a aa aa a aa aa aa aaa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a a aa aa a a a a aa aa a aa aaaaa aa a a aa a a a a aa a a a a aa a a a a a a a aa aa aa aa aa a aa aa a aa aa a aa aa aa aa aa a a a a a a a aa aaaaa a aa aa a a a a aa a a a a a a a a a a a aa a aa a aa a a a a aa aa aa aa aa aa a a aa aa a a a a a a aa aa a a aa aa a a aa a a a a a a a aaaa aa a aa a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a a a a a a a a aa aa a a a aa a a a a a a a a a aa a a a aa a a a a a aa a aa aa a a a a a a a a a a a aa aa aa a aa aa a a aa aa aa a a a a aa aa a a aa aa a a a a a a a a a a a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a a aa aa a a a a aa aa a aa aaaaa aa a a aa a a a a aaaa a a a a aa a a a a a a a aa aa aa aa aa a aa aa a aa aa a aa aa aa a a a a aa aaaaa a aa aa a a a a aa a a a a a a a a aaa a a aa a aa a aa a a aa aa aa aa aa aa a a aa aa aa a a a a a aa aa a a aa aa a a aa a a a a aa a a aaaa a a a a a a a a a a a a a a a a a a a a a a aa a a aa a a aa a a a a a a a aa aaaaa a aa a aa aa aa aa aa aa a aa aa aa aa aa aa aa aaa a a a a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a a aa aa a a a a aa aa a aa aa a aa a a a a a a a a a a a a a a a a aa a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a a aa aa a a a a aa aa a a aa a a a a a a a a a a a a a a a aa aa aa a aa aa aa a a a aa aa aa a a a a a aa aa a a a a aa aa a a aa a a a a a a a a a a a a a a a a a a a a a aa aa aa a aa aa aa a a a aa aa aa a a a a aa aa a a a aa aa a a a a a a a a a a a a a a a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a a a a aa aa a a a a aa aa a a aa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aa aa a a a a a aa aa a a a a a a a a a a a a

Figure 4: Layout of the asynchronous counter for W=12

2 The CUMULUS array As the counter cell presented above has been optimized for size, as much as 1024 may be easily integrated on the same chip. This arises the question how the serial binary data streams sk and tk can be provided for each cell on the chip. As applications necessitate the comparison of one pattern with many prototypes, one may use the same stream t and perform comparisons with a dierent stream si for each counter. However, this one-dimensional array of counters still necessitates 1024 high-speed connections to a large memory. As on-chip memory is not large enough for most applications, we have to provide 1024 pins, which is not practicable today. In general, we therefore look for an architecture which is scalable and minimizes the number of memory connections. For an array of M  N counting elements with a boundary of 2M + 2N we cannot provide O(M  N ) connections, but only O(M + N ). This leads us to the 2-dimensional array structure of the chip which is depicted in gure 5. This architecture allows a simultaneous comparison of each of M row bit streams sik , (i = 1; :::; M ) with each of the N column bit streams tjk , (j = 1; :::; N and k = 1; :::; L):

ij =

L X k=1

sik  tjk ;

or ij =

L X k=1

sik  tjk ;

(2)

Furthermore, after the accumulation phase, each column can determine independently the minimum of the counter values cij using a special logic within each counter cell and a column-wide \or":

Cj = i=1min c ;:::;M ij

(3)

This is also done in a bit-sequential way, starting from the highest position Cj(W ?1) down to the lowest position Cj(0). Here and further on, a high index in parentheses shall notify the binary position, for example:

aa aaa aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a a a a a a a a a a a a a a a a a a aa a a a aa aaa aa aa a aa a a a aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aaaa a a aa aa aa a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a aa aaaa aa aaa aaa aaa aaa aaa aaa aaa aaa aa aa a a a a a a a a a a a

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aa aa a aa a a a aa aa a aa a a a aa aaa a a aa aa a aa a aa aaa aa aa a aa a a a aa aa a aa a a a aa aaa a aa aaa aa aa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa a a aa a

aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a aaa a a a a a a a a aaa a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a a a aa aa aa aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a a a aa aa aa aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a a a a aa aaa aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a aaaaaaaaaaaaaaaaaaaa aa aa aa a aaaaaaaaaaaaaaaaaa a a aa aaaaaaaaaaaaaaaaaaaa aaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa a aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa a a a a aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aa a aa a a a aa aaa aa aa a aa a a aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a aa a a a a a a a a a a a a a a a a a a a a aa

aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aa aaaaaaaaaaaaaaaaaaaaa a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aaaaaaaaaaaaaaaaaaa aa aaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aaa aa a aa aa

aaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aa aa aa aa a a a a a aaa aaa a a aa aa aa aa a a aaa aaa a a a a aa aa aa aa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a aa aa aa aa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a a aa aaa aa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a a a aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aa aa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aa aa aa aa a aaa aaa a a aa aa aa aa a a aaa aaa a a a a aa aa aa aa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a aa aa aa aa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a a a aaa aaa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a a a aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a aaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aa a a a a aa aaa a a aa aa aa a a aa aaa a a a a aa aa aa aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a a a aa aa aa aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a a a aa aa aa aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a a a a aa aaa aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a a aa aaa aa a a a a a a a a a a a a a a a a a a a aa a aa aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa

aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa a aa aa a a a a a a a a a a a a a a a a a a a a aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aaaa aa

aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa a a a a a a a a a a a a a a a a a a a a a a aa aa aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a aa aa aa a a a a aa aa aa a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa

aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a aaaa a a a a a a a a a a a a a a a a a a aaa aaa aaa a a aa aa a aa aa a aaa aaa a a a a aa aa aa aa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a a a aa aa aa aa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a a a aa aa aa aa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a a a aaa aaa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a a a aaa aaa aaa aaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a aaa a a a a a a a a a a a a a a a a a a a aaa aaa aaa a a aa aa a aa aa a aaa aaa a a a a aa aa aa aa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a a a aa aa aa aa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a a a aa aa aa aa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a a a aaa aaa aaa aaa a a aa aa aa aa a a a a aa aa aa aa a a aaa aaa a a a a a a aaa aaa aaa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a aa a a a a a a a a a a a a a a a a a a a aaa aa aaa a a aa a aa aa a aa aaa a a a a aa aa aa a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a a a aa aa aa a a aa aa aa a a aa aaa a a aa aa aa a a aa aaa a a a a aa aa aa aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a a a a aa aaa aa aaa a a aa aa aa a a a a aa aa aa a a aa aaa a a a a a aa aaa aa aaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa a aa aa a aaa aa a aa a a aa aa a aaa aa a a a aa a aa aa a aa a aa aa a a aaa aa a aa a aa aa a a aaa aa a a a aa a aa aa aaa aa a aa a aa aa a a aaa aa aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aa aaaa a a a a a a a a a a a a a a a a a a aaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a a aaa a aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aa aa aa aa aa aaa aa aaa aa aa aa aaa aa aaa aa aa aa aaa aa aaa aa aa aa aaa aa aa aa a a a aaa a aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aa aa aa aa aa aaa aa aaa aa aa aa aaa aa aaa aa aa aa aaa aa aaa aa aa aa aaa aa aa aa aa a aaa a aaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aa aa aa aaa a aaa a aaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa aaa aa aa aa aa aaa a aaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aa aa aa aaa a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aaa a aaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a a aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa aaa aa aa aaa aaa a aaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aa aa aa aaa a aaa a aaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aa aa aa aaa a aaa a aaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a a a a a a a a a a a a a a a a a a a a a a a a a a a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa a aa aaa aa a aa a a aa aa a aaa aa a a a aa a aa aa a aa a aa aa a a aaa aa a aa a aa aa a a aaa aa a a a aa a aa aa aaa aa a aa a aa aa a a aaaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aa aaaa a a a a a a a a a a a a a a a a a a aaa

aaa aaa aaa aaa aaa aaa a a aaaa a aaa aaa aa a a a a a a a aaa aa a a aa aa a aa a aa aaa aa aa a aa a a a aa aa a aa a a a aa aaa a a aa aa a aa a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aaa aa aaaa aa a a aaa aaa aaa aaa aaa aaa a a aa a a a a a a aaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a a a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aa aa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa a a aaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aa aa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa a aa aa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa a aaa aa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a aaa aa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa aa aa aa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa a aaa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aa aa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a a aaa aa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aa aa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa a aaa aa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa a aaa aa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a a a a a a a a a a a a a a a a a a a a a a a a a a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aa aa aa aa a aaa aaa a a aa aa a aa aa a aaa aaa a a a a aa aa aa aa a a aa aa aa aa a a aaa aaa a a aa aa aa aa a a aaa aaa a a a a aa aa aa aa aaa aaa a a aa aa aa aa a a aaa aa aa aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aa aa aaa a a a a a a a a a a a a a a a a a a a aaa

aa aa aaa aaa aaa aaa aaa aaa aaa aa aa aa a aa aa a a a a a a a a aaa a a aa aa a aa a aa aaa aa aa a aa a a a aa aa a aa a a a aa aaa a a aa aa a aa a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa a aa a a a aa aaa aa aa aaa aa a a aa aaa aaa aaa aaa aaa aaa aaa a aaa a a a a a a aaa

a aaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aaaaaaaaaaaaaaaa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aaaa aaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aa a aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aa aa aa aa a aaaa aaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aa a aa aa aa aaa aa aaa aa aa aa aaa aa aaa aa aa aa aaa aa aaa aa aa aa aaa aa aa aa aa a aaaa aaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aa a aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aa aa aa aaa a aaaa aaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aa a aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa aaa aa aa aa aa aaaa aaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aa a aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aa aa aa aaa a aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aaaa aaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a a a a a a a a a a a a a a a a aa aaaaa aaa aa aa aaaaa aaa aa aa aaaaa aaa aa aa aaaaa aa aaaaa aaaa aa aaa a aaaaa aaa a aaaaa aaa a aaaaa aaaaa a aaa a aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aa aaa aa aa aa aaaa a aaaa aaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a aa a aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aaa aa aa aa aa aaa aa aa aa aaa a aaaa aaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa aaaaa a a a a a a a a a a a a a a a a a a a a a a a a a a a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aa aa aaa aaa aaa aaa aaa aaa aa aa aa a a aa a aa a a a a a a a a a a a aa aa a a aa aa aa aa a a a a aa aa a a a a aa aa a a aa aa a a aa aa aa aa a a a a aa aa aa aa a a a a aa aa aa aa a a a a aa aa aa aa a a a a aa aa aa aa a a a a aa aa a a a a aa aa aa aa a a a a aa aa aa aa a a a a aa aa aa aa a a a a aa aa aa a aaa a aa a aaa aaa aaa aaa aaa aaa aa aa a aaa a a a a a a aa

Figure 5: CUMULUS chip structure

cij =

WX ?1 w=0

c(ijw) 2w

(4)

During the evaluation, the minima can be read out serially onto the T pins. However, one is more interested in additional information like the pattern index or a class number. Therefore, we propose to load L0 new, associated bits, here indexed with negative w, into the counters while their contents is shifted:

cijw

( )

=

(

ij(w) for w = 0; :::; W ? 1 siw for w = ?L0; :::; ?1:

(5)

When the search for the minimal values on

cij =

W X w=?L

0

c(ijw) 2w

(6)

P (w ) w is performed, the resulting minimum in each row Cj = W m=?L Cj 2 contains both the minimal PW (w) w distance Dj = w=0 Cj 2 and the associated information Cj(w) = siw of the row i containing the \nearest" pattern. This associated information may be anything like a class number, the pattern number, or information destinated for other recognition processes, as described in section 6. As shift operations can be inserted between groups of bit accumulation, dierences may be accumulated with a dierent weight Ak : 0

ij =

L X k=1

Ak (sik  tjk );

with i = 1; :::; M;

j = 1; :::; N

(7)

6 This is done very eciently if the weight factors Ak = 2ak are binary powers and already sorted (ak  ak+1 ), as one only needs to shift the intermediate results between dierent weight groups. Furthermore, the information about the weight factors is stored very compactly, as it is essentially contained in the order and the changes between groups. For accumulations with general weighs Ak , i.e. which are no powers of two, the same comparison can be processed several times with a dierent shift position, corresponding to the binary representation of Ak . This general procedure is, of course, less ecient. The chip containing the array of counters will also comprise a pipelined control logic including the automatic generation of the reset signals for shifting and of the clock signals. All operations are controlled for each cycle independently which allows a highly exible pipeline. The control logic will be routed automatically and occupy only a negligible part of the circuit (about 3% for M = N = 32). The main features of this chip will be:

        

Very wide SIMD bit-sequential parallelism. Few input and output connections, fully scalable architecture. One-cycle operations NOP, XOR, PROD, LDS for the accumulation of 0, si  tj , si  tj and si . Shift operation for multiplication of the counter values by powers of 2 during several cycles. Shifting with a bit-sequential determination of the minima within each column. Charging of associated information from the pins si during that process. Bidirectional bus for tj allowing for the search of minima between dierent chips by pull-down. Bidirectional bus for si allowing for a readout of the accumulation in each row. Direct, fast data transfer from the tj pins to the si pins.

Possible applications for the CUMULUS chip will require a large number of bit-counting operations and re ect the two-dimensional structure by exhibiting a two-fold parallelism. This parallelism allows for the ecient usage of the fast memory connections.

3 The system architecture The most interesting property of the presented architecture is the fact that it is completely scalable. The number of I/O for a square N  N array of counters is 2N , i.e. O(N ) and not O(N 2) like the number of counter elements. This applies both for the chip and the system level. An overall view of a complete system is shown in g. 6. It is composed of an M 0  N 0-array of CUMULUS chips and two independent memory banks, S-MEMORY and T-MEMORY providing data words of size M  M 0 and N  N 0 respectively. During the counting phase, the readout is done in block-mode at a rate of one word per cycle. All lines are served by fast CMOS drivers and terminated at the other end.

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aaa aaaa aa aa aa a a aa a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa a a a a a aaa aaa a aaa aaa aaa aaa aaa aaa aaa aaa a a a a a aaa aaa a aaa aaa aaa aaa aaa aaa aaa aaa a a a a a aaa aaa a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa a a a a a aaa aaa a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aa aa aa aa a a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaa aa aa aa aaa aa aaaaaaaaaaaaaaaaa aa aa aa aa aaaaaaaaaaaaaaaaaaaaa aaa aaaa aa aa aa a aa a a aa aaa aaaa a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aaaaaaaaaaaaaaaaaaaa aaaa a a aaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa a aaa aa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aaa aa aa aa aa a a a aa aaaa aaa a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa

aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa

aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaa

aaaaaaaaaaaaaaaaaaa aa aa aa aa aa aaaaaaaaaaaaaaaaaa aa aa aaa a aaaaaaaaaaaaaaaaaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aaaaaaaaaaaaaaaaaaaa aaaa a a aaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaa aaaa aaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa a aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa aa a aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa aa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aaa aaaa aa aa aa aa a aa a a a aaa aaaa aaa aaaa aaa aaaa aa aa aaaaaaaaaaaaaaaaaaaaa aa aa aa aa a aa aaaaaaaaaaaaaaaa a aaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa aa aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaa aa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaa aaa aa aaaaaaaaaaaa aa aaa aaaaaaaaaaaaa aa a aa aa aa aa a a a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa a aa a aa aa aa aa a a aaaa aaa aaaa aaa aaaa aaa aaaa aaa a aa a aa aa aa aa a a aaaa aaa aaaa aaa aaaaaaaaaaaaaaaaaa aaa aaaaaaaaaaaaaaa aaaaaaaaaaaaaaa

aa aaaaaaaaaaaaa aaa aa aaaaaaaaaaaaa aa aaaa a aaaaaaaaaaa a aa aa aa aa a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaaaaaaaaaaaaaaaa aaaa aaaaaaaaaaaaa aaaaaaaaaaaaaaa

aa aaaaaaaaaaaaa aaa aa aaaaaaaaaaaaa aa aaaa a aaaaaaaaaaa a aa aa aa aa a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaaaaaaaaaaaaaaaa aaaa aaaaaaaaaaaaa aaaaaaaaaaaaaaa

aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a a a aa aa aa aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a a a a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aa aa aa aa a a a a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aa aa aa aa a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a a a aa aa aa aa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a a a a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aaa a a a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa a aa aa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa a aaa aaa aaa aaa a a aaa aaa aaa aaa aaa aaa aa aa aa aa a a aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa aaa a a aa aa aa aa a a a aaa aaa aaa aaa aaa aaa a aaa a a a aaa aaa aa aa aa aa a a a aaa aaa aaa aaa a aaa a a a aaa aaa aaa aaa aa aa aa aa a a aaa aaa aaa aaa a aaa aaa aaa aaa aaa aaa aaa aaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaa a aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Figure 6: CUMULUS system structure

The determination of minimal values for each column, but between many chips, is realized by pulling down the corresponding column data line after it has been precharged actively. This sort of column broadcast necessitates more time (5 system cycles) than each bit counting (one cycle). The S and T memories need to be very fast for a readout at consecutive addresses, as this will directly limit the maximum system speed, fmax . One should be able to write back values onto the T-memory at arbitrary positions, but at a frequency of fmax =4 only. The S-Memory may be used to store intermediate results at fmax =2. The S-memory will typically store the database of prototypes and associated information. The T-memory will contain the \states of the activation". The row parallelism does not pose any problems, as databases typically are rather large. In the case of simple pattern recognition, the very large data base will be stored in the S-memory. As the number of prototypes for which the comparison has to be performed is typically much larger than M  M 0, several cycles are performed and each time, the resulting distance and associated information are stored in the T-memory. Additional passes are needed to recharge that information into the counter array and to determine the global minimum. Both data memories and the CUMULUS array need to receive control signals at a high rate of one per cycle, i.e. fmax . As a fast and simple solution, we propose the usage of a third memory, the C-memory, which contains a precompiled table of addresses and control signals and acts as a controller.

8

4 Binary pattern classi cation 4.1

K nearest neighbor search (KNN)

In order to evaluate a possible use of the specialized hardware for pattern recognition, and to test a simple binary \k nearest neighbor" (KNN) algorithm against other more sophisticated methods like feed-forward neural networks, we have implemented a highly optimized calculation of the Hamming distance on a Sun SPARC10 workstation. The nearest neighbor method [3] tries to classify patterns by a simple comparison with a number M of reference patterns. The Hamming distance to all registrated prototypes is calculated and the rst k neighbors are regarded in order to derive the class, for example using a simple majority rule. We have tested only a simple \one nearest neighbor" (ONN) algorithm which outputs directly the class of the nearest neighbor, as higher k do not signi cantly improve the recognition. We performed tests on the NIST database of 223125 segmented handwritten digits [1]. We rst normalized them to images of 16x16 binary pixels either by choosing the same scale factor in x- and y -direction (method 1) or with a free anisotropic scaling, i.e. always taking the full width and height (method 2). The encouraging results from these tests, shown in the following table, are obtained without any further treatment like selection, or alignment: When 167343 patterns, i.e. three quarters of the database, were M= 1000 2000 5000 10000 20000 50000 167343 method 1: 9.5% 7.3% 5.4% 4.0% 3.0% 2.4% 1.50% method 2: 6.8% 4.8% 4.0% 3.0% 2.4% 1.7% 1.05% Figure 7: Error rate for KNN on the NIST database (without any rejection) used for \training" and the tests were performed on the remaining 55781, a recognition rate of about 99% was attained. Although this brute method neglects any topological information, just like Hop eld networks, but unlike some feed-forward network solutions [2], it might be more interesting as it does not involve complex operations like oating-point multiplications. In g. 8, the resulting errors are displayed in a log/log plot over the database size M for the anisotropic scaling in curve A. A power law

d  CM ?! ;

with ! = 0:37

(8)

seems to be a good approximation both for the anisotropic and isotropic normalization. It would be interesting to know, if this behavior is universal, i.e. if for a dierent number of classes or other data, a similar curve results. We would therefore like to investigate many questions of fundamental interest involving statistical properties on a very large data base. Extended simulations could also lead to an improved preprocessing. As no sign of saturation for large M is evident in g. 8, we can expect even better classi cation for larger databases. In comparison with dynamical neural networks, the direct pattern comparison possesses many advantages. If the length of the patterns is L, a recognition by a completely connected Hop eld network necessitates several consecutive iterations, each performing L  L accumulations of real couplings. The number of dierent patterns which can be learned is theoretically limited by 2L. Practically, i.e. for

Figure 8: Pattern recognition error d for dierent database sizes M and dierent algorithms | A: KNN without learning, B/C: optimized binary prototypes on learning set (B) and on test set (C) reasonable learning schemes it is well below L, which means that more than one real or integer coupling is needed to store one pattern bit. The main reasons for employing KNN methods are:

        

Only accumulations of single bits instead of integers are needed. The information is stored much more compactly. There is no complicated dynamics, no arti cial attractors. No complicated iterative learning is necessary. Each prototype is directly inserted. Prototypes can be easily deleted, i.e. forgotten. Highly correlated patterns do not pose any problem. The algorithm is much more ecient, if only few prototypes are given. On the other hand, there exists no upper limitation for the number of patterns. Any associative information can be treated easily.

Finally, it should be stressed that both algorithms, neural networks and KNN, can be easily executed on a parallel SIMD structure.

10 4.2

Learning of binary patterns

The error rates for a simple KNN method are limited by twice the value for the Bayesian classi er [3], one can therefore attain the best classi er for a huge number of samples. In practical applications, however, this number will be limited both by computing power and memory. It becomes evident that \learning" methods which allow to extract an essential set of prototypes from a large number of sample patterns are highly interesting. On the other hand, well-known methods such as \Learning Vector Quantization" (LVQ) [4] involve

oating-point operations and output real-valued vectors. As our architecture is well adapted only to binary accumulation, we are interested in an algorithm which entirely relies on bit counting. It is not evident that an easy discrete learning algorithm exists which converges to a representative set, because we may not perform slight continuous changes in order to reduce a total error function by a steepest descend method. However, we have found that the local minima found by a discrete approximation are sucient to reduce the number of prototypes up to a factor 20. The algorithm can be regarded as a binary version of a \k-means method" [5]. Unlike LVQ methods, it has the advantage, that learning can be performed for each class separately. We will brie y describe it here: M prototypes of binary vectors sik , (k = 1; :::; L) { in our case L = 256 bit images { are modi ed with the aim to well cover the set of N sample patterns tjk all lying within the same class, i.e. to minimize the global error N L X X E = minM sik  tjk : (9)  ; with  = ij i=1 ij j =1

k=1

In order to nd this con guration, every prototype is in uenced by a \cloud" of sample patterns for which it is the nearest. Just like in the standard \k-means" algorithm, each prototype tries to move towards the center of gravity of this cloud. However, it can only attain binary vectors, and each pixel will take on the value of the majority of the values, either \0" or \1". The prototypes are initially set to patterns randomly chosen from the learning base. In each learning sequence, the algorithm sequentially presents all sample patterns. After three passes, an almost optimal con guration is reached, and further passes improve the recognition performances only slightly. Each time a pattern is presented, the nearest of the prototypes is determined and only it is updated. This is done by incrementing or decrementing local counters Cik for each pixel position k depending on the value tjk . These counters, which are initially set to 0, memorize if the majority of presented patterns took a value of 0 or 1 at this position. The prototype is then updated by setting its pixel values sik to 0 or 1 for a negative resp. positive counter value Cik . One may directly update the involved prototype after the presentation, which leads to a fast convergence. However, a global updating after a certain number of pattern presentations is better adapted to our architecture and gives almost the same results. The described algorithm is very fast, as it can handle all classes separately and as it involves only bit counting. One important feature is the fact that it can learn new patterns incrementally, i.e. without to take into account all already known patterns (in fact, they are compressed into the counter values). We have tested the new algorithm for the NIST database. We always took the rst 75% of the database (167343) to derive a number M of prototypes. Then, we measured the recognition error both for the learning set (curve B) in gure 8 and for the test set, the remaining 25% (curve C). Curve B shows the same power law behavior of eqn. ( 8 ) as for the simple KNN algorithm (curve A), however with

a constant about a factor 20 smaller. Thus, given a certain recognition performance, one can reduce the number of prototypes and thereby the total computing requirements to 5%. Curve C shows a saturation for a large number of prototypes M . It is evident that in order to be able to generalize, the algorithm needs a higher number of samples. One could try to develop binary learning algorithms for better separating the dierent classes, which would be a binary version of LVQ. However, our simple algorithm already delivers a satisfying compression and is feasible even for a very large number of prototypes (one million). Finally, one may wish to overcome the worst disadvantage of the simple KNN methods, the fact that a classi cation is not invariant under intrinsic symmetries such as rotations or translations. The simplest way to achieve this, if performance is abundant, is to present each pattern under many dierent aspects, i.e. slightly distorted. All these reasons are strong motivations for the construction of a specialized hardware. It will enable both the development of improved algorithms and the direct exploitation of huge data bases which are directly acquired in practical applications.

5 Further applications As the counter element provides for a bit-multiplication and accumulation, a parallel accumulation of products of integers with binary or Ising-like variables, as needed in Hop eld neural networks, can be executed most eciently. Even integer matrix products are realizable, however at a lower eciency, as the multiplication has to be emulated by several additions. Typically, the S memory will contain integer weights

Sjk =

RX ?1 r=0

s(jkr) 2r ;

r = 1; :::; R;

( 10 )

which are to be multiplied by binary values tjk :

Pij =

L X k=1

Sik tjk =

RX ?1

L X

r=0 k=1

sikr tjk ( )

!

2r =

RX ?1 r=0

ijr

( )

!

2r

( 11 )

Again, we need a two-fold parallelism, i.e. a matrix of binary states, in oder to fully exploit the architecture. For Hop eld networks, this means that N dierent instances of the same system, however with dierent neural Ising-like states, are simulated in parallel. During a learning phase, one may learn many dierent patterns in parallel and keep the coupling values constant between \global updates". Among further applications for CUMULUS are simulation of the binary perceptron and evaluation of cost functions for simulated annealing.

12

6 Networks of associative binary blocks For most applications, the realization of a fully connected neural network or of a single associative memory block are not reasonable for several reasons. The number of incoming and outgoing bits may be much too high to allow a complete connectivity. On the other hand, due to a certain modular or inhomogeneous structure of the problem, the task can often be separated into many dierent agents which can locally treat information with dierent meaning. Finally, the problem may require internal states for an evaluation in several stages. Therefore, a network of many independent associative blocks which exchange informations on well adapted communication lines, seems to be much more exible for many applications. These associative blocks, which are able to perform recognition or classi cation tasks, may be realized using neural networks. As mentioned in the last section, however, KNN algorithms seem to be very well adapted to that task and much less time-demanding. In order to nd the optimal structure for a given cognitive task, a structure which allows many free parameters is indispensable. For a general structure of associative blocks these are:

 The number of associative blocks.  The number of incoming and outgoing bits, for each block independently.  The destination block of each outgoing bit.  The number and the value of prototypes for each block.  The associated information sent to any block for each prototype independently.  Optionally, the weight of each input position. The structure of the CUMULUS architecture allows for all of these choices and all parameters can be changed directly and quickly. As calculations are treated in a bit-serial way, L may take any value, unlike in other hardware realizations, where the processing is performed in parallel on a hardware structure with a xed data path width. Furthermore, many virtual blocks can be simulated by the same structure. No routing in hardware between dierent chips is necessary, as the connections between dierent virtual blocks are realized by indexing. Many changes can be performed \on the y" which demand a recon guration on other non-sequential architectures. In order to exploit the parallelism for tj , many independent systems have to be simulated simultaneously. They have the same con guration including the same prototypes, they are however in dierent states. This sort of parallelism can also be exploited in most learning schemes, where many recognition processes are executed before the prototypes are changed in a \global update step". In order to evaluate possible use of information processing by these structures, a simulator realizing this bit-sequential parallel comparison on a standard processor, is being developed in our group. It will ease the development of a code-generator and the test of structures of associative elements.

7 Realization and Conclusion Two test versions of the CUMULUS chip have been developed by now in 1:5m technology and are being fabricated by ES2 in the Eurochip run of september 1993. The rst version integrates an 8  8 array of counters of width W = 12, and uses standard library cells which are placed and routed manually. It has been extensively simulated in SILOS II and is operative up to fmax = 70 Mhz for worst case delays. A detailed description of the circuit including a de nition of all operational modes will be available in October 1993 as a diploma thesis [6]. The second test chip includes a 4x4 counter array which is based on a full-custom toggle ip op cell to further increase the density by a factor of about 1.6. Otherwise, it has exactly the same modes of operations as the rst test chip. After extensive hardware tests of the small chip by the use of a logic analyzer, a full-scale version will be fabricated in 1:0m technology. Furthermore, we will develop a rst test system employing standard DRAM chips in fast page mode with a large data bus. An optimized version will contain very fast video RAM memory and will probably be realized as an SBUS card. In this paper, we have de ned a most simple and highly ecient architecture which is dedicated for binary pattern recognition and classi cation. In spite of this extreme specialization, this structure is very exible due to the bit-sequential operation. The two-dimensional structure largely reduces the number of high-speed data connection to the external memory and makes the architecture scalable. We hope that the availability of this dedicated and very fast hardware will accelerate the development of more powerful learning algorithms for binary classi cation and encourage the use of huge binary associative memories for very complex cognitive tasks.

Bibliography [1] M. D. Garris, R. A. Wilkinson, NIST Special Database 3: Handwritten Segmented Characters (available on CDROM) (1992) [2] Y. le Cun et al., Proceedings of the ICPR, 1990, Atlantic City, NY [3] P.A. Devijer, IEEE Trans. on Information Theory 25 p. 749 (1979) [4] LVQ T. Kohonen, Self-Organization and Associative Memory, Springer, 1989 [5] J. MacQueen, Proc. of the fth Berkeley Symp. on Mathematics, Statistics and Probabilities I, p. 281 (1967) [6] M. Gumm, Diploma Thesis at the \Intitute for Microelectronic Systems", Technical University of Darmstadt, Germany, 1993