A Survey of Neuromorphic Engineering --Biological Nervous Systems Realized on Silicon Jihong Liu
Chengyuan Wang
College of Information Science and Engineering. Northeastern University, Shenyang, China Email:
[email protected]
Sino-Dutch Biomedical and Information Engineering School Northeastern University, Shenyang, China Email:
[email protected]
Abstract—Neuromorphic systems are inspired by the structure, function and plasticity of biological nervous systems [1]. This field is evolving a new era in computing with a great promise for future medicine, healthcare delivery and industry [2]. The paper focuses on the introduction of neuromophic engineering. It describes the history of neuromophic engineering and the process of developing neuromophic chip, and then introduces the required hardware and AER (Address Event Representation) tools for morphed on silicon. Finally it summarizes the applications of neuromophic engineering based on neuromophic chip. It is shown that the neuromophic engineering has a promising development. Keywords- neuromorphic; address event representation (AER); neural chips; neuromorphic design
I.
INTRODUCTION
The term neuromorphic was coined by Carver Mead, in the late 1980s to describe very-large-scale integration (VLSI) systems containing electronic analog circuits that mimic neurobiological architectures present in the nervous system. In recent times the term neuromorphic has been used to describe both analog, digital or mixed-mode analog/digital VLSI systems that implement models of neural systems (for perception, motor control, or sensory processing) as well as software algorithms. Neuromorphic engineering is a promising interdisciplinary discipline that takes inspiration from biology, physics, mathematics, computer science and engineering to design artificial neural systems, such as vision systems, head-eye systems, auditory processors, and autonomous robots, whose physical architecture and design principles are based on those of biological nervous systems [3]. Neuromorphic engineers are using garden-variety VLSI, complementary metal oxide semiconductor (CMOS) technology to achieve their goal. This effort is facilitated by similarities between VLSI hardware and neural wetware. Both technologies: •
Provide millions of inexpensive, poorly-matched devices.
•
Operate in the information-maximizing low-signal-tonoise/high-band width regime [4].
II.
THE PROCESS OF DEVELOPING NEUROMORPHIC CHIP
The neuromorphic engineering could be divided into neuromorphic modeling, reproducing neurophysiological phenomena to increase the understanding of the nervous systems and neuromorphic computation which uses the neuronal properties to build neuron like computing hardware. Basically, the former provides the knowledge of the biological algorithm while the latter translates the algorithm into electrical circuits. As shown in Fig. 1, this is an iterative process, since the understanding of the biological algorithm is a very complex process. As more knowledge evolves yielding improved algorithm, the electrical circuits are revised and improved. These circuits then pass through all the stages of developing integrated circuit (or chip), which involves the circuit layout, verification, fabrication in foundry and testing and subsequent deployment. A brief explanation of each of these steps is provided below: •
Layout design: This stage involves the translation of the circuit realized in the previous stage into silicon description through geometrical patterns aided by computer aided design (CAD) tools. This translation process follows a process rule that specifies the spacing between transistors, wire, wire contacts and so on. The layout is designed to represent the electrical circuit schematics obtained from the algorithm.
•
Fabrication: Upon satisfactory verification of the design, the layout is sent to the foundry where it is fabricated. The process of chip fabrication is very complex. It involves many stages of oxidation, etching, photolithography, etc. Typically, the fabrication process translates the layout into silicon or any other semiconductor material that is used.
•
Testing: The final stage of the chip development is called testing. Electronic equipment like oscilloscopes, probes, and electrical meters are used to measure some parameters of the chip, to verify its functionalities based on the chip specifications [2].
Supported by the National Natural Science Foundation of China under Grant No.60774097.
978-1-4244-2587-7/09/$25.00 ©2009 IEEE
Algorithms
Chips
Behaving Systems
Figure 1. Illustration of the neural systems development flow
III.
THE HARDWARE AND TOOLS FOR MORPHED ON SILICON
A. The hardware for morphed on Silicon Neuromorphic VLSI systems are the bridging technology between biological neural network models and engineering VLSI hardware. The design of neuromorphic systems dates back to 60's, however, the bulk of activities started in 80's when neural network research was resurrected and VLSI technologies started to mature [5], [6]. Neuromorphic hardware is the term used to describe full custom-designed integrated circuits, or silicon ‘chip’, that are the product of neuromorphic engineering--a methodology for the synthesis of biologically inspired elements and systems, such as individual neurons, retinae, cochleas, oculomotor systems and central pattern generators [7]. Recently, field programmable gate array (FPGA) technology has improved in density to the point where it is possible to develop large scale neuromorphic systems on a single FPGA. Although these are admittedly larger in area, have higher power consumption, and may have lower throughput than the more customized analog VLSI implementations, many interesting neuromorphic signal processing systems can be implemented using FPGA technology, enjoying the following advantages over analog and digital VLSI:
time, wider dynamic range, better stability, and simpler computer interface over analog VLSI implementations [8]. Rapid design time, low cost, flexibility, digital precision, and stability are characteristics that favor FPGA as a promising alternative to analog VLSI based approaches for designing neuromorphic systems. High computational power as well as low size, weight, and power are advantages that FPGA demonstrates over software based neuromorphic systems [9]. B. The tools for morphed on Silicon The AER is an event-driven asynchronous inter-chip communication technology for neuromorphic systems [4], [10]. AER originally proposed as a means to communicate sparse neural events between neuromorphic chips, has proven efficient in implementing large-scale networks with arbitrary, configurable synaptic connectivity [11]. A potentially huge advantage of AER systems is that computation is event driven and thus can be very fast and efficient. Here we describe a set of AER building blocks. The AER building blocks were assembled into a prototype vision system that learns to classify trajectories of a moving object. All modules communicate asynchronously using AER. The building blocks and demonstration system have been developed in the EU funded research project CAVIAR (Convolution AER Vision Architecture for Real-time). The building blocks (Fig. 2) consist of:
•
Shorter design and fabrication time.
•
More robust to power supply, temperature, and transistor mismatch variations than analog systems.
•
A retina loosely modeled on the mangocellular pathway that responds to brightness changes.
•
Arbitrarily high dynamic range and signal-to-noise ratios can be achieved over analog systems.
•
A convolution chip with programmable convolution kernel of arbitrary shape and size.
•
Whereas a VLSI design is usually tailored for a single application, the reconfigurability and reuseability of an FPGA enables the same system to be used for many applications.
•
A multi-neuron 2D competition chip.
•
A spatio-temporal pattern classification learning module.
•
Designs can be optimized for each specific instance of a problem whereas application-specific integrated circuit (ASIC) needs to be more general purpose.
•
A set of FPGA-based printed circuit boards for address remapping and computer interfaces.
•
They can be interfaced more easily with a host computer.
FPGA provides a very flexible platform for the development of experimental neuromorphic circuits and offers advantages in terms of faster design time, faster fabrication
Using these AER building blocks and tools R. Serranogotarredona’s group built the demonstration vision system shown schematically in Fig. 2, which detects a moving object and learns to classify its trajectories. It has a front end retina, followed by an array of convolution chips, each programmed to detect a specific feature with a given spatial scale.
Input: Moving objects
off
Output: ON & OFF Contrast changes
Output: Multiscalele filtering
Output: position & size (x,y,size)
on Output: classification of trajectory Retina chip INI (Zurich)
Convolution chip IMSE (Sevile)
Object chip INI (Zurich)
Delay line chip Learning chip UIO (Osio)
AER Computer Interfaces Monitor, remap, inject AER USE (Sevile) Figure 2. Demonstration AER vision system
The competition or ‘object’ chip selects the most salient feature and scale. A spatio-temporal pattern classification module categorizes trajectories of the object chip outputs [12]. IV.
APPLICATIONS AND COMPARISON OF RESEARCH GROUPS
A. The applications of neuromorphic chip Their applicability to engineering challenges is widespread, and includes biomedical, displays, hearing, imaging, language, locomotion, neural interface, power, processing, robotic, security and vision. Neuromorphic engineering proposes to fill the gap between, on the one hand, computational neuroscience, and, on the other hand, traditional engineering [13]. In the biomedical field, the neuromorphic chip can be used to stimulate locomotion in an animal (tested on a temporarilyparalyzed cat) [14]. Neuromorphic vision systems are ideal for mobile applications because they promise compact computational sensing at lower power consumption compared to the traditional imager systems [15]. The implantable neuromorphic vision chips proposed analogue integrated circuits are to implement four cell functions (photoreceptor cells, horizontal cells, bipolar cells, and ganglion cells) of the retina for implant and all simulation results are successfully verified and consistent with researches on biological retinas [16]. Neuromorphic motion sensors are attractive for use on battery powered robots which require a low payload [17]. Because of its compact hardware and low power dissipation, the neuromorphic vision system developed in the present study is suitable to robotic vision. More interestingly, it provides insights to explore the visual function of the neuronal network of the brain, visualizing neural images inferred from physiological experiments [18]. An on-chip spike time dependent learning circuit is integrated to dynamically adapt weights for odour detection and classification [19]. B. Comparison Europe – USA – Japan European research is a relatively strong interacting community. However, especially compared to the USA, Europe is still lagging behind in several respects. In the USA, interdisciplinary co-operations between life-sciences and
engineering has a much longer tradition and is also more strongly accepted by funding agencies and popular media. Furthermore, given the different economic structure, exploitation of results into commercial products is much more effective and developed in the USA. For instance, while important work on neuronal implants exists in Europe, technology must be seen as more advanced in the US. It is somewhat harder to assess the situation in Japan. While there is a strong tradition in computational neuroscience (e.g. the pioneering works of people like Fukushima or Amari), vision or robotics other communities like cognitive science seem to be less pronounced. Important exceptions are the institute in Tokyo (Kawato) and RIKEN (Ito). While Japan generally shows a large potential for turning research results into engineering solutions, it remains somewhat unclear what their role in areas like neuromorphic engineering is, as compared to Europe. In general, Japanese research is characterised by a large number of concerted efforts toward particular goals. In summary, some of the strengths of European research in the domain of neuromorphic engineering are： •
Leading research in topics like pulsed neural network models, neuronal implants, binding, topographic mapping, etc.
•
Experimental neuroscience: There is a large number of laboratories world-wide, with a trend for the USA to lead, but Europe is very strong in a number of areas, e.g. vision, cortex synchronised oscillation in neuronal networks, hybrid (biological-plus silicon or computer simulated) neuronal networks, patch clamp and multiple single cell recording techniques, psychophysics and neurophysiology of multisensory perception by vision, the vestibular system and at a lesser degree the tactile and proprioceptive systems.
•
Modelling and measuring perception-action interaction on tasks such as reaching, catching, object manipulation, equilibrium maintenance, gaze control, locomotion. The European school for motor control has regular meetings and strong implantation in cooperations with robotics through European centres of
excellence in Italy, England (e.g. the London Institute for Movement Science), Germany, Sweden (Karolinska Institute), France, etc., and many European projects. •
Several good technological advances in the field of sensory-motor sensing, in part because of the impact of the European Space program; Several techniques and methods which have been developed to study sensory adaptation to microgravity could find a relay in fundamental research on perception systems in this initiative. This would be a good use of European efforts. In turn, solving of problems such as telepresence or telemanipulation, or human robot interfaces for space projects could also be (in addition to the classical applications in health or industry) a long term output of this initiative.
Among the weaknesses are： •
There are still too many gaps between the disciplines.
•
In general, the technologies that necessitate costly equipment (e.g. two-photon imaging, genetically expressed molecular tags allowing functional systems to be turn on or off, multiple electrode recording systems) are clearly led by the USA.
•
There are fewer concerned efforts, as compared to Japan and there is a lack of funding, with the exception of a few countries (e.g. Germany, UK) [20]. V.
CONCLUSION
Modern advancement in biology is enabling better understanding of forms, structure, and behavior of biological systems to develop algorithms implemented in analog integrated circuits. These circuits are developed in paralleldistributed architectures with elements of adaptation and learning using low power and high integration density technologies. As the algorithms mature, these chips, neuromorphs, are expected to become more efficient for critical industrial application like animal parts replacements. Effective interdependence collaborations among the key areas of biology, electrical engineering, physiology and computer science are very fundamental to develop and engineer this emerging area of computational biology [2], [3].
[2] [3] [4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14] [15]
[16] [17]
[18]
ACKNOWLEDGMENT The authors thank their colleagues at the Institute of Electronic Science and Technology for providing the good conditions for learning, for helping in collecting the information on the neuromorphic engineering.
[19]
[20]
REFERENCES [1]
C.A. Mead, “Neuromorphic electronic systems," Proceedings of the IEEE, vol. 78 (10), pp. 1629-1639, 1990.
Paula Bach， “Neuromorphics: biological nervous systems morphed on silicon,” unpublished. Wikipedia, [online] http://en.wikipedia.org/wiki/Neuromorphic. K. Boahen, “Point-to-Point connectivity between neuromorphic chips using address events,” IEEE Trans. on Circuits and Systems Part-II, vol. 47 (5), pp. 416-434, May 2000. R.G. Runge, M. Uemura and S.S, “Viglione. electronic synthesis of the neural networks in the pigeon retina,” in Cybernetic problems in bionics, Dayton, Ohio, USA, pp. 791-800, May 1968. Alireza Moini, “Neuromorphic VLSI systems for visual information processing: drawbacks,” 1999 Third international conference on Knowledge-Based intelligent information engineeing systems, Adelaide, Australia, pp. 369–372, 1999. Catherine Breslin and Adrian O’Lenskie, “Neuromorphic hardware databases for exploring structure-function relationships in the brain,” Phil. Trans. R. Soc. Lond. B(2001)356, pp. 1249-1258, 2001. M. P. Leong, Craig T. Jin and Philip H.W. Leong, “An FPGA-Based electronic cochlea,” EURASIP Journal on Applied Signal Processing, pp. 629–638, 2003. Cassidy A., Denham S., Kanold P.and Andreou A., “FPGA based silicon spiking neural array,” Biomedical Circuits and Systems Conference, Montreal, Que, pp. 75–78, 2007. M. Sivilotti, “Wiring considerations in analog VLSI systems, with application to field-programmable networks,” Ph.D. dissertation Pasadena, CA: California Inst. Technol., 1991. R. J. Vogelstein, F. Tenore, R. Philipp, M. S. Adlerstein, D. H. Goldberg, and Cauwenberghs, “Spike timing-dependent plasticity in the address domain,” in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2003. R. Serrano-gotarredona, M. Oster, P. Lichtsteiner and A. Linaresbarranco, “ AER building blocks for multi-layer multi-chip neuromorphic vision systems,” in Advances in Neural Information Processing Systems, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA: MIT Press, vol. 15, Dec. 2005. G. Cauwenberghs , “Neuromorphic learning VLSI systems: A survey,” In: T.S. Lande, Editor, Neuromorphic systems engineering, Kluwer Academic Publishers, 1998, pp. 381–408. Sunny Bains. “Analog chip does job of spinal cord for locomotion,” unpublished. Ralph Etienne-Cummings, Swati Mehta, Ralf Philipp and Viktor Gruev, “Neuromorphic vision systems for mobile applications,” IEEE 2006 Custom Intergrated Circuits Conference (CICC), San Jose, CA, pp. 531– 534, Sept. 2006. C.T. Chiang and C.Y. Wu, “Implantable neuromorphic vision chips,” Electronics Letters, vol. 40 (6), pp. 361–363, 2004. Lukas Reichel, David Liechti, Karl Presser, and Shih-Chii Liu, “Robot guidance with neuromorphic motion sensors,” The 2005 IEEE international Conference on Robotics and Automation, Barcelona, Spain, pp. 3540-3544, April 2005. Tetsuya Yagi and Kazuhiro Shimonomura, “Silicon primary visual cortex designed with a mixed analog-digital architecture,” Proceedings of International Joint Conference on Neural Networks, Orlando, Florida, USA, pp. 2723–2728, 2007. Thomas J. Koickal, Alister Hamilton Tim C, Pearce Su L. Tan, James A. Covington and Julian W. Gardner, “Analog VLSI design of an adaptive neuromorphic chip for olfactory systems,” Circuits and Systems, 2006. ISCAS 2006. Proceedings, 2006 IEEE International Symposium, Island of Kos, pp. 4547-4550, 2006. Georg Dorffner, Jeff Newman,Alain Destexhe,Kirsty Grant and Alain Berthoz, “Life-like perception systems,” unpublished.
