Dynamical principles for neuroscience and ... - Semantic Scholar

Auke Jan Ijspeert, Jonas Buchli, Allen Selverston, Mikhail Rabinovich, Martin Hasler, Wulfram Gerstner, Aude Billard, Henry Markram and Dario Floreano (Editors)

EPFL LATSIS Symposium 2006

Dynamical principles for neuroscience and intelligent biomimetic devices Invited Speakers Abstracts & Poster Abstracts

EPFL Campus, March 8-10, 2006

EPFL LATSIS Symposium 2006

Dynamical principles for neuroscience and intelligent biomimetic devices Auke Jan Ijspeert, Jonas Buchli, Allen Selverston, Mikhail Rabinovich, Martin Hasler, Wulfram Gerstner, Aude Billard, Henry Markram and Dario Floreano (Editors) March 8, 2006 School of Computer and Communication Sciences Ecole Polytechnique Fédérale de Lausanne 1015 Lausanne, Switzerland ISBN 978-2-8399-0134-5

© 2006 EPFL

Contents Preface

I

9

Invited Speakers Abstracts

11

1 Allen I. Selverston – Oscillations and Oscillatory Behavior In Small Neural Circuits 13 2 Sten Grillner – The intrinsic function of a motor system - from ion channels to network and behaviour 15 3 Serge Rossignol – Dynamic reflex interactions during locomotion 17 4 Carmen Sandi – The Impact of Stress on Memory and Synaptic Plasticity 19 5 Jean-Jacques Slotine – Modularity, synchronization, and what we may learn from the brain 21 6 Martin Hasler – Nonlinear Dynamics, Synchronization and Applications in Neuroscience 23 7 G. Bard Ermentrout – What makes a neuron spike? Phase resetting and intrinsic dynamics 25 8 Wulfram Gerstner –Synaptic Plasticity from an Optimality Viewpoint 27 9 Mikhail I. Rabinovich – Reproducible transient dynamics of neural circuits: Generation and processing of sequences 29 10 Misha Tsodyks – Neural network model of primary visual cortex: from functional architecture to lateral connectivity and back 31 11 Wolfgang Maass – Neural Circuits as Analog Computers

33

12 Gwendal Le Masson – Can Biological Neurons and Micro Electronic Devices collaborate for a common and useful task? 35 13 Thierry Bal – Spike transfer properties of thalamic neurons in hybrid networks in vitro 37 14 Peter Fromherz – Neuron-Semiconductor Interfacing – its Nature and Implementation 39 15 Rodney Douglas – Interesting computations performed by collections of recurrently connected neurons, and their implementation in hybrid VLSI 41 5

16 Avis H. Cohen – Control of the spinal cord by an analog VLSI device: on the road to development of a neuroprosthetic device for spinal cord injury patients 43 17 Philippe Renaud – Technological and Bioelectrical Considerations for Neural Interfacing Microelectrode Arrays 45 18 Maria Chiara Carrozza – Towards the development of a cybernetic hand: scientific, technological and clinical issues 49 19 Andrew Schwartz – Neural prosthetics Useful Signals from Motor Cortex 53 20 Yasuo Kuniyoshi – Emergence and Development of Embodied Cognition From Humanoid Robot Body to Humanoid Robot Brain 55 21 Barbara Webb – Integrating insect behaviours in robot models 57 22 Auke Jan Ijspeert – Central pattern generators in animals and robots 59

II

Poster Abstracts

61

23 R. Alessio et al. – Avalanche behaviour in cultures of dissociated neurons 63 24 S. Badel et al. – Biomimetic VLSI for Real-Time Image Processing 65 25 L. Badel et al. – How to measure the instantaneous I-V curve of neurons with in vivo-like voltage fluctuations 67 26 M. Bezzi et al. –Quantitative characterization of information transmission in a single neuron 69 27 Y. Bornat et al. – An Analog/Digital Simulation System for Biomimetic Neural Networks 71 28 D.A. Braun et al. – Optimal feedback control adapted to explain sensorimotor learning phenomena 73 29 D. Calitoiu – New measures for describing the synchronization of bursting neurons 75 30 A. V. Chizhov et al. – Conductance-based neural population model 77 31 J. F. Ferreira et al. – Biomimetic Visuovestibular Artificial Perception Systems for Coping with Independent Motion, Illusions, Conflicts and Ambiguities 79 32 W. Gerstner et al. – Predicting Neuronal Activity with Simple Models of the Threshold Type 81 33 D. Ghezzi et al. – PhotoMEA: A new optical biosensor for neuronal networks analysis 83 34 B. Girard et al. – Using contraction analysis to design a model of the cortico-baso-thalamo-cortical loops 85

35 F. Heer et al. – CMOS Integrated Bidirectional 128-Electrode Array for Electrogenic Cells 87 36 M. Hersch et al. – A Multi-Referential Dynamical System for Reaching 89 37 N. Joehl et al. – Wireless remotely powered telemetry for microelectronic implanted cortical interface recording system 91 38 R. Jolivet et al. – Na+/K+-ATPase-Specific Spike-Frequency Adaptation 93 39 M. Kleiner et al. – A More Precise Sense in Which The Neural Code is Optimal 95 40 M. Kraft et al. – FPGA implementation of ReSuMe learning in Spiking Neural Networks 97 41 T. Kulvicius et al. – Speed Optimization of a 2D Walking Robot through STDP 99 42 R. Kupper et al. – Simulations of a columnar architecture for cortical stimulus processing 101 43 E. de Lange et al. – Mapping Electrophysiological Diversity of Neocortical Neurons on a Simple Mathematical Diversity 103 44 C. Laschi et al. – Experimental Investigation on a Vestibular Natural Interface 105 45 G. Luksys et al. –Effects of Stress & Genetic Background on Meta-parameter Dynamics in a Simple Reinforcement Learning Model 107 46 A. Mercanzini et al. – The Lausanne Neuroprosthesis: A Flexible Polyimide Microelectrode for Acute and Chronic Cortical Recordings 109 47 J.B. Mouret et al. – Evolution of neuro-controllers for flappingwing animats 111 48 G. Neumann – A Reinforcement Learning Toolbox and RL Benchmarks for the Control 113 49 T.U. Otto et al. – Perceptual learning with spatial uncertainties: models and experiments 115 50 B. Petreska et al. – Neural Modeling of Imitation Deficits

117

51 F. Ponulak et al. – ReSuMe learning method for Spiking Neural Networks dedicated to neuroprostheses control 119 52 F. Ponulak et al. – Adaptive Central Pattern Generator based on Spiking Neural Networks 121 53 F. Popescu et al – EEG-based control of reaching to visual targets 123 54 M.J.E. Richardson et al. – A novel deconvolution-reconvolution method for the measurement of closely-spaced post-synaptic potentials 125

55 J. Rickert et al. – Correlations between motor cortical population signals (LFP, EcoG) improve the decoding of movement direction 127 56 L. Righetti et al. – Programmable Central Pattern Generators129 57 E.L. Sauser et al. – Ideomotor Compatibility: Investigating Imitative Cortical Pathways 131 58 C. Schmitt et al. – WalkTrainer(TM): Re-education device for active walking of paraplegic and hemiplegic people 133 59 D. Sheynikhovich et al. – Spatial reorientation and environment geometry: extraction of global parameters from local visual features 135 60 A. von Twickel et al. – Adaptive behaviour of single legs with evolved neural control 137 61 D. Verstraeten et al. – The unified Reservoir Computing concept and its digital hardware implementations 139

The EPFL LATSIS Symposium 2006 Dynamical principles for neuroscience and intelligent biomimetic devices Over the last twenty years, it has become more and more obvious that dynamical principles are central to the multiple types of processing taking place in the central nervous system. This is observable at many levels, at the neuronal level —synaptic plasticity for instance depends tightly on the timing of incoming action potentials—, at the circuit level —where feedback loops are responsible for computation in cortical microcolumns and for producing coordinated oscillations for locomotion in the spinal cord—, and at the systems level —where multiple feedback pathways between sensory and motor brain areas appear to be central for prediction and action selection— to name a few examples. The goal of the conference is to bring together scientists and engineers interested in understanding these dynamical properties of the nervous system, and in taking inspiration from these properties for the design of prosthetic and robotic devices. The conference is interdisciplinary in nature, and aims at bringing together researchers working on similar topics and phenomena but from different backgrounds, in particular neuroscience, mathematics, and engineering. These proceedings contain the abstracts of 23 invited talks and 39 submitted poster presentations. Thanks to the generous support of the Latsis Foundation, we were fortunate to be able to invite some of the most prominent researchers in the areas of neurophysiology, neuroprosthetics, nonlinear dynamics, neuromorphic systems, and biologically inspired robotics. We believe that it is at the intersection of these different areas that some of the most significant advances in understanding the dynamical principles of the brain take place, through interactions between neurophysiological experiments, mathematical modeling, and development of prosthetic and robotic devices. Nonlinear dynamical systems modeling is indeed crucial in order to characterize and understand the dynamical phenomena taking place in the central nervous system. Similarly being able to create interfaces between neurons and electronic circuits is of tremendous importance both for studying neural circuits and for creating prostheses. Finally, robotics can both be very useful tools for testing biological models, and can at the same time significantly benefit from neuroscience in order to create machines which can replicate some of the amazing abilities of the central nervous system. We hope that the conference will provide a fruitful forum for exchanging discoveries and ideas on these exciting topics, and that these proceedings will provide useful reading material and literature pointers. To conclude, we would like to express our deepest gratitude to the Latsis Foundation for its generous financial support of the conference. We also would like to thank the EPFL for its technical and organizational support. We are especially grateful to all the distinguished invited speakers for responding so positively to our invitation. Finally we would like to thank all the participants for submitting posters, and for traveling to Lausanne.

Sincerely yours, Auke Jan Ijspeert, on behalf of all the organizers

Lausanne, 22 February 2006

Organizers: Auke Jan Ijspeert (EPFL), Jonas Buchli (EPFL), Allen Selverston (University of California, San Diego), Mikhail Rabinovich (University of California, San Diego), Martin Hasler (EPFL), Wulfram Gerstner (EPFL), Aude Billard (EPFL), Henry Markram (EPFL), and Dario Floreano (EPFL).

Part I

Invited Speakers Abstracts

11

The EPFL LATSIS Symposium 2006

Oscillations and Oscillatory Behavior In Small Neural Circuits A.I. Selverston Institute for Nonlinear Science, UCSD, La Jolla, CA, USA [email protected]

Abstract

inhibitory synapses produce intermediate phase relationships between the two CPGs.

Two fundamental questions that can be asked about oscillatory activity are: how does it originate and what is its purpose? As will be shown, there are several ways such activity can be generated. It is not always clear what oscillatory function is even when the output leads directly to behavior. We have examined the dynamical properties of two small oscillatory systems, the lobster pyloric and gastric mill central pattern generators (CPGs). These CPGs are useful in providing answers to both questions because they are small, i.e. contain only a few neurons, and each neuron and synapse are repeatedly identifiable. The mechanisms and the infrastructure for producing the two different patterns have been well studied. Each neuron uses specific ionic conductances, particularly IH , IA , IN aP and several ICa currents and synaptic connections with different characteristics to generate a three phase pyloric rhythm at 2 Hz and a six phase gastric mill rhythm at .1 Hz. The patterns can be turned on and off with specific input fibers and continuously regulated by sensory feedback. Modulators applied exogenously or by stimulation of identified neuromodulatory neurons can functionally “rewire” each CPG circuit to produce different stable patterns. What are the major take home messages ?

• Sensory inputs provide feedback to both CPGs and assist in producing effective and smooth oscillatory behavior as well as dealing with cycle-by -cycle modifications due to sensory inputs. • The core circuits, i.e. the minimal network that can generate the pattern, for both the gastric mill and the pyloric rhythm can be modeled and may be best described with a winnerless competition algorithm made up of asymmetric inhibitory synapses. A model CPG circuit, based on principles learned from the stomatogastric system and implemented in analog hardware can act as a controller for a robotic lobster leg. By using presynaptic inhibition and sensory feedback, this small network can provide a closed loop microcontroller for forward and backward stepping.

Biography

• Some synaptically isolated pyloric neurons are chaotic. When the same neurons are synaptically reconnected, their activity becomes regularized. The purpose of this covert chaotic activity is not yet clear but may be to smooth the overall motor pattern as well as make it more adaptable. • Electronic neurons can replace biological neurons that have been inactivated thus helping to determine the role of each neuron in the overall operation of the circuit. Each neuron is unique and each plays a unique role.

Al Selverston received his A.B. from the University of California, Berkeley in 1962, and his PhD from the University of Oregon in 1967. He was a Post-Doc in the laboratory of Donald Kennedy at Stanford University before joining the Biology Department at the University of California, San Diego in 1969. After rising to the rank of full professor, he left UCSD to become the Director of the Institute of Neurobiology in San Juan Puerto Rico in 1997. Returning to UCSD in 2001, he is now research professor at the Institute of Nonlinear Science. Selverston was a Grass Fellow

• Pyloric CPGs from different animals connected with a dynamic clamp show how best to couple separate unit oscillators. Reciprocal inhibitory connections connected to driver cells can produce stable out-of-phase patterns. The same connections to particular non-driver cells produce stable in-phase patterns. Changing the strengths of the

13

Dynamical principles for neuroscience and intelligent biomimetic devices

in Neurophysiology at Woods Hole, a NIH physiology trainee at the University of Oregon and a Postdoctoral Fellow at Stanford. He received a Guggenheim Fellowship to the National Center of Scientific Research in Marseille France in 1975, received a Humboldt Senior Scientist Award to the Max Planck Institute of Comparative Physiology in 1982 and Fulbright and Royal Society of England Fellowships to Cambridge University in 1991. He was an Associate Editor of the Journal of Neurophysiology from 1988 to 1999.

References [1] M. Falcke, R. Huerta, M.I. Rabinovich, H.D.I. Abarbanel, and A.I. Selverston. Modeling observed chaotic oscillations in bursting neurons: the role of calcium dynamics and IP3. Biol. Cybernetics, 82:517–572, 2000 [2] A. Szucs, P. Varona, A. Volkovskii, H.D.I. Abarbanel, M.I. Rabinovich, and A.I. Selverston. Interacting biological and electronic neurons generate realistic oscillatory rhythms. Neuroreport, 11:563–569, 2000 [3] A. Szucs, R.D. Pinto, Rabinovich. M.I., and A.I. Selverston. Synaptic modulation of the interspike interval signatures of bursting pyloric neurons. J. Neurophysiology, 89:1363–1377, 2003 [4] R.C. Elson, A.I. Selverston, H.D.I. Abarbanel, and M.I. Rabinovich. Inhibitory synchronization of bursting in biological neurons: Dependence on synaptic time constant. J. Neurophysiology, 88:1166–1176, 2002 [5] A. Scucs, H.D. Abarbanel, M.I. Rabinovich, and A.I. Selverston. Dopamine modulation of spike dynamics in bursting neurons. Eur. J. Neurosci., 21:763–772, 2005 [6] M. Denker, A. Szucs, R.D. Pinto, H.D. Abarbanel, and A.I. Selverston. A network of electronic neural oscillators reproduces the dynamics of the periodically forced pyloric pacemaker group. IEEE Trans Biomed Eng, 52:792–798, 2005 [7] A.I. Selverston. A neural infrastructure for rhythmic motor patterns. Cell Mol Neurobiol, 25:223– 244, 2005

14


The intrinsic function of a motor system – from ion channels to network and behaviour S. Grillner Nobel Institute for Neurophysiology, Department of Karolinska Institutet, SE-171 77 Stockholm, Sweden [email protected]

Abstract The neuronal networks underlying locomotion in lamprey and the initiation of goal directed motor behaviour have been studied in some detail. The motor activity is affected by fore brain dopamine system, acting on the basal ganglia. The segmental locomotor network consists of ipsilateral excitatory glutamatergic and inhibitory crossed glycinergic interneurons, which are responsible for the fast synaptic interaction, but slow metabotropic receptors (5-HT, GABA, mGluR) also contribute for fine tuning of cellular properties. In addition, there is a sensory movement-related input to the network from ipsilateral excitatory and crossed inhibitory stretch receptor neurons that help adapt the movements to external events. For the pattern generation the intrinsic properties of the different network neurons play a critical role. One focus will be on the role of different subtypes of Ca2+ channels and Ca2+ and Na+ dependent K+ channels for neuronal network function. The modulation of different ion channel subtypes affects neuronal function and causes thereby characteristic changes at the network level. Different modulators like aminergic and peptidergic transmitters often exert neuron and synapse specific effects. Modulators like tachykinins, in addition to short term effects, also have effects on the cellular and network levels that are dependent on protein synthesis and last more than 24 hours. In the lamprey network it is possible to bridge from the cellular to the behavioral level and predict what changes a modulation of a given type of ion channel in a given cell type will have on the network level. The experimental analysis has gone hand in hand with extensive modeling based on a detailed knowledge of the membrane properties of the neurons that generate the motor pattern. The neurons have been modeled with sodium, potassium, calcium and calcium dependent potassium channels making them perform in a similar way as their biological counterparts. The populations of interneurons have been connected with inhibitory (glycine; chloride current) and excitatory (AMPA and NMDA receptors) model synapses to form the segmental and intersegmental network under-

Figure 1: Schematic representation of the brainstemspinal cord network controlling locomotion in lamprey. The core burst generating circuit is the excitatory interneurones (E). The inhibitory (I) neurons generate the left right alternation. The network is activated from the supraspinal command areas, the mesencephalic (MLR) and diencephalic (DLR) locomotor regions via the reticulospinal (RS) system. Sensory input to the network is provided by stretch receptors(SR) that sense the ongoing movements.

15


lying locomotion. In addition, the sensory control has been modeled as well as the neuronal basis of steering. In this way we have been able to generate segmental networks that generate alternating locomotor activity in the same frequency range as found experimentally, and also the coordination underlying the intersegmental phase coupling and steering. These properties have also been included in neuromechanical models of lamprey locomotion.

Institute (NRP, 1989-), Awards: Grass lecturer Society of Neuroscience, Boston 1983; Greater Nordic prize of Eric Fernström (1990), Bristol-Myers Squibb award for Distinguished achievements in Neuroscience Research, New York (1993), Reeve/Irvine Research Medal, New York 2002, Neuronal Plasticity Prize, Fondation Ipsen, Paris 2003 (together with F. Clarac and S. Rossignol, Erlanger lecture, IUPS, San Diego 2005, Ralph Gerard Prize, SFN, Washington, (2005)).

References

Biography

[1] O. Ekeberg and S. Grillner. Philos Trans R Soc Lond B Biol Sci, 354(1385):895–902, 1999 [2] S. Grillner. Nature Rev. Neurosci., 4:573–586, 2003 [3] S. Grillner, J. Hellgren, A. Ménard, K. Saitoh, and M. Wikström. Mechanisms for selection of basic motor programs–roles for the striatum and pallidum. Trends Neurosci., 28(7):364–370, 2005 [4] S. Grillner, H. Markram, E. De Schutter, G. Silberberg, and F.E.N. LeBeau. Microcircuits in action – from CPGs to neocortex. Trends Neurosci., 28(10):525–533, 2005 Sten Grillner has focused on the astounding capability of the brain to control movements, and in particular the cellular bases of vertebrate motor behavior. Early on he demonstrated that networks (CPGs) within the mammalian spinal cord can produce the detailed motor pattern of locomotion, which involves the coordination of hundreds of different muscles, and furthermore that sensory input provides important feedback signals to the CPGs. Subsequently, he and his colleagues took on the task of analyzing the intrinsic function of these networks by developing the lamprey as a vertebrate model system. The cellular basis of locomotion, steering and posture is now understood to a significant degree in this biological (and computational) model system, and the basic design appears conserved from cyclostomes to primates in this respect it is currently the best understood vertebrate motor system. Sten Grillner became Bachelor of Medicine (1962), Dr of Medicine PhD (1969), Docent in Physiology (1969 - 1975), Visiting Scientist Academy of Science Moscow (5mo 1971); Karolinska institutet: Professor Dept. of Physiology III (1975-1986), Director Nobel Institute for Neurophysiology (1987- ); Chairman Department of Neuroscience (1993-2000); He has served in the Nobel Committee and Nobel Assembly for Physiology or Medicine as member and chairman. Academies: Member Academiae Europaeae (1990- ), Royal Swedish Academy of Science (1993-, chairman Section for Biology (2004- )), Member American Academy of Arts and Sciences (2004), Neuroscience

Acknowledgement The Swedish research council VR-M, VR-NT, Wallenberg Foundation and EC commission (Neurobotics).

16


Dynamic reflex interactions during locomotion S. Rossignol Department of Physiology, Center for Research in Neurological Sciences Pavillon P.-G. Desmarais, Université de Montréal, C.P. 6128, succursale Centre-ville Montréal (Québec) H3C 3J7 [email protected]

Locomotion is produced by the dynamic interactions of lumbar and cervical Central Pattern Generators (CPG in Figure 1) which provide specific bilateral signals (i=ipsilateral; co=contralateral) to various limb motoneurones (St, Semitendinosus, a knee flexor; Sart, Sartorius, a hip flexor and knee extensor; VL, Vastus Lateralis, a knee extensor, GM, Gastrocnemius Medialis, an ankle extensor and Tri, Tricep Radialis, an elbow extensor). The CPG is in constant dynamic interactions with afferent inputs from muscles, joints and skin as well as with descending inputs coursing through the Dorsolateral or Ventrolateral Funiculi (DLF & VLF)) and providing either on-off/steering/ correcting commands or general neurochemical modulation [3, 5]. This talk will discuss more specifically how sensory inputs dynamically interact with spinal and supraspinal mechanisms. A key question concerns the role of sensory inputs to the final output locomotor pattern. It has been proposed that proprioceptive afferents may provide up to 50% of the amplitude of extensor bursts. Cutaneous denervation of the t in otherwise normal cats is rapidly compensated by other sensory inputs suggesting that cutaneous inputs normally contribute not much to locomotion [2]. However, after spinalisation at the last thoracic vertebra, although cats recover hindlimb locomotion [5, 6, 7], such bilateral cutaneous denervation impedes correct foot placement [1] suggesting that the dynamic sensorimotor interactions between cutaneous inputs and the the spinal CPG is critical for the correct expression of the locomotor behaviour on the treadmill. Dynamic sensorimotor interactions are best studied during perturbations applied in various phases of the step cycle[8]. Phasic and tonic proprioceptive inputs may participate in the timing as well as in the amplitude of muscle discharges [4, 9, 10]. Such inputs can enhance, reduce, delay or phase advance muscle activity, depending on the phase of the step cycle in which they are applied. Cutaneous inputs arising from a perturbation in different parts of the cycle will generate a variety of responses which are well-adapted to correct the movements of the limb specifically in those phases of the cycle. Thus a perturbation of the dor-

Figure 1: General framework of the quadripartite control of locomotion. sum of the hind foot during swing gives rise to a flexion of the limb so that the foot is firstly withdrawn and placed above and in front of the obstacle. The same stimulus during stance does not generate such unwanted flexion responses but acts rather on extensor muscles (inhibition or excitation). Various mechanisms appear to contribute to such dynamic control. Presynaptic inhibition can completely turn off transmission through some sensory pathways in certain phases of locomotion. Although changes in excitability of the motoneurones, due to the locomotor drive potential and

17


References

non-linear membrane properties, can contribute to the amplitude of reflex responses, they cannot account for the distribution of the reflex responses within the cycle since the reflex responsiveness and the locomotor activity of a given muscle are often dissociated. Finally, and more importantly, interneuronal selection is probably the main mechanism through which appropriate alternative pathways are activated to evoke motor responses which are adequate for the phase of the locomotor cycle. Taken together, these results suggest that the design of a control system for locomotion must include a variety of task- and phase-dependent sets of cutaneous and proprioceptive reflexes to insure adequate responses are generated in various phases of the movement.

[1] L.J.G. Bouyer and S. Rossignol. Contribution of cutaneous inputs from the hindpaw to the control of locomotion: 2. spinal cats. J. Neurophysiol., 90:3640–3653, 2003. [2] L.J.G. Bouyer and S. Rossignol. Contribution of cutaneous inputs from the hindpaw to thecontrol of locomotion: 1. intact cats. J. Neurophysiol., 90:3625–3639, 2003. [3] S. Grillner. Control of locomotion in bipeds, tetrapods, and fish. In J.M Brookhart and V.B Mountcastle, editors, Handbook of physiology. The nervous system II, pages 1179–1236. Amer. Physiol. Soc., Bethesda, Maryland, 1981.

Biography

[4] K.G. Pearson and S. Rossignol. Fictive motor patterns in chronic spinal cats. J. Neurophysiol., 66:1874–1887, 1991. [5] S. Rossignol. Neural control of stereotypic limb movements. In L.B. Rowell and J.T. Sheperd, editors, Handbook of Physiology, Section 12. Exercise: Regulation and Integration of Multiple Systems,, pages 173–216. Oxford University Press, New York,, 1996. [6] S. Rossignol, M. Blanger, C. Chau, N. Giroux, E. Brustein, L. Bouyer, C.-A. Grenier, T. Drew, H. Barbeau, , and T. Reader. The spinal cat. In R.G. Kalb and S.M. Strittmatter, editors, Neurobiology of spinal cord injury, pages 57–87. Humana Press, Totowa, 2000.

S. Rossignol received an M.D. (66) and an MSc (69) from Universite de Montreal and a PhD from McGill in 73 (G. Mevill Jones). After postdoctoral studies with S. Grillner (73-75) he returned to Universite de Montreal. He was named full professor in the Department of Physiology in 83 and was director of the Center for Research in Neurological Sciences from 19922004, director of the FRSQ Group for Research on the Central Nervous System from 1996-2004, director of the CIHR Group in Neurological sciences from 19962003, director of the Quebec Mental Health and Neuroscience Network from 02-05. He received the LéoPariseau Prize in 98 (from ACFAS), the Christopher Reeve Medal and Prize in 99, shared the Ipsen Prize for Neural Plasticity with Grillner and Clarac in 02. He holds a Tier 1 Canada Research Chair on the Spinal cord since 2000. He has authored 125 articles and book chapters and and 226 abstracts.

[7] S. Rossignol, C. Chau, N. Giroux, E. Brustein, L. Bouyer, J. Marcoux, C. Langlet, D. Barthlemy, J. Provencher, H. Leblond, H. Barbeau, and T.A. Reader. The cat model of spinal injury. In L. McKerracher, G. Doucet, and S Rossignol, editors, Spinal cord trauma: regeneration, neural repair and functional recovery, volume 137, pages 151–168. Elsevier, New York, 2002. [8] S. Rossignol, R. Dubuc, and J.-P. Gossard. Dynamic sensorimotor interactions during locomotion,. Physiological. Reviews, 86:89–154, 2006. [9] P. Saltiel and S. Rossignol. Critical points in the forelimb fictive locomotor cycle and motor coordination: evidence from the effects of tonic proprioceptive perturbations in the cat. J. Neurophysiol., 92:1329–1341, 2004. [10] P. Saltiel and S. Rossignol. Critical points in the forelimb fictive locomotor cycle of the cat and motor coordination: Effects of phasic retractions and protractions of the shoulder in the cat. J. Neurophysiol., 92:1342–1356, 2004.

18


The Impact of Stress on Memory and Synaptic Plasticity Carmen Sandi Brain Mind Institute, EPFL, CH-1015, Lausanne, Switzerland [email protected]

Abstract Stress exerts potent and paradoxical effects on brain and cognitive function. Depending on the circumstances, it can either facilitate or impair memory processes. Emotionally arousing experiences generally lead to stronger memories than more ordinary events, whereas high stress levels seem to interfere with the retrieval of previously acquired memories. The findings of Yerkes and Dodson (1908) when performing experiments in mice almost a century ago, led to propose a non-linear inverted-U shape relationship between arousal and learning, a principle well known in Psychology as the Yerkes and Dodson Law (see Figure 1). In this talk, I will discuss experiments aimed to dissociate the main factors accounting for the interactions between stress and memory in a spatial learning task in rats, the Morris water maze. A factorial approach is applied including the following different levels of analysis:

Figure 1: The traditional inverted-U shaped function between arousal and cognitive performance, based on Yerkes and Dodson (1908).

1. The source of stress (i.e., intrinsic or extrinsic to the task) 2. The nature of the stressor (i.e., specific or unspecific to task demands)

Biography

3. The stress intensity (i.e., low, moderate, or high) 4. The stress from the individual (i.e., anxiety trait and state). A stress-related positive linear function on emotional (fear) memory will be proposed to interact with cognitive processes involved in spatial orientation to eventually determine an inverted-U shape function between stress intensity and spatial learning. Finally, the neurobiological mechanisms that translate stress actions into such diverse cognitive outcomes (facilitating vs. impairing effects on memory function) will be discussed, including:

Carmen Sandi is Director of the Laboratory of Behavioral Genetics at the Brain Mind Institute, EPFL, Lausanne, Switzerland, that she joined in 2003. Previously, since 1996, she was Professor of Psychobiology at the National University of Distance Education in Madrid, Spain, where she led the ”Stress and Memory” Research Group. She graduated in Salamanca, Spain, as BSc, MSc in Psychology (1984) and obtained a subsequent Master’s degree, in Clinical Psychology in Madrid. She did her PhD. in Behavioral Neurobiology at the Cajal Institute, Madrid, and obtained the Doctoral degree at the University Autonoma of Madrid (1988). Then, she pursued

• The key modulatory role of stress hormones on memory function; and • Electrophysiological and pharmacological evidence implicating the amygdala on synaptic plasticity processes occurring in the hippocampus and prefrontal cortex.

19


postdoctoral training at the University of BordeauxINSERM and The Open University, U.K, where she underscored the importance of individual differences on the impact of stress on neuroendocrine and immune systems and became interested on the interactions between stress and cognitive function. The work of her group focuses on the role and mechanisms of action of stress on brain and cognitive function. Their work has demonstrated a key role of glucocorticoid hormones on the facilitating actions of stress on memory consolidation. They have also done major contributions to understand the detrimental effects of chronic stress and traumatic experiences in brain function and cognition. In particular, work in her group has implicated cell adhesion molecules as key mediators of stress and memory interactions. She has published more than 70 research articles in prestigious journals, as well as several books on the field of stress and cognitive function. She has received several awards, including a Serono Research Prize. She has been member of the executive committee of the European Brain and Behaviour Society (EBBS; 19992002). She has been organizer of multiple symposia and member of several advisory/program committees for the organization of scientific conferences, including a series of EURESCO conferences on the ”neural mechanisms of learning and memory”, the 35th EBBS annual meeting, the 1st International Haifa Forum for Brain and Behavior, and the 37th annual meeting of the International Society of Psychoneuroendocrinology (ISPNE) Conference.

and endocrine correlates of individual differences in spatial learning ability. Learning and Memory, 11:244–252, 2004 [6] I. Akirav, M. Kozenicky, D. Tal, C. Sandi, C. Venero, and G. Richter-Levin. A facilitative role for corticosterone in the acquisition of a spatial task under moderate stress. Learning and Memory, 11:188–195, 2004 [7] I. Akirav, C. Sandi, and G. Richter-Levin. Differential activation of hippocampus and amygdala following spatial learning under stress. European Journal of Neuroscience, 14:719–725, 2001 [8] M. I. Cordero, J. J. Merino, and C. Sandi. Direct relationship between shock intensity and corticosterone secretion on the establishment and longterm retention of contextual fear conditioning. Behavioral Neuroscience, 112:885–891, 1998 [9] C. Sandi, M. Loscertales, and C. Guaza. Experience and time-dependent effects of corticosterone on spatial memory in rats. European Journal of Neuroscience, 9:637–642, 1997 [10] G. A. Metz, N. M. Jadavji, and L. K. Smith. Modulation of motor function by stress: a novel concept of the effects of stress and corticosterone on behaviour. European Journal of Neuroscience, 22:1190–1200, 2005

References [1] C. Sandi. Stress, cognitive impairment and cell adhesion molecules. Nature Reviews Neuroscience, 5:917–930, 2004 [2] D.M. Diamond, C.R. Park, A.M. Campbell, and J.C. Woodson. Competitive interactions between endogenous ltd and ltp in the hippocampus underlie the storage of emotional memories and stressinduced amnesia. Hippocampus, 15:1006–1025, 2005 [3] B. Roozendaal. Systems mediating acute glucocorticoid effects on memory consolidation and retrieval. Progress Neuropsychopharmacology Biological Psychiatry, 27:1213–1223, 2004 [4] K. Touyarot and C. Sandi. Mid-life stress and cognitive deficits during early aging in rats: Individual differences and hippocampal correlates. Neurobiology of Aging, 27:128–140, 2006 [5] C. Sandi, M.I. Cordero, J.J. Merino, N.D. Kruyt, C.M. Regan, and K.J. Murphy. Neurobiological

20


Modularity, synchronization, and what we may learn from the brain Jean-Jacques Slotine Massachusetts Institute of Technology Cambridge, MA 02139, USA [email protected]

Abstract

• Stable polyrhythmic aggregates of arbitrary size can be constructed recursively, motivated by evolution and developmemt.

Although neurons as computational elements are 7 orders of magnitude slower than their artificial counterparts, the primate brain grossly outperforms robotic algorithms in all but the most structured tasks. Parallelism alone is a poor explanation, and much recent functional modelling of the central nervous system focuses on its modular, heavily feedback-based computational architecture, the result of accumulation of subsystems throughout evolution. We discuss this architecture from a global stability and convergence point of view. We then study synchronization as a model of computations at different scales in the brain, such as pattern matching, temporal binding of sensory data, and mirror neuron response. Finally, we derive a simple condition for a general dynamical system to globally converge to a regime where multiple groups of fully synchronized elements coexist. Applications of such ”polyrhythms” to some classical questions in robotics and systems neuroscience are discussed.

• Just as global synchronization occurs naturally and quantifiably in networks of locally coupled oscillators, it can be turned off by adding a single inhibitory connection. • In vision and pattern recognition, detectors for various types of symmetries can be systematically constructed.

Biography

The development makes extensive use of nonlinear contraction theory, a comparatively recent analysis tool whose main features will be briefly reviewed. In particular, • Global results on synchronization can be obtained using most common models of neural oscillators (such as integrate-and-fire, FitzHugh-Nagumo,or Izhikevich). Jean-Jacques Slotine was born in Paris in 1959, and received his Ph.D. from the Massachusetts Institute of Technology in 1983. After working at Bell Labs in the computer research department, in 1984 he joined the faculty at MIT, where he is now Professor of Mechanical Engineering and Information Sciences, Professor of Brain and Cognitive Sciences, and Director of the Nonlinear Systems Laboratory. He is the co-author of the textbooks ”Robot Analysis and Control” (Wiley, 1986) and ”Applied Nonlinear Control” (Prentice-Hall, 1991). Prof. Slotine was a member of the French National Science Council from 1997 to 2002.

• Since contraction is preserved under most common system combinations (parallel, hierarchies, negative feedback), it represents a natural framework for motor primitives. • In locomotion, the analysis exhibits none of the topological difficulties that may arise when coupling large numbers of phase oscillators, and it guarantees global exponential convergence. • Replacing ordinary CPG connections by filters enables automatic frequency-based gate selection.

21


References [1] W. Lohmiller, J.-J. Slotine. On Contraction Analysis for Nonlinear Systems. Automatica 34(6), 1998. [2] J.-J. Slotine, W. Lohmiller. Modularity, Evolution, and the Binding Problem: A View from Stability Theory. Neural Networks, 14, 2001. [3] J.-J. Slotine. Modular Stability Tools for Distributed Computation and Control. Int. J. Adaptive Control and Signal Processing, 17(6), 2003. [4] W. Wang, J.-J. Slotine. On Partial Contraction Analysis for Coupled Nonlinear Oscillators. Biological Cybernetics, 2004. [5] J.-J. Slotine, W. Wang, K. El Rifai. Contraction Analysis of Synchronization and Desynchronization in Networks of Hybrid Nonlinear Oscillators, Sixteenth International Symposium on Mathematical Theory of Networks and Systems (MTNS), 2004. [6] N.Tabareau, J.-J. Slotine. Notes on Contraction Theory, Technical Report 0503, MIT-NSL, 2005. [7] Q.-C. Pham, J.-J. Slotine. Stable Concurrent Synchronization in Dynamic System Networks, arxiv.org/abs/qbio.NC/0510051, 2005.

22


Nonlinear Dynamics, Synchronization and Applications in Neuroscience Oscar De Feo, Cristian Carmeli and Martin Hasler School of Computer and Communication Sciences Ecole Polytechnique Fédérale de Lausanne (EPFL) CH-1015, Lausanne, Switzerland [email protected]

Abstract

is suitable for the analysis of EEG signals. It measures the degree of a weak form of synchronization. EEG signals are aggregations of huge amounts of different neural activities that are in addition filtered by the skull. Therefore, we cannot expect to observe any strong and clean form of synchronization in EEG signals, whether or not clean synchronization is present among certain populations of neurons in the brain. We will show the results obtained by our synchronization measure for vision experiments on human subjects.

The time-evolution of nonlinear systems has a large variety of possible behaviours, ranging from quite regular behaviour such as convergence to the unique equilibrium point to the most exotic behaviour, namely chaos. In the very regular case, the qualitative behaviour of the motion is the same as for linear systems and quantitatively series expansion approaches allow analyzing such systems quite efficiently. This is by no means the case when the time evolution becomes more distinctively nonlinear. Series expansion methods do not make sense anymore and quantitative analytic results cannot be obtained, except in very special circumstances. Therefore, analysis concentrates on qualitative properties of the motion and quantitative results have to rely on numerical simulation. We shall give a short introduction to the various qualitative behaviours of nonlinear dynamical systems. In neuroscience, nonlinear effects are omnipresent. Therefore, the mathematical discipline of nonlinear dynamics should be able to contribute in an essential way to the understanding of the functioning of the brain. While this discipline certainly has made important contributions to computational neuroscience, in particular with models for single neurons and synapses, at the level of collective dynamics of networks of neurons, the contributions are much more modest. Synchronization is the simplest and the most striking collective behaviour of interacting dynamical systems. Synchronization phenomena are studied in many different scientific and engineering contexts, and it is a subject of mathematics itself. There are various notions of synchronization, ranging from the strongest, namely global complete synchronization, to much weaker forms, such as phase synchronization, or a simple reduction of the size of an attractor. We shall briefly introduce these notions. The question of the role that synchronization plays in information processing in the brain, and what form of synchronization plays a role, has been extensively debated and still needs further research. Our contribution fits into this context. We will introduce a measure for synchronization that

Synchronization among a population of neurons implies an interaction of a certain strength between these neurons. A more direct way to study such interactions is to build functional models from multivariate data. If a single scalar output signal of a nonlinear dynamical system is measured, the system itself can be reconstructed using the delay embedding method. If a number of signals are measured simultaneously, in principle the multi-output system could be identified directly. However, this task becomes quickly intractable and simplifying assumptions have to be made. This means that we move from “black-box” to “grey-box” modelling. For the case of weakly interacting dynamical systems, we can proceed as follows. For each measured signal, we reconstruct separately a nonlinear dynamical system using the delay embedding method. Then we assume linear interaction between these systems and construct the interaction matrix by a classical linear multi-input multi-output system identification method. We shall illustrate this procedure. For the case of a network integrate-and-fire models reconstructed from multivariate spike trains, we will outline a special method that takes advantage of the fact that after each spike the membrane voltage is reset to a rest value. These works belong to the general field of nonlinear dynamical systems identification. An impressive amount of literature has been published on this subject, without leading to definite and general solutions, however.

23


Biography

References [1] S. Strogatz. Nonlinear Dynamics and Chaos. With applications to Physics, Biology, Chemistry, and Engineering. Addison Wesley Publishing Company, 1994 [2] A. Pikovsky, R. Rosenblum, and J. Kurths. Synchronization, A universal concept in nonlinear sciences, volume 12 of Cambridge Nonlinear Science Series. Cambridge University Press, Cambridge, UK, 2001 [3] W. Singer. Putative role of oscillations and synchrony in cortical signal processing and attention. In L. Itti, G. Rees, and J.K. Tsotsos, editors, Neurobiology of Attention, pages 526–533. Elsevier, 2005

Martin Hasler received the Diploma in 1969 and the PhD degree in 1973 from the Swiss Federal Institute of Technology, Zurich, both in physics.

[4] V. Belykh, I. Belykh, and M. Hasler. Connection Graph Stability Method for Synchronized Coupled Chaotic Systems. Physica D, 195(1-2):159– 187, 2004

He continued research in mathematical physics at Bedford College, University of London, from 1973 to 1974. At the end of 1974 he joined the Circuits and Systems group of the Swiss Federal Institute of Technology Lausanne (EPFL), where he was given the title of a Professor in 1984. He became associate and full professor, respectively, in 1996 and 1999. In 2002, he was acting Dean of the newly created School of Computer and Communication Sciences of EPFL.

[5] I. Belykh, E. de Lange, and M. Hasler. Synchronization of Bursting Neurons: What Matters in the Network Topology. Physical Review Letters, 94(18):8101, 2005 [6] C. Carmeli, M.G. Knyazeva, G.M. Innocenti, and O. De Feo. Assessment of eeg synchronization based on state-space analysis. NeuroImage, 25(2):339–354, April 2005

During the 70s, his research was concentrated on filter theory and design, in particular active and switched capacitor filters. Since 1980 his research is centered on nonlinear circuits and systems, including the qualitative analysis of resistive and dynamic circuits, the modeling and identification of nonlinear circuits and systems, neural networks and the engineering applications of complicated nonlinear dynamics, in particular chaos. Chaos is applied to the transmission of information and to signal processing. Among the applications of the modeling and identification of nonlinear systems is the modeling of high-temperature superconductors for energy applications. Very recently, he is interested in the qualitative behavior and the information processing capabilities of networks of nonlinear dynamical systems, be they biological (brain) or technical (computer networks).

[7] O. De Feo and C. Carmeli. Identifying dependencies among multivariate time series. In International Conference on Nonlinear theory and its applications (NOLTA), volume CDROM, 2004 [8] V.A. Makarov, F. Panetsos, and O. De Feo. A method for determining neural connectivity and inferring the underlying network dynamics using extracellular spike recordings. Journal of Neuroscience Methods, 2005

He is a Fellow of the IEEE. He was the chairman of the Technical Committee on Nonlinear Circuits and Systems IEEE CAS Society from 1990 to 1993. From 1993 to 1995 he was the Editor of the IEEE Transactions on Circuits and Systems, Part I. He was the Chairman of ISCAS 2000, Geneva, Switzerland. He was a member of the Board of Governors of the IEEE CAS Society and until the end of 2005 its Vice-President for Technical Activities. He is a member of the Scientific Council of the Swiss National Science Foundation since Jan. 2000.

24


What makes a neuron spike? Phase resetting and intrinsic dynamics G. Bard Ermentrout Department of Mathematics, University of Pittsburgh [email protected]

References

Abstract

[1] B. Ermentrout and D. Saunders. Phase resetting and coupling of noisy neural oscillators. J. Computational Neuroscience, 2006. To appear

What aspects of a stimulus cause a neuron to fire? How do stimulis affect the time of spikes? In this talk, I will discuss what we can learn about neuronal firing patterns by regarding neurons as nonlinear oscillators. The spike-triggered average or reverse correlation method is a common approach for determining what kinds of stimuli make a neuron fire. The poststimulus time histogram is another experimental measurement for describing the affect of a stimulus on the firing pattern of a neuron. Both of these curves should be affected by the membrane properties of the individual neuron of interest. Since this is a huge-dimensional space, we will focus on one property of neurons which has been shown to be tightly coupled to neuronal dynamics: the phase resetting curve (PRC). The PRC describes the shift in the timing of a spike due to a brief stimulus as a function of the time since the last spike. We show that under certain circumstances there is a 1:1 mapping between the STA, the PSTH, and the PRC. Thus, we connect internal dynamics of neurons with their preferred stimuli and their population responses. This work is joint with Boris Gutkin, Alex Reyes, Nathan Urban, Roberto Galan, and Nicolas Fourcaud.

[2] R.F. Galan, N. Fourcaud, N. Urban, and B. Ermentrout. Stochastic synchrony: A simple mechanism for generating beta/gamma oscillations. J. Neuroscience, 2006. To appear [3] R.F. Galan, G.B. Ermentrout, and N.N. Urban. Efficient estimation of phase-resetting curves in real neurons and its significance for neural-network modeling. Phys Rev Lett, 94(15), 2005 [4] B.S. Gutkin, G.B. Ermentrout, and A.D. Reyes. Phase-response curves give the responses of neucrons to transient inputs. J Neurophysiol., 94(2):1623–53, 2005 [5] B. Gutkin, G.B. Ermentrout, and M. Rudolph. Spike generating dynamics and the conditions for spike-time precision in cortical neurons. J Comput Neurosci, 15(1):91–103, 2003 [6] B. Ermentrout, M. Pascal, and B. Gutkin. The effects of spike frequency adaptation and negative feedback on the synchronization of neural oscillators. Neural Comput, 13(6):1285–310, 2001

Biography

[7] B.S. Gutkin and G.B. Ermentrout. Dynamics of membrane excitability determine interspike interval variability: a link between spike generation mechanisms and cortical spike train statistics. Neural Comput, 10(5):1047–65, 1998 [8] P. Goel and B. Ermentrout. Synchrony, stability, and firing patterns in pulse-coupled oscillators. Physica D, 163(3–4):191–216, 2002 [9] Alla Borisyuk, G. Bard Ermentrout, Avner Friedman, and David Terman. Tutorials in Mathematical Biosciences I : Mathematical Neuroscience. Lecture Notes in Mathematics / Mathematical Biosciences Subseries. Springer, NY, 2005

Bard Ermentrout, trained in Theoretical Biology, has been in the Mathematics Department at the University of Pittsburgh since 1982. He has joint appointments in Computational Biology and Neuroscience. He is a University Professor of Computational Biology and heads the Mathematical Biology group at Pitt. He is interested in dogs and parrots.

25


26


Synaptic Plasticity from an Optimality Viewpoint Wulfram Gerstner Laboratory of Computational Neuroscience School of Computer and Communication Sciences and Brain Mind Institute Ecole Polytechnique Fédérale de Lausanne (EPFL) 1015 Lausanne [email protected]

Abstract

Biography

Joint work with: T. Toyoizumi, J.-P. Pfister, and K. Aihara. Synaptic connections between neurons change depending on neuronal activity and other factors. We tried to understand the rules controlling these changes from an optimality viewpoint. Maximization of information transmission by a spiking neuron model predicts changes of synaptic connections that depend on timing of pre- and postsynaptic spikes as well as on the postsynaptic membrane potential. Under the assumption of Poisson firing statistics, the synaptic update rule exhibits all the features of the Bienenstock-CooperMunro (BCM) rule, in particular regimes of synaptic potentiation and depression separated by a sliding threshold. Moreover, the learning rule is also applicable to the more realistic case of neuron models with refractoriness and is sensitive to correlations between input spikes, even in the absence of presynaptic rate modulation. The learning rule is found by maximizing the mutual information between presynaptic and postsynaptic spike trains under the constraint that the postsynaptic firing rate stays close to some target firing rate. An interpretation of the synaptic update rule in terms of homeostatic synaptic processes and Spike Timing Dependent Plasticity is presented.

Wulfram Gerstner has studied physics in Tübingen and Munich. After a couple of short research stays (Berkeley, Oxford, Courant Institute) he moved in 1996 to the EPFL where he heads the Laboratory of Computational Neuroscience.

References [1] T. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner (2005) Generalized Bienenstock-Cooper-Munro rule for spiking neurons that maximizes information transmission Proc. Natl. Acad. Sci. USA, 102:5239-5244 [2] T. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner (2005) Spike-timing Dependent Plasticity and Mutual Information Maximization for a Spiking Neuron Model IN: Advances in Neural Information Processing Systems 17 , edited by L.K. Saul and Y. Weiss and L. Bottou (MIT-Press), pp. 1409-1416

27


28


Reproducible transient dynamics of neural circuits: Generation and processing of sequences Mikhail I. Rabinovich Institute for Nonlinear Science, University of California, San Diego [email protected]

Abstract

Biography

A traditional way of analyzing neural computations is with the use of attractors. This means transformation of a given input - an initial state inside of the basin of an attractor to a fixed desired output. A few years ago the author and his colleges introduced a new concept of computation based on a winnerless competition (WLC) principle and on transient but stable heteroclinic sequences (SHS). According to this principle, the incoming stimulus is transformed into a complex temporal output based on the intrinsic sequential dynamics of a network with WLC. It has been shown in the models that neural circuits with non-symmetric inhibitory synapses demonstrate sequentially switching WLC dynamics that is stable and depends on the incoming information. The WLC principle is able to explain how real neural circuits can solve the fundamental contradiction between stability and flexibility. An appropriate set of mathematical models for competition phenomena is based on generalized Lotka-Volterra models. These models describe the cooperative dynamics of an arbitrary number of competitive agents that can be dynamical elements themselves. The connections between agents can be complex and even random. In general, the stimulus dependent transient dynamics of a microcircuit follows a complicated or even a chaotic trajectory in phase space. Transient sequential representation of olfactory information has been observed in the locust and bee antennal lobes. The interaction of excitatory and inhibitory neural ensembles in a model of brain microcircuits also leads to the SHS. Nonautonomous i.e. sensory dependent activities of central pattern generators of animals can be explained in the framework of transient WLC dynamics also. In particular, it has been shown that the irregular hunting swimming of the marine mollusk Clione is the result of the chaotic dynamics of a gravimetric sensory network, consisting of interconnected inhibitory neurons. The WLC concept looks very promising for the creation of artificial brains that can be used for the control of autonomous biomimetic robots.

Mikhail (Misha) Rabinovich (1941) received the Ph.D. degree in physics from Nizhny Novgorod University, Russia, in 1967. Since 1974 he has been a Full Professor of radiophysics with Nizhny Novgorod University. Currently, he is a Research Scientist with the Institute for Nonlinear Science, University of California, San Diego. He is a member of the Russian Academy of Science (http://inls.ucsd.edu/˜rabin).

References [1] M. Rabinovich, A. Volkovskii, P. Lecanda, R. Huerta, H.D.I. Abarbanel, and G. Laurent. Dynamical encoding by networks of competing neuron groups: Winnerless competition. Phys Rev LEtt, 87(6):8102, 2001 [2] G. Laurent, M. Stopfer, R.W. Friedrich, M.I. Rabinovich, A. Volkovskii, and H.D.I. Abarbanel. Odor encoding as an active, dynamical process: Experiments, computation, and theory. Annual Review of Neursocience, 24:263–297, 2001 [3] V.S. Afraimovich, V.P. Zhigulin, and M.I. Rabinovich. On the origin of reproducible sequential activity in neural circuits. Chaos, 14(4):1123–9, 2004 [4] R. Huerta and M. Rabinovich. Reproducible sequence generation in random neural ensembles. Phys Rev Lett, 93(23), 2004 [5] M. Rabinovich, R. Huerta, and P. Varona. Heteroclinic synchronization: Ultra-subharmonic locking. Phys Rev Lett, 2006 [6] R. Levi, P. Varona, Y. Arshavsky, M. Rabinovich, and A. Selverston. The role of sensory network

29


dynamics in generating a motor program. Journal of Neuroscience, 25(42):9807, 2004 [7] A. Venaille, P. Varona, and M. Rabinovich. Synchronization and coordination of sequences in two neural ensembles. Phys Rev E, 71(061909), 2005 [8] V. Afraimovich, M. Rabinovich, and P. Varona. Heteroclinic contours in neural ensembles and the winnerless competition principle. J. Bifurcation and Chaos, 14(4):1195, 2004 [9] P. Seliger, L. Tsimring, and M. Rabinovich. Dynamics-based sequential memory: Winnerless competition of pattern. Phys Rev E, 67, 2003 [10] P. Ashwin and M. Timme. When instability makes sense. Nature, page 36, 2005

30


Neural network model of primary visual cortex: from functional architecture to lateral connectivity and back M. Tsodyks Dept. of Neurobiology, Weizmann Institute, Rehovot, Israel [email protected]

Abstract

Israel, and joined the group of Haim Sompolinsky and Daniel Amit in Jerusalem that was active in modeling associative memory and sensory processing. After few years in Jerusalem, I moved to the Salk Institute in San Diego, to become a postdoctoral fellow in the lab of Terry Sejnowski. Since returning to Israel in 1995, I hold a faculty position at the Dept. of Neurobiology of Weizmann Institute of Science in Rehovot, where I continue my work in computational neuroscience in collaboration with several experimental laboratories. I held visiting positions at EPFL, Lausanne, on 0204/2004, and at Ecole Normale Supérieure, Paris, on 10/2005-02/2006. Since 2005, I am an adjunct fellow at the Frankfurt Institute for Advanced Studies (FIAS).

Recent optical imaging studies of the primary visual cortex (V1) have shown that activity patterns similar to single condition orientation maps (OMs) emerge during spontaneous activity. It has been argued that the lateral interactions between V1 neurons are responsible for the formation of the OMs in the absence of visual stimulation. We suggest a neural network model of V1 in which the OMs are encoded in the lateral connections. Our proposed connectivity pattern depends on the preferred orientation and, unlike previous models, on the degree of orientation selectivity of the interconnected neurons. An analytical study shows that the network has a ring attractor composed of an approximated version of each of the OMs. Thus, activity patterns similar to OMs can emerge spontaneously when the network is presented with an unstructured noisy input. We perform simulations and show that the model can be applied to experimental optical imaging data and generate realistic OMs. We also study a variation of the model with spatially restricted connections, and show that it can give rise to activity patterns that are composed of several OMs corresponding to different orientations.

References [1] R. Ben-Yishai, R. Lev Bar-Or, and H. Sompolinsky. Theory of orientation tuning in visual cortex. Proc. Natl. Acad. Sci. USA, 92:3844–3848, 1995 [2] M. Tsodyks, T. Kenet, A. Grinvald, and A. Arieli. Linking spontaneous activity of single cortical neurons and the underlying functional architecture. Science, 286:1943–1946, 1999 [3] U. Ernst, K. Pawelzik, C. Sahar-Pikielny, and M. Tsodyks. Intracortical origin of visual maps. Nature Neuroscience, 4:431–436, 2001

Biography

[4] T. Kenet, D. Bibitchkov, M. Tsodyks, A. Grinvald, and A. Arieli. Spontaneously emerging cortical representations of visual attributes. Nature, 425:954–956, 2003 [5] J.A. Goldberg, U. Rokni, and H. Sompolinsky. Patterns of ongoing activity and the functional architecture of the primary visual cortex. Neuron, 42:489–500, 2004

I studied theoretical physics in Moscow, in the Landau Institute of Theoretical Physics. Shortly after completing my PhD, I got interested in neural network theory. This interest developed further when I moved to

31


32


Neural Circuits as Analog Computers Wolfgang Maass Institute for Theoretical Computer Science, Technische Universitaet Graz, A-8010, Graz, Austria [email protected]

Abstract

Biography

1. I will begin with an examination of computational properties of a detailed model for a laminar cortical microcircuit. This model is based on recent data by Alex Thomson et al regarding connections between cortical layers, and data by Henry Markram and Alain Destexhe regarding dynamical models for neurons and synapses with rather high levels of noise that appear to be characteristic for in-vivo conditions. It turns out that this data based neural circuit defines a dynamical system that has interesting computational properties, which distinguish it from a variety of control models that are lacking the characteristic structure of stereotypical cortical microcircuits. Details can be found in [Häusler and Maass, 2006].

Phd (1974) and Habilitation (1978) in Mathematics at the Ludwig-Maximilians-Universitaet in Munich. From 1979 to 1984 research at MIT, the University of Chicago, and the University of California at Berkeley as Heisenberg-Fellow of the Deutsche Forschungsgemeinschaft.

2. I will then examine the theoretical limitations of analog computing by such generic models for neural circuits under the assumption that a few neurons within the circuit are trained for specific tasks (or equivalently, if the circuit receives feedback from suitably trained readout neurons). Both the theoretical analysis and computer simulations of generic cortical microcircuit models suggest that the capacity for analog computing of the resulting dynamical systems is in fact extremely large. We present a mathematical result which implies that under idealized conditions such circuits become universal models for analog computation, i.e. a fixed circuit can be used to simulate any conceivable analog computer (not only every Turing machine). In particular, they are able to process analog input streams in diverse ways according to their present internal state, and they are able to integrate analog information and control movements over behaviorally relevant long time spans in spite of biologically realistic levels of noise. Details can be found in [Maass, Joshi, and Sontag, 2005] and [Joshi and Maass, 2005].

From 1982 - 1986 Associate Professor and from 1986 - 1993 Professor of Computer Science at the University of Illinois in Chicago. Since 1991 Professor of Computer Science at the Technische Universitaet Graz in Austria, and since 1992 Head of the new ”Institut fuer Grundlagen der Informationsverarbeitung” at the Technische Universitaet Graz. Sloan Fellow at the Computational Neurobiology Lab of the Salk Institute (La Jolla, USA) during 1997/98. 9/2002 - 2/2003: Visiting Professor at the BrainMind Institute, EPFL, Lausanne, Switzerland. Since 2005: Adjunct Fellow of the Frankfurt Institute of Advanced Studies (FIAS)

33


References [1] S. Häusler and W. Maass. A statistical analysis of information processing properties of laminaspecific cortical microcircuit models. Cerebral Cortex. In press [2] W. Maass, P. Joshi, and E. D. Sontag. Principles of real-time computing with feedback applied to cortical microcircuit models. In Advances in Neural Information Processing Systems, volume 18. MIT Press, 2006 [3] W. Maass, P. Joshi, and E. D. Sontag. Computational aspects of feedback in neural circuits. 2005. Submitted for publication [4] P. Joshi and W. Maass. Movement generation with circuits of spiking neurons. Neural Computation, 17(8):1715–1738, 2005 [5] A. Kaske and W. Maass. A model for the interaction of oscillations and pattern generation with real-time computing in generic neural microcircuit models. Neural Networks. In press

34


Can Biological Neurons and Micro Electronic Devices collaborate for a common and useful task? Gwendal Le Masson∗ , André Garenne∗ Thierry Bal† Université Victor Segalen Bordeaux 2, INSERM 358- 33077 Bordeaux, France∗ UNIC, CNRS UPR 2191, Institut A. Fessard, 91198, Gif-sur-Yvette, France† [email protected]

Abstract

Biography

A lot of emphasis has been made since the last ten years about interfacing neural tissue and micro electronic circuits or system and many new technologies have been developed for this goal [5, 6]. The usual paradigms to justify these approaches are divided into two groups of questions: (1) how can we chronically, influence, assist or repair an injured nervous system by using an electronic device ultimately implanted. That is the field of Neuro-prosthesis. (2) Can we teach biological neurons to perform engineering tasks, ranging from basic calculation to complex robotic control? Of course theres also a common benefit expected for these researches about what we can learn about the nervous system through real time, close looped and direct neuro-artificial communications. Different anatomic levels have been used to implement hybrid interfaces from the channel level (dynamic clamp [7]) to global signal such as EEG (Brain Machine Interface[1]) or implanted cortical electrodes. Every level imposes its own technological limits, but very common principles seem to govern the long-term dynamics of hybrid networks. To our view, the constant equilibrium between stability and plasticity is a key concept that has to been taken into account for collaboration of neuro-biological and artificial components towards a real and useful task. Many basic homeostatic [3] as well as plasticity rules are involved in such hybrid network and even more specifically when the connections intend to be chronic. Through specific examples of hybrid circuits weve been working on since the last ten years [4, 2], we will try to clarify the technical but also the conceptual limits and perspectives of these research areas.

Gwendal Le Masson is a senior research scientist at the French INSERM as well as a Professor of clinical Neurology at the University of Victor Segalen Bordeaux 2. He obtained both his PhD in Neurosciences and his MD in Bordeaux and did his postdoctoral fellowship in Pr Eve Marder lab in collaboration with Pr L. Abbott in Brandeis University and center for complex systems. He worked very early on the dynamical clamp technique on different in vitro preparation from small invertebrate networks to cortico-thalamic or spinal cord slices. His current research topics related to long term changes in cortical networks in culture maintained chronically into hybrid closed loop interactions. The morphological and functional changes of these long-term perturbations are monitored.

References [1] In M.C. Roco and W.S. Bainbridge, editors, Converging Technolgies for Iproving human Performances. 2002. NSF/DOC sponsored report. [2] D. Derjean, S. Bertrand, G. Le Masson, M. Landry, V. Morisset, and F. Nagy. Dynamic balance of metabotropic inputs causes dorsal horn neurons to switch functional states. at. Neurosci., 6(3):274– 81, 2003.

35


[3] G. Le Masson and L.F. Abbott. Activity dependent regulation of conductances in model neurons. Science, 259:1915–7, 1993. [4] G. Le Masson, S. Renaud-Le Masson, D. Debay, and T. Bal. Feedback inhibition controls spike transfer in hybrid thalamic circuits. Nature, 417(20):854–8, 2002. [5] W.L.C. Rutten. Selective electrical interfaces with the nervous system. Biomed. Eng., 4:407–52, 2002. [6] A.B. Schwartz. Cortical neural prosthetics. Annu. Rev. Neurosci., 27:487–507, 2004. [7] A.A. Sharp, M.B. O’Neil, L.F. Abbott, and E. Marder. The dynamic clamp: artificial conductances in biological neurons. Trends Neurosci, 16(10):389–94, 1993.

36


Spike transfer properties of thalamic neurons in ’hybrid networks’ in vitro Thierry Bal∗, Damien Debay∗, Jakob Wolfart∗, Alain Destexhe∗, Sylvie Renaud† and Gwendal LeMasson‡ ∗ UNIC,CNRS UPR 2191, Institut A. Fressard, 91198, Gif-sur-Yvette, France [email protected] † CNRS UMR 5818, ENSEIRB-Univ. Bordeaux 1, 33405, Talence, France [email protected] ‡ INSERM E358, Institut F. Magendie, Univ. Bordeaux 2, 33076, Bordeaux, France [email protected]

Abstract

ground could be a mechanism by which corticothalamic feedback dynamically regulates the thalamic relay of sensory information. Previous work has shown that the understanding of the responsiveness of central neurons requires a detailed knowledge of their intrinsic properties, which are mediated by various calcium- and voltage-dependent conductances [5]. Our results suggest that background activity alters this responsiveness fundamentally. We suggest that a complete characterization of the properties of central neurons requires the knowledge of intrinsic and synaptic background conductances [6], as well as the amount of conductance fluctuations (”noise”).

The response of thalamic and cortical neurons to individual synaptic inputs is potentially influenced by local synaptic interactions and by the overall state of the network, in a continuum from sleep to waking and attentiveness [1]. We examine these properties in the thalamus using hybrid technology in which a biological neuron is connected in vitro to silicon and computer-based simulated neurons through artificial synaptic connections [2] using dynamic clamp [3] (see also abstract by G. Le Masson). In these hybrid networks, individual membrane currents of the simulated and biological neurons and the properties of their synaptic connections can be selectively and quantitatively controlled throughout their dynamic range. The thalamus is the major gateway for the flow of sensory information to the cerebral cortex. In early stages of sleep, when sensory perception drops, this structure is the source of robust network synchronized oscillations (spindle waves) in the 6 to 14 Hz frequency range (Fig. 1, left). We examine the role of these thalamic oscillations in the gating of synaptic inputs. We show that feedback inhibition from cells of the thalamic reticular nucleus controls spike transfer in thalamocortical cells in a state-dependant manner. During waking and attentiveness (Fig.1, right), thalamocortical neurons relay sensory information to the cortex and receive synaptic feedback, the function of which is unclear. We studied the influence of artificial synaptic bombardment, mimicking the cortical feedback, on the response of thalamocortical cells by injecting stochastically fluctuating mixed excitatory/inhibitory background conductances [4]. The conductance background modulated the input/output gain, increasing the sensitivity to small inputs, and reduced the influence of T-type calcium channels. In addition, it increased the occurrence of burst firing at resting potentials. Therefore, gain modulation via synaptic back-

Biography

Thierry Bal is a senior research scientist at the french CNRS. He obtained a PhD in Neurosciences at the University of Bordeaux, France on the neurobiology of small neural networks in invertebrates. After 4 years of postdoctoral work at the Yale University scholl of medicine, New Haven, USA, in the laboratory of Pr. D.A. McCormick, he joined the CNRS in 1995 and is currently leader of the research group : ”Cybernetics of thalamic and cortical networks” at the Integrative and Computational Neuroscience Research Unit led by Yves Frégnac in Gif-sur-yvette, France (http://www.unic.cnrs-gif.fr/). His current research in-

37


terests include the modulation of sensory information processing in the thalamus and cortex by top-down background synaptic activity, using hybrid biological and neuromimetic neuronal microcircuits and slices that spontaneously generate patterned activity.

References [1] D.A. McCormick and T. Bal. Sleep and arousal: thalamocortical mechanisms. Annu. Rev. Neurosci., 20:185–215, 1997 [2] G. Le Masson, S. Renaud-Le Masson, D. Debay, and T. Bal. Feedback inhibition controls spike transfer in hybrid thalamic circuits. Nature, 417:854–858, 2002 [3] A.A. Prinz, L.F. Abbott, and E. Marder. The dynamic clamp comes of age. Trends Neurosci., 27:218–224, 2004 [4] J. Wolfart, D. Debay, G. Le Masson, A. Destexhe, and T. Bal. Synaptic background activity controls spike transfer from thalamus to cortex. Nat. Neurosci., 8:1760–1767, 2005 [5] R.R. Llinas. The intrinsic electrophysiological properties of mammalian neurons: insights into central nervous system function. Science, 242:1654–1664, 1988

Figure 1: Two functional facets of the thalamic gate. Center: synaptic organization of the thalamocortical network. Red: inhibitory; blue/gray: excitatory; LGNd: dorsal lateral geniculate nucleus; PGN: perigeniculate nucleus. During sleep, the activity of the thalamic cells is dominated by intrinsic membrane properties and robust synchronized slow oscillation symbolized here by a spindle wave (left bottom records). These phenomenon are associated with a drop of conscious sensory perception. Upon arousal, thalamic cells are in high conductance states that change drastically their integrative properties. These “active” states are due to the presence of background synaptic bombardments resulting from the activity of recurrent synaptic networks in cortical and corticothalamic networks. Artificial synaptic “noise” is injected in the cells using dynamic clamp. Graphic shows the modulation of input-output gain by the noise. Modified from [4].

[6] A. Destexhe, M. Rudolph, and D. Pare. The highconductance state of neocortical neurons in vivo. Nat. Rev. Neurosci., 4:738–751, 2003

38


Neuron-Semiconductor Interfacing – its Nature and Implementation Peter Fromherz Department of Membrane and Neurophysics Max Planck Institute for Biochemistry Munich, Germany [email protected]

Abstract

the Max Planck Society. His present interests are the interfacing of semiconductor chips with neuronal systems and the development of voltage-sensitive dyes for brain research.

It is a challenge to join semiconductor chips and neuronal systems on a microscopic level with the goal to develop hybrid processors for scientific, medical and technological applications. On the biological side, we consider the three levels of ion channels, of individual nerve cells from snails and rats and of tissue from rat brain. We analyze the structure and the electrical resistance of the cell-silicon contact with luminescent dyes. To elucidate the mechanism of interfacing, we study the interaction of recombinant sodium and potassium channels with capacitors and transistors of simple silicon chips. On the basis of these results, we implement the two-way interfacing of silicon chips with individual nerve cells, with small neuronal networks and with brain tissue. CMOS chips with thousands of interfacing sites are applied to obtain time-resolved maps of for recording and stimulation on the level of single cells, of small neuronal networks and of brain tissue.

References [1] P. Fromherz. The neuron-semiconductor interface. In I. Willner and E. Katz, editors, Bioelectronics from theory to applications, pages 339– 394. Wiley-VCH, Weinheim, 2005 [2] D. Braun and P. Fromherz. Fluorescence interferometry of neuronal cell adhesion on microstructured silicon. Phys Rev Lett, 81:5241–5244, 1998 [3] R. Gleixner and P. Fromherz. The extracellular electrical resistivity in cell adhesion. Biophys. J. In press. DOI: 10.1529 [4] M. Schmidtner and P. Fromherz. Functional Na+channels in cell adhesion probed by transistor recording. Biophys. J., 90:173–182, 2006

Biography

[5] M.H. Ulbrich and P. Fromherz. Opening of K+ channels by capacitive stimulation from silicon chip. Appl. Phys. A, 81:887–891, 005 [6] R. Weis, B. Müller, and P. Fromherz. Neuronadhesion on a silicon chip probed by an array of field-effect transistors. Phys Rev Lett, 76:327– 330, 1996 [7] R. Schätzthauer and P. Fromherz. Neuron-silicon junction with voltage-gated ionic currents. J. Neurosci., 10:1956–1962, 1998 Peter Fromherz, is a director at the Max Planck Institute for Biochemistry in Martinsried/Munich and professor for Biophysics at the Technical University Munich. He completed his PhD in Physical Chemistry in 1969 at the University Marburg. Subsequently, he lead a research group at the Max Planck Institute for Biophysical Chemistry in Göttingen. In 1981 he became a full professor for Experimental Physics at the University Ulm. Since 1994 he is a scientific member of

[8] M. Voelker and P. Fromherz. Signal transmission from individual mammalian nerve cell to fieldeffect transistor. Small 1, pages 206–210, 2005 [9] G. Zeck and P. Fromherz. Noninvasive neuroelectronic interfacing with synaptically connected snail neurons immobilized on a semiconductor chip. Proc. Natl. Acad. Sci. USA, 98:10457– 10462, 2001

39


[10] A. Lambacher, M. Jenkner, M. Merz, B. Eversmann, R.A. Kaul, F. Hofmann, R. Thewes, and P. Fromherz. Electrical imaging of neuronal activity by multi-transistor-array (MTA) recording at 7.8µm resolution. Appl. Phys. A, 79:1607–1611, 2004 [11] M. Hutzler and P. Fromherz. Silicon chip with capacitors and transistors for interfacing organotypic brain slice of rat hippocampus. Eur. J. Neurosci., 19:2231–2238, 2004

40


Interesting computations performed by collections of recurrently connected neurons, and their implementation in hybrid VLSI Rodney Douglas Institute of Neuroinformatics, ETH and University of Zurich, 8059, Zurich, Switzerland [email protected]

Abstract

general relational processor that, probably like the neocortex, uses the sWTA as its principle element.

Animal brains are much more effective in dealing with real-world tasks than even the most advanced computers. In vertebrates the neocortex is very likely the subsystem of the vertebrate brain most relevant for intelligent and effective interaction with the world, and is one region where we can hope to understand the relationship between neuronal architecture and the computation that it supports. Fortunately, the evidence shows that cortex has a surprisingly uniform architecture [4, 5]. Since the fundamental work of Gilbert and Wiesel on the neuronal circuits of visual cortex, it has seemed likely that the basic architecture and operation of cortex can be understood in terms of relatively few types of excitatory and inhibitory neurons. We have now developed a quantitative description of the circuits formed in cat area 17 by estimating the ’weights’ of the interconnections between different neuronal types [2]. A dominant feature of this circuit is the high degree of connectivity between pyramidal cells in the superficial cortical layers, suggesting that the fundamental computational process of cortex depends on direct recurrence between these pyramids [5]. Populations of such recurrently connected neurons can implement ’soft Winner-Take-All’ (sWTA) circuits that have interesting computational properties, that are quite different to conventional computing circuits [6, 7]. For example analog amplification and digital multistability are generally seen as incompatible functions and are separated into two classes of electronic technology. But in the neocortical circuits multistability can coexist with analog responses. The sWTA circuits exhibit population coding, gain modulation, focal attention (signal selection), and spatiotemporal pattern generation, all of which are characteristics of neocortical computation [3, 1, 11, 10], and they can be fabricated in custom Very Large Scale Integrated electronic circuits composed of either rate- or spiking-neurons [7, 9, 8]. Although the properties of sWTA’s are now well understood, and there are many specific examples of how they can be used in particular neuronal and technological applications, there is little understanding of how one could build a general processor with these circuits. In this talk I will describe our steps towards building a

Biography

Rodney Douglas is Professor of Neuroinformatics, and Co-Director at the Institute of Neuroinformatics of the Swiss Federal Institute and the the University of Zürich. He graduated in Science and Medicine at the University of Cape Town. After obtaining a Doctorate in Neuroscience, he moved to the Anatomical Neuropharmacology Unit in Oxford, where he continued his research on the anatomy and biophysics of the microcircuitry of cerebral together with Kevan Martin. As Visiting Associate, and then Visiting Professor at Caltech, he extended his research interests in neuronal computation to the modeling of cortical circuits using digital methods (together with Christof Koch), and also by the fabrication of analog VLSI circuits (together with Misha Mahowald). In 1996 he and Kevan Martin moved to Zurich to establish the Institute of Neuroinformatics. In 2000, Douglas was awarded the Körber Foundation prize for European Science, together with four colleages: Amiram Grinwald (Weizmann), Christoph von der Malsburg (U. Ruhr-Bochum), Randolph Menzel (FU Berlin) Wolf Singer (MPG Frankfurt). This was the first occasion in which the Körber Prize has been awarded for neuroscience. Douglas’s current research interests include; experimental anatomy and physiology of visual cerebral cortex; theoretical analysis and simulation of cortical circuits; design and fabrication of neuromorphic systems

41


that exploit analog Very Large Scale Integration methods to construct electronic circuits that perform analogous signal processing and computational functions to biological neuronal networks; and the development of neuromorphic robots that use analog VLSI sensory motor chips.

References [1] Ben-Yishai B. and Bar-Or R.L and Sompolinsky H., Theory of orientation tuning in visual cortex. , Proceedings National Academy of Science USA, 92(9)3844-8, 1995. [2] Binzegger, T. and Douglas, R. and Martin, K. A quantitative map of the circuit of cat primary visual cortex, Journal of Neuroscience, 24:(39) 8441-53, 2004 [3] Douglas, R. J. and Koch, C. K. and Mahowald, M. and Martin, K. A. C. and Suarez, H. H, Recurrent Excitation in Neocortical Circuits, Science, 269:981–985, 1995. [4] Douglas,R. J. and Martin K. A. C. and Witteridge D., A canonical microcircuit for neocortex , Neural Computation,1:480–488, 1989. [5] Douglas, R. and Martin, K. Neuronal Circuits of the Neocortex, Annual Review of Neuroscience, 27: 41951, 2004. [6] Hahnloser R., and Douglas, R., and Mahowald, M. and K. Hepp, Feedback interactions between neuronal pointers and maps for attentional processing, Nature Neuroscience, 2:746-752, 1999. [7] Hahnloser, R. and Sarpeshkar, R. and Mahowald, M. and Douglas, R.J. and Seung, S., Digital selection and analog amplification co-exist in an electronic circuit inspired by neocortex, Nature, 405:947–951, 2000. [8] Indiveri, G. and Chicca, E. and Douglas, R. A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity, IEEE Transactions on Neural Networks, 2005, (In Press) [9] Liu, S.-C. and Kramer, J. and Indiveri, G. and Delbruck, T. and Douglas R., Analog VLSI: Circuits and Principles, MIT Press, 2002. [10] Poggio, T. and Bizzi, E., Generalization in vision and motor control, Nature, 43:768-774, 2004. [11] Salinas E. and Abbott L.F, A model of multiplicative neural responses in parietal cortex, Proceedings National Academy of Science USA, 93(21):11956-61, 1996.

42


Control of the spinal cord by an analog VLSI device: on the road to development of a neuroprosthetic device for spinal cord injury patients Jacob Vogelstein∗ , Ralph Etienne-Cummings∗ , and Avis H. Cohen† Johns Hopkins University∗ University of Maryland† [email protected]

Abstract

to be implanted in the patient. At this stage, we expect that prior to control by our device, a stimulator of the type used by Dr. Milan Dimitrijevic, and his group, will first initiate locomotion. Our device will then serve as a controller for the ongoing pattern to guarantee that the motor output is appropriate and adaptive. The bulk of the chip will be analog circuits that are very small and draw small amounts of current. The device will be wholly self-contained with the logic and adaptive synapses to permit learning on chip using floating gate technology.

After spinal cord injury, the spinal cord below the injury remains capable of generating the full motor pattern and some control of sensory responses during locomotion. The main obstacle to locomotion in the absence of spinal regeneration is that there is no way to initiate activity or control it to make it adaptive with regard to the world. We know this to be true even in humans, in which it has now been shown that the spinal cord below a lesion site can be stimulated to produce alternating activity resembling locomotion (Dimitrijevic, cf. below). As one part of multipart system to restore locomotion, my colleagues and I have begun development of a silicon chip that can serve as a neuroprosthetic device for spinal cord injury patients. I will discuss 1) organization of the proposed system, 2) the preliminary work we have done to interface with the wetware, 3) the design of the hardware, and the software 4) and the principles of its operation. We have begun this process working with the relatively simple spinal cord of the primitive lamprey. Indeed, we have been working with the isolated spinal cord, to make the preparation even simpler. With this preparation, we have begun stimulation and analysis of activity from a fictively swimming spinal cord. Ongoing fictive swimming is initiated by bath application of D-glutamate, and recorded from the motor nerves with bipolar electrodes. We present perturbations as single pulses of stimulation to the hemicord of one end of the spinal cord segments. A special purpose stimulator delivers pulses with their timing and pulse characteristics generated by computer. The resulting outputs are analyzed with software developed for the purpose. The analysis employs a modified Phase Response Curve method, or PRC. We expect to incorporate real time analysis of output from motor nerves to allow perturbations at pre-determined phases. This will allow us to predict how the stimulation should alter the rhythm or the phases of the outputs, and test the validity of our algorithms for controlling ongoing activity of locomotion. Ultimately, the entire system: stimulator, recording amplifiers, and controllers will all be fabricated on chip

Biography

1970-77 1977-79 1979-80

1980-83

1983-90

1990-2002

43

PhD student, Cornell University Post-doctoral fellow Karolinska Institute (with Sten Grillner) Research Associate, Department of Physiology and Biophysics, Washington University, St. Louis, MO Research Associate, Section of Neurobiology and Behavior, Cornell University, Ithaca, NY Senior Research Associate, Section of Neurobiology and Behavior, Cornell University, Ithaca, NY Associate Professor, Department of Zoology, University of Maryland at College Park, College Park, MD


1996-1998

19981999-

2002-

2005-09

Founder and Director, Program in Neu- [7] R. Minassian, B. Jilge, F. Rattay, M.M. Pinter, roscience & Cognitive Science, UniverH. Binder, F. Gerstenbrand, and M.R. Dimitrijesity of Maryland at College Park, Colvic. Stepping-like movements in humans with lege Park, MD complete spinal cord injury induced by epiduCo-Director, Telluride Workshop for ral stimulation of the lumbar cord. Spinal Cord, Neuromorphic Engineering 42:401–416, 2004 Member, Institute for Systems Research, University of Maryland at College Park, College Park, MD. Professor, Department of Biology, and Institute for Systems Research, University of Maryland at College Park, College Park, MD. External Reviewer for ”Emergence of Adaptive Motor Function through Interaction between Body, Brain, and Environment - Understanding of Mobiligence by Constructive Approach, a grant from the Japanese Ministry of Education, Culture, Sports, Science and Technology

References [1] A.H. Cohen, L. Guan, V. Pate, and T. Kiemel. Temperature can alter the functional outcome of spinal cord regeneration in larval lampreys. Neuroscience, 90:957–965, 1999 [2] M.A. Lewis, R. Etienne-Cummings, M.J. Hartmann, Z.R. Xu, and A.H. Cohen. An in silico central pattern generator: Silicon oscillator, coupling, entrainment, and physical computation. Biological Cybernetics, 88:137–151, 2003 [3] Y. Fukuoka, H. Kimura, and A.H. Cohen. Adaptive dynamic walking of a quadruped robot on irregular terrain based on biological concepts. Int. J. of Robotics Research, 22:187–202, 2003 [4] A.H. Cohen, M. Abdelnabi, B. Bent, C. Coleman, L. Guan, A. Mitra, M.A. Ottinger, and L. Chakrabarti. Changes in distribution of serotonin induced by spinal injury in larval lampreys: evidence from immunohistochemistry and HPLC. J. Neurotrauma, pages 172–188, 2005 [5] J. Vogelstein, R. Etienne-Cummings, N. Thakor, and A.H. Cohen. Phase-dependent effects of stimulation of the spinal central pattern generator for locomotion. IEEE Transactions on Neural Systems and Rehabilitation Engineering. submitted [6] M.R. Dimitrijevic, Y. Gerasimenko, and M.M. Pinter. Evidence for a spinal central pattern generator in humans. Annals of the New York Academy of Sciences, 860:360–376, 1998

44


Technological and Bioelectrical Considerations for Neural Interfacing Microelectrode Arrays Ph. Renaud, P. Linderholm, A. Mercanzini and K. Cheung Microsyststems Laboratory, LMIS-EPFL Ecole Polytechnique Fédérale de Lausanne, CH-1015 Lausanne, Switzerland [email protected]

Abstract

tial change in the electrical characteristics as function of time (figure 4). Electrical consideration for recording electrodes: For recording electrodes, the magnitude of the electrical signal depends on the total impedance of the path between the source and the electrode in the considered frequency domain. This can be determined by electrical impedance spectroscopy (EIS) which consists of applying a small probe signal and recording impedance characteristics in the frequency range.

Building an interface between neural tissue and instrumentation is a key challenge in elucidating the detailed function of the nervous system and to development of future brain-machine interfaces. The progress of microtechnology has made possible important developments in microelectrode arrays to interface neurons and neural networks with electronic devices. One of the key issues in microelectrode arrays for recording and stimulation is the control of the electrode-tissue interface. In an electrolyte solution the metal electrolyte boundary can be described as a capacitance (due to electrode polarization) in parallel with resistive elements (an electron transfer resistance which is related to electrochemical processes and a Warburg element which is related to mass transport).

Figure 2: Typical EIS spectrum of a 50 µm diameter microelectrode in buffer solutions with three different conductivities (from ref. [3]). As depicted in figure 2, the low frequency range is dominated by interfacial capacitance and the plateau gives the resistance of the current path in the fluid to the return electrode. At higher frequencies, measured impedance is further reduced by coupling stray capacitance. If we consider the useful range for recording information from neuronal spiking activity to be 1 kHz to 10 kHz (1ms 0.1 ms), the impedance should be minimized for this range (e.g. by having a corrugated electrode surface: Pt black, TiN, etc.) Electrical consideration for stimulation electrodes: In stimulation, the charge injection (Re element in figure 1) is related, especially for small electrodes, to a faradaic process (because charges accumulated in the interface capacitance are not sufficient). In chronic stimulation, the electrochemical reaction can generate toxic by-products. Iridium oxide greatly reduces toxic effect because electrochemical reactions are mostly

Figure 1: Simplified electrical model for the electrodetissue interface (in stimulation). The electrode is represented by a capacitance C in parallel with the electrochemical transfer resistance Re . A first layer of cells(+ fibroblast + proteins) have specific resistance R2 which can be very different from plain tissue R1 . Zcell is the impedance of the cell which is only significant at high frequency. When an electrode is in contact with tissue, an impedance exists which is a resistive element in first order consideration. However, for chronic experiments the reaction of tissues in the vicinity of the electrodes induces changes in tissue parameters (new cells, protein layers, voids) which may also provoke a substan-

45


confined within the IrO2 layer and thus the safe charge injection limit is 10 times higher than a Pt surface (4 mC/cm2 ). For a localized stimulation, small electrodes placed in the vicinity of the cells are required. There is thus a compromise between spatial resolution and stimulation threshold. Electric impedance tomography: Because electric impedance of a tissue depends on its cellular morphology and organization, impedance tomography can be used to characterize a tissue sample in-situ. Microelectrode arrays can be used to determine the electrical properties of stratified biological samples and potentially offer a direct measurement of the tissue reaction around an implant. This principle has been demonstrated in-vitro with cell cultures [1]. Subretinal implants and EIS monitoring: Retinal implants require a high density of stimulation electrode. We have designed an integrated circuit to be placed in the sub-retinal space which comprises an array of phototransistors controlling local pulse generators activating a microelectrode [2].

Figure 4: Image of a 14 x 14 oscillating pixel array realized in CMOS technology. The PIN photodiode, the electrode and the metal shielding of the circuit are highlighted on the zoomed image on the right. One pixel measures 75 µm x 75 µm (from ref. [2]). tion channel into the tissue is prevented.

Figure 5: (a) SEM image cross-section of a polyimide probe showing buried microchannels and completely insulated, embedded electrical leads; (b) SEM image of polyimide probe with the two microelectrodes and two fluidic outlets; (c) SEM image of a U-tube polyimide microprobe tip showing four microelectrodes and one channel outlet of the U-tube in close proximity for fluid delivery. (from ref. [4]).

Figure 3: Image of a 14 x 14 oscillating pixel array realized in CMOS technology. The PIN photodiode, the electrode and the metal shielding of the circuit are highlighted on the zoomed image on the right. One pixel measures 75 µm x 75 µm (from ref. [2]).

Biography

The evolution of electrode-tissue interactions following implantation has been accessed by EIS in P23 rats [3]. The first day, the electrode impedance values are related to normal tissue parameters. During the first 30 days, the impedance steadily increases by a factor of 500 times the initial values and then stabilizes. Flexible neural probes with perfusion channels: As an alternative to silicon neural probes, flexible polymer materials have been proposed as a way to minimize tissue damage due to motional stress between the implant and brain and allow bending at the surface of the tissue. One can add microfluidics channels into the probe to locally deliver or collect fluids in the vicinity of the electrodes. Polymer probes with channel cross sections of 20 µm have been realized [4]. A three-way tubing geometry can be used for short injections of sample fluids. By constant circulation of a buffer in the two side channels, uncontrolled diffusion out of the injec-

Philippe Renaud (born in 1958) received his diploma in physics from the University of Neuchâtel, Switzerland (1983) and his Ph.D. degree from the University of Lausanne; Switzerland (1988). His thesis work was dedicated to the theoretical and experimental study of magnetoelastic effects.

46


Postdoctoral researcher at University of California, Berkeley, USA (1988-1989) to develop scanning tunneling microscopes for low-temperature and then at the IBM Zürich Research Laboratory in Switzerland (1990-1991) where he performed measurements of the local STM induced luminescence III-V semiconductors heterostructures. He also performed measurement of the magnetically induced polarization of the emitted light. In 1992, he joined the Sensors and Actuators group of the Swiss Center for Electronics and Microtechnology (CSEM) at Neuchâtel, Switzerland. He was involved in the design and the technology of mechanical microsensors and of micro-mirrors for optical switching. In 1993, assistant professor at the Swiss Federal Institute of Technology (EPFL). Until end of 1994, he remains part-time collaborator of CSEM. In summer 1996, visiting professor at the Tohoku University, Japan. In 1997, he is appointed as full professor at EPFL. His research interests are: microfabrication technologies for MEMS and microfluidics, BioMEMS applications. Since 1998, director of the EPFL Center of MicroNanoTechnology (CMI), a large clean room facility with processing equipment for training and scientific experimentation in microelectronic and microfabrication processes.

References [1] P. Linderholm, J. Vannod, Y. Barrandon, and Ph. Renaud. [bipolar resistivity profiling of 3D tissue culture. Biosensors and Bioelectronics, 2005. Submitted [2] D. Ziegler, S. Ferazzutti, P. Linderholm, M. Mazzab, D. Bertrand, A. M. Ionescu, and Ph. Renaud. An active microphotodiode array of oscillating pixels for retinal stimulation. Sensors and Actuators: A, 110:11–17, 2004 [3] P. Linderholm, J.-L. Guyomard, S. Picaud, and Ph. Renaud. Long-term in-vivo impedance changes for subretinal microelectrodes chronically implanted in P23. to be published (2006) [4] S. Metz, A. Bertsch, D. Bertrand, and P. Renaud. Flexible polyimide probes with microelectrodes and embedded microfluidic channels for simultaneous drug delivery and multi-channel monitoring of bioelectric activity. Biosensors and Bioelectronics, 19(10):1309–1318, 2004

47


48


Towards the development of a cybernetic hand: scientific, technological and clinical issues Maria Chiara Carrozza and Paolo Dario ARTS Lab, Scuola Superiore Sant’Anna Viale Rinaldo Piaggio 34, 56025 Pontedera (Pisa) - Italy [email protected], [email protected]

Abstract

3. Control: the prosthesis should restore perception and action capabilities of the human hand. The users intention must be naturally interpreted and in real time. Proprioception and exteroception abilities must be provided to the user by means of an appropriate artificial motor-sensory system connected to the brain, and in this framework the collaboration between neuroscience and robotics is fundamental for shaping hand system design according to the natural model.

This presentation addresses the problem of functional replacement of upper limb in amputees by implanting an advanced cybernetic prosthetic hand connected to the human brain. The problem of functional replacement of upper limb is an ancient problem: historically humans always have needed to replace the hand lost in war or accidents with a prosthesis, in general for vocational and personal autonomy purposes. Therefore, in contrast to the apparent simplicity of desired properties, and despite of recent technology advancements, it is extremely difficult for engineering design to match a specifications list, basically for technological limitations. It is therefore necessary to define priorities among the different requirements and to address separately some of them that are considered most important. An additional critical issue is that after years of experience in hands developments and several literature reviews, one of the most surprising results is that it is not possible to define an universal design rule for obtaining the priority in requirements for prosthetic hands, because this may depend from the psychological, cultural and geographical conditions of the amputee, and the single subject may have a very personal list of wishes and expectations. The human hand is not only an effective tool for grasping and expressing human intelligence and creativity, but also an ideal instrument to explore the external environment and it is connected with an ideal processing unit, the human brain. The cybernetic prosthetic hand aims to be directly interfaced to the brain in order to improve the performance of the prosthesis in terms of acceptability, functionality, and controllability. In general, to be successfully implanted, hand prostheses must respond to the following ideal requirements:

The ultimate goal is to replicate as much as possible the sensory-motor capabilities of the natural hand in terms of sensing and control. Ideally, the hand must be felt by an amputee as the lost natural limb delivering her/him natural sensory feedback by means of the stimulation of some specific afferent nerves. Biomechatronics and Robotics can be successfully applied to solve the problem of prosthetic hands, thus opening new opportunities in different novel applications such as humanoid robotics and neuro-robotics. The natural hand has three basic functionalities: grasping, manipulation and exploration. To accomplish the goal of restoring these capabilities by implanting an artificial hand, two fundamental modules are necessary: to develop an artificial hand equipped with artificial proprioceptive and exteroceptive sensors, and to fabricate an appropriate interface able to exchange sensory-motor signals with the amputees body and the central nervous system. In this presentation, the main scientific issues and the technological challenges related to prosthetic hand design, fabrication and implant will be introduced and the state of the art in this field will be critically analysed. The authors have already developed some artificial hands for robotic and prosthetic applications and their control system in the framework of the EUFET “CYBERHAND” project. Recent achievements and research efforts towards the realization of a cybernetic hand prosthesis are presented with specific focus on biomechatronic design and on the preliminary neural interface to be implanted in the PNS. In particular, the analysis of the natural low level control mechanisms and of the cognitive feedback information has

1. Functionality: the prosthetic device should perform a stable grasp and manipulation for performing vocational operations and activities of daily living; 2. Cosmetics: the prosthesis should have the same static and dynamic appearance of the human hand.

49


been used for designing the CYBERHAND prosthetic hand.

is currently responsible for the design of an Exoskeleton for functional support and enhancement of the upper limb, in the framework of the NEUROBOTICS project (http://www.neurobotics.org). In these projects, bioinspired design and the fusion between neuroscience and robotics are addressed for going “beyond robotics”. Prof. Carrozza is Member of IEEE RAS and EMBS societies and she is an author of several scientific papers and international patents. In addition, she is promoting industrial innovation and start-up creation, she is co-founder of two spin-off of the Scuola Superiore SantAnna and she is member of their Administrative Boards.

Biography

References [1] P. Dario, M.C. Carrozza, E. Guglielmelli, C. Laschi, A. Menciassi, S. Micera, and F. Vecchi. Robotics as a future emerging technology, biomimetics, cybernetics and neuro-robotics in european projects. IEEE Robotics and Automation Magazine, pages 29–45, June 2005

Maria Chiara Carrozza received the Laurea degree in physics from the University of Pisa, Pisa, Italy, in 1990. Since 2001, she has been an Associate Professor of biomedical robotics at the Scuola Superiore SantAnna, Pisa, Italy. She is Director of the Research Division and Deputy Director of Scuola Superiore SantAnna (http://www.sssup.it). She teaches Biomechatronics and Rehabilitation Engineering to Master students of Biomedical Engineering. at the University of Pisa She is elected Member of the national Board of the Italian association of Biomedical Engineering (Gruppo Nazionale di Bioinegegneria). Prof. Carrozza has been visiting professor at the Technical University of Wien, Austria with a graduate course entitled Biomechatronics, and she is involved in the scientific management of the Italy-Japan joint laboratory for Humanoid Robotics ROBOCASA, Waseda University, Tokyo, Japan where she is responsible for artificial hand design. Prof. Carrozza is the Coordinator of the Advanced Robotics Technology and Systems Laboratory (http://www.arts.sssup.it), founded by prof. Paolo Dario, where more than 50 peoples are involved in research projects aimed at design, simulation and development of biomedical robots for rehabilitation engineering, functional support and humanoid robotics. She is active in several national and international projects in the fields of biomechatronics and biomedical robotics. Her research interests comprise biomedical robotics (cybernetic and robotic artificial hands, upper limb exoskeletons), rehabilitation engineering (neurorehabilitation, domotic, and robotic aids for functional support and personal assistance), and biomedical microengineering (microsensors, tactile sensors). The Arts Lab team coordinated by Prof. Carrozza has designed and developed the CYBERHAND artificial hand (http://www.cyberhand.org) and

[2] L. Beccai, S. Roccella, A. Arena, F. Valvo, P. Valdastri, A. Menciassi, M.C. Carrozza, and P. Dario. Design and fabrication of a hybrid silicon threeaxial force sensor for biomechanical applications. Sensors and Actuators, A 120:370–382, 2005 [3] M.C. Carrozza, G. Cappiello, G. Stellin, F. Zaccone, F. Vecchi, S. Micera, and P. Dario. A cosmetic prosthetic hand with tendon driven underactuated mechanism and compliant joints: Ongoing research and preliminary results. In Proceedings of the 2005 IEEE International Conference of Robotics and Automation (ICRA 2005), pages 2672–2677, April 2005 [4] M.C. Carrozza, F. Sebastiani, C. Suppo, B. Massa, F. Vecchi, R. Lazzarini, M. Cutkosky, and P. Dario. The development of the SPRING hand: a self-adaptive hand prosthesis for restoring natural grasping. Journal of Autonomous Robots, 16(2):125–141, March 2004 [5] E. Cavallaro, G. Cappiello, S. Micera, M.C. Carrozza, P. Rantanen, and P. Dario. On the development of a biomechatronic system to record tendon sliding movements. IEEE Trans. Biomed. Eng., 52(6):1110–1119, June 2005 [6] M. Zecca, G. Cappiello, F. Sebastiani, S. Roccella, F. Vecchi, M.C. Carrozza, and P. Dario. Experimental analysis of the proprioceptive and exteroceptive sensors of an underactuated prosthetic hand. In Z. Zenn and S. Dimitar, editors, Advances

50


in Rehabilitation Robotics, Human-friendly Technologies on Movement Assistance and Restoration for People with Disabilities, Lecture Notes in Control and Information Sciences. SpringerVerlag, 2004 [7] S. Roccella, M.C. Carrozza, G. Cappiello, M. Zecca, H. Miwa, K. Itoh, M. Matsumoto, and A. Takanishi. Design, fabrication and preliminary results of a novel anthropomorphic hand for humanoid robotica: RCH-1. In Proc. of 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 266–271, September 28-October 2 2004 [8] M.C. Carrozza, B. Massa, and et al. Micera, S. The development of a novel prosthetic hand - ongoing research and preliminary results. IEEEASME Trans. of Mechatronics, 7(2):108–114, June 2002 [9] S. Micera, J. Carpaneto, M.A. Umilta, M. Rochat, L. Escola, V. Gallese, M.C. Carrozza, J. Krueger, G. Rizzolatti, and P. Dario. Preliminary analysis of multichannel recordings for the development of a high-level cortical neural prosthesis. In Neural Engineering, 2005. Conference Proceedings. 2nd International IEEE EMBS Conference on March 16-19, pages 136–139, 2005

51


52


Neural prosthetics Useful Signals from Motor Cortex Andrew B. Schwartz University of Pittsburgh School of Medicine [email protected]

Abstract

Pittsburgh where he has continued to develop cortical prosthetics along with investigations of how drawing movement is represented in frontal cortex

Advances in our ability to record activity from relatively large groups of individual neurons and the concurrent evolution in our view of systems neuroscience have opened new horizons in our understanding of high-level brain functions. One important aspect of this is our ability to decode intention from these signals. This can take place either by recognizing and categorizing patterns of activity across the population or by using basis functions on the response of individual neurons and weighting the net output of the population. Examples of this will be shown in the preparatory and ongoing neural activity of motor acts. By intercepting intention, prosthetic devices can replace desired movement in immobile individuals as demonstrated in a selffeeding task controlled entirely by recorded brain signals.

References [1] D.M. Taylor, S.I. Helms Tillery, and A.B. Schwartz. Direct cortical control of 3D neuroprosthetic devices. Science, 296:1829–1832, 2002 [2] D.W. Moran and A.B. Schwartz. Motor cortical representation of speed and direction during reaching. J. Neurophysiol., 82:2676–2692, 1999 [3] A.B. Schwartz, D.W. Moran, and G.A. Reina. Differential representation of perception and action in the frontal cortex. Science, 303:380–383, 2004 [4] A.B. Schwartz. Cortical neural prostheses. Ann. Rev. Neurosci., 27:487–507, 2004

Biography

[5] A.B. Schwartz and D.W. Moran. Arm trajectory and representation of movement processing in motor cortical activity. Eur. J. Neurosci., 12:1851– 1856, 2000

Dr. Schwartz received his PhD in physiology from the University of Minnesota in 1984. As a postdoctoral fellow, he worked with Apostolos Georgopoulos from 1984-1987 at Johns Hopkins, where they studied motor cortical representations of reaching. Schwartz was then appointed to a staff scientist position at the Barrow Neurological Institute where he developed a behavioral paradigm to study cortical activity during drawing movements. During this period and with collaborators at Arizona State University, he began work on cortical neural prosthetics which he continued after he moved to the Neurosciences Institute in 1995. Since 2002, he has been a professor at the University of

53


54


Emergence and Development of Embodied Cognition From Humanoid Robot Body to Humanoid Robot Brain Y. Kuniyoshi Dept. of Mechano-Informatics, School of Information Science and Technology The University of Tokyo [email protected]

Abstract

Biography

A new framework of robot-synthetic approach to developmental cognitive science is proposed, which attempts to provide a working model of development starting from embryo, reaching early social interaction. Accumulating evidences suggest that cortical representations self-organize in early development. Computationally, self-organization reflects the structure of incoming information. In very early development including the fetal period, bodily movements and corresponding sensory signals are the major source of information which guides the cortical development. In this paper, we first show that physics of human-like body in action already provide certain information structure. It is the intrinsic property of human-like embodiment that can be exploited for computationally inexpensive and robust ways of action control and perception. Experiments with a whole-body humanoid robot and on human perception are presented. Then, a novel model of coordinated motor pattern generation is presented. It is based on multiple chaotic elements that are coupled through body physics, and explores the potential bodily information structures, aka. body affordances. This model corresponds to the motor pattern generation in very early development. A robotic simulation of motor development of fetuses and neonates is presented. It consists of musculo-skeletal body model, the environment within uterus, a neural model of spine and medulla, and a self-organizing neural network model simulating the developing sensory and motor cortical areas. The neural networks self organizes to capture the information structure of the body. Finally, another sensory-motor learning model is presented which attempts to explain how fetal sensor-motor learning can give rise to neonatal imitation. The above approach aim at modelling the human development starting from subcortically driven motor exploration and reaching cognitive and social interactions.

Yasuo Kuniyoshi received his M.Eng. and Ph.D. degrees from the University of Tokyo, Japan, in 1988 and 1991, respectively. From 1991 to 2000, he was a Research Scientist and then a Senior Research Scientist at Electrotechnical Laboratory, AIST, MITI, Japan. From 1996 to 1997 he was a Visiting Scholar at MIT AI Lab. From 2001, he has been an Associate Professor at the University of Tokyo. Currently he is a Professor of the University of Tokyo at Department of Mechano-Informatics, School of Information Science and Technology. Yasuo Kuniyoshi is the author of over 200 technical publications, editorials and books. He received Outstanding Paper Award from International Joint Conference on Artificial Intelligence, Best Paper Award from Robotics Society of Japan, Sato Memorial Award for Intelligent Robotics Research and other awards. His research interests include Emergence and development of embodied cognition, Human action understanding systems, and Humanoid robots. He is a member of IEEE, Robotics Society of Japan, Japan Society for Artificial Intelligence, Japanese Society of Baby Science, and other societies.

References [1] Y. Kuniyoshi et. al. From humanoid embodiment to theory of mind. In Embodied Artificial Intelligence, volume 3139 of Lecture Notes in Artificial Intelligence. Springer, 2004 [2] Y. Kuniyoshi and H. Inoue. Qualitative recogni-

55


tion of ongoing human action sequences. In Proc. Intl Joint Conf. on Artificial Intelligence, pages 1600–1609. 1993 [3] Y. Kuniyoshi, M. Inaba, and H. Inoue. Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Trans. on Robotics and Automation, 10(6), 1994 [4] Y. Kuniyoshi and L. Berthouze. Neural learning of embodied interaction dynamics. Neural Networks, 11(7):1259–1276, 1998 [5] Y. Kuniyoshi et al. From visuo-motor self learning to early imitation – a neural architecture for humanoid learning. In Proc. IEEE Int. Conf. on Robotics and Automation, pages 3132–3139. 2003 [6] Y. Kuniyoshi et al. Embodied basis of invariant features in execution and perception of whole body dynamic actions — knacks and focuses of roll-and-rise motion. Robotics and Autonomous Systems, 48(4):189–201, 2004 [7] Y. Kuniyoshi and S. Suzuki. Dynamic emergence and adaptation of behavior through embodiment as coupled chaotic field. In Proc. IEEE Int. Conf. on Intelligent Robots and Systems, pages 2042– 2049. 2004 [8] S. Yonekura and Y. Kuniyoshi. Emergence of multiple sensory-motor response patterns from cooperating bursting neurons. In IEEE Int. Conf. on Intelligent Robots and Systems, pages 1377– 1382. 2004 [9] C. Nabeshima, M. Lungarella, and Y. Kuniyoshi. Timing-based model of body schema adaptation and its role in perception and tool use: A robot case study. In IEEE Intl Conf. on Development and Learning (ICDL-05), pages 7–12, 2005 [10] K. Ishiguro, N. Otsu, and Y. Kuniyoshi. Intermodal learning and object concept acquisition,. In IAPR Conf. on Machine Vision Applications, pages 148–151. 2005 [11] A. Pitti, M. Lungarella, and Y. Kuniyoshi. Quantification of emergent behaviors induced by feedback resonance of chaos. In Proc. of the 2nd Australian Conf. on Artificial Life. 2005 [12] Y. Kuniyoshi et al. Haptic detection of object affordances by a multi-fingered robot hand. Intl Journal of Humanoid Robotics. In press

56


Integrating insect behaviours in robot models Barbara Webb Institute for Perception Action and Behaviour University of Edinburgh, Edinburgh EH9 3JZ, United Kingdom [email protected]

Abstract

Biography

Robots can be successfully used to model biological systems and improve our understanding of the mechanisms underlying successful control [3, 4]. Isolated sensorimotor behaviours of insects, such as visual cues for flight control, or taxis towards attractive stimuli, can often be well characterised and may involve relatively small circuits of neurons. Hence they have been a productive area for such modelling. However, like most animals, insects need to integrate multiple sensorimotor loops to achieve coordinated and adaptive behaviour, and few behaviours really occur in isolation. For example, the sound-localising behaviour of crickets [1] can be affected in several ways by visual cues, including: course stabilisation using the optomotor response [6]; associating the relative direction of landmarks to the sound source; competition between the attractiveness of sound and visual cues; and changes in the relative attractiveness of songs depending on the visual environment. Similarly flying insects use multiple visual cues and several different forms of proprioception to stabilise flight; navigating insects need to co-ordinate and store multiple cues. The capabilities displayed seem to require more complex coordination mechanisms than simple vector summation, suppression of one behaviour by another, or behaviour switching. One issue of interest is whether insects use predictive control mechanisms based on efference copy [5, 2]. I will discuss some criteria for assessing this possibility and some behavioural and neurophysiological evidence for it, and present a simple example of how it can be implemented in a spiking neural model for combining robot phonotaxis and optomotor reflexes.

BarbaraWebb received her B.Sc. degree in Psychology from the University of Sydney in 1988 and her Ph.D. degree in Artificial Intelligence from the University of Edinburgh in 1993. She held lectureships in Psychology at the universities of Nottingham and Stirling, and since 2003 has a readership at the School of Informatics at Edinburgh, in the Institute for Perception, Action and Behaviour. Her research focus has been the use of robots as models for exploring issues in neuroethology, in particular the problem of sound localisation in the cricket. She also has an interest in theoretical issues of methodology; in particular the problems of measurement, modeling and simulation.

References [1] R. Reeve and B. Webb. New neural circuits for robot phonotaxis. Philosophical Transactions of the Royal Society A, 361:2245–2266, 2002. [2] P. Russo, B. Webb, R. Reeve, P. Arena, and L. A Patane. ricket-inspired neural network for feedforward compensation and multisensory integration. In IEEE Conference on Decision and Control and European Control Conference. 2005. [3] B. Webb. Can robots make good models of biological behaviour? Behavioural and Brain Sciences, 24(6):1033–1050, 2001. [4] B. Webb. Robots in invertebrate neuroscience. Nature, 417:359–363, 2002.

57


[5] B. Webb. Neural mechanisms for prediction: do insects have forward models? Trends in Neurosciences, 27:278–282, 2004. [6] B. Webb and R. Reeve. Reafferent or redundant: How should a robot cricket use an optomotor reflex? Adaptive Behaviour, 11:137–158, 2003.

58


Central pattern generators in animals and robots Auke Jan Ijspeert School of Computer and Communication Science EPFL (Ecole Polytechnique Fédérale de Lausanne), CH-1015, Lausanne, Switzerland [email protected]

Abstract

quency adaptation can be added to a variety of nonlinear oscillators [6], and be used for learning arbitrary rhythmic signals [7] and for adapting the intrinsic frequency of CPG models to resonant frequencies of compliant robots [8]. Finally, I will present how CPG models can be used to control the locomotion and movements of a variety of robots, from snake-like robots to humanoid robots [9, 10]. A common aspect of these different projects is to explore the usability of CPGs by animals and robots. CPGs are sometimes viewed as producing rigid and stereotyped patterns that are only useful for steadystate locomotion. Our results tend to show that CPGs are well suited for also controlling non steady-state locomotion with constant changes of speed, direction and type of gaits, and for dealing with perturbations from the environment.

The agility and efficiency of animal locomotion tend to fascinate engineers. The skills to coordinate multiple degrees of freedom (DOFs), using compliant actuators (muscles and tendons), and massively parallel control (the central nervous system), give animals an agility and energy efficiency rarely replicated in manmade robots. One of the most impressive features of animals is how they effortlessly deal with multiple redundancies: redundancies in the number of articulated joints, redundancies in the musculature (there are multiple muscles acting on a single joint, and often single muscles acting on multiple joints) and redundancies in muscles (a single muscle is decomposed into multiple motor units). To a large extent, the problem of dealing with these redundancies is solved by central pattern generators (CPGs), i.e. neural networks capable of producing coordinated patterns of rhythmic activity without any rhythmic inputs from sensory feedback or from higher control centers [1]. Even completely isolated CPGs in a Petri dish can produce patterns of activity, called fictive locomotion, that are very similar to intact locomotion when activated by simple electrical or chemical stimulation [2, 3]. Typically, varying simple stimulation allows modulation of both the speed and direction of locomotion. From a control point of view, CPGs therefore implement some kind of feedforward controller, i.e. a controller that ”knows” which torques need to be rhythmically applied to obtain a given speed of locomotion. Interestingly, CPGs combine notions of stereotypy –steady state locomotion tends to show little variability– and of flexibility –speed, direction and types of gait can continuously be adjusted. In this talk, I will present the results of several projects related to the modelling of vertebrate CPGs, their characterization as systems of coupled non-linear oscillators, and the use of CPG-like controllers in robotics. The projects on modelling vertebrate CPGs are done in collaboration with Jean-Marie Cabelguen (University of Bordeaux), and aim at getting insights into the spinal mechanisms of locomotion control in salamander, in particular the gait transitions from swimming to walking [4, 5]. The projects on systems of coupled non-linear oscillators explore how fre-

Biography

Auke Ijspeert is a SNF (Swiss National Science Foundation) assistant professor at the EPFL (the Swiss Federal Institute of Technology at Lausanne), and head of the Biologically Inspired Robotics Group (BIRG). He has a BSc/MSc in Physics from the EPFL, and a PhD in artificial intelligence from the University of Edinburgh. He carried out postdocs at IDSIA and EPFL, and at the University of Southern California (USC). Before returning to the EPFL, he was a research assistant professor at USC, and an external collaborator at ATR (Advanced Telecommunications Research institute) in Japan. He is still affiliated as adjunct faculty

59


[10] A. J. Ijspeert, J. Nakanishi, and S. Schaal. Learning control policies for movement imitation and movement recognition. In S. Thrun S. Becker and K. Obermayer, editors, Neural Information Processing System 15 (NIPS2002), pages 1547– 1554, 2003.

to both institutes. His research interests are at the intersection between robotics, computational neuroscience, nonlinear dynamical systems, and adaptive algorithms (optimization and learning algorithms). He is interested in using numerical simulations and robots to get a better understanding of the functioning of animals (in particular their fascinating sensorimotor coordination abilities), and in using inspiration from biology to design novel types of robots and adaptive controllers.

References [1] F. Delcomyn. Neural basis for rhythmic behaviour in animals. Science, 210:492–498, 1980. [2] A.H. Cohen and P. Wallen. The neural correlate of locomotion in fish. ”fictive swimming” induced in a in vitro preparation of the lamprey spinal cord. Exp. Brain Res., 41:11–18, 1980. [3] S. Grillner. Neural control of vertebrate locomotion – central mechanisms and reflex interaction with special reference to the cat. In W.J.P. Barnes and Gladden M.H., editors, Feedback and motor control in invertebrates and vertebrates, pages 35–56. Croom Helm, 1985. [4] A.J. Ijspeert. A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander. Biological Cybernetics, 84(5):331–348, 2001. [5] A.J. Ijspeert, A. Crespi, and J.M. Cabelguen. Simulation and robotics studies of salamander locomotion: Applying neurobiological principles to the control of locomotion in robots. NeuroInformatics, 3(3):171–196, 2005. [6] L. Righetti, J. Buchli, and A.J. Ijspeert. Dynamic hebbian learning in adaptive frequency oscillators. Physica D, 2006. In press. [7] L. Righetti, J. Buchli, and A.J. Ijspeert. From dynamic hebbian learning for oscillators to adaptive central pattern generators. In Proceedings of 3rd International Symposium on Adaptive Motion in Animals and Machines - AMAM 2005, 2005. [8] J. Buchli, L. Righetti, and A.J. Ijspeert. A dynamical systems approach to learning: a frequencyadaptive hopper robot. In Proceedings of the VIIIth European Conference on Artificial Life (ECAL 2005), Lecture Notes in Artificial Intelligence, Springer Verlag, pages 210–220, 2005. [9] A. Crespi, A. Badertscher, A. Guignard, and A.J. Ijspeert. AmphiBot I: an amphibious snake-like robot. Robotics and Autonomous Systems, 50– 4:163–175, 2005.

60

Part II

Poster Abstracts

61


Avalanche behaviour in cultures of dissociated neurons R. Alessio, L. Cozzi, P. D’Angelo and V. Sanguineti Department of Informatics, Systems and Telematics (DIST), University of Genoa Via Opera Pia 13, 16145 Genoa – Italy [email protected] [email protected] [email protected] [email protected] Recorded spikes

Avalanche dynamics has been described in many natural phenomena [1, 2]. In avalanches, propagation of activity is not continuous or wave-like, but is organized into finite cascades. Recently, avalanches have been observed in the spontaneous neural activity of organotypic cultures from rat cortex and in acute slices, placed on micro-electrode arrays (MEAs) [3, 4]. These behaviours display dynamical properties that are typical of self-organizing branching processes (SOBPs) [5]. More specifically, the statistical distributions of avalanche size and life time follow a power-law, with exponents of, respectively, -3/2 and 2, which correspond to a near-critical state [6]. Criticality is an important property, which has been associated to maximal information transmission and efficiency of information storage [3, 4]. We applied avalanche analysis to cultures of dissociated cortical neurons from rat embryos. In these preparations, neurons rearrange into a twodimensional network that lacks the organization found in organotypic cultures or acute slices and is characterized by high, random synaptic connectivity. In contrast with previous work based on the analysis of local field potentials (LFPs), in our case we could look at spike trains and therefore we could potentially look at the micro-structure of avalanches. Spontaneous activity, recorded extracellularly from up to 60 microelectrodes (MEAs), was characterized by bursts of firing activity, which spreaded over the whole neural population. A preliminary look at raster plots suggested a self-similar structure (see Figure 1). What appeared to be simultaneous firing revealed, at a finer temporal resolution, a complex spatiotemporal pattern of firing. These patterns of activity were identified as avalanches. For purposes of analysis, we adopted the following definition of avalanche: the recordings (spike trains from up to 60 electrodes) were subdivided into time bins of width ∆t. We then defined an avalanche as a set of frames (spatial pattern of activity from all electrodes in a time bin) in which spikes occur, preceded and followed by at least one frame in which all electrodes are silent. The ‘optimal’ bin size was calculated as the ratio of the inter-electrode distance (200 µm) in our MEAs and the average propagation

60

Electrode number (n)

50 40 30 20 10 0 140

150

160

170 Time (s)

180

190

200

Recorded spikes 60

Electrode number (n)

50 40 30 20 10 0 159.95

160

160.05 Time (s)

160.1

160.15

Figure 1: Raster plot of activity from all electrodes shows correlated periods in which spikes do not appear on all electrodes at the same instant, but are scattered in time

velocity of neural activity, estimated from cultured neurons on MEAs (∼50 mm/s), and resulted into ∆t = 4 ms [3]. We also e defined life time and size of an avalanche. Life time was simply the temporal duration of the avalanche. Avalanche size can be defined in different ways. For brevity, here we report the results that refer to the following definition: size is the number of spikes of an entire avalanche. We analyzed the probability distributions of life time and size, for all the avalanches detected in our recordings from the 9

63

Dynamical principles for neuroscience and intelligent biomimetic devices 10

Life t im e dis t rib ut ion

0

However, the slopes were not the same in all preparations: in 4 preparations (out of 9) the slopes in the distributions of life time and size were similar to those found in experiments with slices and organotypic cultures, and consistent with critical SOBP dynamics. Instead, the remaining 5 preparations displayed greater slopes (in absolute value), thus suggesting that large and long-living avalanches are less frequent. Figure 2 shows the power law distributions of size and life time for two experiments that display different behaviours. Cultures of dissociated neurons represent a reduced model system that can be used to investigate the emergent collective and functional properties of the nervous system in order to understand how the brain represents, stores and processes information. Avalanche dynamics seems to be a crucial property of the spontaneous activity of neuronal populations, as it has been related to efficient information processing and storage. The variety of power law exponents that were observed in the statistical distributions of avalanche size and life time in different dissociated cultures suggests that at least some of these preparations are fundamentally different from their acute or organotypic counterparts. In particular, they seem to exhibit a sub-critical behaviour (larger and long-living avalanches are less likely). This may be related to the lack of structure in the connectivity of dissociated cultures (in particular, the reduction of long-range connections). Other factors (e.g. age, density of the culture) could be implicated as well. These results also suggest to relate the indicators of avalanche behaviours to other descriptors of the population behaviour, like the ability to encode information.

s lo pe = -2

P(t)

10

10

10

10

-1

-2

-3

-4

10

10

0

10

Life t im e (t/ t im e bin )

1

A valanc he s iz e2 dis t rib ut io n

0

s lo pe = -3 / 2 t im e bin = 4m s

P(size)

10

10

10

10

-1

-2

-3

-4

10

10

0

10

1

2

10 S iz e (# s pik es )

10

3

Life t im e d is tribu t ion

0

s lop e= -2

P(t)

10

10

10

10

-1

-2

-3

-4

10

10

0

10

L ife t im e (t / t im e bin)

1

A valanc he s iz e2 d is tribu t ion

0

References

s lo pe = -3 /2 t im e b in = 4 m s

P(size)

10

10

10

10

-1

[1]

B. D. Malamud, G. Morein, and D. L. Turcotte, "Forest Fires: An Example of Self-Organized Critical Behavior," Science, vol. 281, pp. 1840-1842, 1998.

[2]

T. E. Harris, The theory of branching processes. New York: Dover Publications, 1989.

[3]

J. M. Beggs and D. Plenz, "Neuronal avalanches in neocortical circuits," Journal of Neuroscience, vol. 23, pp. 11167-11177, 2003.

[4]

J. M. Beggs and D. Plenz, "Neuronal avalanches are diverse and precise activity patterns that are stable for many hours in cortical slice cultures," J Neurosci, vol. 24, pp. 5216-29, 2004.

[5]

J. X. de Carvalho and C. P. Prado, "Self-organized criticality in the olami-feder-christensen model," Phys Rev Lett, vol. 84, pp. 4006-9, 2000.

[6]

P. Bak, C. Tang, and K. Wiesenfeld, "Self-organized criticality: An explanation of the 1/f noise," Physical Review Letters, vol. 59, pp. 381-384, 1987.

-2

-3

-4

10

0

1

10 10 S iz e(# s pik es )

2

10

3

Figure 2: Avalanche life time and size distribution for two different experiments (A, B). A: slopes do not match those predicted by SOBP theory (broken lines). B: slopes are very close to the critical values of -2 and -3/2 for, respectively, life time and size

available experiments, and fitted that with power laws (i.e., a line in log-log scale). In all the experiments both distributions displayed a close-to-linear relationship in log-log coordinates. However, all alternative definitions of avalanche size produced similar results.

64


Biomimetic VLSI for Real-Time Image Processing S. Badel, A. Schmid and Y. Leblebici Microelectronic Systems Laboratory, Swiss Federal Institute of Technology EPFL CH-1015, Lausanne, Switzerland {stephane.badel,alexandre.schmid,yusuf.leblebici}@epfl.ch ΦΦ11

Introduction Bio-mimetic microelectronic integrated circuits take inspiration from analytical or empirical models of biological phenomena in order to solve in a nonconventional or non-Boolean way intricate problems, where an accurate system model is very often impossible to derive. The competitive Hamming artificial neural network (ANN) described in this short paper has been integrated to provide fast, reliable and low power image processing capability to an autonomous embedded system. The Hamming ANN [1] is composed of a quantifier network, where neurons perform the Hamming distance calculation between an input pattern and a previously stored pattern. In a subsequent phase, the neuron with smallest Hamming distance is selected as the winner by the discriminator network composed of a winner-take-all (WTA) circuit. The Hamming ANN always converges towards one of the stored patterns, and is therefore adequate to process classification tasks in signal or image processing fields.

Φ1 ΦΦ22

VVDD DD V DD

row node

dendrite

V r r C C PP

C PP

Φ1

ΦΦ11

C CS S

ΦΦ11

V Si V Si

Φ3

Φ 33 V

Φ2

Φ2

Φ2

VDi V Di

P

C C SS V VSnSn

output stages

Φ2 V Dn V Dn synapses

Offset

Perturbation

Regular Synapses

Figure 1: Charge-based neuron circuit with perturbation column. At end of φ2 dendritic or row voltage is affected according to Equation 1. Pn ∆Vr =

i=1 (VDi

− Vsi0 )CST OT + (VP − VDD )CP CST OT + CP

where CST OT =

n X

C Si

i=1

Architecture and Operation The capacitive implementation of all neural operations has been selected, using the Capacitive Threshold Logic (CTL) design technique previously presented in [2]. The schematic of one neuron is presented in Figure 1. The circuit operation is based on a two phases scheme, consisting of precharge and evaluation phases. Two non-overlapping clock signals are required to drive the phase sequencing. Charge conservation is assumed throughout all phases; this holds true as long as the raw voltage drop due to current leakage does not exceed the precision interval dictated by the output stage. System clock speed must be adapted accordingly. During the precharge phase driven by φ1 , the dendritic voltage is precharged to a reference voltage Vref which is equal to the inverter threshold voltage, while the synaptic nodes are all precharged to a nominal value Vsi0 . At end of φ1 ,

Thus, at end of the evaluation phase, the dendritic row voltage is increased (or decreased) of the sum of n synaptic contributions which are proportional to the corresponding variation of synaptic voltage, as well as to the corresponding synaptic capacitance values. Any variation in these parameters allows controlling the individual connection weights, which can represent positive and negative strength values. The output value is formed by comparison of the row voltage against the threshold value of the first output stage inverter Vth .  ∆Vri < −δ ⇒ VOU T = VDD    ∆Vri > +δ ⇒ VOU T = GN D  −δ < ∆Vri < +δ ⇒ limit of circuit   precision The original concept of Hamming network has been extended in the proposed architecture in order to provide increased signal processing capabilities. A third phase called perturbation phase takes places as soon as all row voltages have settled in the evaluation phase, and is driven by clock φ3 . An external signal is equally applied to the capacitive synapses of a number of i

Vref = Vri = Vth and Vsi = Vsi0 During the evaluation phase driven by φ2 , input voltages are applied to the synapses, eventually resulting in charge being transferred to the synaptic capacitances.

65

(1)

Dynamical principles for neuroscience and intelligent biomimetic devices Precharge

Evaluation

Perturbation

φR Horizontal Row

Output signals

horizontal row V th1

ΦE

Analog Control Signals

ΦR ΦE Sel Even n

Inverter threshold

OUT

Hamming Network

Vertical Row

Sel Even n Sel Odd n

Row votages

Decision Network

Sel Odd n Data

Data

Basic Cell Circuit

Basic Cell Circuit

Perturbation and Offset Capacitors

Input Drivers

VLSI Development The analog core of a 16×16 Hamming ANN has been integrated in a 0.35µm CMOS technology as a proof-of-concept. The relatively low number of synapses is not dictated by any technological limitation, and can be increased up to 256×256 [2]. The size of the circuit is equal to 600×600µm2 including capacitors of unit value equal to Cs = 40f F . The circuit has been extensively characterized to be fully functional, with an observed nonlinearity remaining inside acceptance interval for operational monotonicity. The measured response time is in the order of 10ns, and the power dissipation is less than 50µW per neuron.

neurons arranged in an array. Consequently, the ANN outputs will switch in accordance with the row voltage reached at the beginning of perturbation. This process translates the analog row voltage into a time-domain binary value. To illustrate this process, a SPICE post-layout simulation of a capacitive neuron is shown on Figure 2 Subsequent digital processing is applied to the ANN outputs in order to process various signal processing applications such as WTA, loser-take-all (LTA), kWTA also referred to as winner-share-all, k-LTA, vector rank ordering, true Hamming distance extraction. Several absolute or relative modes of operation are applied in order to perform these functions, resulting in various hardware implementations, which are discussed in detail in [3]. A two-dimensional arrangement of Hamming silicon neurons is proposed to achieve image processing applications. The circuit architecture is depicted in Figure 3, where the detail of a unit cell is also presented. The microphotograph details the main silicon structures in a silicon neuron. Based on the two-dimensional structure, a precision alignment application system has been developed. Fast convergence of the system using linear angular and position corrections algorithms could be demonstrated for a symmetric embossed cross shape [3]. Parasitic capacitances formed by layer overlaps which can not be avoided in the two-dimensional array must be accounted for in the sizing of the unit capacitance. The contribution of a unit overlap is given in 2, from which Cs , worse case, can be extracted. XX 1 Cs Cc ∆Vij 2 Ctot 1 − n (Cc /Ctot ) i j

Hamming ANN Circuit

Figure 3: Two-dimensional Hamming ANN circuit architecture, and microphotograph detail of the one neuron.

Figure 2: Operation of 16 Hamming neurons, showing precharge, evaluation and perturbation phases, and a ramp perturbation signal.

δVc =

Analog or Digital Post -Processing

Conclusion The two-dimensional Hamming ANN proves to be a versatile, fast and energy-efficient hardware block for processing of several complex signal processing functions. The bio-inspired ANN analog model allows one-shot processing of algorithms that would require thousands of cycles on a microprocessor architecture. The inclusion of image sensors into the Hamming ANN is foreseen to develop an autonomous embedded image processing system.

References [1] Lippmann, R. P., “An Introduction to Computing with Neural Nets,” IEEE ASSP Magazine, Vol. 4, No. 2, pp. 4–22, 1987. [2] Ozdemir, H. Kepkep, A. Pamir, B. Leblebici, Y. and Cilingiroglu, U., “A capacitive threshold-logic gate,” IEEE JSSC, Vol. 31, No. 8, pp. 1141–1150, 1996. [3] Badel, S., Schmid, A. and Leblebici, Y., “A VLSI Hamming artificial neural network with k-winner-take-all and k-loser-take-all capability,” Proc. IJCNN 2003, Vol. 2, pp. 977–982, 2003.

(2)

66


How to measure the instantaneous I-V curve of neurons with in vivo-like voltage fluctuations L. Badel∗, S. Lefort†, C. Petersen†, W. Gerstner∗ and M.J.E. Richardson∗ ∗ ´ Ecole Polytechnique Fédérale de Lausanne (EPFL), Laboratory of Computational Neuroscience, School of Computer and Communication Sciences and Brain Mind Institute, CH-1015 Lausanne, Switzerland laurent.badel, wulfram.gerstner, [email protected] † ´ Ecole Polytechnique Fédérale de Lausanne (EPFL), Laboratory of Sensory Processing, Brain Mind Institute, CH-1015 Lausanne, Switzerland sandrine.lefort, [email protected]

Background. The spike-response of a neuron to synaptic input depends on its subthreshold dynamics. A number of experimental protocols are available to probe the subthreshold response of a neuron, including the application of step pulses from fixed voltages that probe the steady-state I-V relationship. Here, a novel method is developed that allows for the measurement of the instantaneous subthreshold response of a neuron to in vivo-like input. The method is able to capture both the subthreshold range and the region of spike initiation, which is crucial for the input-output properties of the neuron. This technique provides a simple and efficient way to measure the I-V relationship of neurons under realistic conditions and over a wide voltage range.

potassium components; at the onset of the spike it is the activation variable of the sodium component m∞ that has the dominant effect on the spike dynamics. Using the usual Hodgkin-Huxley notation and neglecting the potassium term that terminates the spike, the following approximation can be considered reasonable

Model. The voltage dynamics of neurons can be described using a conductance-based formalism [1]. The neuronal membrane is modelled as a capacitor C, in parallel with subthreshold Isub and spike-generating Ispike voltage-gated currents,

where VT and ∆T are related to the position of the spike threshold and its width respectively. This argumentation followed that used for the derivation of the exponential integrate-and-fire model [3].

C

dV = Isub + Ispike + Iapp , dt

(1)

that is charged by the synaptic drive or artificially applied current Iapp . Model Predictions. The subthreshold current is typically dominated by a leak term of conductance gL and reversal potential EL Isub = gL (EL − V ).

Ispike ≃ gN a m3∞ h(EN a − V ) ∝ m3∞

where gN a is the maximum conductance and EN a the reversal potential for this sodium current. Because the activation variable of m∞ is initially exponential [1], the effect of the spike generating current on the subthreshold voltage can be expressed as Ispike ∼ e(V −VT )/∆T

(4)

Method. The aim is to find an experimental protocol that activates the neuron in an in vivo-like state and allows for the simultaneous measurement of the subthreshold properties of the neuron. Measuring the neuronal response to injected noise, which is also a model of fluctuating synaptic drive, provides such a method. By dividing by the capacitance, the conductance-based equation (1) can be written in the following form dV = F (V ) + S(t) dt

(2)

but can also comprise voltage-gated currents such as the h-current. In the experimental protocol to be used here such currents become tonically activated and, to a reasonable approximation [2], can be absorbed into a shift of the conductance and reversal in equation (2). The spike-generating currents comprise sodium and

(3)

(5)

where F (V ) is the sum of the subthreshold and spikegenerating currents and S = Iapp /C is proportional to the noisy applied current. Consider now a small time step from tk to tk+1 where the voltage hops from Vk to Vk+1 due to the random current pulse Sk . Vk+1 − Vk = ∆F (Vk ) + ∆Sk

67

(6)


Figure 1: A. Sketch of the method. Top graph: The voltage response of a neuron to white noise current. Each time the voltage crosses a fixed value (red and green horizontal lines), the value of the voltage at the next time-step V+1 is registered. Bottom graphs: The distribution of V+1 is shown for two voltage values (-80mV and -60mV), the average of which determines the direction and magnitude of the instantaneous current experienced by the neuron F (V ). These values are then plotted as a function of the voltage to construct the I-V curve. B and C. Experimental results obtained from L5 pyramidal cell of the mouse barrel cortex. The I-V curve is probed for two different levels of depolarisation so as to cover the subthreshold voltage range (B) and to explore the region of spike initiation (C). Near the spike threshold, the exponential form (4) is found to accurately fit the experimental data. this can be arranged to solve for F (Vk ) F (Vk ) = (Vk+1 − Vk )/∆ − S(tk ).

(7)

Now all the cases in the trace where the voltage before the time step is close to some value V are grouped together and the average value of the corresponding voltage at the end of the time step V+1 calculated. F (V ) = hV+1 −V i/∆ − hS+ i = hV+1 −V i/∆

(8)

where hS+ i is the average value of the noisy input for each of these time steps. Because the amplitudes of the current pulses are uncorrelated, this average is zero. This means that choosing a voltage V , finding all the places it appears in the trace and calculating the average next time step V+1 directly yields the total current at the voltage V , i.e. F (V ) ∝ Isub + Ispike . This procedure is then repeated for the whole voltage range to build up a complete picture of the instantaneous subthreshold I-V curve. Examples of this method are provided in the figure where it is shown that the I-V curve does indeed comprise a linear part and exponential part for spike initiation, as predicted from equations (2) and (4) respectively.

Experimental protocol. Intracellular recordings in whole-cell patch clamp configuration were made from mouse L5 pyramidal cell in the C2 barrel related column. White-noise currents were injected into the cells and the capacitance and access resistance artifacts removed in the post-measurement analysis.

References [1] A.L. Hodgkin and A.F. Huxley, “A quantitative description of membrane current and its application to conductance and excitation in nerve.” J. Physiol. , Vol. 117, pp 500-544, 1952. [2] M.J.E. Richardson, N. Brunel and V. Hakim, “From Subthreshold to Firing-Rate Resonance” J. Neurophysiol., Vol. 89, pp 2538-2554 2003. [3] N. Fourcaud-Trocmé, D. Hansel, C. van Vreeswijk and N. Brunel, “How spike generation mechanisms determine the neuronal response to fluctuating inputs.” J. Neurosci. , Vol. 23, pp 11628-11640, 2003. [4] G.B. Ermentrout, “ Type I membranes, phase resetting curves, and synchrony.” Neural Comput., Vol. 8, pp 9791001, 1996

68


Quantitative characterization of information transmission in a single neuron M. Bezzi‡, T. Nieus†, A. Arleo∗, A. D’Errico† E. D’Angelo† O. J.-M.D. Coenen∗ ∗

†

Neuroscience Group, Sony CSL, F-75005 Paris, France Dept. of Cellular-Molecular Physiological and Pharmacological Sciences, Univ. of Pavia, and INFM, I-27100, Pavia, Italy ‡ Accenture Tech Labs, F-06600, Sophia-Antipolis, France [email protected]

Neurons transform input spike information in complex ways and many factors enter into play. Synapse dynamics include numerous interacting mechanisms as neurotransmitter release, diffusion and post-synaptic receptor activation, and intrinsic electroresponsiveness. Many of these factors may undergo activity dependent changes, which have major effects on the neuron transmitting properties. This complex neural processing can be analyzed by comparing the information content in the neuron inputs with its output, i.e. by quantifying how much information the neural responses convey about the input stimuli. In this framework, neurons are treated as stochastic communication channels and information theory [1] provides the mathematical tools, e.g. mutual information (MI), to measure their transmitting properties. Assessing MI requires to determining the probability distribution of the output spike trains given any input spike train . In general, this is impracticable even for a single neuron due to: (i) the multiple mechanisms of nonlinear integration at individual synapses, (ii) the large number of synapses (typically 103 -104 ), and (iii) their location on wide dendritic trees with complex electrotonic and active properties1 . A remarkable exception is provided by cerebellar granule cells (GCs). These neurons are characterized by a compact electrotonic structure [3] and a very low number of synapses receiving mossy fiber (MF) afferents (4 on average). Moreover, their mechanisms of synaptic transmission, plasticity and intrinsic excitability have been intensely investigated [4]. In this study we present an information theoretic analysis (mutual information and stimulus-specific information) of transmission properties of a single granule cell using both a detailed mathematical model and

whole-cell patch recordings in slices. Our aim was to understand how information is transmitted by the GC through the MF-GC relays, and how it is affected by LTP. Induction of LTP at MF-GC synapses alters the spike train response of GCs [4]. We have measured, the information transmitted through a single GC before and after induction of long-term synaptic plasticity at MFGC synapses, a condition which modifies the release probability p at the MF synaptic terminals [5]. Experimental data were obtained by in vitro whole-cell patch recordings of GCs. To measure MI, one to four of the MFs were stimulated by a set of spike trains that mimicked the discharge of GCs following punctuate tactile stimulation in vivo [6]. Moreover, we have developed an Hodgkin-Huxley model of a single granule cell, where the stochasticity of neurotransmission is accurately reproduced [4]. The same stimulation protocol was employed to run numerical simulations and MI was measured while varying the release probability p at the formal MF-GC synapses. MI increases as a function of p for both experimental and simulated data. This corroborated the findings that LTP enhances MI and suggested that optimal transmission may correspond to large p values. To further investigate this hypothesis, we have performed a series of simulations to analyze the effect of long-term synaptic plasticity upon the information transmitted by the subset of the most informative stimuli [7]. To assess the contribution of a single stimulus to MI we used the stimulus-specific information (surprise). Their surprise, as function of the average release probability p, after an initial rapid growth, saturated and leveled at a plateau for p = 0.5. Therefore, although on average MI was maximized by LTP, the efficient transmission of the most informative stimuli already occurred at intermediate p values, comparable to those measured in vitro in standard conditions [5].

1 Previously, approximations via dimensionality reduction have been attempted, focusing on the effect of an individual synapse while considering the rest of the dendritic inputs as background noise [2].

69


It also suggests that at higher p values, it is the lesser informative stimuli that increased their overall contribution to MI and not the most informative ones. These theoretical findings suggest that neurons as well as synaptic plasticity mechanisms may have evolved for optimizing the transmission of a limited set of relevant stimuli. We examined the patterns of input spike trains to identify features (e.g. discharge frequency and spatiotemporal structure) that characterize efficient information transfer in GCs. Measuring single stimulus contribution to information as function of the correlation across the MFs, we found that the most informative stimuli were characterized by the presence of multiple coincident spikes across the four MFs, whereas no coincident spikes were observed in the least informative inputs. This result indicated that correlated activity across the four MF afferents may largely contribute to information transmission, extending previous studies [3] showing that GC firing requires the co-activation of two or more MFs. In conclusion, we have analysed from an information theoretic point of view (mutual information, and stimulus-specific information) the transmission properties of a single granule cell using both a mathematical model and whole-cell patch recordings in slices. Our results indicate that a major amount of information is conveyed by the spike time correlations across the inputs and that short- and long-term synaptic plasticity affects the information transmission process significantly. Interestingly, long-term synaptic potentiation increases the average amount of information transmitted, but not necessarily the contribution of the most informative set of stimuli.

[6] P. Chadderton, T. W. Margrie, & M. Hausser, Integration of quanta in cerebellar granule cells during sensory processing. Nature 428, 856-60, 2004. [7] M. Bezzi, T. Nieus, A. Arleo, , E. D’Angelo, , O. J. M. Coenen, Information transfer at the mossy fiber-granule cell synapse of the cerebellum. Soc Neurosci Abs, 827.5, 2004.

References [1] C.E. Shannon, A mathematical theory of communication. Bell System Technical J, 27,379–423, 1948. [2] M. London, A. Schreibman, M. Hausser, M. E. Larkum, & I. Segev, The information efficacy of a synapse. Nat Neurosci 5, 332-40, 2002. [3] E. D’Angelo, G. De Filippi, P. Rossi, V. Taglietti, Synaptic excitation of individual rat cerebellar granule cells in situ: evidence for the role of NMDA receptors. J Physiol,484, 397–413, 1995. [4] T. Nieus, E. Sola, J. Mapelli, E. Saftenku, P. Rossi, and E. D’Angelo LTP Regulates Burst Initiation and Frequency at Mossy Fiber-Granule Cell Synapses of Rat Cerebellum: Experimental Observations and Theoretical Predictions, J Neurophysiol, 95: 686, 2006. [5] E. Sola, F. Prestori, P. Rossi, V. Taglietti, & E. D’Angelo. Increased neurotransmitter release during long-term potentiation at mossy fibre-granule cell synapses in rat cerebellum. J Physiol 557, 843-61, 2004.

70


An Analog/Digital Simulation System for Biomimetic Neural Networks Y.Bornat1, S. Renaud1, J. Tomas1, S. Saighi1, Q. Zou2, A. Destexhe2 IXL Microelectronics laboratory, CNRS, ENSEIRB, Université Bordeaux 1, Bordeaux, France [email protected] 2 Integrative and Computational Neuroscience Unit (UNIC), CNRS, Gif-sur-Yvette,France [email protected]

1

This work presents a tool addressing the investigation of spiking neural networks, in a tentative understanding of the temporal coding of information by such networks. Detailed and biologically-realistic models of neurons can then be implemented to simulate neural dynamics at the single cell level. Networklevel simulations are also possible, including the computation of adaptive functions (connectivity and plasticity rules). We will present a system we engineered to allow real-time simulations of neural networks. It combines analog VLSI artificial neurons with software modules, with enough flexibility on the models to allow a systematic exploration of the neural network dynamics, including adaptation rules. Various computational approaches can be used to understand how neural networks process information. Different levels of precision can be considered for the models; the computation mode itself can be classical software, but also digital and/or analog hardware [1],[2],[3],[4], [5]. The final choice results from a necessary compromise between precision and technical performance, such as computational speed or power consumption. The system we designed proposes an original architecture for the temporal simulation of neural networks; conductance-based models of ionic currents for the neurons and synapses are computed in real-time and in analog hardware; synaptic interactions, subject to short-term and long-term mechanisms, are digitally processed using hardware or software medium. The whole simulation system (PAX) is organized in 4 layers; to globally process a neural network simulation, the layers communicate in different formats and at different levels (see figure 1). The analog hardware layer runs the continuous and real-time computation of the neurons and synapses ionic currents. The models are conductancebased, following the Hodgkin-Huxley formalism [6], to provide a precise computation of neurons electrical activity. They reproduce the two main neuronal types in the neocortex: excitatory (regular spiking) and inhibitory (fast spiking) neurons [7]. The analog integrated circuits (ASICs) are controlled by the digital hardware layer. This hardware is in charge of receiving spike events information from the analog neurons, and of controlling the synaptic connectivity back to

the analog hardware. Predefined stimulation patterns can also be applied to individual neurons; the results section will illustrate the use of such patterns, useful to emulate background cortical activity. The next layer includes the software driver and interface, in charge of controlling the data bi-directional transfer to the software via a PCI bus. Finally, a PC running a realtime operating system hosts software functions to compute the connectivity dynamics functions in the neural network. The software also includes user interface functions to control the off-line and on-line simulation configuration.

Figure 1: System architecture and data flow. We ran on that system a set of experiments to in various configurations, to validate the computation principle and evaluate the system characteristics. The experiments we present are benchmarks, and do not address specific neuroscience questions. They were designed to illustrate the system features, and check its performances when processing well-known singlecell or neural network configurations. Figure 2 presents a simple neural network simulation, processed by the system in real and continuous time. 4 neurons of 2 types (excitatory and inhibitory, respectively 3 and 4 conductances) are connected by nonplastic conductance-based synapses. The resulting bi-

71

Dynamical principles for neuroscience and intelligent biomimetic devices [4] J. Schemmel, S. Hoffman, K. Meier, F. Schurman, “A mixed-mode analog neural network using currentsteering synapse”, Analog Integrated Circuits and Signal Processing, vol.38, pp.233-244, 2004.

stable activity pattern illustrates 2 key features of the simulation system: 1) the bursts of activity, that result from the activation of the slow-kinetic conductance in one of the excitatory neurons; such bursts could not be simulated by a simple integrate-and-fire (IF) model; 2) the variable length of the burst pattern is due to the analog computation mode, where intrinsic noise at the transistor level has an effect on the spike occurrence; this feature, if correctly controlled, is an interesting way to move hardware simulation closer to biological reality at the single-neuron level. Other experimental results will be displayed and commented in the poster.

[5] G. Indiveri, E. Chicca and R.J. Douglas, “A VLSI reconfigurable network of integrate–and–fire neurons with spike–based learning synapses”, in Proc. European Symposium on Artificial Neural Networks, 2004. [6] A. Hodgkin, A. Huxley, “A quantitative description of membrane current and its application to conduction and excitation in nerve”, Journal of Physiology, Vol.117, pp.500-544, 1952. [7] B. Connors, M. Gutnick, “ Intrinsic firing patterns of diverse neocortical neurons”, Trends Neuroscience, Vol.13, pp. 99-104, 1990.

Figure 2: Architecture and activity of a bi-stable network of 4-neurons.

References [1] T. Bal, G. Le Masson, S. Le Masson, A. Laflaquière, D. Dupeyron, “Analog circuits for modeling biological neural networks: design and applications”, IEEE Transactions on Biomedical Engineering, vol.46-6, pp.638-645, 1999. [2] R.Hahnloser, R. Sarpeshkar,. M. Mahowald, R. Douglas, S. Seung, “Digital selection and analog amplification co-exist in an electronic circuit inspired by neocortex” , Nature, vol.405, pp. 947-951, 2000. [3] R. J. Vogelstein, F. Tenore, R. Philipp, M. S. Adlerstein, D. H. Goldberg, and G. Cauwenberghs, “Spike timing-dependent plasticity in the address domain”, in Advances in Neural Information Processing Systems 15 (S. Becker, S. Thrun and K. Obermayer, eds), pp. 1147-1154. MIT Press, Cambridge, MA, 2003.

72


Optimal feedback control adapted to explain sensorimotor learning phenomena D.A. Braun1,2,3, A. Aertsen1,3, R. Paz4, E. Vaadia4, S. Rotter1,3,5, C. Mehring1,2 1

2

Bernstein Center for Computational Neuroscience Freiburg, Germany 3 Inst. of Biol. I / Inst. of Biol. III, Albert-Ludwigs-Univ., Freiburg, Germany 4 Hassadah Medical School, Hebrew Univ., Jerusalem, Israel 5 IGPP, Freiburg, Germany

Recently it has been shown that it is possible to explain a wide range of motor psychophysical findings on the basis of optimal control theory [1,2]. Here we extend the optimal control framework to allow for adaptive responses to environmental changes. We test predictions of our adaptive optimal control model for a sensorimotor learning experiment in monkeys [3] and humans, where subjects had to adapt to visuomotor rotations.

biomechanical hand/cursor system x(t) given the best available estimate of the environment at the time and the noise characteristics of the system. As known from psychophysics [6], the control command itself increases the noise in the system in a multiplicative manner. Additionally, the controller receives noisy and delayed feedback from sensory receptors. In an adaptive stochastic system, therefore, not only the plant state but also system parameters have to be estimated simultaneously. Following the certaintyequivalence principle the best available estimate is employed to adjust the controller as if it was the true system parameter, i.e. the uncertainty of the estimate is ignored by the controller.

Generally, adaptive optimal control laws can be computed neither analytically nor numerically due to the immense mathematical complexity of the posed problem. Therefore, simplifying assumptions and approximations are inevitable for all practical purposes. One of the most prominent assumptions in the adaptive control literature is the certaintyequivalence principle [4]. An optimal controller subject to the certainty-equivalent principle is called a self-tuning regulator [5], which is depicted in figure 1. The controller (i.e. the motor cortex) is designed optimally with respect to a certain performance criterion of a particular motor task and issues an optimal command signal u(t) to control the

In a second step, we applied this model to a sensorimotor learning experiment. When subjects were exposed to unexpected visuomotor rotations in a center-out arm reaching experiment they had to adapt on-line to the altered visuomotor map in order to successfully control a cursor on a computer screen. We show that the traditional desiredtrajectory learning model

Movement Goal & Cost Function

CONTROLLER

disturbance

u(t)

PLANT x(t)

State Estimator Parameter Estimator Figure 1. Adaptive optimal control scheme as explained in the text.

73

y(t) Noisy Receptor Measurement


(error-feedback learning [7-8]) cannot account for the observed on-line adaptation effects in monkeys and humans. However, the proposed adaptive optimal control scheme is able to reproduce these swift on-line adaptations. The model is also able to describe slower over-trial learning effects that reflect the acquisition and adaptation of an internal model by including biologically inspired radial basis function networks. This way, generalization in workspace as measured by so-called aftereffects can be related to underlying basis function width, which may be interpreted in terms of neural tuning functions. Thus, the model suggests the existence of two different computational mechanisms: a swift on-line adaptation and a slower over-trial learning that reflects progressive retention of the internal model of the visuomotor transformation. Finally, we have also found potential neural correlates of these two mechanisms in differential modulations of the neural signals

recorded from motor cortical area M1 of the monkeys during the learning process. The present study employs and extends adaptive optimal control methods as they have been developed in the engineering sciences and applied them to a sensorimotor learning experiment. The model is able to reproduce characteristics of the basic behavioural findings: adaptive trajectory deviations, overtrial learning and generalization effects. The model suggests two computational mechanisms for learning, of which we have found potential neural correlates. The presented framework should be applicable to a wide range of motor learning paradigms and could yield particularly interesting results in conjunction with neural recordings [2]. Supported by the WIN-Kolleg of the Heidelberg Academy of Science, BMBF-DIP D3.2 & BMBF 01GQ0420 to BCCN-Freiburg.

References [1] Todorov E, Jordan M.I (2002) Optimal feedback control as a theory of motor coordination, Nat. Neurosci. 5:1226-1235 (2002) [2] Scott SH (2004) Optimal feedback control and the neural basis of volitional motor control, Nat. Neurosci. 5:534-546 [3] Paz R, Boraud T, Natan C, Bergman H, Vaadia E (2003) Preparatory activity in motor cortex reflects learning of local visuomotor skills, Nat. Neurosci. 6:882-890 [4] Åström KJ, Wittenmark B (1995) Adaptive Control, Addison-Wesley, Reading, Massachusetts [5] Åström KJ, Wittenmark B (1973) On self-tuning regulators, Automatica 9:195-199 [6] Harris CM, Wolpert DM (1998) Signal-dependent noise determines motor planning, Nature 394:780-784 [7] Kawato M, Furuwaka K, Suzuki RA (1987) A hierarchical neural network model for the control and learning of voluntary movements, Biol. Cybern. 56:1-16 [8] Wolpert DM, Ghahramani Z (2000) Computational principles of movement neuroscience, Nat. Neurosci. 3, 1212-1217

74


New measures for describing the synchronization of bursting neurons Dragos Calitoiu∗ ∗ School of Computer Science, Carleton University, K1S 5B6, Ottawa, Canada [email protected]

1

Introduction

synchronized behavior state. However, the outputs of the systems can be uncorrelated. The model proposed for investigation is a network of neurons that generate bursting (firing) behavior. The output of the neurons are chaotic and uncorrelated, although the systems are behaviorally synchronized. We propose two new measures that resemble frequencies for describing this phenomenon.

The nervous system is perpetually active, creating its own dynamics, including periodic rhythms as gamma (30-80 Hz) and beta (12-30 Hz). These rhythms are controversial, partly because they are technically difficult to induce and to spot. Some open questions remain: (a) How does the brain make use of these rhythms and how are they generated? (b) What determines frequency? (c) Why do the same collections of cells sometimes display a 50 Hz rhythm and sometimes1 an 18 Hz rhythm? (d) What causes activity in some populations of neurons to be, at least temporarily, coherent? Mathematics can play a central role in the part of neuroscience that has to do with these dynamics. The above questions are very difficult to solve because of the complexity of the underlying equations and the large number of interacting dynamic processes with a large range of time scales. The objective of the mathematics is to explain how the biophysics of the cells and synapses work together to create coherent synchronous rhythms. The problem of describing the synchronization of nonlinear oscillators was studied for a very long time; Winfree and Kuramoto elaborated very detailed analysis in this field. Synchronization2 is coordination with respect to time. Many kinds of synchronizations were defined as Amplitude synchronization or Phase synchronization3 . In this paper, we explored a new category of synchronization, namely the “behavioral synchronization”. Instead of describing the signal, this category refers to the behavior. Two systems can be, each of them, in function mode A or B. If both of them are simultaneously in the same mode A or B, they are in

2 The Model of Bursting Neuron Our network consists in Bursting neurons proposed by Rulkov[1] and described formally by two dimensional maps as: x(n + 1) =

α + y(n), [1 + x(n)2 ]

y(n + 1) = y(n) − σx(n) − β

(1) (2)

where x(n) and y(n) are, respectively, the fast and slow dynamical variables of the “oscillator” neuron. The slow evolution of y(n) is due to the small values of the positive parameters β and σ, which are of the order of 0.001 [1]. Depending on the parameter α, the neuron demonstrates two qualitatively different regimes of chaotic behavior, namely, continuous chaotic oscillations and chaotic bursts. This bursting dynamics was confirmed in experiments done with biological neurons [2].

3 The network of neurons Having now introduced individual Bursting neurons, we consider a network built with N such “oscillating” neurons [1], which are coupled to each other through the variable x(n). Observe that this coupling is similar to the electrical coupling used in the construction of the HH network. In this case, the variables4 X(n) and

1 The

same piece of tissue may be capable of multiple rhythms, with transitions between them. The hippocampal slice provides an example as shown by Whittington (1997). If the slice is stimulated, provoking activity in both excitatory and inhibitory cells, it displays a transient gamma rhythm. At higher level of stimulation, the rhythms starts off at gamma, undergoes a period of 150-200 ms of incoherence, and then switches to a slower beta frequency rhythm. 2 The definition from Merriam-Webster dictionary states that to synchronize is to happen at the same time. 3 Phase synchronization is the process by which two cyclic signals not only converge to a single common frequency, but also tend to oscillate with a common phase angle.

4 We mention here that the notation we used earlier describes an individual neuron with state variables x(n)and y(n). In the case of the network of Bursting neurons, the corresponding notation used for the ith neuron is Xi (n) and Yi (n), respectively.

75


Y (n) obey: Xi (n + 1) =

N α ² X + Y (n) + Xj (n), i [1 + Xi (n)2 ] N j=1

Yi (n + 1) = Yi (n) − σXi (n) − β

(3) (4)

where Xi (n) and Yi (n) are, respectively, the fast and slow dynamical variables of the ith neuron. The coupling between neurons influences the fast dynamics of each neuron. During this process of generating bursts, we observe that the bursts themselves get synchronized and that the fast chaotic oscillations corresponding to each neuron tend to become asynchronous. Thus, the oscillations are asynchronous and only the behavior of the neurons is the same. We shall refer to this process as “behavioral synchronization”. An example of this phenomenon for a network of two coupled neurons is presented in Figure 1. The reader should compare this process with the behavior of an uncoupled network, as presented in Figure 2.

4

Figure 1: The variations of X1 (n) and X2 (n) for a network of two neurons which are coupled.

The problem and results

In order to characterize the “behavioral synchronization” phenomenon, we propose two new measures that resemble frequencies: (a) The quantity F 1, called the pseudo high frequency of the signal. We compute this measure as the ratio of the average of the number of zero crossings to the length of the chaotic part of the signal. (b) The quantity F 2, called the pseudo slow frequency of the signal. This is computed as the average of the inverse of the time between the ends of two consecutive bursts. We intend to explore the variation of these two measures along with the cross correlation (CC) as a function of the size of the coupled network. We present in the Figure 3 the evolutions of F1, F2 and CC. The graph represents the relative variations (in percents) of these measures for the case of 2, 3 and 4 coupled neurons compared with the case of 2 uncoupled neurons (denoted with 1 on the horizontal axis). For example, the ration of F1 for the 2 coupled neurons and F1 for the 2 uncoupled neurons is expressed in percents. The conclusions of these evolutions are interpreted in comparison with biological values.

Figure 2: The variations of X1 (n) and X2 (n) for a network of two neurons which are not coupled.

References [1] N. F. Rulkov, “Regularization of Synchronized Bursts” Physical Review Letters, Vol. 86, No. 1, pp. 183-186, 2001.

Figure 3: The relative variations (in percents) for F1, F2 and CC, for the case of 2 coupled neurons (point 2 on the horizontal axis), 3 coupled neurons (point 3) and 4 coupled neurons (point 4) compared with the case of two uncoupled neurons (point 1).

[2] R. C. Elson, A. I. Selverston, R. Huerta, N. F. Rulkov, M. I. Rabinovich, H. D. I. Abarbanel, “Synchronous Behavior of Two Coupled Biological Neurons” Physical Review Letters, Vol. 81, No. 25, pp. 5692-5695, 1998.

76


Conductance-based neural population model ∗

Anton V. Chizhov and L. J. Graham Laboratory of Neurophysics and Physiology, University Paris-5, 75006, Paris, France [email protected]

A neural population model is proposed, built on the refractory density approach applied for conductancebased neurons [1], [3]. Threshold conductance-based single-neuron model. We begin with a single-cell conductance-based model of a CA1 pyramidal cell [2], with the fast ionic currents IN a , IDR , IA , ID , IH and the adaptation current IM . We then incorporate this model into the refractory density approach by using a new threshold model that replaces the fast sodium current with an explicit threshold criterion for action potential events based on the derivative of the membrane potential. The threshold model approximates well the sub-threshold potential and spike timing (see Figure 1). Population model. We define a population as an infinite number of similar neurons with both a common input and a noisy input that is unique to each neuron. The population is considered as a continuum and described by distributions of the neural density, average membrane potential, and ionic channel gating variables, in a one-dimensional space characterized by the time elapsed since the last action potential. The neural density, membrane potential and gating variables are governed by a set of partial differential equations. A source term in the density equation is proportional to a probability density of firing, or a hazard function, which is a function of the membrane potential and the threshold potential depending on the first temporal derivative of the membrane potential. The derived hazard function well approximates the solution of the nonstationary Fokker-Planck equation for the activity of integrate-and-fire neurons led by the changing effective potential and gaussian white noise. This approximation represents a sum of particular solutions for the cases of slow and fast changes of the effective potential. Parameters. No parameter fitting is needed for the population model. Some parameters of the threshold single-neuron model, namely, reset parameters of the gating variables were measured using a single trace of the full neuron model. Simulations. Responses of an ensemble of unconnected neurons to stimulation by current step and sinusoidal inputs (see Figure 2 and 3) are simulated and compared with simulations of large numbers of discrete individual neurons. The amplitude of noise for the both

full model proposed model

Ia=500 pA

mV

0 -20 -40 -60 0

100

ms

200

300

Figure 1: Comparison of spike times and voltage curves obtained by the full Hodgkin-Huxley model and the proposed threshold model in the case of injected current Ii = 500 pA starting at t = 0. The similar responses for the two models hold over a wide range of input currents (data not shown). models is taken such that the voltage dispersion at rest is equal to 2 mV. The number of individual neurons was equal to 4000, and a 1ms time bin for the population rate calculations is used. As shown in the Figures, the population firing rates in the two models are similar. A synaptically connected population model is also evaluated and compared with a model network of discrete neurons (data not shown). Conclusion. We have generalized the known refractory density approach for the case of realistic, adaptive, neurons. This model of a single population of neurons can be used as a core of a population model of cortical tissue that can be quantitatively fitted to intracellular experimental recordings. The reduced evaluation time of the proposed refractory density approach should facilitate modeling more complex neural networks, as compared to the evaluation of networks based on explicit individual neuron models. Thus, the refractory density approach may be an important tool for the implementation of truly large-scale models of the networks in the brain.

References [1] A.V.Chizhov, “The model of neuron populations as a unit of a large-scale network”. Neurocomputers: Design And Application, No.2-3, pp. 60-68, 2004. [2] L. Borg-Graham. “Interpretations of data and mechanisms for hippocampal pyramidal cell models”. In P. S.

77

Dynamical principles for neuroscience and intelligent biomimetic devices Ulinski, E. G. Jones, and A. Peters, editors, Cerebral Cortex, Vol.13, pp. 19-138, Plenum Press, 1999.

200

[3] A.V.Chizhov, L.J.Graham, A.A.Turbin. “Simulation of neural population dynamics with a refractory density approach and a conductance-based threshold neuron model”. Neurocomputing, (accepted for publication), 2006.

Ia=500 pA Individual HH-neurons

Rate, Hz

150

Population model

100 50 0 0

100

200

300

Time, ms

Figure 2: The transient response of the population firing rate to a rapid change in input, beginning at t = 0, where the excitatory input current to the uncoupled neurons of a single population is stepped up to 500 pA. The firing rate transiently jumps up before returning to a new steady-state response. The population model firing rate (solid line) compares well with the averaged firing rate of individual Hodgkin-Huxley neurons undergoing white gaussian noise (dotted line).

350

1000 pA, 10 Hz

Individual HH-neurons Population model

300 Rate, Hz

250 200 150 100 50 0 0

50

100 150 Time, ms

200

250

Figure 3: The responses to 10 Hz oscillating input current of 1000 pA amplitude are shown. The population model firing rate (solid line) compares well with the averaged firing rate of individual Hodgkin-Huxley neurons (dotted line).

78


Biomimetic Visuovestibular Artificial Perception Systems for Coping with Independent Motion, Illusions, Conflicts and Ambiguities ∗

João Filipe Ferreira∗ †, Jorge Lobo∗, Miguel Castelo-Branco† and Jorge Dias∗ Institute of Systems and Robotics, University of Coimbra, 3030-290 Coimbra, Portugal {jfilipe,jlobo,jorge}@isr.uc.pt † Institute of Biomedical Research in Light and Image, University of Coimbra, 3000-548 Coimbra, Portugal {jfilipe,mcbranco}@ibili.uc.pt

moving observer

In biological vision systems, inertial cues provided by the vestibular system play an important role, and are fused with vision in the early processing stages of image processing (e.g, the gravity vertical cue). Artificial perception systems for robotic applications have since recently been taking advantage from low-cost inertial sensors for complementing vision systems, using both static and dynamic cues.

{I } {C }

vertical features gravity field {W }

ground plane

We have thus reached a point in which the next step will be to perform psychophysical studies of human/biological models of the visuovestibular framework so as to take artificial perception to the next level: from bioinspired to biomimetic. In other words, we propose to bring accumulated knowledge of artificial perception technology together with the psychophysical models studied under a Bayesian perspective in implementing novel artificial perception systems which will attempt to closely follow its biological counterparts. These will not only represent a significant advance in artificial sensing, for example in the solution of ubiquitous and difficult problems such as ego-motion or independent motion segmentation, but they will also present a challenge in allowing further insight on aiding human beings to surpass their own perceptive limitations, helping disambiguation and coping with illusions or conflicts arising in extreme conditions where humans are prone to failure, namely: in extreme environments (e.g., in space exploration), where humans are displaced from normal conditions and factors such as 1G gravity; in perceptive pathologies (i.e. perceptionimpaired patients).

3D segmented depth map

Figure 1: Stereo vision system with an inertial measurement unit used on robotic system, frames of reference and its 3D segmented depth map output.

tial sensors. These motion parameters can also be inferred from the image flow and known scene features. Combining the two sensing modalities simplifies the 3D reconstruction of the observed world. Inertial sensors also provide important cues about the observed scene structure, such as vertical and horizontal references. Inertial navigation systems obtain velocity and position by integration, and do not depend on any external references, except gravity. The inertial-sensed gravity vector provides a unique reference for image-sensed spatial directions. More specifically, previous work has shown that the use of visual sensors together with IMUs can be used to estimate camera focal distance [1] or to perform crosscalibration [2]. Knowing the vertical-reference and stereo-camera parameters, the ground plane can be fully determined. The collineation between image ground-plane points can be used to speed up groundplane segmentation and 3D reconstruction. Using the inertial reference, vertical features starting from the ground plane can also be segmented and matched across the stereo pair, so that their 3D position is determined. The inertial vertical reference can also be used after applying standard stereo-vision techniques;

Inertial sensors attached to a camera can provide valuable data about camera pose and movement. Micromachining enables the development of low-cost single-chip inertial sensors that can be easily incorporated alongside the camera’s imaging sensor, thus providing an artificial vestibular system. Figure 1 shows a stereo-camera pair with an inertial measurement unit (IMU) mounted on a mobile robotic platform. The 3Dstructured world is observed by the visual sensor, and its pose and motion are directly measured by the iner-

79

Dynamical principles for neuroscience and intelligent biomimetic devices Perception Psychophysical Study Model Analysis

⇒

Human/Biological Observer (Vision+Vestibular)

Biological Perception, Bayesian Model

3D Scene w/ Static Objects & Moving Objects Sensation

Re el ion od at M alu ev

Artificial & Biological Model Output Comparison

Ego-Motion Illusions, Conflicts & Ambiguities Artificial Observer (Vision+Inertial)

el od sis M the n Sy

taking the ground plane as a reference, the fusion of multiple maps reduces to a 2D translation and rotation problem, and dynamic inertial cues may be used as a first approximation for this transformation, providing a fast depth-map registration method (Figure 1) [3]. In addition, inertial data can be integrated into optical flow techniques, through compensating camera ego motion, improving interest-point selection, matching the interest points, and performing subsequent imagemotion detection and tracking for depth-flow computation. The image focus of expansion and centre of rotation are determined by camera motion and can both be easily found using inertial data alone, provided that the system has been calibrated. This information can be useful during vision-based navigation tasks. In recent work, we have pursued the solution to the specific problem of independent motion segmentation for moving observers of background static scenes with some moving objects. The moving observer has stereo vision and inertial and magnetic sensors to provide a rotation update. Having compensated for rotation, translation can be obtained from a single tracked image feature. Fully registered depth maps can therefore be obtained from the moving system (Figure 1). The depth flow that remains in the resultant map is due to the system covering new scenes, or to moving objects within the overlap volume of successive observations. Mismatches between depth from stereo and depth from optical flow indicate possible independent motion. This can be used to better segment moving objects in the overlap volume and avoid artefacts from slow moving objects. Reymond et al. [4] describe a computational model for the sensory perception of self-motion, considered as a compromise between sensory information and physical coherence constraints. This compromise is realized by a dynamic optimization process minimizing a set of cost functions. This general scheme leads to a straightforward representation of fundamental sensory interactions. The model is tuned and assessed using a range of well-known psychophysical results, including off-vertical axis rotations and centrifuge experiments. The ability of the model to predict and help analyse new situations is illustrated by a study of the vestibular contributions to self-motion perception during automobile driving and during acceleration cueing in driving simulators. This work demonstrates the usefulness of introducing models which mimic biological systems of perception; in the process, it also showcases the limitations of biological perception posed by the physiological characteristics of biological motion sensors, which in certain situations yield partial or ambiguous information [4]. It shows that biological motion sensors do not directly signal the real motion of the body and that some robust estimation process is implemented at the central nervous system level to perform efficient perception. Moreover, biological vision systems take into

Sensor Readings

Artificial Perception, Bayesian Model Artificial Perception

Figure 2: Biomimetic artificial perception research proposal schematic (human observer image courtesy of 3DScience.com). account ego-motion, and deal well with independent motion segmentation. In spite of this, however robust, biological perception estimation processes are prone to suffering from illusions, conflicts and ambiguities. We propose in our work to perform psychophysical studies, such as in [4], of human visuovestibular models under a Bayesian framework, to implement these models as closely as possible using the technology presented on [1, 2, 3] in a robotic-based artificial perception system, to tackle 3D structure perception (specifically independent motion segmentation in the presence of self-motion), and to test the possibilities opened by the robustness of artificial sensor technology as opposed to biological sensory solutions on extreme perception tasks (see Figure 2). In the case of independent motion segmentation, we will address the use of inertial dynamic data to improve the optical flow consistency check, without depending on any tracked feature for the translation, and on combining the two methods to improve robustness.

References [1] Jorge Lobo and Jorge Dias. Vision and Inertial Sensor Cooperation Using Gravity as a Vertical Reference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(12):1597–1607, December 2003. [2] João Alves, Jorge Lobo, and Jorge Dias. Camera-Inertial Sensor modeling and alignment for Visual Navigation. In 11th International Conference on Advanced Robotics, pages 1693–1698, July 2003. [3] Jorge Lobo and Jorge Dias. Inertial Sensed Ego-motion for 3D Vision. In International Conference on Advanced Robotics, pages 1907–1914, July 2003. [4] G. Reymond, J. Droulez, and A. Kemeny. Visuovestibular perception of self motion modelled as a dynamic optimization process. Biol. Cybern., 87:301–314, 2002.

80


Predicting Neuronal Activity with Simple Models of the Threshold Type W. Gerstner∗, R. Jolivet∗, R. Brette†, C. Clopath∗, A. Rauch‡ and H.-R. Lüscher§ ∗

Brain Mind Institute, EPFL, 1015, Lausanne, Switzerland {wulfram.gerstner,renaud.jolivet,claudia.clopath}@epfl.ch † Equipe Odyssée, Ecole Normale Supérieure, 75230, Paris, France [email protected] ‡ MPI for Biological Cybernetics, 72012, Tübingen, Germany [email protected] § Institute of Physiology, University of Bern, 3012, Bern, Switzerland [email protected] A Membrane voltage (mV)

Evaluating the predictive power of simple models of the Integrate-and-Fire-type [1] and developing systematic methods to construct such models from actual recordings has experienced a great deal of popularity in recent years [2, 3, 4, 5, 6, 7]. Several groups reported that this type of model yields accurate quantitative predictions of the activity of real neurons. Rauch and colleagues have shown that Integrate-and-Fire-type models with adaptation reliably predict the mean firing rate of cortical pyramidal cells [3]. Keat and colleagues have shown that a similar model is able to predict almost exactly the timing of spikes of neurons in the visual pathway [2]. However, the question is still open of how the predictions of Integrate-and-Fire-type models compare to the precise structure of spike trains in the cortex. Indeed, cortical pyramidal neurons are known to produce spike trains whose reliability highly depends on the input scenario [8].

50

0

-50

-100 1600

1700

1800

1900

Time (msec)

B

Extending and improving preexisting methodologies, our laboratory has developed a technique to extract relevant parameters from in vivo-like voltage traces [4]. The framework has been extended to deal with various types of stimulations including the increasingly popular dynamic-clamp technique [5]. Our methodology was successfully applied to actual recordings of cortical neurons (see Figure 1) and we were able to confirm and extend the results of Rauch and colleagues using a similar dataset. We found that a simple Integrate-and-Fire-type model is able to accurately predict both the subthreshold fluctuations and the exact timing of spikes within the limits imposed by the input-dependent intrinsic reliability of the neuron [9]. More specifically, we evaluated the reliability of spike timing in cortical neurons and compared it to the predictions of our model using the same quality measure and found that model predictions are always close to the best accessible prediction level.

C

Figure 1: Performances of an Integrate-and-Fire-type model as described in [9]. A. Prediction of the model (black line) is compared to the spike train of the corresponding neuron (thick white line). B. Zoom on the subthreshold regime. This panel corresponds to the first dotted zone in A (horizontal bar is 5 ms; vertical bar is 5 mV) C. Zoom on a correctly predicted spike. This panel corresponds to the second dotted zone in A (horizontal bar is 1 ms; vertical bar is 20 mV). The model slightly undershoots during about 4 ms after the spike (see [9] for further details).

midal neurons under random current injection behave very much like Integrate-and-Fire neurons including a spike-frequency adaptation process. This is a result of importance. Indeed, the Integrate-and-Fire-type models are extremely popular in large scale network studies. Our results can be viewed as a strong a posteriori

Our results suggest that layer 5 neocortical pyra-

81


justification to the use of this class of model neurons. They also indicate that the picture of a neuron combining a linear summation in the subthreshold regime with a threshold criterion for spike initiation is good enough to account for much of the behavior in an in vivo-like lab setting. This should however be moderated since several important aspects were neglected in this study (see [9] for a detailed discussion). In particular, one very important aspect is that our experimental paradigm used somatic current injection. Thus, all dendritic non-linearities, including backpropagating action potentials and dendritic spikes are excluded. However, results from Larkum and colleagues suggest that similar threshold models could still be used in this context given that a multi-compartment model is considered [10]. Our results illustrate as well the importance of two features of neuronal dynamics. Firstly, like in [3], we found that adaptation is a necessary component in the model to connect between various driving regimes. Secondly, while it is relatively easy to correctly predict the subthreshold dynamics even with a simple leaky integrator, it is difficult to find an efficient criterion to decide when to elicit spikes. A threshold model as proposed by Brette and Gerstner therefore seems ideally suited to deal with these issues [6]. It includes an additional mechanism that can be tuned to model spike-frequency adaptation but that is not restricted to this specific neuronal feature [11, 12]. Moreover, the balance equation for voltage includes an exponential term as proposed by Fourcaud-Trocmé and colleagues [13] which describes early activation of voltage-gated sodium channels. This last addition allows to model specific behaviors like delayed spike initiation and offers flexibility at the level of the threshold mechanism. Application of existing mapping techniques [4, 6] to recordings of cortical pyramidal neurons is under study at the moment.

[5] R. Jolivet and W. Gerstner “Predicting spike times of a detailed conductance-based neuron model driven by stochastic spike arrival” Journal of Physiology–Paris, Vol. 98, pp. 442–451, 2004. [6] R. Brette and W. Gerstner “Adaptive Exponential Integrate-and-Fire Model as an Effective Description of Neuronal Activity” Journal of Neurophysiology, Vol. 94, pp. 3637–3642, 2005. [7] J. Pillow, L. Paninski, V. Uzzell, E. Simoncelli and E. Chichilnisky “Prediction and Decoding of Retinal Ganglion Cell Responses with a Probabilistic Spiking Model” Journal of Neuroscience, Vol. 25, pp. 11003– 11013, 2005. [8] Z. Mainen and T. Sejnowski “Reliability of Spike Timing in Neocortical Neurons” Science, Vol. 268, pp. 1503–1506, 1995. [9] R. Jolivet, A. Rauch, H.-R. Lüscher and W. Gerstner “Predicting spike timing of neocortical pyramidal neurons by simple threshold models” To appear in the Journal of Computational Neuroscience, 2006. [10] M. Larkum, W. Senn and H.-R. Lüscher “Top-down Dendritic Input Increases the Gain of Layer 5 Pyramidal Neurons” Cerebral cortex, Vol. 14, pp. 1059–1070, 2004. [11] E. Izhikevich “Simple Model of Spiking Neurons” IEEE Transactions on Neural Networks, Vol. 14, pp. 1569–1572, 2003. [12] E. Izhikevich “Which model to use for cortical spiking neurons?” IEEE Transactions on Neural Networks, Vol. 15, pp. 1063–1070, 2003. [13] N. Fourcaud-Trocmé, D. Hansel, C. van Vreeswijk and N. Brunel “How Spike Generation Mechanisms Determine the Neuronal Response to Fluctuating Inputs” Journal of Neuroscience, Vol. 23, pp. 11628–11640, 2003.

References [1] W. Gerstner and W. Kistler, Spiking Neurons Models: Single Neurons, Populations, Plasticity, Cambridge University Press, 2002. [2] J. Keat, P. Reinagel, R. Reid and M. Meister “Predicting Every Spike: A Model for the Responses of Visual Neurons” Neuron, Vol. 30, pp. 803–817, 2001. [3] A. Rauch, G. La Camera, H.-R. Lüscher, W. Senn and S. Fusi “Neocortical Pyramidal Cells Respond as Integrateand-Fire Neurons to In Vivo-Like Input Currents” Journal of Neurophysiology, Vol. 90, pp. 1598–1612, 2003. [4] R. Jolivet, T. Lewis and W. Gerstner “Generalized Integrate-and-Fire Models of Neuronal Activity Approximate Spike Trains of a Detailed Model to a High Degree of Accuracy” Journal of Neurophysiology, Vol. 92, pp. 959–976, 2004.

82


PhotoMEA: A new optical biosensor for neuronal networks analysis D. Ghezzi*, A. Pedrocchi*, A. Menegon†, S. Mantero*, F. Valtorta† and G. Ferrigno* * Bioengineering Department, Politecnico di Milano, 20133, Milano, Italy [email protected] [email protected] [email protected] [email protected] † DIBIT, San Raffaele Scientific Institute and “Vita-Salute” University, 20132, Milano, Italy [email protected] [email protected] electrodes) and poor spatial resolution both in stimulation and recording (MEA). In addition to traditional electrophysiology techniques, optical methods for stimulating (using Caged Compounds) and recording (using Voltage-Sensitive fluorescent Dyes) neuronal activity have been used separately for a long time. Typically light stimulations are combined with electrical recordings, whereas optical recordings with electrical stimulations. First experiments of optical stimulation and imaging were done with a microscope-based set-up (Figure 1).

Light stimulations and optical recordings of neuronal activity are two promising approaches for investigating the molecular mechanisms at the basis of neuronal physiology. In particular, flash photolysis of caged compounds [1] offers the unique advantage of allowing to quickly change the concentration of either intracellular or extracellular bioactive molecules, such as neurotransmitters or second messengers, for the stimulation or modulation of neuronal activity. Moreover optical recordings of neuronal activity by Voltage-Sensitive Dyes (VSDs) [2] allow to follow changes of neuronal membrane potential with highspatial resolution. This enables the study of the subcellular responses and that of the entire network at the same time. In the last decade, studies on neuronal physiology and plasticity have provided a detailed picture of the molecular mechanisms underlying the modulation of neuronal activity; on the other hand, the molecular mechanisms which control the network properties remain poorly understood, and represent a new frontier in neuroscience. Two different approaches can be followed for the study of neuronal functions: a largescale approach aiming at understanding the activity of many neurons interacting in a complex network and a micro-scale approach aiming at providing detailed behavioural models of the molecular systems which actively contribute to the generation and modulation of the neuronal activity. A new breakthrough in neuroscience would be the possibility to stimulate and modulate a single neuron, or selected parts thereof, and study its influence over the functioning of the entire network. In this manner, the micro-scale meets the large-scale approach, allowing the understanding of how the mechanisms that influence the physiology of single neuronal units are able to alter the behaviour of the entire network. At present, experiments are carried out by electrical stimulations and recordings using intracellular or extracellular electrodes as well as MicroElectrode Array devices. These systems have yielded important results but show some limits, e.g. in terms of mechanical damage of the cell (intracellular

Figure 1: This figure represents the set-up for a total optical analysis of neuronal networks. Three pathways were sown: that of light stimulation (Red line), that of MEA recordings (Green line) and that of optical recordings (Black line).

In this set-up MEA recordings were used as validation of the optical stimulations. Figure 2 (Top panel) shows a neuronal spike evoked with a 100msec UV stimulus for activating the caged-glutamate (MNIcaged-L-glutamate, Tocris Bioscience, Bristol, UK) used in a 100uM concentration. Figure 2 (Bottom panel) shows the Cai2+ variations measured using the Ca2+-sensitive fluorescent ratiometric indicator fura2AM (Invitrogen, San Giuliano Milanese, Italy). This panel shows the temporal analysis obtained both for a neuronal cell (Black line) and a glial cell (Red line). The first period of 2min shows that the intensity of light for fluorescent excitation of fura-2 (340/380nm)

83


and to record the fluorescence modification from the same areas (Figure 3).

is too low for activating caged glutamate.

Figure 3: Prototype allows comparing the recordings of the PhotoMEA optical system with those of standard MEA system, assumed as validating reference. This is possible because, instead of the slide, we have used a MEA under which optical fibres are glued. Besides, in order to get an optimal check of the system’s functioning, every optical fibre is glued exactly next to a MEA electrode, in such a way as to guarantee a high-correspondence between recorded signals.

Actual work is focused on the development of an improved version of the PhotoMEA device. In fact, the new biosensor will be based on innovative optical microtechnologies, such as integrated waveguides for the stimulation and detectors for the imaging, that combine in a single chip both local light stimulations and high-spatial resolution fluorescent optical recordings over the entire culture. The main advantages of PhotoMEA arise from the possibility to avoid electrical stimulations and use light to achieve precise temporal and spatial activation of different regions of a neuronal network, which can be sized to single neurons or a part of it. Moreover, optical recordings allow the possibility to monitor at the same time the activity of the sub-neuronal compartments, the neuronal cell and the whole network with high-spatial resolution. The merging of micro-scale and large-scale approach offers a unique opportunity to follow the effects of local neuronal pathways on neuronal network activity, for instance during pharmacological and toxicological treatments.

Figure 2: (Top panel) Neuronal spike evoked with a single UV light stimulus. (Bottom panel) the Cai2+ variations measured using the Ca2+-sensitive fluorescent ratiometric indicator fura-2, evoked by three light pulse of different duration (10msec, 100msec, 1sec). Figure 2 also shows that the elevation in Ca2+ recorded in the experiment was the effect of the optical stimulation and not a pure artefact of such stimulus. This appears from the tracking activity of a glial cell close to the neuron where no change was recorded (red trace in Fig. 2). This experiment shows that the optimal pulse duration for this set-up is about 100msec. Brief light pulses (10msec) could be unable to stimulate neuronal cells and too long pulses (1sec) could produce a saturated response of the neuron that could be not easy to recover. Recently, a new device for neuronal cultures analysis, PhotoMEA, has been proposed (Italian patent pending number MI2005A000114) [3]. First prototype of PhotoMEA proposes a solution for the integration of the two optical methods by avoiding the use of both microscopes. The innovative concept is based on the use of optical fibres, which are used to lead the stimulation light directly on defined positions of the coverslip

References [1] E. M. Callaway, R. Yuste, “Stimulating neurons

with light” Curr. Opin. Neurobiol., Vol. 12, pp. 587-592, 2002. [2] M. Zochowski, M. Wachowiak, C. X. Falk, L. B. Cohen, Y. W. Lam, S. Antic, D. Zecevic, “Imaging membrane potential with voltage-sensitive dyes” Biol. Bull. Vol. 198, 1-21, 2000. [3] Patent pending number MI2005A000114, assigned to Politecnico di Milano, Technical University.

84


Using contraction analysis to design a model of the cortico-baso-thalamo-cortical loops B. Girard∗, N. Tabareau∗, J.-J. Slotine† and A. Berthoz∗ ∗ LPPA, UMR 7152, Collège de France - CNRS, 75005, Paris, France {benoit.girard,nicolas.tabareau,alain.berthoz}@college-de-france.fr † Nonlinear Systems Laboratory, MIT, 02139, Cambridge, MA, USA [email protected]

The basal ganglia (BG) are a set of interconnected subcortical nuclei (detailed in Fig. 1) present in all vertebrates. They are thought to constitute a generic selection circuit, interacting with various cortical and subcortical systems involved in sensorimotor, cognitive or limbic processes. The interaction of the BG with the cortex takes place in parallel cortico-baso-thalamocortical (CBGTC) loops (Alexander et al., 1986). Depending on the involved circuits, the role of the BG can be to select the most appropriate behaviour in a given context, the target of a saccade among the multiple points of interest present in the visual field, the piece of information to be stored in working memory, etc. Each of these elements competing for selection is represented by a channel inside the BG. The selection mechanism is disinhibition (Chevalier and Deniau, 1990). At rest, the output of the BG tonically inhibits the circuits enabling the activation of the competing elements. When one of them wins the competition, the inhibitory output of the corresponding channel is removed and the target circuit can be activated. Numerous computational models of the basal ganglia have been proposed in the last ten years (Gillies and Arbruthnott, 2000, for a review). However most of them rely on the outdated “direct/indirect pathways” scheme proposed by Albin et al. (1995), which doesn’t take into account numerous connections. Even the latest and most complete ones (Gurney et al., 2001; Frank et al., 2000) neglect some interesting projections. Moreover, despite the fact that the cortico-basothalamic circuitry contains numerous internal loops susceptible to generate various dynamic behaviours, the model’s dynamics was not analysed. Consequently, we propose a new model of the cortico-baso-thalamo-cortical loops including usually neglected connections and prove the stability of its operation using contraction analysis (Lohmiller and Slotine, 1998). Contraction analysis is an extension to nonlinear systems of the stability analysis for linear systems. It is well adapted to study the dynamics of artificial neural networks made of nonlinear components. Moreover, contraction has the advantage of being pre-

served through basic system combinations (hierarchies, feedback, etc.). The details of the basal ganglia part of our model were previously presented (Girard et al., 2005) (see Fig. 1, Basal Ganglia dashed box). In accordance to neurobiological data (Parent et al., 2000; Kita et al., 1999), it includes projections from the external globus pallidus (GPe) to the striatum, which are usually neglected, moreover the projections from the GPe to the subthalamic nucleus (STN), the internal globus pallidus (GPi) and the substantia nigra pars reticulata (SNr) are considered diffuse. It was proved to be contracting and to perform efficient selection. Given known thalamus anatomy and relationships with cortex and basal ganglia (Pinault, 2004), we add a simple thalamo-cortical module to the existing BG model (see Fig. 1, Thalamus and cortex dashed boxes) in order to close the loop. The module itself is contracting and thanks to contraction combination properties, the proof of the contraction of the resulting circuit is very simple. Depending on the CBGTC loop considered, the thalamic nuclei involved (ventro-lateral, medio-dorsal, etc.), as well as the sensory and frontal cortical areas, may vary. They are respectively represented by the TH nucleus and the SCtx and FCtx areas in Fig. 1. The excitatory TH-FC loop is proposed to have a role of amplification of the sensory signal, it is however under the inhibitory control of the thalamic reticular nucleus (TRN). Thus, as projection weights are constrained so that the module is contracting, it is proved that the activity in the loop cannot saturate and self-sustain indefinitely. The system thus avoids getting locked in a given state, which would not subsequently be influenced by any changes in the external input. The basal ganglia inhibitory input to the thalamus selectively controls the amplification process: only the selected channel is amplified, while the low level signal in the other channels is preserved as the BG don’t inhibit the cortex directly. Consequently, even if the signal corresponding to the winning channel only reaches the subcortical targets of the BG, the whole information is kept in the cortex.

85

Dynamical principles for neuroscience and intelligent biomimetic devices Neurobotics project funded by the European Community, grant FP6-IST-001917.

Basal Ganglia Cortex

Dopamine D2 Striatum

GPe

References

SCtx Thalamus

STN

FCtx

TRN Dopamine

D1 Striatum

GPi/SNr

Disinhibition of channel 2

TH

Figure 1: Cortico-baso-thalamo-cortical loop model with three competing channels represented (second channel shaded). Projections from the second channel neurons only are represented. Boxes : subcortical nuclei or cortical areas; circles: artificial leakyintegrator neurons; white arrowheads: excitatoty projections; black arrowheads: inhibitory projections; D1 and D2: neurons of the striatum with two respective types of dopamine receptors; STN: subthalamic nucleus; GPe: external segment of the globus pallidus; GPi/SNr: internal segment of the globus pallidus and substantia nigra pars reticulata; TH: thalamic nucleus (depends on the loop considered); TRN: thalamic reticular nucleus; FCtx: Frontal cortical area involved in the loop; SCtx: Sensory cortex providing the external input to the circuit. Dopamine level has a modulatory effect on the striatal input.

Concerning neuromimetism, the model proposed still omits two BG nuclei projections, from the STN to the striatum D1 and D2 (Parent et al., 2000) and from the D1 striatum neurons to the GPe (Wu et al., 2000). The STN neurons projecting to the striatum constitute a population distinct from those projecting to the GPe, GPi and SNr. We plan to investigate the possible role of such a specific interconnection in future versions of the model. The D1-GPe projection could improve the quality of the selection, as it did for the Gurney et al. model (Gurney et al., 2004), nevertheless, this adds an new loop whose contraction must be assessed. Finally, the inhibitory interneurons of the striatum were not modelled and might also add some selectivity. This model is a part of a larger work aiming at modelling interactions of the various cortical and subcortical components of the saccadic circuitry. For this purpose, we designed a contracting superior colliculus/brainstem saccade burst generator model (unpublished yet). As the basal ganglia also form loops with the superior colliculus (McHaffie et al., 2005), future work will aim at connecting the two models, using contraction analysis to simply address the dynamics of the resulting intricate loops. B.G., N.T. and A.B. acknowledge the support of the

Albin, R. L., Young, A. B., and Penney, J. B. (1995). The functional anatomy of disorders of the basal ganglia. Trends in Neurosciences, 18(2):63–64. Alexander, G. E., DeLong, M. R., and Strick., P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9:357–381. Chevalier, G. and Deniau, M. (1990). Disinhibition as a basic process of striatal functions. Trends in Neurosciences, 13:277–280. Frank, M. J., Loughry, B., and O’Reilly, R. C. (2000). Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cognitive, Affective and Behavioral Neuroscience, 1:137–160. Gillies, A. and Arbruthnott, G. (2000). Computational models of the basal ganglia. Movement Disorders, 15(5):762– 770. Girard, B., Tabareau, N., Slotine, J.-J., and Berthoz, A. (2005). Contracting model of the basal ganglia. In Bryson, J., Prescott, T., and Seth, A., editors, Modelling Natural Action Selection: Proceedings of an international workshop, pages 69–76, Brighton, UK. AISB Press. Gurney, K., Humphries, M., Wood, R., Prescott, T., and Redgrave, P. (2004). Testing computational hypotheses of brain systems function: a case study with the basal ganglia. Network: Computation in Neural Systems, 15:263– 290. Gurney, K., Prescott, T. J., and Redgrave, P. (2001). A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biological Cybernetics, 84:401–410. Kita, H., Tokuno, H., and Nambu, A. (1999). Monkey globus pallidus neurons projecting to the neostriatum. NeuroReport, 10:1467–1472. Lohmiller, W. and Slotine, J. (1998). Contraction analysis for nonlinear systems. Automatica, 34(6):683–696. McHaffie, J., Stanford, T., Stein, B., Coizet, V., and Redgrave, P. (2005). Subcortical loops through the basal ganglia. Trends in Neuroscience, 28(8):401–407. Parent, A., Sato, F., Wu, Y., Gauthier, J., Lévesque, M., and Parent, M. (2000). Organization of the basal ganglia: the importance of the axonal collateralization. Trends in Neuroscience, 23(10):S20–S27. Pinault, D. (2004). The thalamic reticular nucleus: structure, function and concept. Brain Research Reviews, 46(1):1– 31. Wu, Y., Richard, S., and Parent, A. (2000). The organization of the striatal output system: a single-cell juxtacellular labeling study in the rat. Neuroscience Research, 38:49–62.

86


CMOS Integrated Bidirectional 128-Electrode Array for Electrogenic Cells F. Heer1, S. Hafizovic1, W. Franks1, U. Frey1, F. Greve1, A. Blau2, T. Ugniwenko2, C. Ziegler2, and A. Hierlemann1 (1) Physical Electronics Laboratory, ETH Zurich, Hoenggerberg HPT H 6, 8093 Zurich, Switzerland [email protected] (2) Department of Physics & Biophysics, Erwin-Schroedinger-Str. 56, 67663 Kaiserslautern, Germany

We report on a CMOS-based microelectrode array chip (6.5 by 6.5 mm2) for bidirectional communication (stimulation and recording) with electrogenic cells. The integration of on-chip circuitry, which includes analog signal amplification and filtering stages, analogto-digital (A/D) converters, digital-to-analog (D/A) converter, stimulation buffers, temperature sensors, and a digital interface for data transmission, notably improves the overall system performance. Measurements with cardiomyocytes and neuronal cells were successfully carried out, and the circuitry characterization evidenced a total equivalent input noise of 5.9 mVRMS (10 Hz - 100 kHz) at a gain of 1,000.

Fig. 2: Schematic of the chip architecture and the electronic components. The stacked frames indicate that these subunits are repeated for each electrode or each row. The chip also includes a temperature sensor. HPF denotes high-pass filter, LPF low-pass filter.

Fabrication The chip is fabricated in industrial complementary metal-oxide semiconductor (CMOS) technology [1]. After the CMOS process, a 2-mask post-processing procedure is required to cover the Al electrodes with biocompatible platinum and to protect the Al from undesirable electrochemistry using a 1.6 mm thick passivation stack of alternating silicon nitride and silicon oxide [2]. The electrode (Fig. 1) diameter (10 to 40 mm) and location (pitch 50 to 500 mm) is defined during these post processing steps. Additionally the electrodes can be electroplated with porous platinum black to reduce the electrode impedance [2].

In-house Platinum In-house Passivation Pt-Electrode

Al Bulk Silicon

Post processing

Repeated electrode unit 250 µm

Fab

Chip size: 6.5 x 6.5 mm2

System Description Each of the 128 electrode circuitry repeating units in the array includes a high-pass filter with 20 dB gain, a low-pass filter, a 30-dB amplifier, and a stimulation buffer (Fig. 2). Important advantages arise from the modular architecture with buffers and filters implemented at each electrode in comparison to other CMOS electrode arrays published so far [3,4]: (i) The signal is

CMOS-Al contacts

Fig. 1: Micrograph of the electrode array chip, close-up, and transducer schematic. Left: The chip features the 8by-16 electrode array in the center part and 16 A/D-converters and the digital block at the right-hand side. Center: Close-up of the 128-fold repeated circuitry unit. Right: Schematic of the platinum-electrode processing.

87

Dynamical principles for neuroscience and intelligent biomimetic devices Amplitude / µV

Electrode 60 50 40 30 20 10 1

2

3

4

5

6

Time / s

Time / ms

Fig. 4: Exemplary biological measurements of beating of primary cardiomyocytes from neonatal rats after 3 days in vitro. Left: Raster plot of events vs. time and electrode. Clearly distinguishable is a synchronous beating, of the culture at 6 Hz. Further analysis revealed the existence of a natural pace maker region in the cell culture. Right: action-potential recorded from a spontaneously firing neural culture after 56 days in vitro. The amplitude of the signal is about 500 µVPP, and the noise level is 27 µVRMS. amplified and filtered in close vicinity of the electrodes, which makes the design less sensitive to noise and distortion picked up along connection lines; (ii) a stimulation buffer makes the stimulation signal independent of the number of activated electrodes; (iii) the high-pass filter allows for immediate signal amplification; (vi) the low-pass filter limits the noise bandwidth and acts as an anti-aliasing filter. Additionally, the high-pass filter has a reset in order to ensure operability immediately after stimulation. Finally the signal is multiplexed for 20-kHz, 8-bit A/D-conversion. Total amplification is selectable, either 1,000 or 3,000.

ulating interest in their work. Funding has been generously provided by the European Information Society Technology (IST) Future and Emerging Technologies program, and the Swiss Bundesamt für Bildung und Wissenschaft (BBW), contract number IST-200026463. FPGA board with USB 2.0

Counter Electrode

To manage the enormous data rates (3.2 MB/s out, 0.4 MB/s in), an FPGA in conjunction with an USB 2.0 chip has been employed. I/O buffering and digital signal processing like averaging and event detection are implemented on the FPGA to reduce the data volume transmitted to the PC. Furthermore, the overall microelectrode array system is very compact. Fig. 3 depicts all needed components: a laptop computer, a USBpowered FPGA card, and a simple PCB to provide references voltages and power supply stabilization. Stimulation circuitry elicits action potentials in a spatiotemporally controlled manner. Typically a rectangular bipolar voltage signal of, e.g., 10 kHz with amplitudes of up to ±1.5 V is applied to an arbitrary subset of electrodes. The digital 8-bit, 60-kHz stimulation pattern comes from the external PC, is D/A-converted and finally buffered at each electrode.

Packaged chip

Power and digital I/O lines

Fig. 3: Photo of the system setup except for the Faraday cage.

[1]

[2]

Results Exemplary biological measurements include measurements from neuronal cell cultures with amplitudes of 500 µV and from cardiomyocytes with amplitudes of 1.3 µV (Fig. 4).

[3] [4]

Acknowledgments: The authors are grateful to Prof. Henry Baltes for sharing laboratory resources and for his ongoing stim-

88

Reference Standard 0.6-mm triple-metal, double-polysilicon CMOS process as provided by XFAB, Dresden, Germany. F. Heer et al., Biosens. & Bioelect., Vol. 20/2, pp. 358-366, 2004. B. Eversmann et al., IEEE J. Solid-State Circuits, Vol. 38, pp. 2306-2317, 2003. G.T.A. Kovacs, Proc. of the IEEE, Vol. 91, No. 6, pp. 915-929, 2003.


A Multi-Referential Dynamical System for Reaching M. Hersch and A. G. Billard LASA Laboratory, EPFL, 1015 Lausanne Switzerland {micha.hersch, aude.billard}@epfl.ch

1 Introduction After decades of research in robotics and control, computer-controlled systems are far from exhibiting the robustness and adaptability of biological systems. It has been convincingly argued that biological systems control their movement through dynamical systems that have attractive or cyclic properties, which explains the observed robustness of those movements [3]. Other scholars have argued that multiple frames of reference are being used during the control of human reaching movements [4]. In the following sections, we bring together those two principles (dynamical system and multi-referential control) in order to design a controller for robot reaching movements. It is expected that using those two principles that seem to be key to biological motion control, will produce a robust and adaptive controller.

2 The VITE model The VITE model for reaching movements was originally developed in [1]. In a slightly modified version, it can be expressed by the following equation: ¨ r = α(−˙r + β(rT − r))

(1)

where r is the present position vector, rT is the target position vector and α, β are scalars between 0 and 1. It can be easily verified that this dynamical system creates a stable attractor at the target location, and that the present position will reach the target with a straight line and a roughly bell-shaped velocity profile and stay there.

3 Concurrent dynamical systems 3.1

Description

One way to combine multi-referential control and dynamical system control is to have two dynamical systems acting in two different frames of reference. Here we take the VITE dynamical system and apply it once in the arm configuration (or joint angle) space and once in the end-effector cartesian location space. We thus have two dynamical systems acting in parallel. As the

Figure 1: The robot reaching for a target tracked by a stereovision system.

end-effector location is uniquely determined by the arm configuration by a non-linear relationship, some kind of constraints must be enforced so that both dynamical systems remain in compatible states. In other words, we must make sure that the end-effector location x corresponds to the arm configuration θ. Moreover, we want x˙ and θ˙ to be as close as possible from desired the end-effector velocity x˙ d and joint angle velocity θ˙d given by the two individual dynamical systems. This amounts to solving the following constrained optimization problem: Min (θ˙ − θ˙d )T Wθ (θ˙ − θ˙d ) + (x˙ − x˙ d )T Wx (x˙ − x˙ d ) ˙x ˙ θ,

u.c.

˙ x˙ = Jθ,

where J is the jacobian of the kinematic function K and the diagonal matrices Wθ and Wx control the influence of each of the dynamical systems. The solution of this optimization problem is given by: θ˙ = (Wθ + JT Wx J)−1 JT Wx x˙ d + Wθ θ˙d . (2) By modulating the two parameters W θ and Wx , one can vary the control strategy from a pure cartesian control (Wθ = 0) to a pure joint angle control (Wx = 0). Other configurations correspond to a hybrid controller. A more detailed description of the model can be found in [2].

89


θ

x

W =0

170

x

W =0

θ

y[mm]

z[mm]

140 130 120

50

150 140

y[mm]

100

−100

90 −60

−40

−20

0

20

40

60

80

100

3

0 −50

110

handpath target position at target switch

1

160

100

150

80 −80

170

Target

W /W =0.1 150

160

−150 −150

−100

−50

x[mm]

0

50

100

150

y[mm]

130 120

1

2

110

Figure 2: Hand-paths for two reaching tasks, one in the workspace center (left) and one nearby workspace boundaries (right). The three trajectories correspond to a pure joint angle controller (dotted line), a pure cartesian controller (dashed line) and the hybrid controller (solid line).

joint limit

3

SFE

angle value [rad]

2.5 2

EB

1.5 1 0.5

SAA

0

SHR

−0.5

Wx = 0 Wx/Wθ = 0.1 Wθ = 0

−1 −1.5 0

20

40

joint limit

60

80

100

time steps

120

140

160

180

200

Figure 3: Joint angle trajectories for the movement depicted in figure 2, right. SFE, SAA, SHR, EB correspond to the four dofs of the arm. Not all dashed trajectories (pure cartesian controller) are smooth.

3.2

Results

This system has been implemented to control the arms of the Hoap2 robot shown in Fig 1. Those arms have four degrees-of-freedom (dofs). As illustrated on Figs. 2 and 3, the hybrid controller combines the advantages of the joint angle controller (smooth trajectories) and of the cartesian controller (short hand-paths). Thanks to the dynamical system approach, the robot can smoothly and accurately reach any target lying in its workspace, even in the face of sudden perturbations (see Fig. 4). This controller has a couple of interesting properties. First it does not have any singularity, as it can be seen as a generalization of the Damped Least-Squares method [5] which avoids singularities. Indeed, one can notice that the inverse of equation 2 always exists. Second, it can be shown that by a clever weight modulation strategy, joint limit avoidance can be obtained.

3.3

90

0

1

10

2 20

30

40

50

60

70

80

90 100

x[mm]

Figure 4: The system is at the position indicated by the circle, when the target switches to the corresponding square. The system smoothly adapts its trajectory.

Joint Angle Trajectories

3.5

−2

3

100

Discussion

The controller described above can be seen as a VITE dynamical system in the joint end-effector location and arm configuration space, whereby the position is constantly projected on the manifold of coherent positions

given by x = K(θ). The projection is defined by the weight matrices Wθ and Wx . It must be noted that the convergence properties of a linear dynamical system (such as VITE) are not necessarily preserved when a projection on a non-linear manifold is added to the system. And indeed, spurious attractors appear when Wx becomes too big compared to Wθ . So the weights act as bifurcation parameters of the system. Below a particular threshold, the uniqueness of the target as attractor can be guaranteed, but above that threshold the system may miss the target. In sum, the use of dynamical systems in multiple frames of references has lead us to simple solutions to classical robotics problems such as singularities, joint limit avoidance and trajectory adaptation. The non-linear relationship between the frames of reference complexifies the nature of the resulting movements and imposes some constraints on the weight modulation, that is, the interplay of the two dynamical systems.

References [1] D. Bullock and S. Grossberg. Neural dynamics of planned arm movements: Emergent invariants and speed-accuracy properties during trajectory formation. Psychological Review, 1988. [2] M. Hersch and A.G. Billard. A biologically-inspired model of reaching movements. In Proceedings of the 2006 IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. In press. [3] J.A.S. Kelso. Dynamic Patterns: The Self-Organization of Brain and Behavior. MIT Press, 1995. [4] J. Paillard, editor. Brain and Space. Oxford University Press, 1991. Chapters from Arbib, Berthoz and Paillard. [5] C.W. Wampler. Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods. IEEE Transactions on Systems, Man, and Cybernetics, 16(1):93–101, 1986.

90


Wireless remotely powered telemetry for microelectronic implanted cortical interface recording system Norbert Joehl*, Catherine Dehollain*, Alexandre Schmid†, Yusuf Leblebici† and Michel Declercq* * Electronics Laboratory (LEG), †Microelectronic Systems Laboratory (LSM) Ecole Polytechnique Fédérale de Lausanne (EPFL), Station 11, CH-1015 Lausanne, Switzerland Contact person: [email protected], Tel. 0041 (0)21 693 69 71 ber of channels, the sampling frequency and the ADC resolution. The implanted electronic circuit has to be remotely powered through magnetic coupling using a transformer [4] to avoid the use of battery inside the skull. The maximum sustainable heat transfer is equal to 80mW/cm2 to avoid damage of human tissues [5]. Therefore, it is mandatory to propose an automatic regulation system to control the power of the output signal delivered by the external transmitter which is connected to the primary coil of the transformer. All four aforementioned constraints constitute the foundation of the proposed development specifications. The telemetry unit, with a special focus on the power regulation system, which holds a central position, will be explained in following Section.

Introduction Cortical implants based on microelectronic systems have emerged in the recent years thanks to advances in integration technology allowing microsensors and readout electronics to be fabricated in a very compact and power-efficient silicon realization. Target applications for implanted microelectronic systems include study of cortical waveforms in specific areas of the brain, study and understanding of brain operation, information encoding process, altered brain operation affected by disease, and possible stimulation. Longer term goals include brain-machine interfaces allowing disabled people to recover lost functionality by the means of mechanical, micromechanical or micro-electronic prosthetic devices operating in closed-loop control mode. Such a system can be used as a brain interface in the development of (e.g) a neuroprosthetic arm [1]. Functionally operating microelectronic cortical interface systems have been demonstrated earlier [2]. In this work, we address a set of specific constraints required to increase system safety, robustness as well as patient comfort. The goal of the wireless remotely powered biomedical system is to transmit the information/signals delivered by cortical neurons to the external world by the means of a wireless RF link [3]. Telemetry is intended to permit actual implantation of the developed microelectronic interface system. The massive increase in the number of recording sites is a current research issue, which affects communication channels in terms of minimal data rate. One hundred recording sites, to be sampled with a 30kHz frequency and a resolution of 8 to 12 bits, are considered to provide an acceptable balance between the achievable data rates, and the amount of information that can be exploited. In this context, no compression of cortical waveforms is undertaken. The transfer of information from the outside world to the implanted electronic circuit is defined as the uplink communication, and the one from the brain to the outside world as the downlink communication. In practice, the data rate of the uplink communication (lower than 100 kbit/s for system configuration) is much lower than the one of the downlink communication (from 10 to 50 Mbit/s encoding operative signals). The downlink data rate is directly related to the num-

Telemetry unit

Figure 1: Block diagram of the telemetry unit. The system depicted in Figure 1 consists of two galvanically separated parts. In addition to carrying out specific tasks represented by a simple block called here Application, it has two main purposes. First, the external part provides and regulates the power needed

91


ous VDD value to be propagated outside, in order to adapt the amplitude of the 4MHz signal accordingly. This reference value is encoded into the frequency of a signal generated by the voltage controlled oscillator (VCO) located in the internal part and which is driven by the observed VDD voltage. After is has been transmitted through T2 to the external part, this signal is reshaped and then compared with a reference signal through a phase-frequency comparator. The resulting signal is filtered and finally used to control the gain of the amplifier, which in turn sets the amplitude of the 4MHz signal. The regulation loop is thus closed. High-speed, digitized neural data transmission from the inside to the outside (downlink) is performed by modulating the amplitude of the VCO signal. Demodulation is performed in the external part by an amplifier coupled with an automatic gain control (AGC) signal, which also allows reshaping the amplitude regulation signal. The selection of a high RF frequency (typically 100 to 500MHz) is related to the high data rate which is required in this direction (10 to 50 Mbit/s).

by the internal part. Second, the two parts must be able to communicate in both ways with different data rates. The galvanic isolation and the required power level dictates a communication between both parts by the means of magnetic coupling. In Figure 1, this communication is achieved through two pairs of inductive coils (or transformers) T1 and T2. Pair T1 transmits power and data from the outside to the inside part (uplink). For this purpose, the primary coil (outside) of T1 is stimulated by a 4MHz sinusoidal signal. This signal is transmitted to the secondary coil, where it is rectified across a four-diode bridge and filtered by capacitor C. Electric charges accumulated on the capacitor plates create the supply voltage VDD which powers the whole internal part. A pulse-width modulation type (PWM) is applied to the primary signal (4MHz) using switch S to transmit data. On the inside part, a data slicer, which is directly connected to the coil of T1 demodulates the received signal and sends data to the application. The interruption of the 4MHz signal caused by the operation of switch S applied to modulate data must be short and staggered to avoid excessive perturbation of the VDD supply formed on the inside part. The selection of a 4MHz frequency is dictated by the relatively low data rate required in the uplink direction (less than 100kbit/s), as well as by the fact that a signal penetrating organic tissues is less attenuated at lower frequency. The first purpose of pair T2 consists of high-speed downlink data transmission of digitally converted neural signals, whereas its second purpose is related to transmission of a reference value allowing the closed loop regulation of power transmitted from the outside to the inside. In order to perform properly, the circuit on the internal part of the system must be supplied with a stable voltage. The VDD value depends on the coupling between the coils, on the current consumed at the secondary coil, on the value of the capacitor C, and on the amplitude of the signal applied at the primary coil. Coupling factor between the coils is strongly dependent on to their distance and/or alignment. Similarly, but to a lesser extent, the current consumed at the secondary coil may vary according to the application workload. Therefore, regulation of VDD must be performed. A linear or series regulation system type, which would necessarily have to be located on the internal part can not be considered due to the poor efficiency of this type of regulation, which is prone to cause a dangerous dissemination of heat towards body tissues. The selected solution consists of dynamical adaptation of the amplitude of the 4MHz signal applied to the primary coil of T1. In this way, regulation occurs in the external part of the system where power transmitted to the internal part is limited to power required by the internal part at any given time. Unecessary heat dissemination within the body can hence be avoided. The external power regulator needs a reference value proportional to the instantane-

Conclusion The principle of a high-speed and power efficient telemetry unit, has been described in this article, with a special emphasis on the power regulation system. A similar automatic regulation system can also be used in applications such as radio frequency identification (RF ID) backscattering remotely powered passive tags [6] to regulate the RF output power delivered by the transmitter of the interrogator.

References [1] M.A. Nicolelis, Brain-machine interface to restore motor function and probe neural circuits, Nature Reviews Neuroscience, Vol. 4, pp. 417-422, May 2003. [2] K. D. Wise, D. J. Anderson, J. F. Hetke, D. R. Kipke, K, Najafi, Wireless implantable microsystems: high-density electronic interfaces to the nervous system, Proc. IEEE, Vol. 92, No. 1, January 2004, pp. 76-97. [3] P. Mohseni, K. Najafi, S.J. Eliades, X. Wang, Wireless multichannel biopotential recording using an integrated FM telemetry circuit, IEEE Transactions on neural systems and rehabilitation engineering, Volume 13, No 3, Sept. 2005, pp. 263 – 271. [4] C. Sauer, M. Stanacevic, G. Cauwenberghs, N. Thakor, Power harvesting and telemetry in CMOS for implanted devices, IEEE Transactions on Circuits and Systems I, Volume 52, No 12, Dec. 2005 pp. 2605 – 2613. [5] E. H. Liu, G. M. Saidel, H. Harasaki, Model analysis of tissue response to transient and chronic heating, Annals of Biomedical Engineering, Vol. 31, 2003, pp. 1007-1014.

[6] J.P. Curty, N. Joehl, C. Dehollain and M. Declercq, Remotely Powered Addressable UHF RFID Integrated System, IEEE Journal of Solid-State Circuits, Volume 40, No 11, Nov. 2005 pp. 2193 – 2202.

92


Na+ /K+ -ATPase-Specific Spike-Frequency Adaptation Renaud Jolivet∗ and Pierre J. Magistretti∗ Brain Mind Institute, EPFL, 1015, Lausanne, Switzerland [email protected] [email protected]

Spike-frequency adaptation is exhibited by almost all neurons that generate action potentials. It is a widespread phenomenon present in peripheral and central systems of both vertebrates and invertebrates. Beyond filtering out slow changes in stimulus, it was recently shown to play a specific role in information processing in the weakly electric fish by separating transient signals from background oscillations [1]. On the modelling side, it is a necessary mechanism for quantitative neural models to connect between different stimulation regimes [2, 3]. Spike-frequency adaptation may originate from many different processes most of which are wellknown and having been extensively studied in in vitro preparations as well as in computational models (see e.g. [4]). We wanted to explore more specifically adaptation that may originate from the activity of the electrogenic Na+ /K+ -ATPase pump. Following sustained activation, sodium accumulates in the neuron and can reach a very high level above the resting state (8 − 15 mM), up to 100 mM on some locations [5]. High intracellular sodium concentrations increase the activity of the electrogenic Na+ /K+ -ATPase pump which is responsible for an outward current INaK [6, 7, 8] as well as increased metabolic demand [9]. Using an Hodgkin-Huxley-type model including an mAHP current (ImAHP ) plus an Na+ /K+ -ATPase pump [10, 11], we studied the neuronal response to long sustained tonic stimulation. We found that INaK induces spike-frequency adaptation with a long time scale of the order of a few seconds to a few tens of seconds (Figure 1). This is essentially due to the time scale for sodium extrusion (31.4 s) that allows to integrate the output down to a very low critical frequency of ∼ 0.03 Hz. However, this is not the sole reason. The effective late adaptation time constant is the result of a complex interaction between the mAHP current and the electrogenic pump (Figure 1C). Interestingly, this interaction takes place even below frequencies where calcium accumulates with consecutive spikes. Overall, the resulting pattern of instantaneous frequency that is generated is very similar to what is observed in in vitro recordings. For weak stimulations, INaK induces phasic spiking.

A

50 mV -70 mV

B

10 mM

8 mM

100 nM 1 nM

0

50

100

150

200

Time (s)

30

20

adaptation

τ

Instantaneous

(s)

C

frequency (Hz)

∗

0 0

τ

Ca

(ms)

350

0 0

50

100

Time (s)

Figure 1: INaK induces spike-frequency adaptation. A. From top to bottom, applied current (Istim = 1 µA/cm2 ) and voltage response. B. Intracellular Na+ (top) and Ca2+ concentrations (bottom). In A and B, arrowheads indicate the baseline level. C. Instantaneous frequency during stimulation (symbols) and fitted double exponentials for the complete model (solid line) and when ImAHP is blocked (dashed line). The longest apparent adaptation time constant τadaptation is highly dependent on the time constant for Ca2+ extrusion τCa (inset). The dependence is plotted for increasing stimulation intensities: Istim = 1 µA/cm2 (solid line), Istim = 2 µA/cm2 (short dots) and Istim = 5 µA/cm2 (dots).

93


a potential role for the Na+ /K+ -ATPase pump in signal processing at frequencies accessible in vivo.

A

References

50 mV -70 mV

B

[1] J. Benda, A. Longtin and L. Maler “Spike-frequency adaptation separates transient communication signals from background oscillations” Journal of Neuroscience, Vol. 25, pp. 2312–2321, 2005.

2 mM

[2] A. Rauch, G. La Camera, H.-R. Lüscher, W. Senn and S. Fusi “Neocortical Pyramidal Cells Respond as Integrateand-Fire Neurons to In Vivo-Like Input Currents” Journal of Neurophysiology, Vol. 90, pp. 1598–1612, 2003.

8 mM

100 nM 1 nM

0

5

10

100

150

[3] R. Jolivet, A. Rauch, H.-R. Lüscher and W. Gerstner “Predicting spike timing of neocortical pyramidal neurons by simple threshold models” To appear in the Journal of Computational Neuroscience, 2006.

200

Time (s)

# spikes

C

40

60

30

f

20

ss

[4] J. Benda and A. Herz “A Universal Model for SpikeFrequency Adaptation” Neural Computation, Vol. 15, pp. 2523–2564, 2003.

(Hz)

0 0

10

[5] C. Rose and A. Konnerth “NMDA receptor-mediated Na+ signals in spines and dendrites” Journal of Neuroscience, Vol. 21, pp. 4207–4214, 2001.

10

0 0.7

0.8 I

stim

µ

[6] P. Sokolove and I. Cooke “Inhibition of impulse activity in a sensory neuron by an electrogenic pump” Journal of General Physiology, Vol. 57, pp. 125–163, 1971.

2

( A/cm )

Figure 2: INaK induces phasic spiking for weak stimulations. A. From top to bottom, applied current (Istim = 0.8 µA/cm2 ) and voltage response. B. Intracellular Na+ (top) and Ca2+ concentrations (bottom). In A and B, arrowheads indicate the baseline level. C. The number of spikes produced in the phasic spiking regime is plotted versus the applied current Istim for the complete model (solid line) and when ImAHP is blocked (dashed line). For currents ≥ 0.83 µA/cm2 , the neuron fires steadily with a continuous gain function, i.e. the steady-state frequency fss is a continuous function of the applied current Istim , like in a type I neuron (inset) [12].

[7] A. French “Ouabain selectively affects the slow component of sensory adaptation in an insect mechanoreceptor” Brain Research, Vol. 504, pp. 112–114, 1989. [8] D. Parker, R. Hill and S. Grillner “Electrogenic pump and a Ca2+ -dependent K+ conductance contribute to a posttetanic hyperpolarization in lamprey sensory neurons” Journal of Neurophysiology, Vol. 76, pp. 540–553, 1996. [9] L. Pellerin and P. J. Magistretti “Glutamate Uptake into Astrocytes Stimulates Aerobic Glycolysis - a Mechanism Coupling Neuronal-Activity to GlucoseUtilization” PNAS, Vol. 91, pp. 10625–10629, 1994. [10] R. Heinrich and S. Schuster, The Regulation of Cellular Systems, Chapman & Hall, 1996.

Spiking stops after a few seconds even though the stimulation is maintained (Figure 2) [6]. Interestingly, this type of behavior cannot be obtained with the mAHP current alone. While calcium entry is entirely dependent on spiking, sodium continues to flow in the neuron through voltage-gated channels even after spiking has stopped if the membrane is sufficiently depolarized. This process approximately linearly converts the stimulus amplitude in a finite number of spikes (Figure 2C). For stronger stimulations, the model behaves as a type I neuron.

[11] X. Wang “Calcium coding and adaptive temporal computation in cortical pyramidal neurons” Journal of Neurophysiology, Vol. 79, pp. 1549–1566, 1998. [12] A. Hodgkin “The local electric changes associated with repetitive action in a non-medullated axon” Journal of Physiology, Vol. 107, pp. 165–181, 1948.

These results illustrate the importance of sodium as a messenger for long-term signal integration and point to

94


A More Precise Sense in Which The Neural Code is Optimal ∗

Marius Kleiner∗ and Bixio Rimoldi∗ School of Computer and Communication Sciences Ecole Polytechnique Fédérale de Lausanne CH-1015 Lausanne, Switzerland [email protected]

We look at a biological subsystem in terms of its ability to reproduce, at its output, the essence of an input. The input may be an external stimulus or the signal produced by a neuron. The output may look very different from the input, yet contain the information needed to describe the input precisely or approximately. We are not assuming that the goal is to reconstruct the input from the output. For instance, from the output we may only be interested in deciding which action to take (e.g. fight or flight). The elements of our system are a source, an encoder, and a channel. The source is assumed to be specified by means of a statistical model. The encoder is some processing that produces signals that are well suited to carry the intended information across the channel. It is not always clear how to break the system into encoder and channel. We think of the channel as a given bottleneck and the encoder as that part that processes the source so as to produce an appropriate representation (the neural code) which suitable for the channel. The encoder (or equivalently the neural code) is the subject of interest to us. A question that we would like to answer is whether or not the encoder (or equivalently the code) is optimal. A reasonable way to attempt an answer is to define a measure for the “badness” of the channel output. If one knew which feature of the input the output is supposed to convey, then one could define a distortion measure by means of a function that maps any pair (s, y) of input-output signals to a positive number: the larger this number, the harder it is to infer the quantity of interest from the output y when the input is s. Given such a function we could watch the system in action and record the average distortion. We could then raise the question whether or not there exists a different encoder for which the average distortion could be reduced without increasing the average cost. The cost is measured by a function that assigns a positive number to each possible channel input signal. This number represents how much it costs the system to send that signal. It could be, for instance, the metabolic cost.

distortion measures be the result of our investigation rather than being part of the specified model. This can be done if we assume that the system is optimal (in the sense specified above). Indeed we know from previous work that if we specify the source, the encoder and the channel, there is a well defined class of cost and distortion measures for which the system is optimal. This line has been previously followed in [1]. In the present work we will go one step further by tightening our optimality criterion. Then only one function (as opposed to a class of functions) will make the system optimal. We have a general method of describing this function in a compact mathematical form. Before proceeding we point out that our Achilles’ heel is the assumption that the system is optimal. If the system is not optimal then our cost and distortion measures are meaningless since they need not be related to what the biological system cares about cost and distortion. Ultimately, the confirmation that our assumption is correct (or not) shall rest in the verification that the cost and distortion measure that we obtain are (or not) backed up by biological evidence. We are not there yet. For now or assumption of optimality relies on the fact that the neural code has remained the same over 550 million years. The fact that nature has not been able to improve the neural code for such a long time is strong evidence that the code may be optimal in some sense. In [2, 3] the results of [1] have been applied to a simple and well-known model of the cricket cercal sensory system, depicted on Figure 1. The system input is the wind direction, and the encoder (by definition the part whose optimality is under consideration) consists of four primary interneurons in the cricket cercal sensory system. These are modeled by four characteristic tuning curves. The channel that separates the encoder from the point of observation is assumed to simply add independent Gaussian noise to each of the four outputs, with the slight modification that the output signals must remain non-negative. Hence, this may be thought of as a simplistic firing rate model. For this model we evaluated (numerically) and plotted sample cost and distortion measures for which the tuning curves of the interneurons are optimal. The model used in [2, 3] assumes that there is exactly

The difficulty with this approach is that nobody knows what the right cost and distortion measures are. We take a related approach that bypasses this difficulty. In fact we turn the problem around and let the cost and

95

Dynamical principles for neuroscience and intelligent biomimetic devices n1 x1 ? - k

-

-

Computational and Systems Neuroscience, (Salt Lake City, USA), March 17-20 2005.

y1

n2

[3] B. Rimoldi and M. Gastpar, “A precise sense in which the neural code is optimal.” Computational Neuroscience Meeting, (Madison, WI, USA), July 17-21 2005.

x2 ? - k

-

-

y2

[4] W. B. Levy and R. A. Baxter, “Energy efficient neural codes,” Neural Comput., vol. 8, no. 3, pp. 531–543, 1996.

n3

s x3

? - k

-

-

y3

-

y4

n4 x4 -

? - k

Figure 1: The model for the cricket cercal sensory system considered consists of the system input (the wind direction) connected to four primary interneurons, modeled by four characteristic tuning curves. The channel that separates the encoder from the point of observation is assumed to add independent Gaussian noise to each of the four outputs. one channel use per source symbol. A stronger criterion of optimality is to allow the possibility of varying the number of channel uses per unit of time (which is equivalent to varying the number of channel uses per source symbol) while keeping the average cost per unit of time fixed. The latter provision is needed since otherwise when we increase the number of channel uses we automatically increase the cost per unit of time. A system is now declared optimal if it is better than any system even including those that use the channel more or less frequently. The work presented here considers this possibility. This approach is related to [4] where it is shown that neurons do not fire at the highest possible rate; they fire at the rate that makes the best out of the objective to improve the information transfer and the conflicting objective to minimize costs per unit time. A key difference is that our approach is end-to-end: a code is optimal if it is as good or better than any competitor from an end-to-end perspective rather than for its ability to send bits.

References [1] M. Gastpar, B. Rimoldi, and M. Vetterli, “To code, or not to code: lossy source-channel communication revisited.” IEEE Transactions on Information Theory, vol. 49, no. 5, pp. 1147–1158, 2003. [2] M. Gastpar and B. Rimoldi, “Cercal sensory system: A precise sense in which the tuning curves are optimized.”

96


FPGA implementation of ReSuMe learning in Spiking Neural Networks∗ M.Kraft, F.Ponulak and A.Kasiński Institute of Control and Information Engineering, Poznań University of Technology, 60-965, Poznań, Poland {Marek.Kraft, Filip.Ponulak, Andrzej.Kasinski}@put.poznan.pl Recent simulation experiments with ReSuMe learning in Spiking Neural Networks (SNN) indicate that the networks of spiking neurons can be successfully applied to control neuroprostheses1 . However, when considering efficient, portable neurocontrollers, one has to deal with the constraints defined by the task at hand, that is the strict requirements for the real-time operating of the controller, its low weight, low-energy consumption, programmability. These requirements bring our attention to the implementation of the SNN-based controllers in FPGA circuits [1]. The main advantage of the FPGA systems over processor-based designs is their true parallel mode of operation. This induces high ability of FPGA to perform the required information processing in the realtime. FPGA processors are easily programmable and reusable. Tested algorithms can eventually be implemented in ASIC microcircuits. All these facts, together with the easy accessibility and the low costs of the FPGA circuits make them attractive tools for the neurocontrollers implementation. In this paper we describe the implementation of spiking neurons and ReSuMe algorithm in FPGA. We discuss the properties of the implemented models, describe assumptions and requirements defined over the models. Finally we demonstrate an experiment in which the spiking neurons implemented in FPGA are successfully trained with ReSuMe method to produce a sequence of spikes with the predefined timing. Leaky Integrate and Fire (LIF) neuron model was chosen for the implementation due to its computational simplicity. LIF model is purely phenomenological, yet it still preserves the essential properties of the biological neurons. FPGA implementation accounts for all important parameters of the LIF model. Moreover, the implemented neuron is fully parametrized – the values, such as a number of inputs, the threshold (Uth ), resting (Ur ) and after-hiperpolarization (AHP) potential, neuron time

constant, maximum and minimum weight values, as well as the resolution of weight and membrane potential in bits can be easily modified. While not excited, the membrane potential Um of the implemented neuron tends toward the resting value as the exponential function of time. The synaptic excitations, however, are assumed to result in the instantaneous changes of Um so as to reflect the short time constants of the synapses. The change of Um is proportional to the synaptic weights w. Whenever Um crosses the threshold, the neuron’s output is set to ’1’ for a single time step and the neuron is assumed to fire a spike. Next, Um is reset to the AHP value and the neuron becomes insensitive to excitations until Um increases to Ur . The examples of the Um traces at the FPGA neuron model are depicted in Fig.2.A and B. In our implementation synapses are represented only by their weight values. The synaptic delays are not accounted for. Weights are modified according to the ReSuMe learning algorithm [2]. The learning method is based on the spike coincidence. According to ReSuMe algorithm the excitatory synapses are facilitated if they deliver spikes directly before the desired spike times. The same synapses are depressed if they respond directly before an output spike. For the inhibitory synapses, the opposite relationship is defined. The amplitudes of weight changes for the particular case are determined by the learning windows defined over the difference between the pre- and postsynaptic spike times (sl = tpost− tpre ) as well as between the presynaptic and the desired spike times (sd = td −tpre ). For the sake of simplicity the learning windows W (sl ) and W (sd ) implemented in FPGA are assumed to be linear functions of sl and sd , respectively (instead of exponential functions used originally in ReSuMe definition): ( for s ∈ h0, T i , c·A· T−s T (1) W (s) = 0 otherwise,

∗ The work was partially supported by the Polish State Committee for Scientific Research, project 1445/T11/2004/27. 1 Suitability of SNN to movement control is discussed in the accompanying papers: “ReSuMe learning method for Spiking Neural Networks dedicated to neuroprostheses control” and “Adaptive Central Pattern Generator based on Spiking Neural Networks”

where c = −1 for s = sl and c = +1 for s = sd, parameter A is a maximal amplitude and T is a width of the learning window. Our experiments demonstrate that the introduced simplifications do not influence the convergence of the learning process.

97


A diagram of a single neuron model with the input synapses and the ReSuMe modules is depicted in Fig.1.

Figure 1: Block diagram of a learning unit model implemented in FPGA. We present the results of our preliminary experiment with ReSuMe learning implemented in FPGA (Fig.2). A single neuron model with 100 synaptic inputs was trained to reproduce at its output the desired sequence of spikes S d (t), defined by the teacher signals. There are no special constraints for the input signals. As an illustration we consider the case where every synapse propagated only a single spike to the modeled neuron. The synaptic excitations and the teacher signals were fed to the system via FPGA inputs. The neuron was trained for 90 epochs. The time courses of the membrane potential before and after training are shown in Fig.2.A and B, respectively. The desired spike train S d (t) and the resulting sequence of spikes S l (t) are represented by the bold vertical bars. The evolution of the synaptic weights during the learning process is illustrated in Fig.2.C. According to the ReSuMe rules, the most significant changes in the weight values should be encountered in the case of synapses that delivered spikes to the learning neuron just before the spike times in S d (t). This is observed also in Fig.2.C (synaptic weights marked with 1,2,3). On the other hand some synaptic weights were depressed during the training (marked with 4). This occurred in the case of synapses that were activated whenever a neuron fired at other times than desired. Starting from the 86th epoch of learning, which corresponds to obtaining the desired signal at the neuron’s output, the synaptic weights settle at the fixed values. This indicates the stability of the obtained result of learning. The precision of S d (t) approximation is constrained mainly by the time and membrane potential discretization. However, presented results show that the errors of approximation induced by the discretization factors are an order smaller then the minimal inter-spike intervals in S d (t). In many applications such errors are negligible.

Figure 2: A single neuron model with 100 synaptic inputs was trained to reproduce the desired spike train S d (t) at the neuron’s output (S l (t)). Time courses of a membrane potential before (A) and after training (B) are presented. (C) Evolution of the synaptic weights during learning. As a conclusion we state that the presented results demonstrate fast learning convergence and a high quality of the desired signal approximation in the modeled neuron.

References [1] Andrzej Kasiński and Marek Kraft. The Design of a Compact LIF-Neuron Circuit in FPGA to Enable Implementation of Large-Scale Spiking Neuron Networks with Learning Capabilities. Submitted to ICAISC’2006. [2] Filip Ponulak. ReSuMe - new supervised learning method for Spiking Neural Networks. Technical Report, Institute of Control and Information Engineering, Poznan University of Technology, 2005. Available at http://d1.cie.put.poznan.pl/˜fp/.

98


Speed Optimization of a 2D Walking Robot through STDP Tomas Kulvicius∗ , Tao Geng† , Bernd Porr‡ and Florentin Wörgötter∗,† ∗ Bernstein Centre for Computational Neuroscience, University of Göttingen, Bunsenstr. 10, 37073 Göttingen, Germany {tomas,worgott}@chaos.gwdg.de † Department of Psychology, University of Stirling, Stirling FK9 4LA, Scotland, UK [email protected] ‡ Department of Electronics & Electrical Engineering, University of Glasgow, Glasgow GT12 8LT, Scotland, UK [email protected]

given in [1]. Neural network. We use a hybrid neural network for control consisting of two components: 1) The motor control circuit (inside dashed box in Fig. 2) which operates with linear, Hopfield-type neurons and 2) the learning control circuit (outside box) which uses spiking neurons to more realistically emulate plasticity. The motor control circuit contains motor neurons (EM, FM), which, being linear, can send their signals unaltered to the motors. Ground contact sensors (GL, GR) influence all motor neurons of both legs. Stretch receptors, sensitive to the anterior extreme position of the hip (AL, AR), influence each joint individually (joint level) and extensor as well as flexor sensor neurons (ES, FS), sensitive to joint angles, only operate on their respective motor neuron (intra-joint level). The output of the motor-neurons directly drives the motors of the joints, not employing any kind of position or trajectory tracking control algorithms. Details of the controller and its basic parameters are described in [1]. Learning scheme. The learner has inputs x1 from the left and x0 from the right hip which converge onto the learning unit L (Fig. 2) where signals from the left leg (x1 ) preceed signals from the right (x0 ) as shown in Fig. 3 A. We use a copy of input signal x1 delayed by a time delay τ to be able to employ STDP. A time delay T between x0 and delayed signal x1 depends on a walking speed of the robot. When walking slowly, time difference T between x0 and x1 is relatively large. When walking speed is increasing, T is getting smaller and when the robot reaches a desired speed specified by the time delay τ of the input signal x1 , the time difference T equals 0 and according to STDP synaptic weights stop changing [3]. Inputs x0 and x1 feed into aPsummation unit v. The output is calculated by v = j ρj uj , where u = h ∗ x is a convolution of input x with resonator h. We define h(t) = 1b eat sin(bt), p where a = −πf /Q and b = (2πf )2 − a2 , with f =

Introduction. To achieve adaptive and fast walking in artificial bipeds is still a very difficult problem, the solution of which should contribute to our understanding of human locomotion [5]. We approach this problem by combining a novel biomechanical design of a small robot with a neural controller that is based only on sensor-inputs and does not use CPGs [2] or specific trajectory planning. This way the robot achieves a very high walking speed and is rather robust against fast parameter changes [1]. This allows us to implement on-line adaptation using spike timing-dependent plasticity (STDP) [3] to gradually change the robot’s walking speed.

Figure 1: A picture of the planar robot. Design of the robot. Our robot is 23 cm high, foot to hip joint axis (see Fig. 1). It has four joints: left hip, right hip, left knee, and right knee. Each joint is driven by a modified RC servo motor. We constrain the robot sagitally by a boom (planar robot). All three axes (pitch, roll and yaw) of the boom can rotate freely and have no influence on the dynamics of the robot in the sagittal plane. A detailed description of the robot is

99

Dynamical principles for neuroscience and intelligent biomimetic devices Left Hip Motor ES

FS

EM

FM

A

Right Hip Motor ES

FS

FM

EM

V

I1

Left

y

FES

I2

Right

yR GR

yL

t

0

x1

L r

t

1

X1

T>0 EM

FM

FM

FS

Left Knee Motor

FS

AR

ES

t

x1 x0

x0 x1

4

D120

ρ1

3 Weight

AL

x1 x0

T 0, µ = 4 × 10−6 . The behaviour of this rule and its convergence properties are discussed in [4]. Walking speed of the robot depends mostly on two parameters of the hip: the threshold of the extensor sensor-neuron θES and the gain of the motor-neuron GM (see Fig. 2). Initial values are ΘES = 120 deg and GM = 1.8. The learner unit L essentially excerts disinhibition at neurons EM, F M and ES and this disinhibition increases as soon as weight ρ1 grows leading to the desired changes in ΘES and GM and, hence, to a speed increase. Results. Learning results are shown in Fig. 3 B-E. Synaptic weight ρ1 is presented in panel B and stabilizes as soon as the desired speed is reached (in around 40 s) because at that point the order of the spikes becomes reversed (see Fig. 3 A). The robot reaches maximum speed (more than 90 cm/s) after 40 s and afterwards oscillates around the speed of 80 cm/s (see panel C). Changes of the controller parameters are presented in panel D and E, which stabilize with a remaining small oscillation as soon as T ≈ 0 is obtained. Due to the symmetry of the circuitry, equivalently, the robot would slow down if it is started with a high speed. Conclusion. In this short paper we have shown that it is possible to combine neural control with learning in a fast walking robot and that the targeted control parameters will converge when implementing an STDP rule to increase the robot’s speed. Hence, similar selfstabilization should also be possible with other parameters, which will allow investigating more general adaptive properties, like adaptation to changing terrain.

100

110 105 100

C 90

E Gain

80 70 60 50 0

95 90

0 Speed (cm/s)

Figure 2: A hybrid neural network: control circuit (inside dashed box) and learning circuit (outside box). Triangles and circles denote excitatory and inhibitory synapses, respectively

QES

115

20 40 Time (sec)

60

3 GM 2.8 2.6 2.4 2.2 2 1.8 0

20 40 Time (sec)

60

Figure 3: Results of the learning experiment. A) Inputs of the learning unit. Φ - joint angle of left and right hip, Ψ - threshold, inputs x0 and x1 , τ - time delay of x1 , T - time difference between x1 and x0 . B) Synaptic weight ρ, C) change of the walking speed, D) threshold θEM and E) gain GM .

References [1] T. Geng, B. Porr, and F. Wörgötter. Fast biped walking with a reflexive neuronal controller and real-time online learning. Int. Journal of Robotics Res. (in press), 2006. [2] A. Lewis and G. Bekey. Gait adaptation in a quadruped robot. Autonomous Robots, 12:301–312, 2002. [3] H. Markram, J. Lübke, M. Frotscher, and B. Sakmann. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science, 275:213–215, 1997. [4] B. Porr and F. Wörgötter. Strongly improved stability and faster convergence of temporal sequence learning by utilising input correlations only. Neural Comp. (in press), 2006. [5] J. Pratt. Exploiting Inherent Robustness and Natural Dynamics in the Control of Bipedal Walking Robots. PhD thesis, Massachusetts Institute of Technology, 2000.


Simulations of a columnar architecture for cortical stimulus processing∗ Rüdiger Kupper, Andreas Knoblauch, Marc-Oliver Gewaltig, Ursula Körner, and Edgar Körner Honda Research Institute Europe GmbH, D-63073, Offenbach/Main, Germany [email protected]

How does the brain, most notably the visual system, manage to process and ultimately “understand” the immense amount of data, that is picked up by our sensors in each second of everyday life? What strategies, what neural algorithms does it use to interpret the sensory input in terms of what it “knows”, and how does it decide when to learn and memorize new content? Questions like these still go largely unanswered, when we come to view brain function as a whole – in spite of the overwhelming amount of detailed neurophysiological data that is available, and in spite of our progress in modeling and explaining individual brain functions in specific areas of the brain. The brain is probably not just a collection of highly specialized neural circuits, which provide individual optimized solutions at the various stages of processing, but it re-uses the same set of generic and powerful processing strategies over and over again. Thus, answering the above questions based on the available physiological data is virtually impossible, without having a useful hypothesis of brain function, ranging from local circuitry to the brain as a whole. We aim to answer these questions, founding on a concisely drawn functional model of a reappearing cortical circuitry, which is the very basis of cortical stimulus processing and understanding. In [3] we have put forward a hypothesis of computation in neocortical architecture. It bridges the gap between processing of signals at the single-neuron level, and the processing of cognitive symbols at the level of knowledge representation: This model proposes the cortical column as a basic, generic building block of cortical architecture. The same columnar circuit is reused all over the cortex, applying a generic algorithm to varying sensory data. This model gives a detailed functional interpretation of the six-layered columnar cortical architecture (fig. 1) and related sub-cortical (thalamic) structures. It hypothesizes three intercommunicating columnar processing systems at each stage of the cortical hierarchy: The “A-system” (including the middle cortical layers IV and lower III) accomplishes fast bottom-up processing. Computation in this bottomup pathway is heavily based on a spike-latency code, which is able to reliably encode stimulus properties in the timing of individual spikes [4]. In the A-system, the first wave of spikes traveling upwards in the corti∗ Submission

to the EPFL-Latsis-Symposium 2006, Lausanne.

Figure 1: Layered model of a cortical column as proposed in [3]. Three different subsystems at different vertical locations (layers) are intertwined within each cortical column. The A-system (middle layers) accomplishes fast bottom-up processing of sensory signals. The B-system (superficial layers) represents the input from the A-system in a refined way by exchanging information with neighboring columns. The C-system (deep layers) develops representations related to action/behavior and predictions fed back to lower levels. cal hierarchy can activate a coarse initial “local hypothesis” on the contents present in the stimulus. In the “Bsystem” (superficial layers II and upper III), this initial hypothesis is refined by slower processes, involving iterative exchange of information between columns both at the same (horizontal connections) and at different hierarchical levels. Finally, the “C-system” (deep layers V and VI) represents the local interpretation of the input signals that results from the local integration of bottom-up, lateral, and top-down signals. The local interpretation of the C-system is then fed back to the Bsystem of a lower level, inducing expectations, predictions, and consequently revised interpretations of the input signals at this stage. Subsequently, input signals that match the local prediction are suppressed, and only differences between predicted and actual signals can reach the next higher cortical level (cf. [5]). Thus, stimulus content is effectively expressed in terms of previously achieved knowledge (self-reference). Learning of new representations is induced, if the remaining activity is too large, and if the difference signal reaches the highest level of cortical integration, the hippocampal formation.

101

Dynamical principles for neuroscience and intelligent biomimetic devices COREtext model: setup

HF

mod.

IT - B

visual domain

IT - A2 IT - A1

COREtext model

“IT”

objects

words

“V2”

corners, angles, object parts

syllables

“V1”

edges

characters

V2 - B Example of receptive field construction: mutter

V2 - A1 IT

stimulus

0

rate neurons 1

2

mut

V1 - B V1 - A2

V2

V1 - A1

V1

l

0

3

4

1

2

3

3.0 4

5

mu tter 0

1

2

3

transfer function

5

ter

mod.

V2 - A2

4

5

0.2

setup as suggested by neurobiology and model considerations reduced to the minimum of necessary subsystems and connections

Figure 2: The “COREtext” model implements three cortical levels (denoted V1, V2, IT for convenience) including the columnar A and B subsystem to explore the neural activation dynamics and the “switching-off” mechanism (inhibition from B to A2), as proposed in [3]. We use text as a simplified input space, giving exact rules for the construction of receptive fields. l

At the Honda Research Institute, we substantiate this model on several levels of detail. At the single neuron level, we investigate, under which conditions a spikelatency code can reliably be generated and maintained in the visual system [4], and we propose, how the visual system can immediately profit from the use of a spike-latency code, implementing homogeneity detection [1]. At the level of several cortical columns, we examine the information flow inside the column, and between columns of different cortical areas. We simulate a model prototype, that demonstrates the formation of a fast initial stimulus hypothesis, and its subsequent refinement by inter-columnar communication in a hierarchy of three cortical areas. In this reduced (but instructive) simulation, we implement word recognition from a string of characters (fig. 2). The three cortical areas represent letters, syllables, and words. Focusing on the intra- and inter-columnar dynamics, we show how the different processing systems interact in order to switch off expected signals and accomplish symbolic recognition of words, and how representations for new words can be constructed based on old representations (self-reference). At the level of the visual hierarchy, we implement a large-scale simulation of main parts of the visual system, involving several primary and higher visual cortical areas (V1, V2, V6, IT), as well as parts of the hippocampal formation (HF), and sub-cortical structures involved in generating eye saccades (fig. 3). In this model, we simulate the interplay of visual areas in object recognition. Area V4 exemplarily features the detailed columnar setup. It is embedded into the hierarchy of other visual areas, which are modeled as topographic feature maps and associative memories [2]. Using this model we can demonstrate trans-saccadic

102

Figure 3: Layout of our visual model of saccadic object recognition. The model consists of various visual areas (R, V1, V2, V4, V6, IT), auditory areas (AC), hippocampal formation (HF), saccade related areas (SC, S1, S3), and some auxiliary areas triggering learning and the execution of saccades (LX, SacX). Currently only area V4 implements the full columnar model. object classification and the learning of new object representations, based on the incremental refinement of an object hypothesis during a saccadic sequence. In our contribution, we will give an overview of our different modeling approaches, ranging from the single-spike level, over investigations of the columnar dynamics, to a large-scale simulation of main parts of the visual hierarchy.

References [1] Marc-Oliver Gewaltig, Ursula Körner, and Edgar Körner. A model of surface detection and orientation tuning in primate visual cortex. Neurocomp., 52–54:519–524, 2003. [2] A. Knoblauch and G. Palm. Pattern separation and synchronization in spiking associative memories and visual areas. Neur. Netw., 14:763–780, 2001. [3] Edgar Körner, Marc-Oliver Gewaltig, Ursula Körner, Andreas Richter, and Tobias Rodemann. A model of computation in neocortical architecture. Neur. Netw., 12(7–8):989–1005, 1999. [4] Rüdiger Kupper, Marc-Oliver Gewaltig, Ursula Körner, and Edgar Körner. Spike-latency codes and the effect of saccades. Neurocomp., 65–66C:189–194, 2005. Special issue: Computational Neuroscience: Trends in Research 2005 – Edited by E. de Schutter. [5] R. P. N. Rao and D. H. Ballard. Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neurosci., 2(1):79–87, 1999.


Mapping Electrophysiological Diversity of Neocortical Neurons on a Simple Mathematical Diversity Enno de Lange and Oscar De Feo School of Computer Science, EPFL, CH-1015, Lausanne, Switzerland [email protected] The neocortex is the structure in the brain that differentiates mammals from other vertebrates. It is thought that it is responsible for the evolution of intelligence. The large majority (70-80%) of the neurons in the neocortex are excitatory pyramidal neurons [1], with relatively simple electrophysiological behavior. The remaining 20-30% are interneurons providing local connectivity. Interneurons, mainly inhibitory, show a surprising diversity in morphological and electrophysiological properties [2]. In the quest for understanding of the neocortex there exist grosso modo two complementary approaches. On one hand, detailed modelling of neurons and even simulation of large networks with these detailed models are used to study neural circuitry and information processing in the brain1 . On the other hand, simple neuron models that focus on the essence of the neuron’s behavior, are important to perform analytical studies of networks of neurons. An example of such a simple neuron model is the Integrate-and-Fire (I&F) model [3]. The I&F model can unfortunately only reproduce spiking behavior. It is believed that interneurons, with their rich gamma of electrophysiological responses other than spiking, such as bursting, irregular spiking and chattering, are vital in understanding the complex cognitive functions of the neocortex. Modelling these interneurons requires a more complicated model. It is possible to extend the I&F model to mimic bursting behavior and even chaos [4, 5, 6]. Another possibility is to take continuous models, i.e. without a threshold condition, of three or more differential equations. These model are much slower in simulation, but have the advantage of being analytically more tractable; one can apply standard techniques from nonlinear dynamics and bifurcation theory. A popular example of a model capable of producing bursting and chaotic behavior is the Hindmarsh-Rose (HMR) model [7]: 1 A good example of a project that uses these detailed models to simulate neural circuitries, is the Blue Brain project: http://bluebrainproject.epfl.ch

x˙ = y − ax3 + bx2 + I − z, y˙ = c − dx2 − y, z˙

(1)

= r (s (x − x1 ) − z)

This model is often used as a paradigm of (chaotic) bursting behavior in mathematical studies of synchronization [8, 9] There are many analytical studies of HMR model ([7, 10, 11]), but it seems that reports on a complete numerical bifurcation analysis are still lacking. Here we present a bifurcation analysis through numerical simulation, numerical continuation and normal form theory [12]. Such a bifurcation analysis can be used as a route map when the model is to be employed for the identification, or imitation of complex neuronal responses. In (1), x represents the membrane potential and y and z can be seen as summarizing the dynamics of the fast (sodium) and slow (calcium) currents. The parameter I is the current (injected) into the membrane. Parameter r is small, such that the system can be decomposed in a fast and a slow subsystem according to the adiabatic approach. The three most important parameters, or at least those that are sufficient for obtaining all possible interesting kinds of responses, are I, b and, to a lesser extent, the adiabatic parameter r. We limit ourselves therefore to a classification of the parameter space (b, I) for different values of r. As an example for the type of results we obtained, we show the result of a simulation analysis for r = 1E − 2 (Fig. 1). The parameter space is divided into four distinct regions. First, in the bottom right triangle the neuron is quiescent. When we increase the input current, at I ≈ 2 a cycle is born and the model gets into spiking mode (one spike per cycle). This cycle appears suddenly and with non-zero frequency, which is typical for type I neurons. On the left side of the figure more complex behavior exists. Starting at the bottom at b ≈ 2.6, the shades get darker for increasing current. In the beginning there is (regular) bursting behavior with

103


References [1] J. DeFilipe and I. Fari˜ nas, “The pyramidal neuron of the cerebral cortex: Morphological and chemical characteristics of the synaptic inputs,” Prog Neurobiol, vol. 39, pp. 563–607, 1992. [2] M. Toledo-Rodriguez, A. Gupta, Y. Wang, C. Z. Wu, and H. Markram, “Neocortex: Basic neuron types”. In The handbook of brain theory and neural networks, ed. M. A. Arbib, pp. 791–725. Cambridge, MA, USA: MIT Press, 2nd ed., 2002.

Figure 1: Classification by exhaustive simulation analysis of the HMR model. Every point in the figure represents a parameter duplet (b, I), here shown for r = 1E − 2. Gradient of the point is proportional to the number of distinct spikes per cycle.

two or three peaks per burst and, as the current increases, spikes are added to each cycle, hence the term “period adding”. In the middle region, where dark shades dominate, there are complex solutions. These darkest shades mean in general that there were more peaks per cycle than the program could count. This is an indication of chaotic behavior and indeed, when we make a phase-space plot of solutions in these regions it shows a strange attractor with a teacup form – a well known phenomenon in adiabatic systems [13]. The chaotic solutions in each black peak have a generating cycle corresponding to a solution in the period adding region. In results not shown here we show that the system does not have to be slow-fast decomposable to exhibit chaotic behavior and that in the biologically more plausible case of a moderately adiabatic system the chaotic regions are actually larger. We also did a numerical bifurcation analysis in which we study the homoclinic bifurcations of codimension one and two that organize the number of spikes (bursts) in the model. Furthermore we show that the complex transition between spiking and bursting behavior can only be explained by an analysis of the complete system; they do not exist in the fast subsystem. Referring to the classification of neuron models described in [14] and [2], but only looking at steady state behavior, since the onset behavior is a transient, which is not part of our static bifurcation diagram, we give a route map to obtain the following types of behavior: • • • •

[3] W. Gerstner and W. Kistler, Spiking neuron models. Cambridge, UK: Cambridge University Press, 1st ed., 2002. [4] B. Ermentrout, “Type I membranes, phase resetting curves, and sychrony.,” Neural Comp, vol. 8, p. 979, 1996. [5] E. M. Izhikevich, “Simple model of spiking neurons,” IEEE Trans Neural Networks, vol. 14, pp. 1596–1572, 2003. [6] R. Brette and W. Gerstner, “Adaptive exponential integrate-and-fire model as an effective description of neuronal activity,” J Neurophysiol, vol. 94, pp. 3637–3642, 2005. [7] J. Hindmarsh and R. Rose, “A model of neuronal bursting using three coupled first order differential equations,” Proc R Soc Lond B, vol. 221, pp. 87– 102, 1984. [8] M. L. Rosa, M. Rabinovich, R. Huerta, H. Abarbanel, and L. Fortuna, “Slow regularization through chaotic oscillation transfer in an unidirectional chain of hindmarsh-rose models,” Physics Letters A, vol. 266, no. 1, 2000. [9] I. Belykh, E. de Lange, and M. Hasler, “Synchronization of bursting neurons: What matters in the network topology,” Physical Review Letters, vol. 94, no. 18, p. 8101, 2005. [10] E. Izhikevich, “Neural excitability, spiking and bursting,” Int. J. Bifurcation and Chaos, vol. 10, no. 6, pp. 1171–1266, 2000. [11] V. Belykh, I. Belykh, M. Colding-Jørgensen, and E. Mosekilde, “Homoclinic bifurcations leading to the emergence of bursting oscillations in cell models,” The European Physical Journal E, no. 3, pp. 205–219, 2000. [12] Y. A. Kuznetsov, Elements of Applied Bifurctation Theory. Springer Verlag, New York, 2nd ed., 1998. [13] Y. Kuznetsov, O. De Feo, and S. Rinaldi, “Belyakov homoclinic bifurcations in a tritrophic food chain model,” SIAM Journal of Applied Mathematics, vol. 62, no. 2, pp. 462–487, 2001. [14] A. Gupta, Y. Wang, and H. Markram, “Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex,” Science, vol. 287, pp. 273–278, January 2000.

Regular and Fast Spiking (RS and FS) Irregular Spiking (IS) Intrinsic Bursting (IB) Stuttering (STUT)

104


Experimental Investigation on a Vestibular Natural Interface Cecilia Laschi*, Eliseo Stefano Maini*, Paolo Dario§*, Alain Berthoz† * ARTS Lab, Scuola Superiore Sant'Anna, 56127, Pisa, Italy [email protected], [email protected] § CRIM Lab, Scuola Superiore Sant'Anna, 56127, Pisa, Italy [email protected] † Laboratory of Perception and Action, Collège de France, 75005, Paris, France [email protected] This work addresses the problem of developing novel interfaces for robotic systems that can allow the most natural transmission of control commands and sensory information, in the two directions. Traditional robot interfaces are based on user’s actions on an input device; typically, a cortical remapping is required, from the motion intention of the person to the different geometry and kinematics of the input device. Furthermore, detecting the user’s action on the input device and transmitting it to the robot introduce a delay from when the movement is planned in the human brain to when it is accomplished by the robot. A more suitable approach to the development of natural interfaces is based on the detection of the user’s motion intention. This can be detected as it originates in the brain, by means of brain-machine interfaces [1], or when the control signal is transmitted in the nervous system to peripheral districts [2]. Nevertheless, it is argued in neuroscience that, in humans, simple movements anticipate to some extents other complex sensory-motor behaviours. Many authors have investigated various types of anticipation. For example, Land et al. [3] reported that during everyday activities, gaze fixations always are close to the object being manipulated, and very few fixations are irrelevant to the task occurred. Moreover, gaze arrives at the object to be manipulated some 0.5 seconds before any indication of manipulation. Johansson et al. [4] demonstrated that, in manipulation tasks, gaze consistently fixates future object contact points, well before the hand reaches these locations, and it anticipates reaching trajectory via-points. In a similar way, head movements are believed to anticipate body motions, such as turning while walking [5,6]. Such anticipatory movements may be used in a context-dependent manner for building natural and intuitive interfaces. The two main advantages of this approach are: (1) the detected movements are naturally associated with motor behaviours and as such they do not put any additional cognitive burden on the person; (2) the detected movements occur well in advance of motor behaviours and therefore they would help obtain a timely reaction in the controlled robotic system.

This work is aimed at validating the hypothesis that head movements can be used to detect, slightly in advance, a person’s intention to steer, during locomotion, and that a natural interface can be developed for controlling the navigation of a robotic artifact, based on this principle. Specifically, preliminary experiments have been conducted to validate the occurrence of anticipatory head movements in case of driving robotic artifacts, in locomotion tasks [7]. A prototype ‘vestibular’ interface has been developed to this purpose, based on a 3-axial artificial vestibular system, developed by some of the authors for humanoid robotics applications [8]. Three different experimental sessions have been carried out by using: (1) a driving video-game; (2) a robotic endoscope, with a 2-DOF steering tip; and (3) a mobile robot with a camera on-board. 22 subjects, in total, were asked to perform driving tasks, by using the traditional input interfaces of the different devices (joystick and gamepad) and by receiving a visual feedback from on-board cameras, through a binocular wearable display (I-Glasses by Video Pro 3D). The prototype ‘vestibular’ interface was mounted on top of the subjects’ heads. The 6 signals from the vestibular interface (3 linear accelerations and 3 angular velocities along 3 axes) were recorded and compared with the steering commands recorded from the input devices. Fig. 1 shows overviews of the three experimental set-ups. Among the 6 signals coming from the vestibular interface, the signal corresponding to the angular velocity of the head during rotation (yaw axis) resulted to have a good correlation with the signals corresponding to right and left steering, in all the experiments. 3. Mobile robot vestibular interface view from onboard camera 1. Rally videogame

Input device wearable display

2. Robotic endoscope

Figure 1: Overview of the three experimental sessions

105


Fig. 2 shows the two signals recorded in one of the trials with the driving video-game. If looking at the zero-crossing of the steering commands, it is clear how they are always anticipated by head rotations. The time of anticipation is in average close to 1 sec. Also, the versus of the head rotation is coherent with the corresponding steering command even if the amplitude of the two signals is not always proportional. Moreover, it may be noted that the head movement is almost completed in the very beginning of the steering command.

Acknowledgments This work has been partly supported by the EU within the NEUROBOTICS Project (The fusion of NEUROscience and roBOTICS, IST-FET Project #2003-001917).

Angular velocity of the head rotation Steering command 20

15

10

Angular velocity [deg/s]

reduced in the second scenario, due to the smaller dimensions of the image, the narrow field of view, the low speed, the unfamiliar environment. In conclusion, we argue that a vestibular natural interface, detecting head motion, can be developed for controlling the locomotion of a robotic artefact, in tele-presence scenarios.

5

0

References

-5

-10

-15

-20

-25 210

215

220

225

time [s]

Figure 2: Compared angular velocity of head rotation and movement of the input device in right-left steering, in one of the trial with the driving game

In the case of the robotic endoscope, the movements of the head during the experimental task were negligible. We envisage two main possible reasons explaining the lack of anticipatory movements of the head in this experimental scenario: first of all, the smaller dimensions of the image, the narrow field of view of the endoscope, and the unfamiliar environment (i.e. a mock-up of a human spine) are such that the subject could not have a real ‘tele-presence’ perception, and the task was perceived more as a precision task, than as a navigation task. Secondarily, the navigation speed was much lower than in the other scenarios and we argue that a relation exists between locomotion speed and the elicitation of anticipatory head movements. In the case of the mobile robot, head rotations anticipate steering of nearly 1 s and are concluded even before the steering commands start, similarly to the case of the driving game. This experimental study validated the hypothesis that head movements can be used to detect, slightly in advance, a person’s intention to steer, during locomotion. The results obtained with different experimental set-ups showed that anticipatory head rotations take place also when driving a remote system, instead of being walking. It is important to note that for one of the experimental scenario this result was not obtained. A comparative analysis of the three cases suggests that a critical role is played by the perception that the person can have of self-locomotion. This was in fact

106

[1] J.M. Carmena, M.A. Lebedev, R.E. Crist, J.E. O’Doherty, D.M. Santucci, D.F. Dimitrov, P.G. Patil, C.S. Henriquez, M.A.L. Nicolelis, "Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates", PLoS Biology, Volume 1, Issue 2, 2003, pp. 193-208. [2] X. Navarro, T.B. Krueger, N. Lago, S. Micera, P. Dario, T. Stieglitz, “A critical review of interfaces with the peripheral nervous system for the control of neuroprostheses and hybrid bionic systems”, J Pher Nerv Sys, 2006 (accepted). [3] Land M., Mennie N., Rusted J. “The roles of vision and eye movements in the control of activities of daily living”. Perception 28(11):1311-1328, 1999. [4] R.S. Johansson, G.B. Westling, A. Backstrom, J.R. Flanagan, “Eye-hand coordination in object manipulation”, Journal of Neuroscience, 21:6917-6932, 2001. [5] A. Berthoz, The Brain's Sense of Movement: Perspectives in Cognitive Neuroscience, Harvard University Press, June 2000. [6] R. Grasso, P. Prévost, Y.P. Ivanenko, A. Berthoz, “Eye-head coordination for the steering of locomotion in humans: an anticipatory synergy”, Neuroscience Letters 253 (1998) 115-118 [7] C. Laschi, E.S. Maini, F. Patane', L. Ascari, G. Ciaravella, U. Bertocchi, C. Stefanini, P. Dario, “A vestibular interface for natural control of steering locomotion of robotic artifacts: preliminary experiments with a robotic endoscope”, ISRR 2005 - International Symposium on Robotics Research, San Francisco, USA, October 12-15, 2005. [8] F. Patanè, C. Laschi, H. Miwa, E. Guglielmelli, P. Dario, A. Takanishi, “Design and development of a biologically-inspired artificial vestibular system for robot heads”, IROS 2004, IEEE/RSJ International Conference on Intelligent Robots and Systems, Sendai, Japan, September 28 – October 3, 2004.


Effects of Stress & Genetic Background on Meta-parameter Dynamics in a Simple Reinforcement Learning Model G. Luksys*,†, C.Sandi*, and W.Gerstner† Laboratory of Behavioral Genetics, EPFL, 1015, Lausanne, Switzerland † Laboratory of Computational Neuroscience, EPFL, 1015, Lausanne, Switzerland [Gediminas.Luksys, Carmen.Sandi, Wulfram.Gerstner]@epfl.ch *

Animals (and humans) choose their actions based on learned reward predictions for different environmental stimuli and motivational drives toward specific rewards. Different aspects of learning are known to be influenced by stress [1] and genetic factors [2]. Stress effects are thought to be dependent on its type (extrinsic, intrinsic task-specific or unspecific), intensity, timing, and the type of learning involved (spatial/episodic vs. procedural learning). While it is known that stress affects memory by modulating plasticity through stress hormones and neuromodulators, there is by far no integrative model that would accurately predict and explain differential stress effects. Although stress can be described in objective measures (e.g. level of food deprivation or uncertainty in reward delivery schedule in a task), its effects on learning and memory are strongly influenced by how an animal subjectively perceives it. Behavioural traits such as anxiety, impulsivity, and novelty reactivity, which are often coming from genetic background, play a key role in modulating stress effects. Temporal difference (TD) reinforcement learning models [3], often used to simulate learning in simple conditioning tasks, are suitable for evaluating the effects of stress on procedural learning. Discrete TD models contain different states, which the agent may occupy, and different actions that it may undertake in order to acquire rewards. The values of expected future reward (Q-values) are learned for each stateaction pair. How these values are updated and how actions are chosen based on them is controlled by certain meta-parameters such as learning rate, future reward discount factor, memory decay/interference factor, and exploitation-exploration factor. Different values of these parameters may lead to the evolution of different behavioural strategies for dealing with the environment. Besides dopamine (DA), whose levels are known to be related to the reward prediction error in TD framework [4], other neuromodulators such as serotonin (5-HT), norepinephrine (NE), and acetylcholine (ACh) are hypothesized to be the neural substrates of these meta-parameters [5]. Since stress indirectly acts through neuromodulators, it should affect respective meta-parameters in reinforcement learning models. Genetic differences, especially those related to neuromodulatory receptor/transporter phenotypes, are expected to influence the meta-parameters as well. To learn more about influences of stress and genotype on processes of learning and action choice, we

carried out 5-hole-box light conditioning experiments with C57BL/6 and DBA/2 inbred mouse strains, which are known to have profound differences in anxiety and other behavioural traits [6]. We exposed animals to different kinds of stress to evaluate its effects on immediate performance, and also tested their long-term memory after a break of 26 days. Then, we used a simple discrete state TD model to formalize their behaviour. For each experimental day, we estimated a set of model meta-parameters that produced the best fit between the model and the animal’s performance, including a number of different performance measures. Finally we analyzed how estimated meta-parameters evolved at different stages of learning, and which differences occurred between the genetic strains of mice (C57 vs. DBA) and between stressed vs. non-stressed groups of animals. The results in estimation of meta-parameters indicated an average-to-good fit between the model and animal behaviour. The goodness-of-fit values varied for different performance measures (see Figure 1), however did not significantly depend on mouse strain, stress group or experimental day. Generally, the most reliable estimates were those of the learning rate and the exploitation-exploration factor.

Figure 1: Comparison between several experimental and model performance measures.

107


During the course of learning, the learning rate followed an inverse-U shaped curve, peaking mostly during days 4-6 of the 8 day block (Figure 2). This could potentially be related to task-specific attention, which cannot be formed early because of the lack of familiarity with the task, and which decreases towards the end, when the task performance becomes habitual. The exploitation factor showed progressive increase with learning, reaching the peak at the end of the learning block (Figure 2). It might be inhibited by uncertainty (as represented by high reward prediction errors), which become smaller with learning. Consistently, the onset of uncertainty in reward delivery schedule resulted in decreased exploitation factors.

Figure 3: Differences in estimated learning rates and exploitation-exploration factors between C57BL/6 and DBA/2 mouse strains.

Figure 2: Dynamics of estimated learning rates and exploitation-exploration factors. Estimated meta-parameters differed significantly between the 2 strains of mice (Figure 3). C57BL/6 mice, being faster learners and more accurate performers, had slightly higher learning rates (ANOVA with repeated measures, p = 0.048), and much higher exploitation-exploration factors (p = 2·10–11). Long-term memory of the mice was affected by stress, as the memory inference/decay factors (ε), estimated for the first session after the break were higher for animals that underwent different stress conditions than in controls (Figure 4). Many of estimated meta-parameter dynamics were consistent with experimentally observed levels of corresponding neuromodulators and/or with optimal learning principles, but some others were contradictory. The existence of significant differences between mouse strains indicates a link between underlying genetic polymorphisms and behavioural parameters. This might be a useful tool in identifying the specific roles of gene-related neuromodulatory systems, involved in controlling learning dynamics.

108

Figure 4: Memory decay/interference factors for different genotypes and different stress conditions.

References [1] Kim & Diamond, “The Stressed Hippocampus, Synaptic Plasticity and Lost Memories”, Nature Reviews Neuroscience, 2002 [2] Wehner et al., “Quantitative Genetics and Mouse Behavior”, Annual Reviews Neuroscience, 2001 [3] Sutton & Barto, “Reinforcement Learning: An Introduction”, MIT Press, 1998 [4] Schultz et al., “A Neural Substrate for Prediction and Reward”, Science, 1997 [5] Doya, “Metalearning and Neuromodulation”, Neural Networks, 2002 [6] Holmes et al., “Behavioral Profiles of Inbred Strains on Novel Olfactory, Spatial and Emotional Tests for Reference Memory in Mice”, Genes, Brain, and Behavior, 2002


The Lausanne Neuroprosthesis: A Flexible Polyimide Microelectrode for Acute and Chronic Cortical Recordings André Mercanzini, Karen Cheung, Arnaud Bertsch and Philippe Renaud Microsystems Laboratory, Ecole Polytechnique Fédérale de Lausanne, Switzerland [email protected]

The field of neuroprosthetics promises to introduce new capabilities in restoring CNS functions lost to disease and will be one of the most important tools for neuroscience researchers elucidating the brain’s functions. Yet today their development remains limited due to the formation of a cellular encapsulating sheath around the device after implantation. The sheath creates a high electrical resistance between the probe and tissue and renders the probe unusable a few weeks following implantation. This project involves the design, implementation, and in vivo use of neural probes that take advantage of recent advances in slow release polymer coatings and microfabrication in order to create a probe which remains stable after implantation. We have developed a polymer microfabrication technique for chronic implantable devices. Thin, flexible polyimide foils containing planar microelectrodes are the basis for recording. Polyimide has been shown to be biocompatible and provides good dielectric strength in addition to high mechanical flexibility. Our microarrays consist of alternating layers of polyimide-platinum-polyimide which are patterned using a clean room process. They are fabricated on a solid substrate (see Figure 1) and are released following the anodic dissolution of a sacrificial aluminum layer underneath the polyimide. The final structures are 15 µm in thickness, with 200-nm metal layers. Different designs are possible: Figure 2 demonstrates a probe with 4 tetrodes and Figure 3 demonstrates part of a linear array of 16 electrodes with 100 µm spacing. The polymer microelectrode arrays are extremely robust, withstanding hairpin bends, yet maintain enough rigidity for insertion into the brain without the use of a guide. In-vivo acute recordings from the mouse were evaluated and results indicate that although the electrode’s intrinsic impedance (~ 1 MOhm) (see Figure 4) is higher than Michigan electrodes, these neuroprostheses give larger responses and good local pickup. The microfabricated electrodes have also been used in chronic recordings.

Figure 1 - A batch processed wafer with 50 flexible polyimide electrode arrays

50 um 150 um

Figure 2 – Four tetrodes with 150 um spacing. Electrodes are 25 um in diameter and are spaced 50 um apart.

100 um

Figure 3: A linear electrode array. Electrode sites are 25 um in diameter and spaced 100 um.

109


Figure 4: Electrical Impedance characteristics of the polyimide neuroprosthesis in vitro. (adapted from [1])

Figure 5: Demonstration of controlled drug release from a cellulose nitrate film doped with dexamethasone. Before release into PBS (left) and after 3 weeks in PBS (right). 0.7

The most common problem for devices implanted in the brain is the tissue-electrode interface and the eventual degradation of recording quality which renders the electrode unusable [2]. The first step of the initial inflammatory response is the covering of the polymer surface by proteins [3]. Cells then wall off the implant after a time lag depending on the material and on the implantation site. A capsule forms and engulfs the implant within 7-10 days thereby increasing its impedance. A sustained response is fully developed by 6 weeks [4]. For neural prostheses to become reliable tools for neuroscience researchers and clinically viable therapies for lost CNS functions, it is important that devices and procedures be developed to minimize or eliminate the formation of the cellular encapsulating sheath [4]. We are currently developing slow release drug coatings which will be integrated with these devices in order to locally deliver an anti-inflammatory drug to minimize the effect of the immunological response to device implantation. Our approach is to combat the initial immunological response using highly localized drug delivery from the probe itself. We developed a cellulose nitrate based coating which is doped with dexamethasone, an anti-inflammatory that has been demonstrated to alleviate the brain’s immunological response to implanted devices [5]. Figure 5 demonstrates preliminary results of a cellulose nitrate film which released dexamethasone into PBS solution over several days at therapeutic concentrations.

110

0.6 0.5 Absorbance

Compared to stiff metal or silicon electrodes, the flexible polyimide probes may reduce micromotion shear-induced inflammation and scar tissue formation, and also reduce mechanical mismatch between implanted probes and neural tissue. Histology of the tissue surrounding the implanted polymer structures has shown inflammation that is comparable to that seen around the implantation sites of smaller silicon structures. Therefore, reduction of the cross-sectional area of the polymer probes may improve the tissue reaction.

0.4 0.3 0.2 0.1 0 1

2

3

4

5

6

Days

Figure 6: Absorbance of dexamethasone in PBS. Release profile over 6 days.

Acknowledgements This project is supported by the Healthy Aims project IST-2002-1-001837. Andre Mercanzini gratefully acknowledges support from the Natural Sciences and Engineering Research Council of Canada.

References [1] S. Metz, A. Bertsch, D. Bertrand, and P. Renaud, "Flexible polyimide probes with microelectrodes and embedded microfluidic channels for simultaneous drug delivery and multi-channel monitoring of bioelectric activity," Bionsensors and Bioelectronics, vol. 19, pp. 1309-1318, 2004. [2] A. B. Schwartz, "Cortical Neural Prosthetics," Annual Review of Neuroscience, vol. 27, pp. 487-507, 2004. [3] E. Fournier, C. Passirani, C. N. Montero-Menei, and J. P. Benoit, "Biocompatibility of implantable synthetic polymeric drug carriers: focus on brain biocompatibility," Biomaterials, vol. 24, pp. 3311-3331, 2003. [4] D. H. Szarowski, M. D. Andersen, S. Retterer, A. J. Spence, M. Isaacson, H. G. Craighead, J. N. Turner, and W. Shain, "Brain Responses to micro-machined silicon devices," Brain Research, vol. 983, pp. 23-35, 2003. [5] L. Spataro, J. Dilgen, S. Retterer, A. J. Spence, M. Isaacson, J. N. Turner, and W. Shain, "Dexamethasone treatment reduces astroglia responses to inserted neuroprosthetic devices in rat neocortex," Experimental Neurology, vol. 194, pp. 289-300, 2005.


Evolution of neuro-controllers for flapping-wing animats Jean-Baptiste Mouret∗ and Stéphane Doncieux∗ ∗ AnimatLab / LIP6, Université Pierre et Marie Curie (Paris VI), Paris, France [email protected] [email protected]

1

Introduction

Being both highly manoeuvrable and able to glide, birds demonstrate the potential capabilities of a flapping-wing unmanned aerial vehicle (UAV). However, the control of such an artificial bird would require the generation of carefully synchronized rhythmic movements, which have to be modified in accordance with sensor inputs. In this work, which was done within the general framework of the Robur project [1], we focused on the evolution of wing-beat controllers for an animat flying horizontally at constant speed, even in the presence of air mass perturbations. A good starting point to design such controllers is to draw inspiration from real birds that, like most animals, probably base their locomotion behaviors on Central Pattern Generators (CPG). To bootstrap the evolutionary process and let it the chance of discovering such CPG, we chose to constrain our wing-beat controllers to call upon both non-linear oscillators and traditional McCulloch and Pitt’s neurons. Thus, the task of the evolutionary algorithms that were used was to combine such building blocks, to design the structure of the overall controller and to optimize its inner parameters. This work relied on a realistic aerodynamic simulator that made it possible to compare the efficiency of various controllers. In this simulator, an animats wing was modelled with two rigid panels and its body was built using cones and cylinders. A wings internal panel was linked to the body by two joints (twist and dihedral), while its external panel could be moved along two degrees of freedom (twist and sweep). The neural networks generated by the evolutionary process could control each of these degrees of freedom and could rely on an ideal air-speed sensor as input.

2

Evolutionary algorithm

The design of a good controller for a flapping-wing platform requires a trade-off between several aspects. The problem being intrinsically multi-objective, we used MOGA, a multi-objective evolutionary algorithm. To code the possible organization of any neural controller, we used ModNet, a modular encoding scheme

that manipulated traditional artificial neurons and nonlinear oscillators, thus making it possible to re-use in several places of the overall controller any specific combination of such elementary modules if it proved to be efficient for the evolved behaviour. The evaluation of the fitness relied on a two-step process. The first stage assessed the ability to fly horizontally at constant speed in a non-perturbed air mass. This evaluation took place in a simulated wind-tunnel in which the aerodynamic forces and moments were recorded. The second stage was designed to evaluate how well the artificial bird could keep its flying speed constant using its sensor. The simulated bird was mounted on a simulated rail with an initial speed. Randomly occurring horizontal forces were used to slow down the bird. During this process, best controllers are those which maintain the speed of the simulated bird as close as possible to the wanted speed.

3

Results

Two efficient control strategies emerged that both relied on ways to synchronize degrees of freedom similar to those observed in real birds. According to the first strategy (figure 1), the amplitude of the wing beat is amplified when the artificial bird has to speed up. For the second one (figure 2), the neuro-controller is split in two sub-networks. One part is an open-loop wingbeat generator. The other part adapts the external twist in order to generate the traction needed to maintain the speed. Videos of some of the evolved behaviors can be downloaded from our website1 .

References [1] Doncieux, S., Mouret, J., Muratet, L., and Meyer, J.-A., “The ROBUR project: towards an autonomous flappingwing animat” Proceedings of the Journes MicroDrones, 2004.

1 http://animatlab.lip6.fr/pages/ RoburEvolvingEn

111


80

An g le (d e g re e ) --->

60 40 20 0

-2 0 -4 0

-6 0 1 .5

1 .6

1 .7

d ih e d ra l e xt . t wis t

2 .1 2 .2 1 .9 2 .0 Tim e (s )---> e xt . s we e p speed t wis t h o riz. p e rt . 1 .8

2 .3

2 .4

speed = 15

(b) (a) Figure 1: (a) Neural controller for a typical individual that implements the first strategy (adapting amplitude). Rectangles are non-linear oscillators. Ovals are traditional artificial neurons. i0 is the input linked to the speed sensor. i1 is a constant input. Outputs: Di = Dihedral, ETwist = External Twist, ESweep = External Sweep, Twist. Modules being circled using dashed lines, the two darker sub-sets of oscillators and neurons are two copies of the same module. (b) Angular positions of the five joints of the right wing as a function of time. The upstroke starts when the dihedral is at its maximum positive value and ends when it is minimal. The amplitude of the oscillations increases when the artificial bird is slowed down (t = 1.8s to t = 2.1s).

60

An g le (d e g re e ) --->

40 20 0

-2 0 -4 0

-6 10 .2

1 .4 d ih e d ra l e xt . t wis t

1 .6

1 .8 Tim e (s )--->

e xt . s we e p t wis t

2 .0

speed h o riz. p e rt .

2 .2

2 .4

speed = 15

(b) (a) Figure 2: (a) Neural controller for a typical inidividual that implements the second strategy (adapting external twist). Modules are circled using dashed lines and similar gray levels denote copies of same modules. (b) Angular positions of the five joints of the right wing as a function of time. The upstroke starts when the dihedral is at its maximum positive value and ends when it is minimal. The external twist is adapted at each time step. The main twist (internal twist) is constant. The sweep is synchronized like in the individual depicted on figure 1. Although this is not displayed on the figure, the speed of the bird reaches the desired value (15m.s−1 ) about one second later.

112


A Reinforcement Learning Toolbox and RL Benchmarks for the Control of Dynamical Systems ∗

G. Neumann∗ Institute for Theoretical Computer Science, Graz University of Technology, A-8010, Graz, Austria [email protected]

We introduce the “RL Toolbox”, a new software tool for Reinforcement Learning. The RL Toolbox contains many different RL algorithms with emphasis on algorithms for learning continuous control problems. In addition to the toolbox, we made exhaustive benchmark tests with the implemented algorithms. These tests give a good overview about the capabilities of the algorithms, and, in combination with the RL Toolbox, simplify the work with RL considerably. A new algorithm, called the “Actor-Critic PolicyGradient” (PG-AC) algorithm is also introduced and compared to the other algorithms.

1

The Reinforcement Learning Toolbox

The Reinforcement Learning Toolbox (RLT 2.0) is a general C++ Library for all kind of reinforcement learning problems. The RL Toolbox is currently one of the most extensive library for RL, and is already well known in the research community. This can be seen by the visit counter of the home-page ([1], more than 6000 visits in 2 years) and also by searching in Google for the general term ”Reinforcement Learning”. The toolbox is already listed on the beginning of the second page. Additionally, there have already been two Master Thesis, for which the toolbox have been used ( [5] and [4]). There is also a manual, a class reference and several tutorials for the toolbox, which are available at the home-page.

1.1

Structure of the Toolbox

The Toolbox uses a modular design, so the Toolbox can be easily extended with new algorithms and learning problems. The class system of the toolbox consists of four main parts: the environment, the agent, the agent listeners and the agent controller. The learning problem is defined by the environment class. For a new learning problem this is the only class which have to be implemented by the user. The interaction of the class system can be seen in figure 1, for more details see [3].

As a result of the modular design, it is simple to exchange the environment, the learning algorithm or the policy.

Figure 1: The structure of the learning system, with the agent, the environment, the agent listeners as interface for the learning algorithms and the agent controllers

1.2

Algorithms

Nearly all common RL algorithms are included in the Toolbox as well as special algorithms for continuous state and action spaces which are particularly suited for optimal control tasks. These algorithms are TD(λ)Learning with value function approximation, continuous time RL, Advantage Learning and many more. For a complete list of the algorithms which are included in the toolbox see [3]. In addition to these already existing algorithms, a new Actor-Critic algorithm is introduced. This new algorithm is called Policy-Gradient Actor-Critic (PGAC) learning. The PGAC algorithm is a model-based ActorCritic algorithm which can be used for continuous action spaces. One main advantage of the PGAC algorithm is that planning can be used to update the policy, which makes the policy updates more robust.

1.3

Function Approximators

Most of these algorithms can be used with different kinds of function approximators, we implemented constant normalized RBF networks (GSBFNs, see [?]) and

113


feed forward neural networks (FF-NNs). RBF networks are usually easier to learn because they are linear function approximators and they only use local update rules. The advantage of feed forward neural networks is that they do not suffer from the curse of dimensionality.

2

Reinforcement Learning Optimal Control Tasks

for

The Benchmark Problems

We tested the RL algorithms on three different benchmark problems with different levels of difficulty. All the benchmark problems are commonly used in literature. The first task is called the pendulum swing up task. The goal of this task is to swing up a pendulum. The second task is called the cart-pole swing up task. Here, again, we have a pendulum, but now the pendulum is hinged to a cart. The goal of the task is to swing up the pendulum and to stay within a certain region with the cart. The third and hardest task is called the acrobot swing-up task. In this case, we have a two-link pendulum, again, we have to swing up both links by applying a force at the middle-joint.

2.2

(b) Cart-Pole

(c) Acrobot

Figure 2: The benchmark problems

We evaluated the learning algorithms for 3 different continuous control tasks, and analyzed how they can cope with different function approximators. The benchmark tests are done for most of the algorithms which are implemented in the Toolbox. The need for benchmark data for RL is apparent when we look at the papers published in this field. Many researchers use slightly different learning problems and even if there is a comparison of a new algorithm to other algorithms, it is hard to say how well this new algorithm really works because the performance of the compared algorithms strongly depends on the parameter settings which has been used. This issue has also been discussed in the Reinforcement Learning Benchmark Bake-offs ([2]) workshop at the NIPS conference 2005. But this workshop addressed RL in general and not for optimal control tasks and the benchmark problems which have been used were also more simple. To the best of our knowledge, our work contains the most exhaustive benchmark tests for learning optimal control tasks with RL.

2.1

(a) Pendulum

tricks to improve the performance of the algorithms. We incorporated the use of planning, directed exploration and hierarchic learning and compared the results.

2.3

Results

Due to the large collected data there are naturally many results, some are encouraging, some a little bit frustrating. All algorithms had a poor performance on the acrobot task. For the other two benchmark problems, VFunction learning approaches with RBF networks performed very well. We produced the best results with the use of planning, Hierarchical RL could also improve the performance considerably. For the RBF networks, the new PGAC algorithm had an average performance. Non-linear function approximation schemes had their difficulties; they usually needed a very long time to learn a good policy, if they succeeded at all. For the cartpole task, the V-Function learning already took over 100000 episodes to converge to a good policy. In this case the PGAC algorithm seems to have an advantage over all other algorithms, it managed to learn a good policy using a FF-NN as critic already after 20000 episodes. Learning was also observed to be more stable and more robust to the used parameters of the algorithm.

References [1] http://www.igi.tu-graz.ac.at/ril-toolbox/, Homepage of the RL Toolbox. [2] http://www.cs.rutgers.edu/˜mlittman/topics/nips05mdp/, Neural Information Processing Systems Workshop: RL Comparisons [3] G. Neumann, “The Reinforcment Learning Toolbox, RL for optimal Control Tasks”, Master Thesis, 2005

The Benchmark Tests

The algorithms were tested with constant normalized RBF networks and feed forward neural networks as far as it was possible. The influence of certain parameters of the algorithms like the learning rates or the λ parameter (used for the eligibility traces) was also evaluated. Further we investigated how we can use certain

114

[4] V. Hamburger, “Standing up with motor primitives”, Master Thesis, TU Augsburg, 2005 [5] M. Ullerstam, “Teaching robots behavior patterns by using reinforcement learning”, Master Thesis, Shibaura Institute of Technology in Tokyo, 2003


Perceptual learning with spatial uncertainties: models and experiments. T.U. Otto1∗, L. Zhaoping2 , M. Fahle3 & M.H. Herzog1 1

Laboratory of Psychophysics, EPFL, 1015 Lausanne, Switzerland ∗ [email protected] 2 Psychology Department, University College London, London WC1E 6BT, UK 3 Institute for Human Neurobiology, University of Bremen, 28211 Bremen, Germany A

B

Introduction Perceptual learning is the ability to improve perception per se. One example is the improvement of bisection discrimination [1, 2]. Perceptual learning is specific for many stimulus parameters. For example, no transfer of learning occurs when the orientation of stimuli changes. After the change, observers have to re-train to improve performance. Perceptual learning has gained increasing interest in recent years since it is concerned with the creation of the building blocks of perception. We perceive the world, at least partly, according to how we have learned to perceive it. However, the mechanisms underlying perceptual learning are still unknown [3]. Most models of perceptual learning (for a review see [4]) implicitly assume that stimuli are projected exactly to the same retinal position at each trial (Fig. 2A). However, this assumption is not met in the experimentation because of eye tremor, drifts, and microsaccades. Nevertheless, our perceptual performance often achieves a spatial acuity much finer than the typical magnitudes in the positional uncertainties of the stimulus. This is, for instance, the case in the bisection task studied in this contribution. As shown in Fig. 1, observers are asked to judge whether a middle element is closer to one or the other of two outer elements. Thresholds in these tasks are in the order of a few tens of arc seconds, whereas the microsaccades are in the order of tens of arc minutes. A recent ideal observer analysis showed that introducing positional uncertainties in the stimuli changes the underlying perceptual decision making problem from a linear to a nonlinear problem which is approximately quadratic for small positional uncertainties [5]. It was proposed that the nonlinear computation could be achieved by recurrent connections between the nonlinear sensory neurons before the neural responses are sent to the next, linear decision stage (Fig. 2B). This non-linearity implies three predictions.

Third prediction. Improvement of performance can be diminished or even abolished if bisection stimuli with two different outer element distances are presented randomly interleaved during training.

First prediction. If the bisection stimulus is presented at random positions, improvement of performance is comparable to when the bisection stimulus is presented constantly at one position.

Results To test the first prediction, we presented a line bisection stimulus with its position randomly chosen in an area of twice the size of the stimulus itself. As predicted by the model, the human brain can improve

1200’’

Figure 1: The bisection task is to judge whether the central element is closer to the left or to the right outer delineator. A. Line bisection stimulus with an outer line distance of 1200”. B. Dot bisection. A

B

Decision unit: S w(x) a(x) > 0 ?

Decision unit Feedforward weights W Peak width, determines performance

Weight function w(x) or decision template

Recurrent weights J

Sensory neurons

a(x)

Neural activity a(x) -1

e

1

Spatial location x

-1+y

e+y

1+y

x

Figure 2: Models. A. Feedforward network. The decision depends only on neurons responding to the central line, i.e. the weights w(x) = 0 for neurons responding to the outer lines. B. Recurrent network. The decision network is augmented by recurrent connections J, and the feedforward weights w(x) examine inputs from neurons responding to all lines. Second prediction. Training with a bisection stimulus of one given outer element distance can yield negative transfer to bisection stimuli with other outer element distances.

115


Summary Perceptual learning, particularly for hyperacuity tasks, requires the ability of the human brain to cope with positional uncertainties. Indeed, this was experimentally found. Mathematically, coping with spatial uncertainties changes the decision problem from linear to non-linear [5]. A nonlinear model suggests that learning two outer line distances simultaneously would require the creation of two sets of recurrent connections, one for each outer line distance, on the same neural population. Since these two sets of connections are likely to conflict with each other, learning would be difficult or impossible as found in our data. However, the prediction of negative transfer, implied by this model, could not be verified. Hence, while nonlinear recurrent networks may achieve positional invariance, the brain is able to prevent negative transfer by learning mechanisms not considered by our model.

116

A

B 80

1.5

Threshold ratio (post/pre)

Threshold (arc sec)

9 observers

60

40

20 9 observers

rLLL

1.0

0.5

0 0

2

4

6

8

10

12

14

rLLL

Block number

LLL

rDDD

oLLL

Figure 3: Position uncertainty. A. Bisection acuity as a function of training with line bisection stimuli whose position was randomly varied (means and s.e.m.). B. We determined performance before and after the training. Performance improves significantly for the trained ‘jittered’ line stimulus (rLLL) as well as for its constant version (LLL). Transfer of learning occurs neither to dot (rDDD) nor orthogonal line stimuli (oLLL) indicating a specific learning (means and s.d.). A

B 80

90

Percent correct (%)

Threshold (arc sec)

performance with line bisection stimuli even though this positional variation is about a factor of 50 larger than bisection acuity thresholds (Fig. 3). This experiment clearly demonstrates that perceptual learning occurs under position variant conditions even if subjects would fixate perfectly. Moreover, it is surprising that bisection learning is very unspecific for the exact position of training but very specific for the exact stimulus type. To test the second prediction, observers trained line bisection at a constant stimulus position. We determined the degree of transfer to a number of line bisection stimuli with different outer line distances. We were particularly interested whether or not negative transfer occurs with one of these distances. However, bisection acuity remains constant or even seems to improve slightly in some untrained conditions (results not shown). Hence, we do not find evidence for the prediction of negative transfer. To test the third prediction, observers trained with bisection stimuli with two different, randomly interleaved outer line distances. The model suggests that different, even conflicting, recurrent networks are needed if bisection stimuli with more than one outer line distance have to be trained. Since perceptual learning involves learning the optimal recurrent network for each condition, one might expect that interleaving the conditions during learning makes it impossible to learn either network and, thus, no or diminished improvement of performance should result in either condition. This is what we found (Fig. 4). This result is in good agreement with the findings showing that in a contrast discrimination task no learning occurs if the reference contrasts are unpredictably interleaved [6]. However, in contrast to an improvement of templates, our computer simulations favor an explanation in terms of recurrent connections [5].

60

40

20 1800’’ 1200’’

80

70

60 1800’’ 1200’’

5 observers

0

4 observers

50 0

2

4

6

8

10

12

Block number

14

16

0

4

8

12

16

20

24

Block number

Figure 4: Stimulus uncertainty. Bisection performance as a function of training. Line bisection stimuli with two different outer line distances (1200” and 1800”) were randomly interleaved during training. Improvement of performance is weak compared to Fig. 3A (means and s.e.m.). A. Thresholds. B. Percent correct.

References [1] Crist et al. (1997). Perceptual learning of spatial localization: specifity for orientation, position, and context. J Neurophysiol, 78:2889-94. [2] Westheimer et al. (2001). Configuration specificity in bisection acuity. Vision Res, 41:1133-8. [3] Fahle M. & Poggio T. (Eds.) Perceptual Learning. MIT Press (2003). [4] Tsodyks & Gilbert (2004). Neural networks and perceptual learning. Nature, 431:775-81. [5] Zhaoping, Herzog & Dayan (2003). Quadratic ideal observation and recurrent preprocessing in perceptual learning. Network, 14:233-47. [6] Yu C., Klein S.A. & Levi D.M. (2004). Perceptual learning in contrast discrimination and the (minimal) role of context. J Vision, 4(3), 169-82.


Neural Modeling of Imitation Deficits B. Petreska∗, M. Adriani†, O. Blanke† and A. G. Billard∗ ∗ Learning Algorithms & Systems Laboratory, LASA, EPFL, 1015 Lausanne, Switzerland {biljana.petreska, aude.billard}@epfl.ch † Laboratory of Cognitive Neuroscience, LNCO, EPFL, 1015 Lausanne, Switzerland {michela.adriani, olaf.blanke}@epfl.ch This abstract addresses the question of human imitation through convergent evidence from neuroscience. We look at deficits in imitation following brain lesion, such as apraxia. We believe that looking at how imitation is impaired can unveil its underlying principles. We also take inspiration from numerous brain imaging studies to ground the functional architecture and information flow of our model. In the end we will use findings from monkey brain neurophysiological studies to implement the details of our processing modules. We aim at developing a model of visuo-motor imitation using tools from neural networks and dynamical systems. The model should account for some of the behaviors observed in faulty imitation. At this stage we have implemented a somatotopically organized neural network with probabilistically impaired transfer of information that simulates lesions at the level of the parietal cortex (a brain center for sensorimotor integration). To validate the model against human motion experimental data, we conduct, in collaboration with the Geneva University Hospital (HUG), kinematic studies with brain damaged adults specifically disabled in gesture imitation. The model will motivate the realization of computer-based rehabilitation tools. Introduction. Apraxia is generally defined as the inability to perform voluntary movements that cannot be explained by elementary motor, sensory or cognitive deficits (not caused by weakness, ataxia, akinesia, deafferentation, inattention to commands or poor comprehension). Some apraxic patients are impaired for imitation of meaningless gestures, which is believed to test the integrity of a direct route from visual perception to motor control, not mediated by semantic representations or verbal concepts. Knowledge about the human body is also relevant as apraxic patients are unable to map body configurations to their own body or to a mannikin[2]. Kinematic studies show that patients exhibit either a completely normal kinematic profile, but abnormal final position; or kinematic abnormalities (slowing and repeated changes of direction of movement), with correct target [4]. Spatial parapraxias seem to arise from a basic deficit that might concern the mental representation of the target position and kinematic abnormalities from the strategy of online visually con-

trolled movements that cope with it. Experimental Study. A seminal study of imitation of meaningless gestures [3], by Goldenberg, was of particular interest to us (Fig. 1). A patient that suffers from a disconnection between the two hemispheres (following callosal lesions) was asked to imitate hand postures in relation with the head. The study shows that the pattern of errors varies as a function of the visual field to which the stimuli to imitate are displayed and the hand used to execute the imitative movement. Imitation was perfect only in the right visual field right hand condition, indicating a lateralization of the processing to the left hemisphere and a non-uniform information flow across the two hemispheres.

Figure 1: Goldenberg’s experiment of imitation of meaningless gestures and an example of errors made by the patient, from [3]. As we did not have access to data on errors in imitation of apraxic patients (lesion studies provide only statistical data of the correctness of the imitation), we decided to replicate Goldenberg’s experiment of imitation of meaningless gestures in collaboration with the HUG [3]. As we were interested in providing a quantitative data of the deficit, we extended the experiment to record the movement kinematics and hand posture with motion tracking sensors. Computational Model. A neural model of imitation of meaningless gestures would encompass several regions dedicated to specific functions, shown in Fig. 2. We concentrate our modeling work on the parietal cortex, considered to be the center for visuo-motor

117


and multimodal (somatosensory, visual, auditive and vestibular) integration (lower-order computations than in the frontal cortex). Moreover, lesions in the parietal cortex lead to imitation deficits. We have implemented Action Execution 'Body Schema'

VITE: Biologiccaly inspired dynamical model for joint motion during point to point reaching motion:

Somatotopically organised neural network with a representation of the face

Figure 3: A model of the face with tactile sensors, which are the input to our SOM.

Motor Cortex Dorsal Premotor Cortex (BA 6)

* Parietal Cortex (BA 7)

* ?

Parietal Cortex (BA 40)

?

Visuomotor Flow

'EBA' and MT/V5 areas (BA 19/37)

'Body Image' Visual analysis of the action (not directly addressed in this thesis)

Figure 2: Our neurocomputational model of imitation of meaningless gestures. The functional architecture and connectivity of our model is inspired by data from human brain imaging studies [1, 5]. We have modeled area BA 40 (a sensorimotor ’body schema’ module) as a face somatotopically organized neural network, locus of simulated lesions. An ’action execution’ module (in the motor cortex) is necessary for validating the model against experimental data and implements a biologically inspired dynamical model for reaching (VITE). a computational model of this region to simulate focal and diffuse lesions of the transfer of information between its left and right parts (see Fig. 4). We decided to use leaky integrate and fire neurons, which is a simple dynamic model that accounts for variations in the neuron membrane potential mi of neuron Ni over time: X τi · dmi /dt = −mi + wi,j xj (1) where xj represents the neuron’s short-term average firing frequency, τi is a time constant associated with the passive properties of the neuron’s membrane, and wi,j is the synaptic weight of a connection from neuron Nj to neuron Ni . The input is both visual (the stimulus to imitate) and somatosensory (a departing posture and target in terms of relations of body-parts). Therefore it seemed natural to have a somatotopic organization of the information, as is the case in several parietal regions. We trained a neural network to respond to particular regions of the face (Fig. 3), using Kohonen’s algorithm. In the end we obtain a somatotopic representation of the face: parts close to each other on the face have close representations in the neural space and parts more important than others (such as the eyes and the mouth) have larger representations. As the patient converges to the correct response with time, we

118

Figure 4: Neural network applet for training the somatotopically organized network (SOM) related to the face. The activation from neurons in the left network is transmitted to the right network with probability pi from Eq. 2 simulating the brain lesion.

decided to simulate the lesion in a probabilistic way. We suppose that the information does not always fail to transfer, thus each neuron is assigned a probability pi of firing and transmitting the value of the membrane potential to the corresponding neuron in the other hemisphere (related to the severity of the lesion): P (xj = (1 + e−mj +bj )−1 ) = pi ,

xj = 0 otherwise (2)

with 0 ≤ pi ≤ 1. Varying lesion parameters (type, size, locus and severity) induce different patterns of errors.

References [1] J. Decety, et al. Brain activity during observation of actions. Brain, 120:1763–1777, 1997. [2] G. Goldenberg. Imitating gestures and manipulating a mannikin - the representation of the human body in ideomotor apraxia. Neuropsychologia, 33(1):63–72, 1995. [3] G. Goldenberg, K. Laimgruber, and J. Hermsdörfer. Imitation of gestures by disconnected hemispheres. Neuropsychologia, 39:1432–1443, 2001. [4] J. Hermsdörfer, et al. Kinematic analysis of movement imitation in apraxia. Brain, 119:1575–1586, 1996. [5] M. Mühlau, et al. Left inferior parietal dominance in gesture imitation: an fMRI study. Neuropsychologia, 43:1086–1098, 2005.


ReSuMe learning method for Spiking Neural Networks dedicated to neuroprostheses control F.Ponulak and A.Kasiński Institute of Control and Information Engineering, Poznań University of Technology, 60-965, Poznań, Poland {Filip.Ponulak, Andrzej.Kasinski}@put.poznan.pl In this paper we consider ReSuMe - a Remote Supervised Method [1] for precise learning of spatiotemporal patterns of spikes in Spiking Neural Networks (SNN) [2, 3]. The learning method is dedicated to neuroprostheses control. Neuroprosthetic systems aim at producing functionally useful movements of the paralysed organs by stimulating muscles or nerves with the sequences of short electrical impulses [4]. Controllers of such systems are supposed to be robust and flexible. A special emphasis should be put on their good learning abilities and the adaptability to non-stationary character and nonlinearities of the human neuro-musculo-skeletal system. Spiking Neural Networks (SNN) exhibit properties that make them particularly suitable to control neuroprostheses. SNN are not only highly adaptive and computationally very powerful. They are also particularly suitable to process information encoded in time [2]. Moreover, the representation of signals transmitted through- and produced by SNN is very similar to that required to stimulate muscles or nerves. However, the analysis of the recent supervised learning methods for SNN [5] revealed that the existing algorithms were not sufficient for the task at hand. This led to wider exploration of other approaches, which resulted in inventing ReSuMe. ReSuMe takes advantage of the spike-based Hebbian processes [3] and integrates them with a novel concept of remote supervision introduced in [1]. In this approach the efficacy w of any synaptic connection between a presynaptic neuron nin and a postsynaptic neuron nl is modified according to the following rule: Z ∞ d w(t) = S d (t)−S l (t) a+ W (s) S in (t−s) ds , dt 0 where S d (t), S in (t) and S l (t) are target, pre- and postsynaptic spike trains, respectively. The spike trains are defined here by the sums of the firing times [3]. The parameter a expresses the amplitude of the noncorrelation contribution to the total weight change, while the convolution function represents the Hebbianlike modifications of w. The integral kernel W (s) is known as a learning window defined over a time delay

s between the spikes occurring at the synaptic sites [1]. In the case of excitatory synapses the term a is positive and the learning window W (s) has the shape similar as in STDP [3]. In the case of inhibitory synapses a is negative and W (s) is defined similarly as for the antiSTDP rules. For the complete introduction to ReSuMe we refer to [1]. ReSuMe enables supervised learning while still inheriting interesting properties of unsupervised Hebbian approach, i.e. the locality in time and space, simplicity and the suitability for online processing. On the other hand, ReSuMe avoids drawbacks of the Hebbian- and, so called, supervised-Hebbian methods [5]. ReSuMe has been successfully applied to feedforward, recurrent and hybrid (e.g. Liquid State Machine [6]) network architectures. The learning properties of ReSuMe have been investigated in the extensive simulation studies accompanied by the theoretical analysis. In [7] it has been demonstrated that ReSuMe can effectively learn complex temporal and spatio-temporal spike patterns with the desired accuracy and that the method enables imposing on the SNNs the desired input/output properties by learning multiple pairs of input-output patterns. In addition, it has been shown that ReSuMe is able to successfully train the networks consisting of different models of neurons (from simple LIF, to complex biologically realistic models) [7]. In all experiments it was observed that ReSuMe learning process converged very quickly. In [8] we demonstrated the generalization properties of the spiking neurons trained with ReSuMe. This property supports the thesis that SNN can be trained with ReSuMe to become an effective model of the reference objects, such as biological neural or neuromuscular structures. ReSuMe proved to be applicable not only to the modeling, but also to the control tasks. In [9] we consider an experiment in which ReSuMe was successfully applied to generate movement of the 2-DOF model of leg equipped with 4 muscles. A spiking network was trained to reconstruct the spatio-temporal patterns of impulses corresponding to the patterns of activity in the pools of motoneurons. Each pool, consisting of 40 neu-

119


rons, activated the particular muscle model. The model of a limb, driven by the SNN, was able to follow the desired trajectory of movement with a high precision. That study is recently put a step further. In a project on the adaptive Central Pattern Generators (CPG) the spiking networks are trained to produce the desired spike patterns resulting in the rhythmic movements of the limb models1 . ReSuMe is also applied to control the limb model in a feedback-loop system. In this task the network acts not only as a pattern generator but also as a controller. The network has to correctly react to the command signals and to the possible errors between the desired and the actual plant state. The results of the closed-loop control experiment are illustrated in Fig.1. The quality of approximation of the desired trajectory is slightly worse as compared to the results obtained in the open loop case. On the other hand the closed-loop system demonstrates higher robustness to the external perturbations. In parallel to the simulation studies, we develop the hardware system for the neuroprostheses control. So far ReSuMe method has been implemented and tested on FPGA matrices2 . All results discussed above indicate potential suitability of the spiking networks trained with ReSuMe to control the neuroprosthetic systems. An important aspect of the future work on ReSuMe is to verify this ability in the real-world applications.

Acknowledgement The authors would like to express gratitude to Mr Krzysztof Walas for his contribution to the presented results. The work was partially supported by the Polish State Committee for Scientific Research, project 1445/T11/2004/27.

References [1] Filip Ponulak. ReSuMe - new supervised learning method for Spiking Neural Networks. Technical Report, Institute of Control and Information Engineering, Poznan University of Technology, 2005. Available at http://d1.cie.put.poznan.pl/˜fp/. [2] Wolfgang Maass. Networks of spiking neurons: The third generation of neural network models. Neural Networks, 10(9):1659–1671, 1997. [3] Wulfram Gerstner and Werner Kistler. Spiking Neuron Models. Single Neurons, Populations, Plasticity. Cambridge University Press, Cambridge, 2002. [4] Dejan Popović and Thomas Sinkjaer. Control of Movement for the Physically Disabled. Springer-Verlag, London, 2000. 1 Details are given in an accompanying paper “Adaptive Central Pattern Generator based on Spiking Neural Networks” 2 Details in an accompanying paper: “FPGA implementation of the ReSuMe learning method for SNN”

120

Figure 1: SNN trained to control a 1-DOF leg model in a feedback-loop. (A) Desired leg trajectory q d (t) obtained as an effect of the contractions τed (t), τfd (t) at the extensor and flexor muscle models, respectively (B). (C) Required neural activity Ade (t) and Adf (t) for pools of extensor and flexor ’motoneurons’. Ade (t) and Adf (t) are the spike patterns to be learned by SNN. (D) Contraction τe (t), τf (t) of the muscle models resulting from the spike pattern Ae (t) and Af (t) generated by the trained SNN. (E) Resulting movement trajectory q(t) and the corresponding error: e(t) = q d (t)−q(t). [5] Andrzej Kasiński and Filip Ponulak. Comparison of Supervised Learning Methods for Spike Time Coding in Spiking Neural Networks, 2005. Submitted for publication. Preprint available at http://d1.cie.put. poznan.pl/˜fp. [6] Wolfgang Maass, Thomas Natschlaeger, and Henry Markram. Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations. Neural Computation, 14(11):2531–2560, 2002. [7] Andrzej Kasiński and Filip Ponulak. Experimental Demonstration of Learning Properties of a New Supervised Learning Method for the Spiking Neural Networks. In Proc. ICANN’2006: Biological Inspirations, volume 3696 of LNCS, pages 145–153, 2005. [8] Filip Ponulak and Andrzej Kasiński. Generalization Properties of SNN Trained with ReSuMe, 2006. Submitted to ESANN’2006. Preprint available at http: //d1.cie.put.poznan.pl/˜fp. [9] Filip Ponulak and Andrzej Kasiński. A novel approach towards movement control with Spiking Neural Networks. In Proc. AMAM’2005, Ilmenau, 2005. (Abstract).


Adaptive Central Pattern Generator based on Spiking Neural Networks∗ F.Ponulak, D.Belter and A.Kasiński Institute of Control and Information Engineering, Poznań University of Technology, 60-965, Poznań, Poland {Filip.Ponulak, Andrzej.Kasinski}@put.poznan.pl In this paper we present a new, adaptive model of the Central Pattern Generator (CPG) [1] based on Spiking Neural Networks (SNN) [2]. The model has the ability to learn the desired rhythmic patterns from demonstration. Central Pattern Generators (CPG) play the principle role in such processes as locomotion, breathing or heart-beating of animals. For this reason they are of great interest for scientists. Many models of CPGs have already been proposed [3]. However, most of these models are suitable only for the individually designated tasks. It is desirable to construct the generic CPG model with the universal ability to learn the task from the given, desired motor-patterns. An interesting approach to such a programmable CPG has been proposed in [4]. In that approach a CPG model has been based on Hopf oscillators. Here, we introduce a model of an adaptive CPG based on SNN. Our model can learn the desired motorpattern to be generated in each cycle of CPG operation. The learning process is performed according to the Remote Supervised Method (ReSuMe)1 introduced in [2]. We describe our approach in an example of a CPG model with 2 outputs. This corresponds e.g. to the CPG generating the alternating flexor-extensor activity responsible for limb coordination during locomotion. Our model can be divided into two functional parts: rhythm and pattern generator [1] (Fig.1). The rhythm generator produces the basic oscillations and controls their frequency. This part consists of 2 reciprocally connected networks (A, B) of spiking neurons. The networks have the sparse recurrent organization. In our recent study we are concerned only with reproducing the desired shape of the motor-patterns. Hence, we assume that the parameters of the rhythm generator, such as the frequency of oscillations, the duty cycle and the phase relationship are not modified ∗ The work was partially supported by the Polish State Committee for Scientific Research, project 1445/T11/2004/27. 1 General overview of ReSuMe and its applications to movement control are given in an accompanying paper “ReSuMe learning method for Spiking Neural Networks dedicated to neuroprostheses control”

by learning. They are adjusted only once at the beginning of an experiment. The pattern generator produces the actual motor patterns in response to the driving inputs from the rhythm generator. The pattern generator consists of 4 populations of neurons (C,D,E,F). Only populations C and D receive inputs directly from the rhythm generator. The role of these populations is to form a reservoir of the various neural dynamics to be utilized by the remaining populations E and F. Populations C and D consist of multitude of neurons, where each neuron is initialized with the random parameter values, so as to obtain individual dynamic properties.

Figure 1: Network architecture of the proposed CPG model. The populations E and F can be thought of as the pools of motoneurons. These populations are subjects to learning. They are supposed to reproduce the neural activities Ade (t), Adf (t) determined by the desired tension profiles τed (t), τfd (t) of the muscle models (Fig.2). Given the desired tension τ d (t) the corresponding neural activity Ad (t) can be obtained e.g. from the ENG multichannel recordings or from the inverse muscle model, if available. However, in our experiment we use a simple conversion to approximate Ad (t). Namely, the spatio-temporal neural patterns are evaluated from the desired tension profile by assuming that the instantaneous frequency of spikes at the particular neurons is proportional to the actual value of τ d (t) (Fig.2.B,C). For each neuron the proportion ratio is selected ran-

121


Figure 2: Results of an experiment. (A) Desired tension profiles at an extensor τed (t) (black) and a flexor τfd (t) (grey) muscle model. (B,C) Required neural activities Ade (t) and Adf (t) corresponding to τed (t) and τfd (t), respectively. (D,E) Neural activities Ae (t) and Af (t) reproduced by the trained CPG. (F,G) Resulting tension profiles τe (t), τf (t) and (H,I) the corresponding errors ee (t) = τed (t)−τe (t) and ef (t) = τfd (t)−τf (t). domly according to the normal distribution with a relatively small variance. The errors introduced by this conversion diminish as the number of units in the neural population increases. We assume that the level of errors obtained for the populations E and F with 100 neurons in each one, is acceptable. The desired neural activities Ade (t) and Adf (t) serve as the reference (teacher) signals for populations E and F during learning (inputs with labels ’Tchr E’ and ’Tchr F’ in Fig.1). In the experiment presented in Fig.2 the training was performed for 20 seconds of the simulated time. During that time the desired patterns were presented to the learning neurons for 100 times. The neural activities Ae (t) and Af (t) produced by the trained network are shown in Fig.2.D,E and the resulting muscle contractions τe (t), τf (t) are given in Fig.2.F,G (for clarity we present the results in a short time range only). The comparison of the desired and generated neural activities (Fig.2.B,D,C,E) shows that most of the desired firings were correctly reproduced by the CPG model. The approximation errors ee (t) = [τed (t)−τe (t)] and ef (t) = [τfd (t)−τf (t)] (Fig.2.H,I) are of the order smaller then the amplitudes of the corresponding tension profiles. This indicates the good performance of the learning process. The results presented here confirm the ability of the proposed CPG model to learn and to rhythmically generate the desired motor-patterns. In the considered experiment the networks A and B consisted of 400 neurons, the number of neurons in C and D was set to 3000 and the populations E and F were composed of 100 motoneurons. All neu-

122

rons were represented by the Leaky-Integrate-and-Fire model. The simulations were performed in a CSIM toolbox (http://www.lsm.tugraz.at). An interesting extension of the presented approach is the application of the learning procedure to the rhythm generator. Thus not only the shape, but also frequency, duty cycle or phase relationship could be learned from demonstration. Another interesting issue concerns learning multiple motor-patterns in a system under consideration and utilizing external signals to trigger the particular behaviors of the CPG model.

References [1] Scott L. Hooper. Central Pattern Generators. In F.Moss and S.Gielen, editors, Encyclopedia of Life Sciences. Macmillan Reference, 2001. Electronic press at www.ELS.net. [2] Filip Ponulak. ReSuMe - new supervised learning method for Spiking Neural Networks. Technical Report, Institute of Control and Information Engineering, Poznan University of Technology, 2005. Available at http://d1.cie.put.poznan.pl/˜fp/. [3] Eve Marder and Dirk Bucher. Central Pattern Generators and the Control of Rhythmic Movements (Review). Current Biology, 11:R986–R996, 2001. [4] Ludovic Righietti, Jonas Buchli, and Auke Jan Ijspeert. From Dynamic Hebbian Learning for Oscillators to Adaptive Central Pattern Generators. In Proc. of 3rd International Symposium on Adaptive Motion in Animals and Machines, Ilmenau, 2005. Full text on CD.


EEG-based control of reaching to visual targets F. Popescu, Y. Badower, S. Fazli, G. Dornhege and K.-R. Müller* * Fraunhofer –FIRST.IDA, 12489, Berlin, Germany [email protected], [email protected], [email protected], [email protected], [email protected] Abstract Research on non-invasive brain computer interfaces (BCI) has shown that electroencephalograhy (EEG) on-line signal extraction can be used for communication (spelling), computer game playing and for sensorassisted navigation. In this study we attempt to quantify reaching movement performance using EEG and gaze tracking signals. To achieve this the Berlin Brain Computer Interface has been linked to an eye and head tracker. The task studied was typing at a virtual keyboard, with a data information transfer rate of the resulting BCI of 70 bits/s, demonstrating that noninvasive BCI designs can provide useful means to command robotic devices for Brain Machine Interface (BMI) reaching tasks.

2 seconds and trajectory accuracies on the order of 2 cm [4, 6]. While there are many other valid performance measures, even after restricting criteria to those based on task performance, given that some BMI designs go so far as orienting grippers and grabbing objects, it is point-to-point movement speed and accuracy that remains the most basic of motor performance measures which can be expected to affect performance in more complex tasks. We have set up an experiment in which the accuracy of a single reach is limited by the performance of gaze tracking and the speed is limited by the performance of a non-invasive BCI design. Using typing as a test task, we aimed to measure the achievable speedaccuracy of a non-invasive brain to robot interface.

Introduction BCI interfaces present a unique opportunity for the restoration of motor and communicative function for patients challenged by severe paralysis [1]. As the clinical causes of impairment can greatly vary, so can the residual level of motor ability and the specific need of assistive technology. In the most affected patients, the ‘locked-in’ group, there is no residual motor ability. As there are no other means available to the patient to communicate with outside world, both invasive and non-invasive BCI use is warranted, within the limits posed by limited patient consent and surgical risks. Nevertheless the relative number of these cases is rare: much more common are cases of spinal trauma induced tetraplegia, in which arm function is lost, but facial and eye muscle control remain intact. In such cases, non-invasive means of restoration of reaching and grasping promises to offer significant benefits at limited risk and cost and is addressed in this study. The kinds of tasks that EEG BCI designs have been applied to include spelling for communication for ALS and locked-in patients [1], computer games in normal subjects for purposes of BCI development [2] and navigation of nearly autonomous intelligent robots [3] Meanwhile, invasive BCI designs have shown effective restoration of grasp function in monkeys [4, 5] and are currently being tested in human patients. In the comparison of risks and benefits of various BMI designs, one of the significant performance metrics to consider is the expected speed-accuracy tradeoff for reaching movements. Some invasive BMI studies for monkeys report robot movements as fast as

Methods. A single, non-impaired volunteer subject was seated at a standard PC workstation. The subject wore a 64channel EEG cap connected to an amplifier system (BrainAmp128DC, Munich, Germany) sampling at 1KHz. The subject wore a pair of eye tracking cameras (ViewPoint Eye Tracker, Arrington Research, Scottsdale, AZ) fixed with respect to the cranium and to a 6 DOF head tracker (3Space Fastrack, Polhemus, Colchester, VT) by means of elastic band strapped glasses. The combination of stereo eye tracker and head tracker was calibrated to locate the point of gaze on an LCD monitor. A picture of the experimental setup is found in Figure 1. The EEG classification was based using the common spatial patterns algorithm [7], in a three class paradigm, consisting of a ‘left’ handed movement imaginations and a ‘relax’ class. Parameters were chosen such that there was considerable bias towards the rest class. Deviations from the rest class were then used to trigger desired commands if gaze was steady at that particular time. The subject, after the standard 30min BBCI training procedure, was instructed to type at a virtual keyboard shown on a computer monitor. Its layout was based on the QWERTY arrangement, keeping only the letters, ‘space’ and ‘delete’ keys. The subject was asked to focus on the letter he wished to type, and while doing so, to imagine a left handed movement. When this movement imagination was detected, the letter being fixated was added to the sentence being typed, which is shown on the screen, slightly below the keyboard. A key press event blocked the BCI for the next 1s. The dimensions of the keys were under

123


1.5x1.5cm except space and delete which were 4cm wide. The distance from eye to screen was roughly 60cm. The cursor was visible and the screen also showed a horizontally moving ball providing feedback of the BCI classifier state to the subject.

Figure 1: The experimental set-up.

Results The results are shown below for a typical sentence.

real or prosthetic, into doing so. The intuitive link and qualitative experience, we hope, would be a motivating factor for the continued and successful use of such a BMI by the patients whose lives can be positively affected by it. Certainly, useful everyday arm movements involve more than just point-to-point reaching: concurrent grasping and hand orientation are also important and remain to be tested for BMI designs. Much of the benefit assessment of assistive technology will depend upon upcoming ‘realistic setting’ studies of long enough duration to provide reliable feedback from disabled users and their physicians. Although the current study limited itself to 2D target identification, it is easy to imagine how the gaze/BCI procedure can be extended to pick out 3D targets on physical objects for a physical robot to reach to. The question remains as to what 3D target accuracy stereo gaze tracking can provide vs. the 2D accuracy reported herein, which is common but is aided by a priori knowledge of distance of gaze point from the eyes. Future improvements require better online classifications of ‘rest’ vs. several ‘active’ states to improve responsiveness and perhaps control multiple motor parameters at once via BCI.

References

Figure 2: A typical sequence of key presses vs. time. On average, 68.4% of keys ‘pressed’ were intended in the sense of ‘next character in the intended sentence’. However, if the ‘delete’ key can be counted as ‘intended’, 84.2% of key presses were correctly detected. The process resulted in a typing rate of 14.2 correct chars/min (equiv. to 70.5 bits/min) for the 3 repeated sentences tested. Discussion As a demonstration of the efficacy and simplicity of combining eye tracking and EEG for BMI design, we believe that this pilot study was successful. Yet one may ask why EEG-BCI is necessary at all, and the move command or set of commands is not instead given by eye blinks, facial EMG or a voice command, if these abilities are present in the target patient set. The answer is quite simple: producing a movement by imagining it is quite different than talking one’s arm,

124

[1] J. R. Wolpaw, McFarland, D.J., Vaughan, T.M. and Schalk, G., "The Wadsworth Center Brain-Computer Interface (BCI) Research and Development Program.," IEEE Transactions on Neural Systems & Rehabilitation Engineering, vol. 11, pp. 204-207, 2003. [2] S. Lemm, B. Blankertz, G. Curio, and K.-R. Müller., "Spatio-spectral filters for improved classification of single trial EEG.," IEEE Tranactions on. Biomedical Enineering., vol. 52, pp. 1541-1548, 2005. [3] J. d. R. Millán, F. Renkens, J. Mouriño, and W. Gerstner:, "Brain-actuated interaction.," Artificial Intelligence vol. 151, pp. 241-259, 2004. [4] J. M. Carmena, M. A. Lebedev, R. E. Crist, J. E. O'Doherty, D. M. Santucci, D. F. Dimitrov, P. G. Patil, C. S. Henriquez, and M. A. Nicolelis, "Learning to control a brain-machine interface for reaching and grasping by primates," PLoS Bioogyl, vol. 1, pp. E42, 2003. [5] A. B. Schwartz, "Cortical neural prosthetics," Annu Rev Neurosci, vol. 27, pp. 487-507, 2004. [6] W. Wu, Y. Gao, E. Bienenstock, J. P. Donoghue, and M. J. Black, "Bayesian population decoding of motor cortical activity using a kalman filter," Neural Computing, vol. 18, pp. 80-118, 2006. [7] G. Dornhege, B. Blankertz, G. Curio, and K.R. Mueller, "Boosting bit rates in non-invasive EEG single trial classifications by feature combination and multi-class paradigms," IEEE Transactions on Biomedical Engineering, vol. 51, pp. 993-1002, 2004.


A novel deconvolution-reconvolution method for the measurement of closely-spaced post-synaptic potentials ∗

M.J.E. Richardson∗, G. Silberberg†, H. Markram‡and W. Gerstner∗ ´ Ecole Polytechnique Fédérale de Lausanne (EPFL), Laboratory of Computational Neuroscience, School of Computer and Communication Sciences and Brain Mind Institute, CH-1015 Lausanne, Switzerland [email protected], [email protected] † Karolinska Institute, Department of Neuroscience, Nobel Institute of Neurophysiology, Stockholm, SE-17177, Sweden [email protected] ‡ ´ Ecole Polytechnique Fédérale de Lausanne (EPFL), Laboratory of Neural Microcircuitry, Brain Mind Institute, CH-1015 Lausanne, Switzerland [email protected]

Background. Extracting synaptic amplitudes from intracellular voltage traces is a central activity of electrophysiological analysis; their measurement is required for the classification of short-term synaptic plasticity, for the statistical analysis of vesicle-release events as well as for the timing of closely-spaced synaptic events in vivo. This extraction, however, can be difficult for post-synaptic potentials (PSPs) separated by time scales of the order of the membrane time constant. In such cases the voltage amplitude cannot be directly read off the voltage trace due to the overlapping preceding pulses. It will be demonstrated that a simple deconvolution-reconvolution method can be used to great advantage in the analysis of PSP amplitudes. The method essentially reverses the filtering of the neuronal membrane to reveal the underlying synaptic drive with high temporal resolution. It is straightforward to apply and yields much information that is otherwise difficult to extract from intracellular voltage traces. It is therefore proposed as a useful addition to the tool-box of electrophysiological analysis. Motivation. The synaptic signal is mediated via a conductance change with a reversal potential specific to the synaptic type. From the soma, which is where the neuronal voltage is typically measured experimentally, this is seen as a momentary change in the equilibrium potential away from its resting value. Calling the dynamic equilibrium potential ED (t), and following standard modelling approaches [1], the voltage obeys dV = ED (t) − V (1) dt where τ governs the decay of the PSPs and, for this passive model, is identical to the membrane time constant. Though neurons with more complex response τ

properties such as those expressing the h-current are not treated in this abstract, standard modelling techniques [1, 2, 3] for neurons with voltage-gated channels can be employed to extend the method to such cases. Equation (1) states that the somatic voltage is a filtered version of the underlying synaptic drive ED (t) blurred over a time scale τ . It is this filtering that allows for the integration of synaptic input from different sources. However, in the context of experimental voltage recordings, the filtering prevents access to the fine detail of synaptic events. Deconvolution. A rather simple method can be found, however, for reversing the membrane filtering in the voltage recording thus allowing for the somaticallymeasured synaptic drive ED (t), with its high temporal resolution, and the voltage to be simultaneously known. The method is found by re-arranging equation (1) as follows dV +V (2) dt and takes the form of an algorithm that can be applied to the voltage trace after measurement: from equation (2) it is seen that to calculate the underlying drive ED (t) all that is necessary is to take the derivative of the measured voltage, multiply it by τ and then add this back onto the voltage. The only parameter is the filter time constant τ which can be found from a direct fit to the tail of the PSP. In figure panel 1A an EPSP train from a layer 5 pyramidal-to-pyramidal connection is plotted. The EPSP decay constant was measured to be τ = 40ms. Using this constant in equation (2) yields the deconvolved trace shown in panel 1B. As can be seen, the EPSPs have been resolved into well-spaced pulses with decay constants of 2ms consistent with this ED (t) = τ

125

Dynamical principles for neuroscience and intelligent biomimetic devices A

C crops

voltage V(t) 1mV

decay: τ=40ms

voltage reconvolution sum

B

deconvolution ED(t)

D

EPSP Amplitude

50ms

least-squares fit reconvolution

1 0.5 0

1 2 3 4 5 6 7 8 Pulse Number

τd=2.1ms

reconvolved EPSPs

Figure 1: The deconvolution-reconvolution method applied to a pyramidal-to-pyramidal connection. (A) The averaged (160 sweeps) post-synaptic voltage triggered by 8 pre-synaptic action potentials separated by 50ms (black). The membrane filtering of the EPSPs can be fit with a τ = 40ms decay constant. (B) This constant is used to generate the deconvolved trace using Eq (2). The inset shows the superposition of the 8 deconvolved pulses. Their decay constant is 2ms, consistent with this AMPA-mediated connection. (C) The 8 deconvolution pulses cropped 5ms before and 15ms after the pulse from the trace. (D) The lower set of green pulses are the isolated EPSPs from the reconvolution of the cropped pulses using Eq. 3. Just above is the sum of the isolated EPSPs (also green) which can be seen to be almost fully superimposed on the original voltage trajectory (black). The inset demonstrates that the EPSP amplitudes measured from the reconvolved trace are identical to that measured using more complex methods, such as a least-square fit. AMPA-mediated connection. The deconvolved trace is reminiscent of a voltage-clamp current, however, it was algorithmically derived from a current-clamp measurement and the voltage is simultaneously known. Reconvolution. To measure the EPSP amplitudes in panel 1A the decay of the the preceding EPSPs must be accounted for. However, isolated EPSPs can be generated from the deconvolved pulse by the following process. First, each deconvolved pulse is cropped out of the voltage deconvolution, see panel 1C. These can then be individually reconvolved by reversing the process in equation (2). This can be implemented via the following integral Z t ds −(t−s)/τ e EDc (s) (3) Vc (t) = 0 τ where EDc is one of the cropped traces of panel 1C. This yields a set of voltage waveforms Vc (t) in which each of the EPSPs is fully isolated from its neighbours, as seen in the lower traces of panel 1D. The amplitudes can now be measured directly off the reconvolved pulses and can be seen to be in excellent agreement with more complex methods, such as a least-square fit. Experimental details. The experimental data used in this study comes from previous in vitro experiments [4]. The synaptic connection was recorded between

126

layer-5 pyramidal neurons in a rat somatosensory cortical slice using simultaneous whole-cell patch recordings. The pre-synaptic cell was induced to produce an eight-pulse spike train, with the pulses separated by 50ms, and the post-synaptic voltage measured over many sweeps to produce an averaged response.

References [1] A.L. Hodgkin and A.F. Huxley, “A quantitative description of membrane current and its application to conductance and excitation in nerve.” J. Physiol. , Vol. 117, pp 500-544, 1952. [2] C. Koch, “Cable theory in neurons with active linearized membrane” Biol. Cybern., Vol. 50, pp 15-33, 1984 [3] M.J.E. Richardson, N. Brunel and V. Hakim, “From Subthreshold to Firing-Rate Resonance” J. Neurophysiol., Vol. 89, pp 2538-2554, 2003. [4] G. Silberberg, C.Z. Wu and H. Markram “Synaptic dynamics control the timing of neuronal excitation in the activated neocortical microcircuit” J. PhysiolLondon Vol. 556, pp 19-27, 2004


Correlations between motor cortical population signals (LFP, ECoG) improve the decoding of movement direction J. Rickert*1,2, T.Ball2,3, S. Cardoso de Oliveira2, A. Schulze-Bonhage2,3, E. Vaadia4, A. Aertsen1,2, C. Mehring*2,5 1

Neurobiology & Biophysics, Institute of Biology III, Albert-Ludwigs-University, Freiburg, Germany Bernstein Center for Computational Neuroscience Freiburg, Albert-Ludwigs-University, Freiburg, Germany 3 Epilepsy Center, University Clinics, Albert-Ludwigs-University, Hugstetter Straße 49, 79095 Freiburg, Germany 4 Department of Physiology, Hadassah Medical School, The Hebrew University of Jerusalem, Jerusalem, Israel 5 Neurobiology & Animal Physiology, Institute of Biology I, Albert-Ludwigs-University, Freiburg, Germany * These authors contributed equally mailto: [email protected]

2

Results We decoded the amplitudes of distinct frequency bands using a linear discriminant analysis. To quantify the effect of correlations on the decoding, we computed the decoding power for two distinct cases: a) using the full covariance matrix (correlations included) and b) enforcing a diagonal covariance matrix (correlations excluded). The result is depicted in Figure 1 for LFP in monkeys and Figure 2 for ECoG in humans. It shows that, despite an increased uncertainty due to more decoding parameters, excluding correlations lead to a significantly smaller

decoding power (p

Dynamical principles for neuroscience and ... - Semantic Scholar

Dynamical principles for neuroscience and ... - Semantic Scholar

Suggest Documents

Dynamical principles in biological processes - Semantic Scholar

Neuroscience Networks | Semantic Scholar

Dynamical Neuroscience XX - Google Groups

Dynamical Localization for Discrete and ... - Semantic Scholar

Cultural neuroscience - Semantic Scholar

Neuroscience nanotechnology - Semantic Scholar

Integrative Neuroscience - Semantic Scholar

Reporting Checklist for Nature Neuroscience - Semantic Scholar

Machine learning for neuroscience - Semantic Scholar

Reporting Checklist for Nature Neuroscience - Semantic Scholar

Ten challenges for decision neuroscience - Semantic Scholar

Neuroscience of foraging - Semantic Scholar

The neuroscience and psychophysiology of ... - Semantic Scholar

Social cognitive neuroscience and humanoid ... - Semantic Scholar

Educational neuroscience, constructivism, and the ... - Semantic Scholar

Challenges and Design Principles for ... - Semantic Scholar

Dynamical Systems for the Behavioral ... - Semantic Scholar

Dynamical fingerprints for probing individual ... - Semantic Scholar

Dynamical Subgrid-scale Parameterizations for ... - Semantic Scholar

Interval Predictors for Unknown Dynamical ... - Semantic Scholar

Symmetry-breaking dynamical pattern and ... - Semantic Scholar

surrogate and nonlinear dynamical modeling - Semantic Scholar

Structural and Dynamical Properties of ... - Semantic Scholar

dynamical recurrent networks - Semantic Scholar