Mar 17, 2016 - for encoding information in the nervous system â especially with respect ... In this thesis, a simple recurrent neural network model is introduced ...
Robust Encoding of Aperiodic Spatiotemporal Activity Patterns in Recurrent Neural Networks
A thesis submitted to the Graduate School of the University of Cincinnati in partial fulfillment of the requirements for the degree of
Master of Science (M.S.)
in the Department of Electrical Engineering & Computing Systems of the College of Engineering & Applied Science by
Muhammad Furqan Afzal
B.S., Lahore University of Management Sciences, 2014
March 17, 2016
Committee Chair:
Ali A. Minai, Ph.D.
Committee Members:
Carla Purdy, Ph.D. Raj Bhatnagar, Ph.D.
Abstract
Complex processes of perception, cognition and action are thought to emerge from the selforganized dynamics of different brain networks. Humans and many different animals exhibit a vast repertoire of complex movements in response to environmental stimuli. The neural basis of this voluntary motor control in particular is not very well understood. Understanding the basis of these movements is of interest to biologists, neuroscientists, engineers and computer scientists. Once modeled, they can be useful in applications such as robotics, biomorphic chips, and machine learning. Spatiotemporal patterns of neural activity have increasingly come to be seen as important for encoding information in the nervous system – especially with respect to the control of movement. For a given individual, any specific voluntary movement is ultimately encoded as an aperiodic spatiotemporal pattern of activation across a set of muscles, and presumably in spinal, and perhaps cortical, motor neurons. Different individuals can be seen as having different stereotypical patterns of activity which determine the “style” of specific movements – e.g., how they walk or write. Experimental studies also indicate that these activity patterns may themselves be constructed as linear combinations of a few fixed spatiotemporal basis patterns of activity called motor synergies. For this to work, it is essential that neural systems be able to represent spatiotemporal activity patterns that are stimulus-specific, aperiodic (i.e., not rhythmic), transient (i.e., lasting only briefly), and robust (i.e., at least somewhat tolerant of errors and noise). In this thesis, a simple recurrent neural network model is introduced motivated by the need to understand the basis of voluntary motor control, arising as a result of synergies. Distinct attractors representing spatiotemporal patterns of activity are embedded in the system having efficient storage and recall. Since the model is motivated by a need to understand motor ii
control, the patterns are also visualized by turning them into observable movement trajectories in space. Different attractors elicit distinct trajectories. The attractors can also be triggered in combination to give new and complex spatiotemporal patterns, resulting in novel trajectories representing movements. The system is shown to exhibit many of the properties needed for the flexible construction of complex, aperiodic movements.
iii
iv
Acknowledgement I would like to express my deepest gratitude to my advisor Professor Ali Minai who supported me throughout the course of this research work. His involvement, guidance and encouragement has played a major role in the completion of this work. He has also helped me set up better goals for my future career related to the field of neuroscience. I would also like to thank Dr. Carla Purdy and Dr. Raj Bhatnagar for agreeing to be a part of my thesis defense committee. Lastly, I want to express sincere thanks to my parents and siblings, without whose support this would not have been possible at all.
v
Contents
List of Figures
viii
1 Introduction
1
1.1
Information Processing in the Central Nervous System . . . . . . . . . . . . . . .
1
1.2
Sensorimotor Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
1.3
Motivation and Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3
1.4
The Scaffolded Attractors Model . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
1.5
Organization of Remaining Chapters . . . . . . . . . . . . . . . . . . . . . . . . .
5
2 Background and Motivation 2.1
Organization of the Motor Cortex 2.1.1
7 . . . . . . . . . . . . . . . . . . . . . . . . . .
8
Integrative Maps of the Motor Repertoire . . . . . . . . . . . . . . . . . . 10
2.2
Rhythmic Movements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3
The Degrees of Freedom Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4
Motor Synergies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.5
Minimum Intervention Principle
2.6
Neural Basis of Muscle Synergies . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.7
Canonical Action Repertoire . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.8
Using Neural Population Code for Movement Directions . . . . . . . . . . . . . . 20
2.9
Models of Writing and Drawing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
. . . . . . . . . . . . . . . . . . . . . . . . . . . 14
vi
2.10 Sequence Learning Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.11 Generation of Aperiodic Spatiotemporal Activity Patterns . . . . . . . . . . . . . 27 2.12 Computational Models of Metastable Neural Dynamics . . . . . . . . . . . . . . . 27
3 Description of the Model
31
3.1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2
Overall Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3
Model Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 3.3.1
Model Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3.2
Parameter Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3.3
Recurrent Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
3.3.4
Classes of Lateral Connectivity . . . . . . . . . . . . . . . . . . . . . . . . 37
3.3.5
Interpretation of the Stored Attractors . . . . . . . . . . . . . . . . . . . . 38
3.3.6
Modeling of Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.3.6.1
Input Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.3.6.2
Additive Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.4
Model Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3.5
Ideal Spatiotemporal Output . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3.6
Reliability Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 3.6.1
3.7
Recall Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Visualization through Motor Response . . . . . . . . . . . . . . . . . . . . . . . . 44
4 Simulations and Results
45
4.1
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.2
Intrinsic Neuronal Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2.1
α Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
4.2.2
a Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
vii
4.3
4.4
4.2.3
b Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.2.4
γ Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Network Behavior 4.3.1
Appropriate Spatial Pattern of Activity . . . . . . . . . . . . . . . . . . . 49
4.3.2
Efficiency of Recall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 4.3.2.1
Recall Quality against Input Error . . . . . . . . . . . . . . . . . 54
4.3.2.2
Recall Quality against Additive Noise . . . . . . . . . . . . . . . 57
4.3.2.3
Overall Trend . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Visualization through Doodles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.4.1
Robustness of Motor Activity to Input Error . . . . . . . . . . . . . . . . 64
4.4.2
Robustness of Motor Activity to Additive Noise . . . . . . . . . . . . . . . 65
4.4.3
Effects of Parameter Modulation . . . . . . . . . . . . . . . . . . . . . . . 65
4.4.4 4.5
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.4.3.1
α Modulation
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.4.3.2
h Modulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.4.3.3
Neuron-Specific Modulation . . . . . . . . . . . . . . . . . . . . . 67
Synergistic Combination of Movement Primitives . . . . . . . . . . . . . . 67
More Spatiotemporal Attractors and Doodles . . . . . . . . . . . . . . . . . . . . 68
5 Conclusions and Future Work
95
5.1
Research Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
5.2
Future Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
viii
List of Figures 1.1
(a) Different functional brain networks; (b) Different structural brain networks (used with permission of the publisher ). Source: [1]
2.1
2
Map of muscle activation due to electrical stimulation in the brain of a monkey (used with permission of the publisher ). Source: [2]
2.2
. . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . .
8
(a) Architecture of the human cerebral cortex, (b) Somatotopic organization of primary motor cortex in humans (used with permission of the publisher ). Source: [3] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3
A) Synchronous muscle synergies found in cats. B) The synergy activation coefficients representing the activity level of synergies. C) EMG curves (used with permission of the publisher ). Source: [4] . . . . . . . . . . . . . . . . . . . . . . . 14
2.4
Synchronous synergies in frogs recorded during jumping, walking and swimming activities (used with permission of the publisher ). Source: [5] . . . . . . . . . . . 15
2.5
Images showing time-varying synergies in frogs recorded during jumping, walking and swimming activities (used with permission of the publisher ). Source: [5]
2.6
. . 15
Time series showing time-varying synergies in frogs recorded during jumping, walking and swimming activities (used with permission of the publisher ). Source: [6] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.7
Recorded muscle data projected onto task-relevant and task-irrelevant subspaces (used with permission of the publisher ). Source: [7]
ix
. . . . . . . . . . . . . . . . 17
2.8
Synergy Model Architecture (used with permission of the publisher ). Source: [8]
2.9
Population coding in saccadic eye movements. A: The number of spikes vs eye
18
movement duration. B: Instantaneous frequency. C: Different phases of nystagmus. D: Frequency vs duration of movements (used with permission of the publisher ). Source: [9] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.10 Cortical circuit model for writing (used with permission of the publisher ). Source: [10]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.11 AVITEWRITE model architecture (used with permission of the publisher ). Source: [11]
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.12 Architecture of the model consisting of different subsystems: encoding, response and modulation (used with permission of the publisher ). Source: [12]
. . . . . . 26
2.13 Activity waves in a rat brain slice. (a) Schematic of a cortical slice. (b) Recording of the propagating waves. (c) Image plot of the activity (used with permission of the publisher ). Source: [13] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.14 The spatiotemporal patterns generated by the neural network in response to two different stimuli (used with permission of the publisher ). Source: [14]
4.1
. . . . . . 29
Single neuron dynamics with α modulation: red, blue and green curves represent u, v and z respectively. (a) α = 1; (b) 1.25; (c) 1.5; (d) 1.75; (e) 2; (f) 2.25; (g) 2.5; (h) 2.75; (i) 3
4.2
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Single neuron dynamics with a modulation: red, blue and green curves represent u, v and z respectively. (a) a = 4; (b) 3.6; (c) 3.2; (d) 2.8; (e) 2.4; (f) 2; (g) 1.6; (h) 1.2; (i) 0.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
4.3
Single neuron dynamics with b modulation: red, blue and green curves represent u, v and z respectively. (a) b = 0.5; (b) 0.6; (c) 0.7; (d) 0.8; (e) 0.9; (f) 1; (g) 1.1; (h) 1.2; (i) 1.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
x
4.4
Single neuron dynamics with γ modulation: red, blue and green curves represent u, v and z respectively. (a) γ = 0.66; (b) 1.33; (c) 2; (d) 2.66; (e) 3.33; (f) 4; (g) 4.66; (h) 5.33; (i) 6
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.5
Attractors 1 through 5 in the absence of noise or error. . . . . . . . . . . . . . . . 50
4.6
ρ in connectivity-gain space (φ = 0). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08 52
4.7
ρ in connectivity-gain space (φ = 0.6). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08 53
4.8
ζ in connectivity-gain space (φ = 0). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08 53
4.9
ζ in connectivity-gain space (φ = 0.6). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08 54
4.10 ρ and ζ shown together in connectivity-gain space, if both ρ and ζ ≥ 0.9 at a certain level in the space, that region is shown white, else black (φ = 0). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08
. . . . . . . . . . . . . . . . . . . . . . 55
4.11 ρ and ζ shown together in connectivity-gain space, if both ρ and ζ ≥ 0.9 at a certain level in the space, that region is shown white, else black (φ = 0.6). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08
. . . . . . . . . . . . . . . . . . . . . . 56
4.12 (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 50 % connectivity in all cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.13 (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 60 % connectivity in all cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
xi
4.14 (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 70 % connectivity in all cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.15 (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 80 % connectivity in all cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.16 (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 90 % connectivity in all cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.17 (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 100 % connectivity in all cases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.18 Doodles generated by the 5 attractors in the absence of input error or noise. . . . 64 4.19 Change in the doodle for Attractor 1 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6 . . . . . . . . . . . . . . . . . . . . 65 4.20 Change in the doodle for Attractor 2 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6 . . . . . . . . . . . . . . . . . . . . 66
xii
4.21 Change in the doodle for Attractor 3 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6 . . . . . . . . . . . . . . . . . . . . 67 4.22 Change in the doodle for Attractor 4 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6 . . . . . . . . . . . . . . . . . . . . 68 4.23 Change in the doodle for Attractor 5 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6 . . . . . . . . . . . . . . . . . . . . 69 4.24 Additive noise, η, variation for Attractor 1, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.25 Additive noise, η, variation for Attractor 2, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.26 Additive noise, η, variation for Attractor 3, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 4.27 Additive noise, η, variation for Attractor 4, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.28 Additive noise, η, variation for Attractor 5, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.29 Modulation of the doodle for Attractor 1 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5 . . . . . . . . . . . . . . 72 4.30 Modulation of the doodle for Attractor 2 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5 . . . . . . . . . . . . . . 72 4.31 Modulation of the doodle for Attractor 3 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5 . . . . . . . . . . . . . . 73 4.32 Modulation of the doodle for Attractor 4 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5 . . . . . . . . . . . . . . 73 4.33 Modulation of the doodle for Attractor 5 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5 . . . . . . . . . . . . . . 74
xiii
4.34 Modulation of the doodle for Attractor 1 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8 . . . . . . . . . . . . . . . . . . . . . . . 74 4.35 Modulation of the doodle for Attractor 2 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8 . . . . . . . . . . . . . . . . . . . . . . . 75 4.36 Modulation of the doodle for Attractor 3 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8 . . . . . . . . . . . . . . . . . . . . . . . 75 4.37 Modulation of the doodle for Attractor 4 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8 . . . . . . . . . . . . . . . . . . . . . . . 76 4.38 Modulation of the doodle for Attractor 5 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8 . . . . . . . . . . . . . . . . . . . . . . . 76 4.39 Interpolation between doodles by the combined activation of Attractors 2 and 4, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.40 Interpolation between doodles by the combined activation of Attractors 1 and 5, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.41 Interpolation between doodles by the combined activation of Attractors 3 and 2, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.42 Interpolation between doodles by the combined activation of Attractors 4 and 1, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.43 Interpolation between doodles by the combined activation of Attractors 5 and 3, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.44 Sample 1: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
xiv
4.45 Sample 2: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 4.46 Sample 3: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 4.47 Sample 4: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4.48 Sample 5: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 4.49 Sample 6: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 4.50 Sample 7: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.51 Sample 8: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 4.52 Sample 9: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 4.53 Sample 10: All attractors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 4.54 Doodles generated by the 5 attractors in sample 1. . . . . . . . . . . . . . . . . . 90 4.55 Doodles generated by the 5 attractors in sample 2. . . . . . . . . . . . . . . . . . 90 4.56 Doodles generated by the 5 attractors in sample 3. . . . . . . . . . . . . . . . . . 91 4.57 Doodles generated by the 5 attractors in sample 4. . . . . . . . . . . . . . . . . . 91 4.58 Doodles generated by the 5 attractors in sample 5. . . . . . . . . . . . . . . . . . 92 4.59 Doodles generated by the 5 attractors in sample 6. . . . . . . . . . . . . . . . . . 92 4.60 Doodles generated by the 5 attractors in sample 7. . . . . . . . . . . . . . . . . . 93 4.61 Doodles generated by the 5 attractors in sample 8. . . . . . . . . . . . . . . . . . 93 4.62 Doodles generated by the 5 attractors in sample 9. . . . . . . . . . . . . . . . . . 94 4.63 Doodles generated by the 5 attractors in sample 10. . . . . . . . . . . . . . . . . 94
xv
Chapter 1
Introduction 1.1
Information Processing in the Central Nervous System
The brain consists of large number of structurally and functionally connected networks interacting together to perform cognitive and motor functions (Figure 1.1). This complex system continually processes information related to events in its environment and generates motor commands to produce useful response behaviors. Most information in the nervous system is thought to be encoded as spatiotemporal patterns of activity. The dynamics of these activity patterns is the basis of all processes of perception, cognition and behavior. As such, it is of great interest to understand how these patterns arise and evolve within the nervous system, and, in particular, how specific neural network structures process activity patterns with attributes necessary for information processing. The activity patterns representing perceptual, cognitive, and behavioral states in the nervous system span a broad range of representational mechanisms, from feature detectors in the sensory cortices and grid cells in the entorhinal cortex to population codes in the motor system. However, it is also clear that all representations must be fundamentally dynamical in nature. Attractor neural networks have been used widely as models of memory in the nervous system. Memories can be thought of as different attractor states that can be recalled robustly through relaxation
1
Figure 1.1: (a) Different functional brain networks; (b) Different structural brain networks (used with permission of the publisher ). Source: [1] dynamics [15, 16, 17, 18]. More complex, time-varying information can be represented through activity sequences [19, 20, 21, 22, 12], neural fields [23, 24], and metastable attractor dynamics [14, 25, 26, 27, 28]. Neurons are known to encode information at multiple levels from single spikes and bursts to modulated patterns of spiking, and theories of large-scale brain function are based increasingly on the assumption of dynamic spatiotemporal activity patterns rather than static feature codes or fixed-point attractors. Novel motor and cognitive patterns can be represented by dynamic codes over recurrent associative networks based on spatiotemporally correlated neural activity with appropriate modulation [29, 30, 31, 32, 33, 34, 35].
1.2
Sensorimotor Integration
Large-scale integration of different specialized regions of the brain is thought to be made possible by synchronized firing of different neuronal groups across many different regions. Complex
2
cognitive and motor behaviors emerge as a consequence of such synchronized activity. Both structural and functional connectivities of different brain structures determine characteristic cognitive and motor patterns in an individual. Spatiotemporal processing is thought to emerge as a result of the interaction between the incoming stimuli and the natural intrinsic dynamics of different brain networks. Several mathematical and computational models for spatiotemporal pattern formation in brain neuronal populations have been proposed, and some of these will be discussed in the next chapter. It is not fully understood how different populations of neurons collect sensory information and then transform it into appropriate motor commands to be executed by muscles and joints. Such sensorimotor integration is critical for voluntary, goal-directed movements in response to stimuli. Population coding appears to be a likely basis for both sensorimotor integration [36, 37] and the representation of motor commands in the cortex [38, 39]. Recent studies suggest that when the brain receives sensory stimuli, it selects from a repertoire of already available action plans, which are modulated in real-time by sensory information. These action plans then compete against each other through different populations of neurons to ultimately single out a certain appropriate action plan. Several models for mapping this motor repertoire onto different areas of the brain have been proposed [40, 41, 42, 43].
1.3
Motivation and Goals
The motivating issue for the research described in this thesis is to understand the neural basis of voluntary motor control. As discussed earlier, populations of neurons give rise to particular spatiotemporal patterns of activity which encode for different movements in an individual. In many cases, the requirement is to produce relatively short (200-300 ms) patterns of activity across a neural population such that specific neural groups within the population become active and inactive at particular phases of the overall pattern. To be useful, such patterns must meet some basic criteria: 3
1. It should be possible to associate the patterns with specific stimuli, and for these stimuli to reliably elicit the correct patterns. 2. Sufficiently similar stimuli – e.g., noisy versions of a nominal stimulus – should elicit the same response pattern. 3. Significantly different stimuli should elicit significantly different response patterns. 4. The recall of the patterns should be somewhat robust against noise in the system.
It may also be required that the recalled patterns should depend not only on the stimulus but also on prior states of the system [34], though this need not always be the case. Ultimately, the neural system is required to learn a specific set of robust and repeatable transients, which poses an obvious dilemma: How can a transient be made robust? In this thesis, a very simple model based on a modification of the standard attractor network models used for associative memory has been developed [16, 17, 18]. This model uses an approach termed scaffolded attractors, where transient spatiotemporal patterns are stabilized by using metastable fixed point attractors as scaffolding. To make the model concrete, it is specifically related to the problem of modeling voluntary motor control tasks such as writing or drawing.
1.4
The Scaffolded Attractors Model
The scaffolded attractors model uses a continuous time recurrent neural network to encode spatiotemporal activity patterns robustly. The network consists of a layer of two types of primary neurons, termed slow and fast neurons, to achieved the desired dynamics. Distinct spatiotemporal patterns are stored in the network, which are elicited robustly in the presence of noise. Different neurons turn on and off at different time points and stay on for different durations as well, giving the desired spatiotemporal dynamics. Since the model is motivated by the desire to understand motor control, the spatiotemporal patterns generated are visualized
4
in the form of trajectories in 2-dimensional space, signifying movement in different directions. Distinct patterns result in distinct motor mappings in the form of doodles, which represent the visible output of the system in each trial. This motor response can be modulated through the use of different parameters in the model signifying onset delays, latencies and deactivation of individual neurons. The related simulations and results are provided in detail. The simulations consider the following issues:
1. The effect of noise on the storage and recall of patterns and the resulting movements. 2. The effect of neural connectivity on the stability and robustness of recall. 3. The effect of the strength of recurrent signals in the network on the robustness of recall. 4. The use of parameter modulation to systematically vary the stored activity patterns.
Simulations show that the model has robust recall even under reasonably high amounts of noise. Different metrics are used to evaluate this robustness at two different levels. First, each activity pattern is required to involve activity only over a specific subset of neurons even in the presence of noise or variation in system parameters. Second, at a nominal parameter setting and a specific pattern, the neurons that are designated for activity must turn on and off at specific times in the presence of noise. The model is significant in that it provides a spatiotemporal attractor framework which can be used to generate and recall activity patterns similar in nature to those found in the central nervous system. The model also provides insight into neural mechanisms for flexible motor control through the modulation of such activity patterns.
1.5
Organization of Remaining Chapters
The rest of the thesis is organized as follows. Chapter 2 offers detailed background related to the experimental, modeling and other research work on the neural basis of voluntary motor control. 5
Chapter 3 provides a detailed description of the scaffolded attractors model. Chapter 4 describes all the simulations performed with the model along with their results and the discussion of these results. Conclusions, significance and the future prospects of the current work are discussed in Chapter 5.
6
Chapter 2
Background and Motivation As is well-known, all movement in vertebrates requires the activation of groups of muscles in specific temporal patterns. This muscle activity is triggered by input from motor neurons in the spinal cord, which are, in turn, driven by afferents from the motor cortex and somatosensory feedback through muscle spindles. This implies the existence of specific spatiotemporal activity patterns in the spinal cord motor pools and in the motor cortex. Most studies on motor control in humans and other animals have focused on rhythmic movements [44, 45, 46]. However, it is much more challenging to understand how non-rhythmic voluntary movements are controlled. Such movements require aperiodic and precisely timed activation of motor neuron populations, i.e., complex spatiotemporal patterns of activity. The aim of the scaffolded attractors model is to demonstrate how such activity patterns can be encoded in the natural dynamics of recurrent neural networks. This chapter provides an overview of the relevant aspects of the motor system, and of previous work on modeling spatiotemporal activity patterns in the nervous system.
7
2.1
Organization of the Motor Cortex
Several studies have investigated the complex topographical organization of the motor cortex of primates. These have led to the view that the motor repertoire is mapped onto two dimensional substrate of the motor cortex in a systematic topographical organization. Several types of organizations of the motor cortex have been proposed, some of which are briefly described below: The first organization suggested a single map of the body. One such mapping was obtained by electrically stimulating the surfaces of monkey brains and observing the type of muscle activity patterns generated by Beevor et al. in 1890. Figure 2.1 shows such a map of the monkey brain [2].
Figure 2.1: Map of muscle activation due to electrical stimulation in the brain of a monkey (used with permission of the publisher ). Source: [2]
Other studies suggested a division of the motor cortex into a primary motor and a premotor area. The primary motor area had clear cut separations between the representations of the body parts and the premotor area had overlapping representations of the body. It was found that injuries to any one of these areas did not hinder movement but injuries to both did. This suggested that the two areas operated with somewhat parallel functionality [43]. Penfield and Welch [3] suggested that the human motor cortex was instead divided into
8
two regions perpendicular to each other, the lateral motor cortex (M1) and a supplementary motor area, SMA (M2). They suggested that these regions are not discretely organized, and both contain overlapping representations of the body parts. The SMA is more sensitive, and stimulation of sites in the SMA can lead to movements of the entire body [43]. Further studies indicated that the motor cortex is divided into a primary motor cortex and many premotor areas. Some of these areas for primates are listed below:
1. PMDr, dorsal premotor cortex, rostral division 2. PMDc, dorsal premotor cortex, caudal division 3. PMVr, ventral premotor cortex, rostral division 4. PMVc, ventral premotor cortex, caudal division 5. SMA, supplementary motor area 6. Pre-SMA, presupplementary motor area
Studies suggest that these areas do not have definitive borders between them, but are involved in certain specialized tasks. For example, (PMVr) is thought to be involved in hand movements and interpretation of hand gestures of other individuals. PMD is thought to be involved in preparation of movement. SMA is suggested to be involved in the internal rehearsals of the movements to be made and action sequence control. Accurate functions of these divisions are still not clearly understood [43]. It was thought initially that, at the level of individual neurons, the representations of body parts might be completely separated. Further research suggested otherwise. It was found that the muscles of fingers, wrist, arm and hands had overlapping representations in the motor cortex. Other body parts in different organisms were also found to have such overlapping representations. This hinted at integrated control of different body parts by the motor cortex. There is evidence for this type of control in many different organisms now [43]. Figure 2.2 shows 9
the architecture and somatotopic organization of human cerebral and primary motor cortex, respectively.
Figure 2.2: (a) Architecture of the human cerebral cortex, (b) Somatotopic organization of primary motor cortex in humans (used with permission of the publisher ). Source: [3]
2.1.1
Integrative Maps of the Motor Repertoire
One of the most comprehensive models of motor organization in the motor cortex and nearby regions has been developed through the work of Graziano and colleagues [47, 48, 43, 49, 50]. This model suggests that topographic maps of a canonical action repertoire exist in multiple regions. Graziano and Aflalo [48, 43] have developed a computational model to generate an artificial motor cortex array taking into account local continuity among areas. This mapping has been found to be very close to the organization of the actual motor cortex and exhibits many of the somatotopic arrangement features such as the maps of limbs found in monkeys. The cortex is thought to maximize the similarity between neighboring areas resulting in continuity. This also describes the division of cortex into discrete areas responsible for distinct yet overlapping functions. Different explanations have been put forward for the proximity of similar areas. One suggests that neurons close to each other share more synaptic connections and wire together according to Hebbian learning, resulting in similar areas. Others propose that evolution helped organize similar areas close to each other because they have to communicate more frequently for 10
generation of cognitive and motor command signals. For more complex stimuli, the mapping is not that simple and is similar to the competing mapping configurations problem described before [48, 43]. The research by Graziano and colleagues is especially relevant to this thesis because it revealed the neural basis of aperiodic movements such as bringing a hand to the mouth or reaching over to the back. This is discussed in a later section below.
2.2
Rhythmic Movements
Much of the work on models of motor control has focused on rhythmic movements such as swimming, walking, finger tapping, etc. The main insight from these studies is that rhythmic movements are driven by activity in central pattern generators (CPGs) – neural circuits that produce periodic activity dynamics as periodic attractors under conditions of simple stimulus [51, 52]. These attractors represent the coordination modes of the system, and enable CPGs to operate robustly in variable and noisy conditions. Transitions between different types of rhythmic movements, e,g., walking and running, reflect transitions in the attractors of the underlying CPGs [45, 46, 52]. CPG networks are very modular and are further composed of simpler modules to assist in fine rhythmic movements by different body parts. CPG networks are also thought to be modulated by the sensory information to react to changes in environmental conditions [44]. The locomotion CPGs for vertebrates are mostly found in the spinal cord, but neural networks with preferred patterns of activity, or attractors, are thought to exist throughout the nervous system, and underlie not only motor control but also memory, decision-making, and other cognitive functions [53]. Typically, these systems encode a limited number of attractors, each with a specific pattern of activity. These attractors can be elicited by relatively simple or noisy stimuli, thus making it possible for the system to generate appropriate activity patterns for representation and/or control simply by “triggering” them through selection, rather than 11
having to construct them in real time. This approach is especially useful in motor control, where complex but precise activity patterns are required continually. In vertebrates, the basal ganglia system is thought to play a crucial role in linking cognitive and perceptual states to behavior by selectively disinhibiting specific activity patterns in the motor cortex [40, 54].
2.3
The Degrees of Freedom Problem
Primates exhibit various complex movements in response to changing environmental stimuli. To make a desired movement, there are infinitely many paths that the specific body parts involved can take. For example, if an individual wants to bring one of his/her hands to the mouth, there are infinitely many paths to attain this configuration. Similarly, various velocities of the parts involved can achieve it. This is sometimes referred to as the degrees of freedom problem [55]. It makes the primate motor system very complex. It is interesting to note that despite having so many degrees available, the motor system makes desired movements appropriately in reasonable time. This means that the system solves a complex DoF problem and chooses a particular path based on the changing environmental stimuli very promptly and accurately. It has been proposed that the degrees of freedom problem is addressed by exploiting the redundancy of the system as a source of robustness, so that the system chooses any of the infinite trajectories in the underconstrained control space opportunistically. A variant of this is to consider optimal feedback control as the basis of the complex tasks accomplished by humans. Observations indicate that desired movements are made reliably without repeating them in all their finer details. Feedback control only corrects those deviations that can significantly hinder in achieving a particular goal. This leads to final attainment of goal, however using varying configuration paths [56]. The system also divides complex motor tasks into simpler modular ones – a strategy that extends to extended to other types of cognitive processes as well [57]. Biological motor control is especially important to study because, in spite of technological advances, it can still outperform artificial robot controllers under conditions of uncertainty and 12
noise [58].
2.4
Motor Synergies
Most voluntary movements involve aperiodic and transient muscle activity. Thus, the challenge in explaining the underlying control signals is to model how a neural substrate can reliably and robustly generate aperiodic, transient patterns of spatiotemporal activity in response to specific stimuli. As discussed above, the degrees of freedom of the musculoskeletal architecture make motor control a very complex problem. Motor synergies (or muscle synergies) are preconfigured sets of canonical patterns of activity across specific muscle groups that are thought to be used as basis functions for the construction of more complex movements. Motor synergies have been found in frogs, cats, humans and other animals [59, 5, 60, 6, 4]. The modularization of vertebrate spinal cord resulting in these muscle synergies helps simplify the control problem significantly. A set of three synergies has been found to explain several movements in frogs [5, 60, 6]. Experimental data from cats indicates that only 4 muscle synergies accounted for almost 95 % of the postural activity patterns exhibited by these animals [4] as shown in Figure 2.3. Muceli et al. carried out some experiments in which they recorded from the muscles of the upper limb of healthy men. They showed that almost all of the movements made by the limb could be produced by linear combinations of as low as four synergies extracted from individual and multiple joints [61]. It is also suggested that there are two types of muscle synergies:
1. Synchronous synergies: Once these synergies are activated, all the corresponding muscles operate together without any delay. Figure 2.4 shows synchronous synergies found in frog muscles while jumping, swimming and walking. 2. Time-varying synergies: These synergies have both spatial and temporal components due 13
Figure 2.3: A) Synchronous muscle synergies found in cats. B) The synergy activation coefficients representing the activity level of synergies. C) EMG curves (used with permission of the publisher ). Source: [4] to the imbalance of activation across muscles and activation delays. Figure 2.5 and 2.6 show time-varying synergies found during the same experiment mentioned before [5].
2.5
Minimum Intervention Principle
Though synergy-based control is the currently dominant theory, it has been criticized [62]. A motor control approach known as the uncontrolled manifold hypothesis or the minimum intervention principle has been proposed as an alternative. One study investigated the variability in motor patterns, and concluded that the central nervous system allows large deviations in the irrelevant dimensions of the particular task being performed and only corrects the relevant portions under variable conditions. This is also closely related to the optimal feedback control.
14
Figure 2.4: Synchronous synergies in frogs recorded during jumping, walking and swimming activities (used with permission of the publisher ). Source: [5]
Figure 2.5: Images showing time-varying synergies in frogs recorded during jumping, walking and swimming activities (used with permission of the publisher ). Source: [5] This approach implies that the central nervous system does not solve a full degrees of freedom problem, but rather deals only with those degrees which are relevant to a particular task being executed [63]. Kutch et al. studied endpoint force fluctuations during force generation tasks. They showed that muscles are recruited flexibly in ways relevant to the action required, rather than same fixed groups of muscles as suggested by the muscle synergy hypothesis [64]. Another study asked participants to generate fingertip force maintaining particular finger postures. Their selective muscle recordings allowed them to account for most of the forces using EMG signals. The analysis showed that the variance in the task relevant domains was significantly smaller than variance in the irrelevant ones, suggesting that the CNS only tries to correct task-relevant variability. Additionally, variance in both the domains was found to be non-negligible, making a case against the idea that the CNS only uses a small fixed group of
15
Figure 2.6: Time series showing time-varying synergies in frogs recorded during jumping, walking and swimming activities (used with permission of the publisher ). Source: [6] muscle synergies to generate the activity patterns [7]. Figure 2.7 shows the task-relevant and task-irrelevant domains after independent component analysis (ICA). It is clear from experimental evidence that both synergies and the minimum intervention principle represent mechanisms for efficient motor control. Further research in these areas can provide more unified and accurate descriptions of the workings of the motor system.
2.6
Neural Basis of Muscle Synergies
While most of the studies cited above have been carried out at the muscle and cortical levels, modeling the underlying neural dynamics has received less attention. For this reason, the neural basis of motor synergies remains poorly understood, though some models have been proposed. Byadarhaly et al. came up with a modular neural model of motor synergies. They propose that functional modularity in the form of muscle synergies reflects structural modularity in the underlying neural substrate. In this model, synergies are configured in the system using
16
Figure 2.7: Recorded muscle data projected onto task-relevant and task-irrelevant subspaces (used with permission of the publisher ). Source: [7] different pools of neurons. A 2-joint, 6-muscle artificial arm is triggered through a synergy group network (SGN). This network is maintained in an inhibited state modeling the inhibition of basal ganglia output layers under resting conditions. A selector system disinhibits particular synergies through higher cortical inputs. Thus, different configurations in the selector network lead to different synergies and different end movements of the joints and muscles [8, 65]. Figure 2.8 shows architecture of their synergy model. It is reasonable to hypothesize that motor synergies correspond to spatiotemporal patterns of activity across motor neurons in the spinal cord and the motor cortex, and that such patterns represent the “natural dynamics” of the neural networks involved. Indeed, it has been proposed that a wide variety of mental functions could be seen as arising from the interaction of such “neural synergies” at multiple hierarchical levels. One such model argues that modularity at the level of synergies and their hierarchical interaction is fundamental to cognition [66]. Yuste et al. proposed that the spinal cord central pattern generator (CPG) circuits of vertebrates are
17
Figure 2.8: Synergy Model Architecture (used with permission of the publisher ). Source: [8] very similar in function to the neocortical circuits. Like CPGs, these circuits are influenced by the sensory information but are also capable of generating output autonomously. They propose that research related to CPGs can give clues on how the relevant cortical circuitry works which would be useful to obtain a unified explanation of the motor system functionality [53]. This can also result in a better neural model of motor synergies. The idea that cognitive and motor repertoires arise from the interaction of neural synergies at different hierarchical levels resonates with the view of the brain-body system as a complex adaptive system, where complex patterns of perception, cognition and action emerge from the emergent coordination of modular canonical patterns [29]. Experimental data on the cortical representations of sequential actions [67] and complex canonical actions [50] also suggests the existence of a synergy hierarchy. Recent studies have suggested that the prefrontal cortex (PFC) is involved in all aspects of the cognitive processing necessary to generate relevant motor behaviors. It is suggested to be
18
the location where all the information gathered by neural circuits, either from previous memory or changing environmental conditions, is integrated to engender meaningful behaviors. The functions in which the prefrontal cortex is involved include the following:
1. Experimental studies with humans and monkeys suggest that prefrontal cortex plays a critical role in associative learning. In this type of learning, the organism maps changing environmental cues to different responses, or adapts already learned stimulus-response pairs to novel situations. Indeed, it is known through experiments that the removal of ventral PFC from monkeys impairs the learning of visuomotor conditional tasks to a large extent [68, 69]. 2. The PFC is thought to be important in reward-based behavioral control. Recent imaging studies suggest that it is active in response to both the reward and penalty levels in their respective contexts [70]. 3. The PFC is known to be critical for behavioral selection or decision. It helps select a particular item from memory after the integration of all the information for relevant behavior execution. Experiments have revealed that the dorsolateral PFC is active during a motor task requiring decisions about precise movement timings [71]. 4. It is also suggested that the PFC guides behavior in response to changing conditions perceived by the animal. A recent report revealed that the activity of PFC neuronal groups led to cross-temporal association of auditory information and motor behaviors which were selective to colors [72, 67].
2.7
Canonical Action Repertoire
In work that is especially relevant to this thesis, Graziano and colleagues [48, 43, 73, 50] showed that stimulating localized regions of the motor cortex and several surrounding areas in macaques
19
resulted in the production of stereotypical complex and non-repeating movements such as bringing a hand to the mouth or raising a hand above the head. In particular, when a site in the precentral gyrus was stimulated, the monkey closed the aperture in its hand resembling a gripping posture, flexed its wrist and rotated its shoulder to ensure smooth movement of the hand to the mouth. When its head was allowed to rotate freely, it moved the head as well to align the mouth and the hand. All these movements were made in coordination as with voluntary movements. Similarly, stimulation of other sites led to other movements such as defense, manipulation, climbing and leaping. They suggest that the monkey motor cortex encodes a repertoire of complex muscle activations corresponding to distinct movements. Since these movements obviously require spatiotemporal patterns of activation in muscles, spinal cord motor neurons and, presumably, motor cortex neurons, it is reasonable to see the process as one where a particular sustained stimulus elicits a transient spatiotemporal activity pattern in a neural population in a repeatable way. This is exactly what the model proposed in this thesis does.
2.8
Using Neural Population Code for Movement Directions
As indicated earlier, the spatiotemporal activity patterns encoded in the scaffolded attractor model are made concrete by having them represent drawings, or doodles, in two-dimensional space. This is done through the simplified form of a strategy used in the brain for encoding movement attributes, i.e., population coding. As is well-known, the population coding is used in the motor cortex and other cortical regions for representing the direction of movement. Georgopoulos et al. (1982) recorded the activities of single cells from monkey motor cortex while they made movements in different directions, starting from the same points each time. They observed a clear order in the cell discharge frequency as a function of the direction of movement, with the activity of each cell tuned strongly to a specific direction with a bell-shaped tuning curve. Different cells had different preferred directions, with the cell population representing a repertoire of overlapping tuning 20
curves. Experiments showed that the vector combination of preferred directions weighted by the activity of the cells could be read as a code for the direction of the animal’s hand movements. This type of code is called a population code [38, 39]. Some studies have also proposed such neural population coding in other organs. One group investigated the pontine reticular formation (PRF) in alert monkeys, a brain area involved in eye movement coordination. They analyzed firing frequencies of different neuronal units in relation to the eye movements using vector descriptions, and found that the number of spikes and overall frequency encoded the eye position changes and eye movement directions, respectively. They suggested that such vector analysis or population coding could be the basis of saccadic eye movements through the PRF [9]. Figure 2.9 shows the relationship between neuronal spikes and saccades.
Figure 2.9: Population coding in saccadic eye movements. A: The number of spikes vs eye movement duration. B: Instantaneous frequency. C: Different phases of nystagmus. D: Frequency vs duration of movements (used with permission of the publisher ). Source: [9]
Schor et al. proposed a similar result in decerebrate cat responses to head tilt from vestibular neurons. They found maximal neuronal discharge frequencies in certain preferred directions of the tilt. They encoded responses of all cells in the form of vectors which predicted the magnitude of the tilt accurately [74] . Another group investigated neck reflexes in decerebrate
21
cats by studying the responses of spinal interneurons. They found similar results and suggested that the spatiotemporal behavior of neurons in the combined neck and vestibular stimulation is guided by a linear summation of the individual responses [75]. Recent studies with humans have also suggested the use of population coding. Cowper-Smith and colleagues used functional magnetic resonance imaging adaptation (fMRI-A) to investigate blood-oxygen-level dependent (BOLD) signals varying in time in the lateral motor cortex (M1) and surrounding areas in response to writing movements by healthy individuals. They showed that the level of the BOLD signal reduced when wrist movements were made in the same direction continuously, and vice versa. This provides a physiological basis of population coding for preferred directions in humans [76]. All these studies suggest that neural population coding for specific movement directions is of general significance and can help explain how directional information is uniquely encoded in the central nervous system.
2.9
Models of Writing and Drawing
Since the work in this thesis uses population-coded motor activity to visualize the systems output in terms of drawing on a 2-dimensional surface, an overview of neurocomputational models for writing and drawing is given. One such model [10] investigates the role of different cortical networks in relation to voluntary arm movements under varying conditions. The neural model provides explanations of roles of different primate motor cortical areas involved in volitional control. It exhibits properties similar to neural groups in at least two areas of the cortex. The properties include response profiles, activation delays, and kinematic and kinetic sensitivities during movement. Different populations of neurons in different cortical areas perform different important functions such as accumulating feedback from muscles, computation of desired movement trajectory and proprioception, modeled in the form of neural circuitry. All these characteristics play an important 22
role while writing or drawing, ensuring reliable and smooth creation of novel pattern/characters or the ones already memorized. Figure 2.10 shows the model just described.
Figure 2.10: Cortical circuit model for writing (used with permission of the publisher ). Source: [10]
Grossberg and Paine [11] have modeled the cortico-cerebellar interactions during learning of handwriting movements. They suggest that most of the sensorimotor integration develops when an individual imitates a motor task, such as drawing or handwriting. These motor tasks are divided across synergies or muscle groups, which overlap to generate smooth continuous movements. They model the parietal and motor cortical mechanisms that interact with cerebellar predictive learning strategies. For movement in a particular direction, the gaze is aligned with the direction and the motor cortical neurons help make the movement. Cerebellar cells learn the changing directional profile which is sent to motor cortex and muscles later. If somehow the direction of movement gets deviated during imitation, compensatory eye movements are made in the correct direction and visual information about the discrepancy is sent back to the muscles. This act is repeated until a reasonably correct imitation is done. The cerebellar learning output can be sent to the motor cortex, and eventually muscles, at a variable rate 23
controlling the speeds with which the hand movements are made. They also provide different relationships between speed, curvature profiles and sizes of the letters/characters drawn during their simulations. Figure 2.11 shows the conceptual diagram of their model.
Figure 2.11: AVITEWRITE model architecture (used with permission of the publisher ). Source: [11]
Although the studies mentioned here provide more detailed models of handwriting, the main focus of the neural network model presented in this thesis is just generation of aperiodic and transient spatiotemporal patterns of activity which can be combined synergistically to give different drawings in 2-dimensional space. The model can also be extended in the future by adding a detailed writing model to the existing one to learn and create more complex drawings. The view of motor control underlying the present model is also related to the issue of “style”. Almost every movement in an individual is characterized by distinctive – and often clearly recognizable – details that represent that individual’s style, e.g., a handwriting, a way of walking, specific hand gestures, etc. This is easily explained within the framework of synergies: The motor synergies used by the individual form the basis of their style, and differences between
24
individuals reflect differences in their motor synergy repertoire. Since synergies represent longlasting, reliably repeatable neural and muscle activity patterns, an individual’s style is both consistent and relatively fixed.
2.10
Sequence Learning Models
While motivated in part by the motor system, the model presented in this thesis addresses a more abstract function: How to reliably generate stimulus-dependent, robust and transient spatiotemporal activity patterns in a neural population. This issue too has received some attention, though most of it has focused on the generation of activity sequences, where a neural network goes through a succession of states with previous states generating subsequent ones. Sequence learning in neural networks is a vast topic [77, 19, 78], but a few examples are discussed below. One study [19], motivated by theories of memory consolidation, models the relationship between the speed of learning and capacity of brain for storing patterns through a two-stage system. One stage corresponds to small chunks of rapid temporary learning of sequential relationships between spatial activity patterns, and the other one to bigger chunks of slow and long-term storage of sequences. The model demonstrates how chunks of memory sequences could get transferred from temporary to the long-term storage. Another approach [20] has modeled the fast learning and storage of temporal sequences. In this model, sequence recall is triggered through the appearance of items similar to it. The speed of recall can also be controlled through a subsystem in the model. This model has also been applied to motor learning for the drawing of shapes. Huerta and Rabinovich [21] have shown how to produce robust sequences from random neural populations. In contrast to the traditional view, their model is not a learning model but is able to generate robust activity sequences if random neural populations are in proximity of an excitatory-inhibitory synaptic balance. 25
It is known that all motor repertoire are first learned at cognitive levels in the form of certain parameters, in regions of frontal cortex, independent of how the end effectors or muscles act. These learned sequences are then transferred to muscles through networks in the spinal cord for the actual movement. A study by Byadarhaly et al. [22] used population coding to learn complex sequences represented as doodle drawings through a neural network model. Despite the similarity and complexity of many different sequences, they were able to show that population coding can be the basis of efficient sequence learning. Vasa et al. [12] proposed a spiking neural network model for learning and recall of sequential patterns. They proposed that considering only the spatial representations in neuronal networks can also be fruitful, leading to sequential decoding of the existing cognitive parameters. Their model consisted of three different subsystems (Figure 2.12) which interacted together for encoding and recall of spatial sequence elements through selective triggering of networks of neurons used as fixed point attractors.
Figure 2.12: Architecture of the model consisting of different subsystems: encoding, response and modulation (used with permission of the publisher ). Source: [12]
26
2.11
Generation of Aperiodic Spatiotemporal Activity Patterns
The model presented in the current thesis is concerned with generating more integrated spatiotemporal patterns, where different neurons (or neuron groups) become active at different phases of the overall pattern, and remain active only for a certain period before subsiding. The same neuron or neural group can be activated multiple times within the pattern, and the activity of different neurons can overlap, making it more difficult to see this as a sequence learning model. A recently popular approach to understanding spatiotemporal activity patterns in neural systems is that of dynamic neural fields. These fields describe the spatiotemporal evolution of activity across populations of neurons. Basic cognitive activities such as memory, attention, decision-making and learning, as well as the higher functions of cognitive control, can be seen as emerging from the neuronal dynamics in these fields modeled by nonlinear differential equations [79]. Amari investigated the nature of dynamic patterns generated using layers of homogeneous neural fields. For a single layer with an excitatory population, the stimulus, once induced, persists as a neural dynamic pattern, even after the subsidence of stimulus. For two layers with both excitatory and inhibitory populations, oscillatory and traveling wave patterns were observed as well [23]. Some suggest that these neural fields describe different biological neural networks and their spatial dynamics with possible population coding [24]. The traveling waves generated due to neural fields are also observed in rat brain slices as shown in Figure 2.13.
2.12
Computational Models of Metastable Neural Dynamics
The model studied in this thesis focuses on producing aperiodic transients in a neural population, and shares many features with previous models developed for both the production and decoding of spatiotemporal activity patterns in the nervous system.
27
Figure 2.13: Activity waves in a rat brain slice. (a) Schematic of a cortical slice. (b) Recording of the propagating waves. (c) Image plot of the activity (used with permission of the publisher ). Source: [13] Rabinovich et al. [14] used a winnerless competition network of Fitzhugh-Nagumo neurons to convert input patterns into dynamic spatiotemporal activity patterns, inspired by olfactory processing in insects. These patterns can be thought of as attractors in the state space of the network [14]. Figure 2.14 shows some of the patterns they generated. Another group investigated spatiotemporal activity patterns generated by dissociated cortical cells. It is known that intact brain cells produce such patterns of activity, but such an exhibition by dissociated cells could be very interesting in that it can provide clues about the universality of spatiotemporal dynamic patterns in emergent self-organizing neural networks. This is exactly what the group found. They prepared cultured networks using rat brain slices and observed these dynamic patterns of activity on extracellular recordings, which persisted for considerable amount of time [80]. Buonomano and Maass [34] proposed that the spatiotemporal patterns seen during cortical processing emerge from the interaction between incoming stimuli and the intrinsic dynamics
28
Figure 2.14: The spatiotemporal patterns generated by the neural network in response to two different stimuli (used with permission of the publisher ). Source: [14] of neural networks. The internal dynamics are determined both by the neuronal activity and changes in synaptic and other cellular properties, which they term as hidden states. They suggest that the complex internal properties of these neurons project the spatiotemporal patterns into high-dimensional representations. These representations are then read-out by neuronal populations by modulating weights between themselves. They also highlight some issues that need to be investigated, such as how read out neurons modulate their weights, the processes by which reproducible spatiotemporal patterns are formed, and the roles of long-term plasticity. They also give an interesting suggestion that these patterns of activity do not encode for the present stimulus only, but each stimulus is represented in context of the previous stimuli, so that past and present sensorimotor information is integrated. Experimental studies can further reveal details related to this idea . Liu and Buonomano [35] proposed a recurrent neural network model which could embed 29
certain spatiotemporal neural trajectories in the network. Different inputs elicit different output patterns in the network. They also determined a learning rule for generation of such network dynamics, which they termed as presynaptic-dependent scaling (PSD). All the models just described tend to be quite complex – especially in their use of spiking neurons and real-time synaptic modulation. While both these features are biologically important and plausible, the goal in this thesis is to develop a simpler model that illustrates a principle, and is potentially suitable for use in computational applications and hardware implementations. To fully capture the phenomenology of spatiotemporal dynamics in the nervous system, this model will require significant extensions, which can be addressed in future work.
30
Chapter 3
Description of the Model 3.1
Introduction
As described earlier in Chapter 1, the focus of the work presented here is to understand how spatiotemporal activity patterns can be encoded in neural networks, with special emphasis on the role of such representations in voluntary motor control. Experiments suggest that canonical spatiotemporal activity patterns – or synergies – in the spinal cord motor pools and the motor cortex activate different groups of muscles, leading to different movement patterns. Complex movements can arise as a result of combination of many canonical patterns. The work described in this thesis uses a scaffolded attractor model approach to explore this process. Any model for useful representation of activity patterns in the neural substrate must meet a minimal set of requirements, which are as follows: • Stimulus-Dependence: Any information processing system must be able to recognize significant differences in its stimuli and generate recognizably different responses to these. In the present system, this means that recognizably different input stimuli should produce distinct spatiotemporal activity patterns – and thus different movements. • Associativity: It should be possible to associate specific stimuli with specific response patterns using neurally plausible learning mechanisms. 31
• Robust Recall: In the nervous system, signals are always corrupted by noise, and the system must be able to recognize somewhat variant versions of a known stimulus and respond identically in these cases, i.e., produce the same – or very similar – activity patterns. • Robustness to Noise: Neural information processing usually occurs in a noisy environment, where the noise affects not only the stimulus but all signals generated during the process. A good system must be able to withstand a reasonable amount of noise, and maintain a high level of functionality in its presence. In the present system, this means that the recall of the correct spatiotemporal activity pattern must occur even when the system dynamics is noisy.
The system described in this thesis meets all these requirements, as the simulation results presented in the next chapter show.
3.2
Overall Approach
Satisfying the requirements listed above poses a special challenge for a system seeking to encode spatiotemporal dynamics. In particular, spatiotemporal patterns of interest are typically aperiodic and of limited duration, which means that simple oscillator dynamics is insufficient to encode them. From a system dynamics viewpoint, they can be seen as embodying transient-like activity. At the same time, they are required to be robust against noise, and to be elicited reliably by noisy stimuli in a stimulus-specific fashion. The scaffolded attractor model is motivated by the need to meet these challenges. It is now well-established that neural systems – both in the brain and in computational models – often achieve their robustness through attractor dynamics embodied in a recurrent network architecture. Recurrently connected networks of nonlinear neurons with symmetric connection weights are guaranteed to converge to one of a small number of equilibrium states 32
of activity – or fixed points – from all possible initial states [16, 17, 18]. As such, they are believed to underlie the representation of associative memories and spatial representations in the hippocampal and cortical regions [15, 81, 82, 83]. This stabilizing effect of recurrent network dynamics is used in the present model to meet the needs of robust recall, since any errors in the stimulus can be removed through attractor dynamics. Desired patterns of activity are embedded into the system using a variant of Hebbian learning, and recalled in response to specific stimuli. However, since the goal is to stabilize time-varying and aperiodic patterns rather than fixed points, simple attractor dynamics is not sufficient. Instead, the proposed model uses a more complex dynamics based on two classes of neurons, each with its own function and slightly different dynamics. The classes are:
• Fast neurons, that rapidly identify the spatiotemporal patterns to be produced based on the stimulus and set up a “scaffolding” for it. • Slow neurons, that are gradually activated by the fast neurons and other slow neurons, and instantiate the relevant spatiotemporal activity pattern.
Essentially, the model can be seen as a mixture of two sub-networks. The network of fast neurons – termed the fast network – receives the external stimulus and functions like a classic attractor network. Even though the input may be noisy, this network cleans it up and recovers a much better representation of it very rapidly. This then acts as a scaffolding for the the network of slow neurons – termed the slow network – to produce a spatiotemporal activity pattern associated with that specific stimulus. As the slow neurons begin to do so, an inhibitory signal from them to the fast network shuts off activity in the latter, allowing the spatiotemporal activity pattern to develop based purely on the interaction among the slow neurons. The temporal variation in the pattern is produced by the fact that slow neurons are self-deactivating, i.e., they can only remain active for a limited time based on specific parameters representing the attributes of the various ion channels that control activation and deactivation 33
in real neurons. As each slow neuron’s activity rises and falls, it influences the increase or decrease of activity in other neurons, thus creating a characteristic response. Very importantly, this response can be varied systematically by modulation of the neurons’ parameters, which provides a substrate for both real-time adjustment and long-term learning – though this issue is not explored in this thesis. Besides shaping the activity pattern, the use of attractor dynamics in the slow network also serves two other purposes. First, it ensures that only the neurons that are supposed to participate in the pattern indicated by the stimulus are activated. Second, even though the slow network does not converge to a fixed point of activity, the interactions among neurons damp the effect of additive noise, thus making the recalled pattern more stable. After a period, all the slow neurons become inactive automatically and the pattern ends. The system then remains inactive until the next stimulus arrives. It is important to note that all the processes described above arise naturally from the dynamics of the network as represented by the systems equations, and not through ad-hoc external interventions. The equations for the model as well as its dynamics are described below in greater detail.
3.3
Model Description
As described above, the network comprises fast and slow neurons. The sets of fast and slow neurons are denoted by FN and SN, respectively. There are nf fast neurons and ns slow neurons in the network, making up a total of n = nf + ns neurons in the system.
3.3.1
Model Equations
The equations for the activities of the neurons include terms and parameters for neuronal activation and deactivation, onset delays, synaptic connection strengths and the modulation of activity. The equations governing the activity of a fast neuron i are given by: 34
X X dvi 1 wij uj + f wik uk − αzi ) + Ii = (−qvi + e dt τi j∈F N
(3.1)
k∈SN
dzi −zi + = dt ai
P
j∈SN
uj
bi
(3.2)
The equations for slow neuron i are:
X X dvi 1 wij uj + h wik uk − αzi ) = (−qvi + g dt τi
(3.3)
dzi −zi ui = + dt ai bi
(3.4)
j∈F N
k∈SN
The output of neuron i from either class is calculated as:
ui = f (vi ) =
1 1+
e−β(vi −µ)
(3.5)
In these equations, vi represents the activity of neuron i, ui is a nonlinear transformation of this activity representing the output of neuron i, zi is a deactivation variable that controls how rapidly a neuron can be activated and how long it can remain active, τi is the time constant for the activity dynamics of a particular neuron and q is a gain parameter for vi . External input, Ii , is provided only to the fast neurons. Though the duration of this input can vary in principle, the simulations in the current work involve providing the input only at the initiation of a trial. Thus, it is used to set the initial state of the fast neurons, after which the network activity evolves autonomously. The network’s output is taken to be the output vector of the slow neurons, i.e., the fast neurons do not contribute directly to the system’s output. The parameters ai and bi are charging and discharging time constants for zi , respectively, e is a gain parameter controlling the influence of recurrent inputs from fast neurons on fast neuron
35
i, f is a gain parameter controlling the influence of recurrent inputs from slow neurons on fast neuron i, g is a gain parameter controlling the influence of recurrent inputs from fast neurons on slow neuron i, and h is a gain parameter controlling the influence of recurrent inputs from slow neurons on slow neuron i. The parameter α is a gain for the deactivation of all the neurons after they have fired for a certain amount of time. It should be noted that the deactivation variable zi for a slow neuron i depends only on its own activity, but zi for a fast neuron i depends also on the total activation of the slow network. As a result, once the SN becomes sufficiently active, the fast neurons turn off, i.e., the scaffolding is removed.
3.3.2
Parameter Values
In the current model, q, e, f and g are set to 1 by default, and h is set to 2.5, unless stated otherwise. The simulations reported in the next chapter all use networks with n = 200 neurons, of which nf = 50 are fast and ns = 150 are slow neurons. The τ value for all fast neurons is set to 1.5, whereas slow neurons have random τ values in the interval [5,15]. The values β and µ in the nonlinear output function f ( ) are set to 40 and 0.2, respectively. The a and b parameters for the fast neurons are set to random values in the interval (0,0.5]. The a and b parameters for the slow neurons are set to random values in the intervals (0,5] and (0,0.5], respectively. Finally, α is set to 1.2 for slow neurons and 1.5 for fast neurons.
3.3.3
Recurrent Connectivity
The recurrent weights between the neurons described before, are assumed to be symmetric (wij = wji ), and are calculated using a covariance Hebbian learning rule [84, 85] to embed NA attractor patterns in the network. Each attractor pattern y k = [y1k y2k ... ynk ] specifies an ‘active’ (1) or ‘inactive’ (0) state for all the fast and slow neurons, and is thus a 200-bit binary vector. The rule for embedding the attractors is:
36
wij = h(yik − hyi i)(yjk − hyj i)ik
(3.6)
where h.i represents a sample mean and k indexes the attractors. The learning rule results in the required excitatory and inhibitory recurrent connections between neurons, to regulate their overall activity.
3.3.4
Classes of Lateral Connectivity
As can be seen from the neuron equations, there are four classes of lateral connections depending on whether a particular neuron is a fast or a slow one. These connections are f ast − f ast, f ast − slow, slow − f ast and slow − slow. They perform different functions as listed:
• Fast-fast: These connections help the fast neurons to rapidly correct any errors in the initial stimulus and determine the spatiotemporal attractor to be generated by the slow neurons. • Fast-slow : These connections elicit the correct spatiotemporal pattern in the slow neurons based on the activity pattern of the fast neurons. • Slow-fast: These connections help in activating or deactivating the fast neurons according to the cumulative slow neuronal activity (Eq. (3.2)). • Slow-slow : These connections actually shape the exact spatiotemporal pattern produced by the slow neurons due to a particular stimulus, encoding for a particular canonical movement.
All the connections have separate gains. In the simulations that follow, all gains except h (slow −slow) were set to 1. The gain h is set to 2.5 for particular representative simulations and is also varied during other simulations to observe its effect in different cases, as the slow − slow
37
connectivity determines the actual spatiotemporal pattern to be produced by the slow neurons. This pattern, in turn, can encode for various movements.
3.3.5
Interpretation of the Stored Attractors
As mentioned above, the attractors embedded in the network are binary patterns. In standard attractor networks [16, 17, 18], the goal of recall is to stabilize network at a point where all the ‘1’ neurons in the target attractor are active and all the ‘0’ neurons are inactive. This is not true in the scaffolded attractor model. The goal here is for the fast neurons to recall their segment of the pattern rapidly, elicit the slow neuron pattern, and shut down. The activity of the slow neurons is then required to evolve over time with periods of activation and deactivation. However, the nominal activity pattern prescribed by the slow neuron component of the target attractor plays a crucial role: All significant activity during the unfolding of the spatiotemporal slow neuron activity pattern must be confined to those slow neurons that were designated to be active in the nominal attractor, and the remaining neurons must stay inactive. Thus, the nominal attractor indicates which slow neurons should participate in the pattern, but not their temporal pattern of activity. Two factors strongly influence whether the right neurons participate in the spatiotemporal activity pattern generated by the network: The recurrent connectivity between the slow neurons, and the recurrent gain h for the SN. The latter simply controls how much recurrent excitation each slow neuron gets, and if it becomes very high, even neurons that should not be on can receive enough excitation to turn on. The recurrent connectivity’s role is to anchor the pattern as an attractor, so that even if the fast neurons have not specified the attractor precisely, the slow neurons can generate the right pattern. In the simulations reported in Chapter 4, the SN connectivity and gain are varied systematically to identify the connectivity-gain combinations that produce appropriate activity.
38
3.3.6
Modeling of Noise
Noise and uncertainty is an unavoidable attribute of any neural system, and can appear in various ways. Robustness against these factors is critical for the system’s effective operation. In this research, robustness is considered relative to two factors, and discussed below.
3.3.6.1
Input Error
The Hebbian learning approach used to embed attractors in the network ensures that if the initial activity in the fast network corresponds to the activity pattern of one of the embedded attractors, it will remain there. However, in real situations, this is not likely to be the situation, and the initial stimulus to the fast neurons can only elicit an approximate, noisy version of the activity pattern for the desired attractor. The simulations reported in this thesis consider this by initializing each trial of the model with input patterns generated by corrupting the stored attractors by a specific amount of error. This is parameterized by a positive value φ – termed the input error – and is produced by setting the jth component of the input as:
Ij = yjk − φcj (2yjk − 1)
(3.7)
where k denotes the index of the attractor and cj is a uniform random number between 0 and 1. Thus, 1-bits in the attractor are decremented by random amounts and 0-bits are randomly incremented. The input is applied to the fast neurons by setting their initial states to the corresponding input values. The slow neurons are all initialized to an activity level of 0.
3.3.6.2
Additive Noise
In addition to the input error, the outputs of slow neurons are also subject to additive uniform noise at every time step, so that the actual output of neuron j becomes u ˆj (t) = uj (t)(1 + δj (t)), where δj is a uniform random number between −η and η. Thus, η parameterizes the level of additive noise in the slow network. The reason for including the additive noise only in the slow 39
network is that, in fact, the fast neurons are relatively impervious to it due to their dynamics.
3.4
Model Dynamics
External inputs stimulate only fast neurons, which act as a scaffolding to set up the attractors, and the system output is carried only by the axons of the slow neurons. The main difference between fast and slow neurons is in their response time. Fast neurons have a lower value of τ and respond faster, reaching their equilibrium values quickly in response to an external stimulus. The slow neurons depend on input from the fast neurons, and other slow neurons for their activity. They have larger and more varied values of τ , which means that they are slower to become activated even if their nominal state in the attractor is active. Both fast and slow neurons have their own activation dynamics. Fast neurons also receive inhibition signal from the slow neurons (Eq. 3.2). As the overall activity of all the slow neurons increases, the deactivation for fast neurons increases and vice versa. When a stimulus is presented briefly to the fast neurons, they respond rapidly, recovering their portion of one of the stored attractors by correcting any errors in the stimulus. This begins to activate slow neurons based on two factors: Whether a particular slow neuron is designated to be active in the current attractor, and its activation time constant. Thus, the activity of the fast neurons acts as a scaffolding to activate the appropriate slow neurons. Once a sufficient number of slow neurons are activated, the inhibition from slow to fast neurons turns off the latter (Eq. 3.2), thus removing the scaffolding and allowing the slow neuron activity to develop on its own, until eventually all the slow neurons get deactivated. At this point, fast neurons can turn on again and the next cycle starts. The time constants of the deactivation variable z and α control the duration of activity for each slow neuron during any such cycle, generating a unique and repeatable spatiotemporal activity pattern.
40
3.5
Ideal Spatiotemporal Output
N distinct binary-valued attractors, y k = [y1k y2k ... ynk ], yjk ∈ {0, 1}, k = 1, ..., N , are embedded in the system. Each of these binary patterns can be divided into the pattern for the fast neurons and that for the slow neurons, referred to as the fast neuron component (FNC) and slow neuron component (SNC), respectively. However, these binary patterns only define which neurons are to be active or inactive in each attractor. The spatiotemporal attractors arise from the interaction between this specification and the natural dynamics of the network. To obtain these, the network is stimulated with the FNC for each attractor and the resulting spatiotemporal pattern across the slow neurons is observed. This pattern, termed the ideal spatiotemporal output (ISO) for each attractor is regarded as the desired output pattern for the corresponding attractors, and is used in assessing the quality of recall under non-ideal conditions. It is denoted as: xk (t) = [xk1 (t) xk2 (t) ... xkns (t)], xkj (t) ∈ [0, 1], k = 1, ..., N . It should be noted that: • The ISO pattern is defined only over the slow neurons. • It is a time-dependent pattern, unlike the corresponding nominal attractor specification.
However, which slow neurons become active at any phase in an attractor is determined by whether that neuron had a value of 1 in the corresponding binary attractor. The ISO patterns for the attractors used for the simulations are shown in the next chapter. As described earlier, the fast neurons turn on quickly for a short duration and then shut off temporarily as soon as a slow neuronal pattern becomes sufficiently developed due to the deactivation described in Eq. (3.2,3.4).
3.6
Reliability Metrics
The system is evaluated for performance at two levels:
1. Neuron Participation Accuracy: The degree to which all those slow neurons des41
ignated for activity in the target attractor actually become active and the rest remain inactive. 2. Pattern Recall Accuracy: The degree to which the recalled slow neuron spatiotemporal activity pattern corresponds to the one obtained under conditions of no input error and no noise.
Different metrics are used to judge the reliability of the system for each case. For neuron participation accuracy, the system is evaluated using two metrics from the classification domain: Sensitivity and specificity. Sensitivity Sensitivity measures the degree to which all the slow neurons that needed to be active in the nominal attractor actually did become active at some point during the spatiotemporal pattern recalled under the no error, no noise situation. It is denoted by ρ:
ρ=
true positives true positives + f alse negatives
(3.8)
where true positives are the neurons which are active in both the nominal attractor and the ISO, and false negatives are those which are active in the nominal attractor but not in the ISO. Specificity Specificity measures the degree to which neurons designated for inactivity in the nominal attractor remain inactive in the recalled spatiotemporal pattern under no error, no noise conditions. It is denoted by ζ:
ζ=
true negatives true negatives + f alse positives
(3.9)
where true negatives are the neurons which are inactive in both the actual attractor and the ISO, and false positives refer to those which are inactive in the nominal attractor but not
42
in the ISO. The next chapter presents extensive simulations with these metrics. The quality of recall for the spatiotemporal pattern under conditions of input error and additive noise is evaluated as follows.
3.6.1
Recall Quality
The parameters for input error and additive noise – φ and η, respectively – are varied systematically to measure the robustness of the system. The system’s performance is quantified by mean attractor similarity, M , which is defined as a measure of the similarity of the recalled activity pattern to the attractor in the presence of different types of noise. It is given by:
P i hui it ri M= P ; i ∈ SN i hui it
(3.10)
where ri gives the Pearson correlation coefficient between ui (t) and u ˆi (t), where u ˆi denotes the output of neuron i in the noisy case and ui is its output in the noise-free case. Here, t ranges over the entire 1200-step time interval from the point of stimulus application to the end of the cycle. Recall quality is measured only over the slow neurons, since they represent the attractor’s output activity pattern. It ranges between -1 and 1, with 1 representing perfect recall including the exact temporal phases, and -1 indicates recall of the complementary pattern. Using a simple Pearson correlation coefficient between corresponding neurons poses two issues:
• It only takes into account the phases of activity and not the actual magnitude, so that different magnitudes of activity at the same temporal phases will give high correlation. This is good in some sense, as it implies that the same spatiotemporal pattern is produced but the overall activity levels of the neurons are different. • The attractors embedded in the network are sparse. So, the neurons that remain almost inactive in both noise-free and noisy cases contribute towards a high correlation value. 43
To avoid getting unnaturally high correlation values due to sparse networks, the correlations are weighted by the mean activities of the neurons in the ideal cases (noise-free), as can be seen in Eq. (3.10). The next chapter also discusses recall stability simulations in detail.
3.7
Visualization through Motor Response
As mentioned earlier, the model is motivated by the motor system. The response patterns generated are visualized by turning them into observable movement trajectories in 2-dimensional space. Specifically, the activity of the slow neurons is mapped onto the 2-dimensional X-Y plane by considering each neuron, i, as coding movement in a fixed preferred direction, θi , which is chosen randomly with a uniform distribution between 0 and 2π. The instantaneous direction of movement coded by a specific pattern of slow neuron activity is given by:
X(t) =
X
ui (t)cosθi
(3.11)
ui (t)sinθi
(3.12)
i∈SN
Y (t) =
X i∈SN
The generated trajectory is seen as a “doodle” produced as a result of the spatiotemporal activity in the network, and is plotted in the X-Y -plane. Different attractors in the network generate distinct doodles. Looking at how the doodles change under different conditions provides the most concrete information about the behavior of the system. The next chapter presents results from extensive simulations addressing all the issues raised above, and some others.
44
Chapter 4
Simulations and Results 4.1
Overview
The model described in Chapter 3 was implemented and studied through extensive simulations with regard to the following specific issues: 1. The effect of parameter and stimulus variation on the response of individual neurons. 2. The ability of the network to encode attractors where only designated neurons participate in activity, and the role of slow neuron connectivity and recurrent gain on this aspect of system function. 3. The effect of input error and additive noise on the quality of pattern recall and the drawing response. 4. The effect of parameter modulation on the drawing response. The results of the simulations for each of these cases are reported below, and discussed.
4.2
Intrinsic Neuronal Behavior
A spatiotemporal activity pattern over a population of neurons ultimately involves controlling the phase and duration of activity for the individual neurons. Thus, before considering the 45
behavior of the network, it is useful to consider how the dynamics of the slow neurons can be modulated by various parameters. To do this, the original equation for neuronal activity (v) is modified in the following way:
dvi 1 = (−qvi + γI − αzi ) dt τi
(4.1)
The recurrent input terms in Eq. (3.1,3.3) are replaced by a simple square wave input γI, with γ controlling the magnitude of excitation to the neuron. The equations for u and z remain the same as before. The simulations are run for 1000 time steps to demonstrate the effect of varying α, a, b and γ on the dynamics of u, v and z when the neuron is stimulated with a square wave pulse of unit height and duration from time step 100 to 400.
4.2.1
α Modulation
Figure 4.1 shows that increase in α increases the effect of deactivation and the neuron remains active for less time.
Level
2
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
1 0
Level
2 1 0
Level
2 1 0 0
500
Time
1000
0
500
Time
1000
0
500
1000
Time
Figure 4.1: Single neuron dynamics with α modulation: red, blue and green curves represent u, v and z respectively. (a) α = 1; (b) 1.25; (c) 1.5; (d) 1.75; (e) 2; (f) 2.25; (g) 2.5; (h) 2.75; (i) 3
46
4.2.2
a Modulation
Smaller values of a inhibit the activity of z, decreasing the deactivation effect on v and resulting in the neuron remaining active for a longer time (Figure 4.2).
Level
2
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
1 0
Level
2 1 0
Level
2 1 0 0
500
1000
0
500
Time
1000
0
500
Time
1000
Time
Figure 4.2: Single neuron dynamics with a modulation: red, blue and green curves represent u, v and z respectively. (a) a = 4; (b) 3.6; (c) 3.2; (d) 2.8; (e) 2.4; (f) 2; (g) 1.6; (h) 1.2; (i) 0.8
4.2.3
b Modulation
Larger values of b inhibit the activity of z, decreasing the deactivation effect on v and resulting in the neuron remaining active for a longer time (Figure 4.3).
Level
2
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
1 0
Level
2 1 0
Level
2 1 0 0
500
Time
1000
0
500
Time
1000
0
500
1000
Time
Figure 4.3: Single neuron dynamics with b modulation: red, blue and green curves represent u, v and z respectively. (a) b = 0.5; (b) 0.6; (c) 0.7; (d) 0.8; (e) 0.9; (f) 1; (g) 1.1; (h) 1.2; (i) 1.3
47
4.2.4
γ Modulation
With rise in γ, the neuron experiences greater excitation. Thus, it turns on quickly and remains
Level
active for a longer time (Figure 4.4).
3 2 1
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
Level
0
3 2 1
Level
0
3 2 1 0 0
500
1000
Time
0
500
Time
1000
0
500
1000
Time
Figure 4.4: Single neuron dynamics with γ modulation: red, blue and green curves represent u, v and z respectively. (a) γ = 0.66; (b) 1.33; (c) 2; (d) 2.66; (e) 3.33; (f) 4; (g) 4.66; (h) 5.33; (i) 6
The simulations above show that, in the presence of a simple input, the activation phase and duty cycle of a single neuron can be modulated through several parameters. In a full network, some of these parameters would be modulated by the state of the system or external control inputs, thus creating a flexible and context-dependent mechanism for controlling the phase and duration of activity for each neuron. Since the purpose of the current network is only to demonstrate the basic capabilities of the model, it is kept simple by using fixed (and randomly set) values for parameters.
4.3
Network Behavior
This section reports results on the behavior of the network as a whole. To do this, five nominal attractors are specified for the network with 50 fast and 150 slow neurons. The network is stimulated with the fast component of each nominal attractor to obtain the corresponding ISO patterns in the slow network. The ISO patterns for the five attractors embedded in the network 48
are shown in Figure 4.5. The patterns are produced over 1200 time steps. It can be seen that all the patterns are distinct, which should lead to distinct motor patterns in the form of doodles (shown later). Different slow neurons turn on and off at different times in the attractors. Deactivation causes almost all slow neurons to become inactive well before the total duration. This is desired because the goal is to produce aperiodic activity patterns of finite duration. Subsequent stimuli can result in more firing if needed, creating chains of activity.
4.3.1
Appropriate Spatial Pattern of Activity
A key part of encoding and recalling specific spatiotemporal patterns of neural activity is to ensure that the spatial pattern, i.e., which slow neurons participate in the activity at all, is as specified by the nominal attractor. As described in Chapter 3, this is done by checking the sensitivity and specificity of the activated neurons with respect to the nominal attractor activity patterns. The simulations presented below consider the effect of two factors on these metrics: The recurrent connectivity of the slow neurons, and the recurrent gain in the slow neuron network. The motivating hypothesis here is that enabling the correct slow neurons for activity in the absence of direct input from the external stimulus or the fast neurons requires sufficient recurrent stimulation within the slow network. One way to look at this is to think of the fast neurons as having set up a sufficient “pattern of potential activity” before switching off, such that the subsequent activity in the slow neurons – though temporally varying – remains confined within this potential pattern due to the latent influence of attractor dynamics [86, 83, 87, 88, 89]. The key parameters underlying this attractor dynamics are the degree and strength of the recurrent connectivity in the slow neuron system. The simulations in this section show what levels of these parameters are necessary to ensure appropriate spatial patterns of activity, and to determine whether attractor dynamics plays a significant role in this. The criterion for an appropriate spatial pattern of activity is chosen to be one where the sensitivity, ρ, and specificity, ζ, are both 0.9 or higher. Figures 4.6 and 4.7 show mean sensitivity
49
Attractor 1 1 0.9
20
0.8
Neuron
40
0.7
60
0.6 0.5
80
0.4
100
0.3
120
0.2 0.1
140
0
Attractor 2 1 0.9
20
0.8
Neuron
40
0.7
60
0.6 0.5
80
0.4
100
0.3
120
0.2 0.1
140
0
Attractor 3 1 0.9
20
0.8
Neuron
40
0.7
60
0.6 0.5
80
0.4
100
0.3
120
0.2 0.1
140
0
Attractor 4 1 0.9
20
0.8
Neuron
40
0.7
60
0.6 0.5
80
0.4
100
0.3
120
0.2 0.1
140
0
Attractor 5 1 0.9
20
0.8
Neuron
40
0.7
60
0.6 0.5
80
0.4
100
0.3
120
0.2 0.1
140
0
200
400
600
800
1000
1200
Time Step
Figure 4.5: Attractors 1 through 5 in the absence of noise or error. 50
values for all the attractors plotted in the connectivity-recurrent gain space for different noise levels, for two different input error levels. It can be seen that high sensitivity values are only observed for connectivities of 50% and greater, and high gain values. Also, input error reduces sensitivity significantly for all the connectivities. Similarly, Figures 4.8 and 4.9 show mean specificity values for all the attractors plotted in the connectivity-recurrent gain space for different noise levels, for two different input error levels. It can be seen that specificity remains very high at all connectivities despite the high input error. The high specificity even in cases of low (or even no) recurrent connectivity in the slow network indicates that the exclusion of inappropriate slow neurons from activity is not dependent on attractor dynamics in the slow network, but is mainly the residual effect of the stimulus from fast neurons before they turn off. However the fact that sensitivity is high only for high levels of connectivity and recurrent gain indicates that inclusion of all appropriate slow neurons in the activity pattern does depend strongly on attractor dynamics. Figures 4.10 and 4.11 show regions in connectivity-recurrent gain space where mean sensitivity and specificity values meet the designated criterion of 0.9 or greater under different noise levels. There are two sets of these images for low and high input error levels (0 and 0.6). The figure for low error level (0) suggests that only connectivities greater than 50% and gains greater than around 3 have high ρ and ζ values as desired. It can also be seen that at zero error level, noise variation results in almost the same high performance regions throughout. This is because the system does not produce many false positives and negatives at such a low error level. The figure for higher error level (0.6) suggests that at very high error levels, the optimal performance region is much smaller, and only very high connectivities and gains give accurate results. It can also be seen that, at such a high error level, noise variation makes a significant difference and high performance regions shrink further with the increase in additive noise. These
51
(a)
(b) 1
1
100
100 0.9
90
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.8
80
Connectivity (%)
Connectivity (%)
80
0.9
90
0.8
0.2
5
0.1
0
0.1
0 0
0.1
1
2.5
3
5
7
0
12
0.1
1
2.5
Gain (h) (c)
3
5
7
12
Gain (h) (d) 1
1
100
100 0.9
90
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.1
0
0.8
80
Connectivity (%)
Connectivity (%)
80
0.9
90
0.8
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.1
0 0
0.1
1
2.5
3
5
7
12
0
0.1
Gain (h)
1
2.5
3
5
7
12
Gain (h)
Figure 4.6: ρ in connectivity-gain space (φ = 0). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08 noise levels were used because the actual noise handling capacity of the network in terms of efficient recall of attractors is 5 − 6% (to be shown later), and the values chosen in these figures are comparable to that. The recall stability breaks down beyond these levels. Similarly, a 0 input error level is the lowest possible and a level of 0.6 is the maximum level chosen for the current simulations. At this high error level as well, the recall breaks down. Based on these simulations, connectivities of 50 − 100% and two gain levels of 2.5 and 12 (signifying low and high gain levels) were chosen for the subsequent recall stability simulations.
4.3.2
Efficiency of Recall
The next step in evaluating system performance is to look at the detailed recall quality for spatiotemporal patterns. Figures 4.12 − 4.17 show the mean recall quality for all the attractors for different slow neuronal connectivities, varying from 50 % to 100 % in different steps (only these connectivities give the desired high ρ and ζ values as shown before). The recall qualities 52
(a)
(b) 1
1
100
100 0.9
90
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.8
80
Connectivity (%)
Connectivity (%)
80
0.9
90
0.8
0.2
5
0.1
0
0.1
0 0
0.1
1
2.5
3
5
7
0
12
0.1
1
2.5
Gain (h) (c)
3
5
7
12
Gain (h) (d) 1
1
100
100 0.9
90
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.8
80
Connectivity (%)
Connectivity (%)
80
0.9
90
0.8
0.2
5
0.1
0
0.1
0 0
0.1
1
2.5
3
5
7
0
12
0.1
1
2.5
Gain (h)
3
5
7
12
Gain (h)
Figure 4.7: ρ in connectivity-gain space (φ = 0.6). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08 (a)
(b) 1
1
100
100 0.9
90
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.8
80
Connectivity (%)
Connectivity (%)
80
0.9
90
0.8
0.2
5
0.1
0
0.1
0 0
0.1
1
2.5
3
5
7
0
12
0.1
1
2.5
Gain (h) (c)
3
5
7
12
Gain (h) (d) 1
1
100
100 0.9
90
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.1
0
0.8
80
Connectivity (%)
Connectivity (%)
80
0.9
90
0.8
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.1
0 0
0.1
1
2.5
3
5
7
12
0
0.1
Gain (h)
1
2.5
3
5
7
12
Gain (h)
Figure 4.8: ζ in connectivity-gain space (φ = 0). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08 53
(a)
(b) 1
1
100
100 0.9
90
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.8
80
Connectivity (%)
Connectivity (%)
80
0.9
90
0.8
0.2
5
0.1
0
0.1
0 0
0.1
1
2.5
3
5
7
0
12
0.1
1
2.5
Gain (h) (c)
3
5
7
12
Gain (h) (d) 1
1
100
100 0.9
90
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.8
80
Connectivity (%)
Connectivity (%)
80
0.9
90
0.8
0.1
0
70
0.7
60
0.6
50
0.5
40
0.4
30
0.3
20
0.2
5
0.1
0 0
0.1
1
2.5
3
5
7
12
0
0.1
1
2.5
Gain (h)
3
5
7
12
Gain (h)
Figure 4.9: ζ in connectivity-gain space (φ = 0.6). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08 are plotted for two levels of slow − slow gain h, 2.5 and 12 (signifying low and high levels for the model). The same quality curves are plotted in two ways, against input error and additive noise, for different values of additive noise and input error, respectively. Each curve within a particular plot represents a different level of η or φ. Each graph is averaged over 15 independent trials. Each figure comprises four subplots: Two for recall quality against input error for different additive noise levels (each for a different gain level), and two for recall quality against additive noise for different input error levels (each for a different gain level).
4.3.2.1
Recall Quality against Input Error
For the images showing recall quality against input error for different additive noise levels with a low gain of 2.5, the recall for smaller additive noise levels is close to 1 for up to 40% input error and drops to around 90% subsequently. The recall drops to around 40-60 % for extremely high noise levels which are close to 5%, at almost all input error levels. For similar plots with 54
(a) 100 90
Connectivity (%)
80 70 60 50 40 30 20 5 0 0.1
1
2.5
3
5
7
12
5
7
12
5
7
12
5
7
12
Gain (h) (b) 100 90
Connectivity (%)
80 70 60 50 40 30 20 5 0 0.1
1
2.5
3
Gain (h) (c) 100 90
Connectivity (%)
80 70 60 50 40 30 20 5 0 0.1
1
2.5
3
Gain (h) (d) 100 90
Connectivity (%)
80 70 60 50 40 30 20 5 0 0.1
1
2.5
3
Gain (h)
Figure 4.10: ρ and ζ shown together in connectivity-gain space, if both ρ and ζ ≥ 0.9 at a certain level in the space, that region is shown white, else black (φ = 0). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08
55
(a) 100 90
Connectivity (%)
80 70 60 50 40 30 20 5 0 0.1
1
2.5
3
5
7
12
5
7
12
5
7
12
5
7
12
Gain (h) (b) 100 90
Connectivity (%)
80 70 60 50 40 30 20 5 0 0.1
1
2.5
3
Gain (h) (c) 100 90
Connectivity (%)
80 70 60 50 40 30 20 5 0 0.1
1
2.5
3
Gain (h) (d) 100 90
Connectivity (%)
80 70 60 50 40 30 20 5 0 0.1
1
2.5
3
Gain (h)
Figure 4.11: ρ and ζ shown together in connectivity-gain space, if both ρ and ζ ≥ 0.9 at a certain level in the space, that region is shown white, else black (φ = 0.6). (a) η=0; (b) η=0.02; (c) η=0.05; (d) η=0.08
56
a high recurrent gain of 12, the quality is very high even for higher noise levels. It remains at a level of around 85% for relatively lower connectivities at high noise levels, whereas, for very high connectivities, it is close to 90% for very high noise levels. It drops to around 60% for high input error levels at almost all noise levels.
4.3.2.2
Recall Quality against Additive Noise
For the images showing recall quality against additive noise for different input error levels with a low gain of 2.5, the recall declines sharply after a noise level of around 3% for almost all input error levels across connectivities. The curves with high input error decline more sharply. With full connectivity, the quality remains around 80% for 3% of noise and then declines to 60%, for smaller error levels. For very high error and noise levels, it declines to around 40% as expected. For similar plots with a high gain of 12, the quality remains at a high level of 90 − 95% for smaller error levels, and declines to around 75 − 80% for higher error levels. Increasing additive noise level also has an effect, but the curves remain much less steeper than the lower gain case.
4.3.2.3
Overall Trend
The overall trend in the curves suggests that high gain makes a lot of difference. The recall quality improves drastically with a high gain value, and also increases somewhat with increasing connectivity, although this increase is not linear. Both these facts show that attractor dynamics in the slow network plays a significant role in the robustness of the recall. The plots also show that the network is more robust to input error than to additive noise. At full connectivity and high gain value, the system has a noise handling capacity of around 5 − 6% which is quite high. Similarly, the system has an error handling capacity of around 40%. Even at these levels of noise and error, the system produces efficient recall of the spatiotemporal patterns.
57
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(a)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.1
0.2
0.3
0.4
0.5
0.6
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(b)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.1
0.2
0.3
0.4
0.5
0.6
Input Error 1.4
?=0
0.1
0.3
0.4
0.5
1.2
(c)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
1.4
?=0
0.1
0.3
0.4
0.5
1.2
(d)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Additive Noise
Figure 4.12: (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 50 % connectivity in all cases. 58
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(a)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.1
0.2
0.3
0.4
0.5
0.6
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(b)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.1
0.2
0.3
0.4
0.5
0.6
Input Error 1.4
?=0
0.1
0.3
0.4
0.5
1.2
(c)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
1.4
?=0
0.1
0.3
0.4
0.5
1.2
(d)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Additive Noise
Figure 4.13: (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 60 % connectivity in all cases. 59
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(a)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.1
0.2
0.3
0.4
0.5
0.6
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(b)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.1
0.2
0.3
0.4
0.5
0.6
Input Error 1.4
?=0
0.1
0.3
0.4
0.5
1.2
(c)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
1.4
?=0
0.1
0.3
0.4
0.5
1.2
(d)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Additive Noise
Figure 4.14: (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 70 % connectivity in all cases. 60
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(a)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.1
0.2
0.3
0.4
0.5
0.6
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(b)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.1
0.2
0.3
0.4
0.5
0.6
Input Error 1.4
?=0
0.1
0.3
0.4
0.5
1.2
(c)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
1.4
?=0
0.1
0.3
0.4
0.5
1.2
(d)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Additive Noise
Figure 4.15: (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 80 % connectivity in all cases. 61
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(a)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.1
0.2
0.3
0.4
0.5
0.6
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(b)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.1
0.2
0.3
0.4
0.5
0.6
Input Error 1.4
?=0
0.1
0.3
0.4
0.5
1.2
(c)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
1.4
?=0
0.1
0.3
0.4
0.5
1.2
(d)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Additive Noise
Figure 4.16: (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 90 % connectivity in all cases. 62
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(a)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.1
0.2
0.3
0.4
0.5
0.6
1.4
2=0
0.008
0.01
0.03
0.05
1.2
(b)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.1
0.2
0.3
0.4
0.5
0.6
Input Error 1.4
?=0
0.1
0.3
0.4
0.5
1.2
(c)
Quality
1
0.8
0.6
0.4
0.2
0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
1.4
?=0
0.1
0.3
0.4
0.5
1.2
(d)
Quality
1 0.8 0.6 0.4 0.2 0 0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
Additive Noise
Figure 4.17: (a) Mean recall quality vs input error at different additive noise levels, h=2.5; (b) Mean recall quality vs input error at different additive noise levels, h=12; (c) Mean recall quality vs additive noise at different input error levels, h=2.5; (d) Mean recall quality vs additive noise at different input error levels, h=12. 100 % connectivity in all cases. 63
4.4
Visualization through Doodles
As described earlier, different spatiotemporal patterns are visualized in X-Y-space giving different types of doodles representing motor activity. Each pattern generates a distinct doodle using the population coding method described in Chapter 3. Figure 4.18 shows the doodles produced by the 5 attractors embedded in the network in the absence of input error and noise. Interestingly, the doodles for the first 3 attractors look somewhat like the letters “L”, “w” and “a”, respectively. In all the simulations that follow, 100 % slow − slow connectivity and a gain (h) of 2.5 are used, unless specified otherwise.
Figure 4.18: Doodles generated by the 5 attractors in the absence of input error or noise.
4.4.1
Robustness of Motor Activity to Input Error
The robustness of the recalled patterns can be evaluated in a concrete way by comparing the doodles generated at various levels of input error, φ, and additive noise, η, with those generated in the error- and noise-free case. Figures 4.19 − 4.23 show what happens to the doodles for all the attractors as φ is varied from 0 to 0.6. The figures remain robust even at high levels of φ.
64
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.19: Change in the doodle for Attractor 1 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6
4.4.2
Robustness of Motor Activity to Additive Noise
Similarly, Figures 4.24 − 4.28 show that as η is increased for all the attractors, the doodles change shape more sharply than in the φ variation case, suggesting that the model is more sensitive to additive noise than to input error.
4.4.3
Effects of Parameter Modulation
As discussed earlier, the forms of the spatiotemporal patterns produced by the system can be modified systematically through several parameters. This is especially important in the motor system because minor and systematic variations of movements – such as writing specific characters – is observed in practice, and is often useful. The modulation also provides a potential control mechanism by which trajectories can be modified in predictable ways to avoid obstacles or to satisfy constraints.
65
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.20: Change in the doodle for Attractor 2 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6 4.4.3.1
α Modulation
A global way to modulate the spatiotemporal output in the proposed system is to vary the deactivation gain, α. Figures 4.29 − 4.33 show the effect of varying α above and below the default value of 1.2 used for slow neurons for all the attractors. With higher values of alpha, due to increased deactivation, the doodles lose their features gradually, which can be desirable in learning new types of doodles.
4.4.3.2
h Modulation
Figures 4.34 − 4.38 show the effects of the variation of slow − slow gain (h) on the doodles. It can be seen that at very low gain values, the activity is very low, and so small doodles are observed. At higher gain values, spatiotemporal patterns with increased neuronal activity are generated, and so the doodles become large in appearance.
66
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.21: Change in the doodle for Attractor 3 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6 4.4.3.3
Neuron-Specific Modulation
Other parameters like τ , a and b can be varied systematically in neuron-specific ways for particular attractors to produce different types of doodles. This is important because it is often desirable to only control and improvise on a particular type of movement without changing other behavioral patterns. This will be addressed in future work.
4.4.4
Synergistic Combination of Movement Primitives
As discussed earlier, the patterns encoded by the model can be seen as representing a repertoire of canonical movement primitives as suggested by experimental studies [60, 5, 4, 50, 61]. Multiple networks encoding a set of such patterns can be triggered in combination to generate patterns that interpolate between these basis movements. Figures 4.39 − 4.43 show the doodles drawn by combining the activity patterns of two different attractors in different proportions. Clearly, there are patterns of interpolation. This means that different attractors when triggered in combination can lead to learning of new and probably useful movements.
67
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.22: Change in the doodle for Attractor 4 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6
4.5
More Spatiotemporal Attractors and Doodles
A large number of spatiotemporal attractors and their corresponding doodles were generated using all the parameter values mentioned before (100 % connectivity and gain of 2.5). Some of these are shown here. Figures 4.44−4.53 show different samples of 5 attractors embedded in the network, and Figures 4.54 − 4.63 show the corresponding doodles for these attractor samples. It can be seen that at these parameter settings, attractors meet the requirements listed in the previous chapter. They turn on and off at different intervals, have different onset delays, and can only fire for short time intervals. The doodles also look interesting and can potentially be seen as different characters. In future work, these doodles can be learned in more systematic ways through pattern specific modulation of parameters.
68
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.23: Change in the doodle for Attractor 5 due to different levels of input error, φ: (a) φ = 0; (b) 0.1; (c) 0.2; (d) 0.4; (e) 0.5; (f) 0.6
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.24: Additive noise, η, variation for Attractor 1, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4
69
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.25: Additive noise, η, variation for Attractor 2, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.26: Additive noise, η, variation for Attractor 3, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4
70
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.27: Additive noise, η, variation for Attractor 4, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.28: Additive noise, η, variation for Attractor 5, (a) 0; (b) 0.02; (c) 0.05; (d) 0.08; (e) 0.1; (f) 0.4
71
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.29: Modulation of the doodle for Attractor 1 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.30: Modulation of the doodle for Attractor 2 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5
72
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.31: Modulation of the doodle for Attractor 3 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.32: Modulation of the doodle for Attractor 4 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5
73
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.33: Modulation of the doodle for Attractor 5 due to different values of deactivation gain, α: (a) α = 0; (b) 0.5; (c) 1.2; (d) 1.5; (e) 2; (f) 2.5
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.34: Modulation of the doodle for Attractor 1 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8
74
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.35: Modulation of the doodle for Attractor 2 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.36: Modulation of the doodle for Attractor 3 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8
75
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.37: Modulation of the doodle for Attractor 4 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.38: Modulation of the doodle for Attractor 5 due to different values of gain (h): (a) h = 0.1; (b) 1; (c) 2.5; (d) 4; (e) 6; (f) 8
76
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.39: Interpolation between doodles by the combined activation of Attractors 2 and 4, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.40: Interpolation between doodles by the combined activation of Attractors 1 and 5, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0
77
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.41: Interpolation between doodles by the combined activation of Attractors 3 and 2, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.42: Interpolation between doodles by the combined activation of Attractors 4 and 1, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0
78
(a)
(b)
(c)
(d)
(e)
(f)
Figure 4.43: Interpolation between doodles by the combined activation of Attractors 5 and 3, combined with weights ψ and 1 − ψ, respectively: (a) ψ = 1; (b) 0.8; (c) 0.6; (d) 0.4; (e) 0.2; (f) 0
79
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.44: Sample 1: All attractors 80
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.45: Sample 2: All attractors 81
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.46: Sample 3: All attractors 82
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.47: Sample 4: All attractors 83
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.48: Sample 5: All attractors 84
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.49: Sample 6: All attractors 85
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.50: Sample 7: All attractors 86
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.51: Sample 8: All attractors 87
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.52: Sample 9: All attractors 88
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
1 0.9
20
0.8 40
0.7
60
0.6 0.5
80
0.4 100
0.3
120
0.2 0.1
140 200
400
600
800
1000
1200
0
Figure 4.53: Sample 10: All attractors 89
Figure 4.54: Doodles generated by the 5 attractors in sample 1.
Figure 4.55: Doodles generated by the 5 attractors in sample 2.
90
Figure 4.56: Doodles generated by the 5 attractors in sample 3.
Figure 4.57: Doodles generated by the 5 attractors in sample 4.
91
Figure 4.58: Doodles generated by the 5 attractors in sample 5.
Figure 4.59: Doodles generated by the 5 attractors in sample 6.
92
Figure 4.60: Doodles generated by the 5 attractors in sample 7.
Figure 4.61: Doodles generated by the 5 attractors in sample 8.
93
Figure 4.62: Doodles generated by the 5 attractors in sample 9.
Figure 4.63: Doodles generated by the 5 attractors in sample 10.
94
Chapter 5
Conclusions and Future Work This thesis has described a recurrently connected neural network model for the representation and cued recall of spatiotemporal activity patterns, with the motivation of understanding the neural basis of voluntary motor control. The model is shown to store spatiotemporal dynamic activity patterns and recall them robustly in response to specific stimuli. The activity patterns can be seen as encoding temporal responses, such as voluntary movements mediated by motor synergies. The utility of the system is demonstrated by mapping its output to doodling movements through population coding. The patterns encoded by the model can be seen as representing a repertoire of canonical movement primitives as suggested by experimental studies. Such movement primitives can be combined to produce more complex or interpolated movements, and this aspect is also observed through the model. The patterns can also be modulated systematically through several parameters that can serve as a locus for learning and real-time adaptation.
5.1
Research Contributions
The work in this thesis has made the following concrete contributions:
• The proposed model was shown to store and recall spatiotemporal dynamic activity pat-
95
terns successfully in the presence of input error and additive noise. These patterns could be elicited reliably through specific stimuli. • The effect of connectivity between the slow neurons on the system’s functionality was explored. Higher connectivities were found to lead to more robust patterns. • A systematic investigation showed that the participation of specific neurons in spatiotemporal patterns could be specified a priori using nominal attractors, and that this specification was faithfully followed by the dynamics if the system had sufficient connectivity and the recurrent weights were strong enough. • Improved performance with increasing connectivity indicated that attractor dynamics play a significant role in stabilizing the spatiotemporal patterns, and ensuring robust recall with higher sensitivities and specificities. • It was shown through simulations that the movements generated through the system were also stable under conditions of input error and noise, and could be changed systematically by modulating specific parameters.
5.2
Future Goals
Future work plans to extend the current model in several ways, including the following: • An explicit model of neural learning will be incorporated into the system, allowing it to learn movements by imitation or practice. • The effect of varying several parameters in pattern-specific ways will be studied as a way to provide generalization in motor learning, i.e., enabling the system to produce novel actions after learning a limited set. • The model will be used to encode motor primitives in a larger system to serve as the basis of the motor repertoire for simulated and real mobile agents and robots. 96
• Potential applications of the model to understand the basis of motor disorders will be explored.
97
Bibliography [1] O. Sporns, “Network attributes for segregation and integration in the human brain,” Current Opinion in Neurobiology, vol. 23, p. 162–171, 2013. [2] C. Beevor and V. Horsley, “An experimental investigation into the arrangement of excitable fibres of the internal capsule of the bonnet monkey (macacus sinicus),” Philos. Trans. R. Soc. Lond. B Biol. Sci, vol. 181, pp. 49–88, 1890. [3] W. Penfield and T. , Rasmussen, The cerebral cortex of man. The Macmillan Company, 1950. [4] L. Ting and J. Macpherson, “A limited set of muscle synergies for force control during a postural task,” Journal of Neurophysiology, vol. 93, pp. 609–613, 2005. [5] A. d’Avella, P. Saltiel, and E. Bizzi, “Combinations of muscle synergies in the construction of natural motor behavior,” Nature Neuroscience, vol. 6, pp. 300–308, 2003. [6] E. Bizzi, V. Cheung, A. d’Avella, P. Salitel, and M. Tresch, “Combining modules for movement,” Brain Research Reviews, vol. 57, pp. 125–133, 2007. [7] F. J. Valero-Cuevas, M. Venkadesan, and E. Todorov, “Structured variability of muscle activations supports the minimal intervention principle of motor control,” J Neurophysiol., vol. 102(1), pp. 59–68, 2009. [8] K. V. Byadarhaly, M. Perdoor, and A. A. Minai, “A neural model of motor synergies,” in Proceedings of IJCNN 2011, pp. 2961–2968, 2011. 98
[9] V. Henn and B. Cohen, “Coding of information about rapid eye movements in the pontine reticular formation of alert monkeys,” Brain Research, vol. 108, pp. 307–325, 1976. [10] D. Bullock, P. Cisek, and S. Grossberg, “Cortical networks for control of voluntary arm movements under variable force conditions,” Cerebral Cortex, vol. 8, pp. 48–62, 1998. [11] S. Grossberg and R. W. Paine, “A neural model of cortico-cerebellar interactions during attentive imitation and predictive learning of sequential handwriting movements,” Neural Networks, vol. 13, pp. 999–1046, 2000. [12] S. Vasa, T. Ma, K. V. Byadarhaly, M. Perdoor, and A. A. Minai, “A spiking neural model for the spatial coding of cognitive response sequences,” in Proceedings of ICDL 2011, pp. 140–146, 2010. [13] D. Pinto, S. L. Patrick, W. C. Huang, and B. W. Connors, “Initiation, propagation, and termination of epileptiform activity in rodent neocortex in vitro involve distinct mechanisms,” J. Neurosci., vol. 25, p. 8131–40, 2005. [14] M. Rabinovich, A. Volkovskii, P. Lecanda, R. Huerta, H. D. I. Abarbanel, and G. Laurent, “Dynamical encoding by networks of competing neuron groups: winnerless competition,” Physical Review Letters, vol. 87, pp. 068102–1, 2001. [15] D. Marr, “Simple memory: a theory for archicortex,” Phil. Trans. Royal Soc. London, vol. 262, pp. 23–81, 1971. [16] J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proceedings of the National Academy of Sciences USA, vol. 79, pp. 2554– 2558, 1982. [17] J. J. Hopfield, “Neurons with graded response have collective computational properties like those of two-state neurons,” Proceedings of the National Academy of Sciences USA, vol. 81, pp. 3088–3092, 1984. 99
[18] D. Amit, Modelling Brain Function. Cambridge, UK: Cambridge University Press, 1989. [19] A. Minai and W. Levy, “Sequence learning in a single trial,” in Proc. World Congress on Neural Networks, San Diego, vol. Vol. IV, pp. 176–180, 1993. [20] B. Ans, Y. Coiton, J. P. Gilhodes, and J. L. Velay, “A neural network model for temporal sequence learning and motor programming,” Neural Networks, vol. 7, pp. 1461–1476, 1994. [21] R. Huerta and M. Rabinovich, “Reproducible sequence generation in random neural ensembles,” Physical Review Letters, vol. 93, pp. 238104–1 – 238104–4, 2004. [22] K. V. Byadarhaly, M. Perdoor, S. Vasa, E. Fernandez, and A. A. Minai, “Learning complex population-coded sequences,” in Proceedings of ICANN 2009, pp. 296–305, 2009. [23] S. Amari, “Dynamics of pattern formation in lateral inhibition type neural fields,” Biological Cybernetics, vol. 27, pp. 77–87, 1977. [24] P. C. Bressloff, “Spatiotemporal dynamics of continuum neural fields,” Journal of Physics A: Mathematical and Theoretical, vol. 45, p. 033001, 2012. [25] A. A. Minai, L. R. Iyer, D. Padur, and S. Doboli, “A dynamic connectionist model of idea generation,” in Proceedings of IJCNN 2009, pp. 2109–2116, 2009. [26] L. R. Iyer, S. Doboli, A. A. Minai, V. R. Brown, D. S. Levine, and P. B. Paulus, “Neural dynamics of idea generation and the effects of priming,” Neural Networks, vol. 22, pp. 674– 686, 2009. [27] N. Marupaka and A. A. Minai, “Connectivity and creativity in semantic neural networks,” in Proceedings of IJCNN 2011, pp. 3127–3133, 2011. [28] N. Marupaka, L. R. Iyer, and A. A. Minai, “Connectivity and thought: The influence of semantic network structure in a neurodynamical model of thinking,” Neural Networks, vol. 32, pp. 147–158, 2012. 100
[29] J. A. S. Kelso, Dynamic Patterns: The Self-Organization of Brain and Behavior. Cambridge, MA: MIT Press, 1995. [30] W. Singer, “Development and plasticity of cortical processing architectures,” Science, vol. 270, pp. 758–764, 1995. [31] F. Varela, J. P. Lachaux, E. Rodriguez, and J. Martinerie, “The brainweb: phase synchronization and large-scale integration,” Nature Reviews Neuroscience, vol. 2, pp. 229–239, 2001. [32] A. K. Engel, P. Fries, and W. Singer, “Dynamic predictions: oscillations and synchrony in top-down processing,” Nature Reviews Neuroscience, vol. 2, pp. 704–716, 2001. [33] S. L. Bressler and E. Tognoli, “Operational principles of neurocognitive networks,” International Journal of Psychophysiology, vol. 60, pp. 139–148, 2006. [34] D. V. Buonomano and W. Maass, “State-dependent computations: Spatiotemporal processing in cortical networks,” Nature Reviews—Neuroscience, vol. 10, pp. 113–125, 2009. [35] J. K. Liu and D. V. Buonomano, “Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner,” The Journal of Neuroscience, vol. 29, p. 13172–13181, 2009. [36] A. Pouget and L. H. Snyder, “Computational approaches to sensorimotor transformations,” Nature Neuroscience Supp., vol. 3, pp. 1192–1198, 2000. [37] S. Deneve, P. E. Latham, and A. Pouget, “Efficient computation and cue integration with noisy population code,” nature neuroscience, vol. 4, pp. 826–835, 2001. [38] A. P. Georgopoulos, J. F. Kalaska, R. Caminiti, and J. T. Massey, “On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex,” Journal of Neuroscience, vol. 2, pp. 1527–1537, 1982.
101
[39] A. P. Georgopoulos, A. B. Schwarz, and R. E. Kettner, “Neuronal population coding of movement direction,” Science, vol. 233, pp. 1416–1419, 1986. [40] A. Graybiel, “Building action repertoires: memory and learning functions of the basal ganglia,” Current Opinion in Neurobiology, vol. 5, pp. 733–741, 1995. [41] N. Fujii and A. M. Graybiel, “Representation of action sequence boundaries by macaque prefrontal cortical neurons,” Science, vol. 301, pp. 1246–1749, 2003. [42] P. Cisek, “Cortical mechanisms of action selection: the affordance competition hypothesis,” Phil. Trans. R. Soc. B, vol. 362, pp. 1585–1599, 2007. [43] M. S. A. Graziano and T. A. Aflalo, “Mapping behavioral repertoire onto the cortex,” Neuron, vol. 56, pp. 239–251, 2007. [44] S. Grillner, “Biological pattern generation: the cellular and computational logic of networks in motion,” Neuron, vol. 52, pp. 751–766, 2006. [45] H. Haken, J. A. S. Kelso, and H. Bunz, “A theoretical model of phase transitions in human hand movements,” Biological Cybernetics, vol. 51, pp. 347–356, 1985. [46] G. Orlovsky, T. Deliagina, and S. Grillner, Neuronal Control of Locomotion:From Mollusc to Man. Oxford University Press, Oxford, UK, 1999. [47] M. Graziano, “The organization of behavioral repertoire in motor cortex,” Annual Review of Neuroscience, vol. 29, pp. 105–134, 2006. [48] T. A. Aflalo and M. S. A. Graziano, “Possible origins of the complex topographic organization of motor cortex: reduction of a multidimensional space onto a two-dimensional array,” Journal of Neuroscience, vol. 26, pp. 6288–6297, 2006.
102
[49] T. A. Aflalo and M. S. A. Graziano, “Relationship between unconstrained arm movements and single-neuron firing in the macaque motor cortex,” Journal of Neuroscience, vol. 27, pp. 2760–2780, 2007. [50] M. Graziano, The Intelligent Movement Machine: An Ethological Perspective on the Primate Motor System. Oxford University Press, Oxford, UK, 2009. [51] S. Grillner, H. Markram, E. D. Schutter, G. Silderberg, and F. LeBeau, “Microcircuits in action – from cpgs to neocortex,” Trends in Neurosciences, vol. 28(10), pp. 525–533, 2005. [52] A. J. Ijspeert, A. Crespi, D. Ryczko, and J. M. Cabelguen, “From swimming to walking with a salamander robot driven by a spinal cord model,” Science, vol. 315, pp. 1416–1420, 2007. [53] R. Yuste, J. Maclean, J. Smith, and A. Lanser, “The cortex as a central pattern generator,” Nature Rev. Neurosci., vol. 6, pp. 477–483, 2005. [54] J. Houk, “Agents of the mind,” Biol. Cybern., vol. 92, pp. 427–437, 2005. [55] N. I. Bernstein, The Coordination and Regulation of Movements. Pergamon, Oxford, 1967. [56] E. Todorov and M. I. Jordan, “Optimal feedback control as a theory of motor coordination,” Nature Neuroscience, vol. 5, p. 1226–1235, 2002. [57] Z. Ghahramani and M. D. Wolpert, “Modular decomposition in visuomotor learning,” Nature, vol. 395, p. 386:392, 1997. [58] K. Doya, H. Kimura, and A. Miyamura, “Motor control: Neural models and systems theory,” Int. J. Appl. Math. Comput. Sci., vol. 11, p. 77–104, 2001. [59] F. Mussa-Ivaldi, “A. modular features of motor control and learning,” Curr.Opin. Neurobiol, vol. 9, pp. 713–717, 1999.
103
[60] M. Tresch, P. Salitel, and E. Bizzi, “The construction of movement by the spinal cord,” Nature Neuroscience, vol. 2, pp. 162–167, 1999. [61] S. Muceli, A. Boye, A. d’Avella, and D. Farina, “identifying representative synergy matrices for describing muscular activation patterns during multidirectional reaching in the horizontal plane,” Journal of Neurophysiology, vol. 103, pp. 1532–1542, 2010. [62] M. Riley, N. Kuznetsov, and S. Bonnette, “State-, parameter-, and graph-dynamics: Constraints and the distillation of postural control systems,” Science & Motricite, vol. 74, pp. 5–18, 2011. [63] M. Tresch and A. Jarc, “The case for and against muscle synergies,” Current Opinion in Neurobiology, vol. 19, pp. 601–607, 2009. [64] J. J. Kutch, A. D. Kuo, A. M. Bloch, and W. Z. Rymer, “Endpoint force fluctuations reveal flexible rather than synergistic patterns of muscle cooperation,” J Neurophysiol., vol. 100(5), pp. 2455–2471, 2008. [65] K. Byadarhaly, M. Perdoor, and A. Minai, “A modular neural model of motor synergies,” Neural Networks, vol. 32, pp. 96–108, 2012. [66] A. A. Minai, M. Perdoor, K. V. Byadarhaly, S. Vasa, and L. R. Iyer, “A synergistic view of autonomous cognitive systems,” in Proceedings of IJCNN 2010, pp. 498–505, 2010. [67] J. Tanji and E. Hoshi, “Behavioral planning in the prefrontal cortex,” Current Opinion in Neurobiology, vol. 11, pp. 164–170, 2001. [68] P. G. F. Browning and D. Gaffan, “Prefrontal cortex function in the representation of temporally complex events,” The Journal of Neuroscience, vol. 28(15), pp. 3934–3940, 2008.
104
[69] P. G. F. Browning, M. G. Baxter, and D. Gaffan, “Prefrontal-temporal disconnection impairs recognition memory but not familiarity discrimination,” J Neurosci., vol. 33(23), p. 9667–9674, 2013. [70] D. J. Barraclough, M. L. Conroy, and D. Lee, “Prefrontaln cortex and decision making in a mixed-strategy game,” Nature Neuroscience, vol. 7(4), p. 404–410, 2004. [71] I. H. Jenkins, M. Jahanshahi, M. Jueptner, R. Passingham, and D. Brooks, “Self-initiated versus externally triggered movements. ii. the effect of movement predictability on regional cerebral blood flow,” Brain, vol. 123, p. 1216–1228, 2000. [72] J. M. Fuster, “Executive frontal functions,” Experimental Brain Research, vol. 133, pp. 66– 70, 2000. [73] T. A. Aflalo and M. S. A. Graziano, “Four-dimensional spatial reasoning in humans,” Journal of Experimental Psychology: Human Perception and Performance, vol. 34, pp. 1066– 1077, 2008. [74] R. H. Schor, A. D. Miller, and D. L. Tomko, “Responses to head tilt in cat central vestibular neurons. i. direction of maximum sensitivity.,” J Neurophysiol., vol. 51, pp. 136–46, 1984. [75] V. J. Wilson, K. Ezure, and S. J. Timerick, “Tonic neck reflex of the decerebrate cat: response of spinal interneurons to natural stimulation of neck and vestibular receptors,” J Neurophysiol., vol. 51, pp. 567–577, 1984. [76] C. D. Cowper-Smith, E. Y. Y. Lau, C. A. Helmick, G. Eskes, and D. A. Westwood, “Neural coding of movement direction in the healthy human brain,” PLoS One, vol. 5(10), 2010. [77] M. Reiss and J. G. Taylor, “Storing temporal sequences,” Neural Networks, vol. 4, pp. 773– 787, 1991. [78] A. Minai, G. Barrows, and W. Levy, “Disambiguation of pattern sequences with recurrent networks,” in Proc. WCNN, San Diego, vol. IV, pp. 176–180, 1994. 105
[79] Y. Sandamirskaya, “Dynamic neural fields as a step toward cognitive neuromorphic architectures,” Front. Neurosci., vol. 7, 2014. [80] J. D. Rolston, D. A. Wagenaar, and S. M. Potter, “Precisely timed spatiotemporal patterns of neural activity in dissociated cortical cultures,” Neuroscience, vol. 148, p. 294–303, 2007. [81] N. Burgess, M. Recce, and J. O’Keefe, “A model of hippocampal function,” Neural Networks, vol. 7, pp. 1065–1081, 1994. [82] M. E. Hasselmo, E. Schnell, and E. Barkai, “Dynamics of learning and recall at excitatory recurrent synapses and cholinergic modulation in hippocampal region CA3,” Journal of Neuroscience, vol. 15, pp. 5249–5262, 1995. [83] S. Doboli, A. A. Minai, and P. J. Best, “Latent attractors: a model for context-dependence place representations in the hippocampus,” Neural Computation, vol. 12(5), pp. 1003–1037, 2000. [84] P. K. Stanton and T. J. Sejnowski, “Associative long-term depression in the hippocampus induced by hebbian covariance,” Nature, vol. 339, pp. 215–218, 1989. [85] A. Minai, “Covariance learning of correlated patterns in competitive networks,” Neural Computation, vol. 9, pp. 667–681, 1997. [86] S. Doboli, A. A. Minai, and P. J. Best, “A latent attractors model of context selection in the dentate gyrus-hilus system,” Neurocomputing, vol. 26-27, pp. 671–676, 1999. [87] S. Doboli, A. Minai, P. Best, and A. White, “An attractor model for hippocampal place cell hysteresis,” Neurocomputing, vol. 38-40, pp. 1185–1191, 2001. [88] S. Doboli, A. A. Minai, and P. J. Best, “A computational model of the interaction between external and internal cues for the control of hippocampal place cells,” Neurocomputing, vol. 52-54, pp. 371–379, 2002.
106
[89] S. Doboli and A. Minai, “Network capacity analysis for latent attractor computation,” Network: Computation in Neural Systems, vol. 14, pp. 273–302, 2003.
107
