Bio-mimetic path integration using a self organizing population of grid cells Ankur Sinha and Jack Wang? Faculty of Engineering and Information Technology The University of Technology Sydney, Australia
[email protected] [email protected].
Abstract. Grid cells in the dorsocaudal medial entorhinal cortex (dMEC) of the rat provide a metric representation of the animal’s local environment. The collective firing patterns in a network of grid cells forms a triangular mesh that accurately tracks the location of the animal. The activity of a grid cell network, similar to head direction cells, displays path integration characteristics. Classical robotics use path integrators in the form of inertial navigation systems to track spatial information of an agent as well. In this paper, we describe an implementation of a network of grid cells as a dead reckoning system for the PR2 robot.
1
Introduction
Navigation is a capability any animal must possess to survive. It is also a capability that must be implemented in a mobile robotic system capable of carrying out any meaningful tasks. It is known that even smaller mammals, such as rats, have sufficiently well developed navigation systems that enable them to carry out tasks necessary for their survival, such as foraging for food or finding shelter. It is, therefore, of great interest to study these biological navigation systems and attempt to implement bio-mimetic navigation systems on robots. Discoveries of neurons that provide information on the agent’s spatial parameters, such as head direction cells[9,15], place cells[12] and grid cells[6], have lead to the formation of a cognitive map theory[10,8,18,13,11] of biological navigation. This theory states that via a combination of neurons with specific behaviours, animals maintain a “cognitive map” of their environment. The collective firing of grid cells gives rise to a regular triangular lattice representation of the location of the animal. These neurons, similar to head direction cells, have path integrator components that integrate velocity signals, like an INS[1]. In section 1.1 , we briefly introduce grid cells and their computational modelling. We then detail our model in section 2. In section 3, we present our results ?
We’re most grateful to Dr. Xun Wang at the “Magic” Lab, The University of Technology, for his inputs and assistance with the PR2 robot.
2
Bio-mimetic path integration using a self organizing population of grid cells
and briefly discuss challenges and our future work plans in section 4. Finally, we summarize and conclude in section 5. 1.1
Grid cells
Grid cells in the dorsocaudal medial entorhinal cortex of rats were observed by Hafting and colleagues[6] when they were looking for spatial information upstream from the hippocampus that could project on to place cells. They observed neurons with multiple discreet fields of similar amplitude in the dMEC and recorded spike activity in the region to ascertain the presence of a map like organization. On observing the autocorrelogram, neurons considered as central peaks were observed to be surrounded by six nearly equidistant neurons at angular separations of 60°, forming a regular triangular grid. The number of activity nodes increased with the size of the environment, suggesting a structure with infinite size. Unlike head direction and place cells where no topographical structure is observed, grid cells exhibit a precise topographical organization. Also unlike neurons in the hippocampus where neuron sets vary per environment, the grid structure appears common across environments. As with head direction and place cells, various computational models have been proposed that attempt to model the grid like firing patterns exhibited by grid cells. In their recent review[5], Giocomo and colleagues have classified these models into two categories: oscillatory-interference models which use changes in membrane potential oscillator frequencies to encode speed and direction information into grid patterns; attractor based networks which use specific activity of a local set of neurons to generate grid patterns. Our model belongs to the latter category since we use Hebbian style learning rules to set up local excitation between neurons to produce the required grid attractor network. We humbly refer the reader to recent reviews for further information on the properties of grid cells[2] and their computational modelling[5].
2
Methods: The model
The model detailed here is a modified version of the self organizing two dimensional attractor network model Stringer et al. proposed for place cells[16]. In the original model, Stringer and colleagues detailed how their one dimensional attractor model for a head direction cell system[17] could be extended to two dimensions to produce place fields. The model successfully set up a two dimensional attractor network using Hebbian style learning rules and displayed spatial firing patterns consistent with biological observations of place cell networks. While using similar Hebbian style learning rules, we make modifications to the structure of the network required for the formation of a triangular lattice of neurons, rather than a traditional co-ordinate map. We also modify the organization process to ensure that the lattice forms a toroid of grid cells, rather than a flat map to work around edge effects.
Bio-mimetic path integration using a self organizing population of grid cells
G 90
G 0
G 1
G 2
G . . . . . j. . . . . .
G 92
G 91
3
G 99
G . . . . . j. . . . . .
G . . . . . j. . . . . .
G n
G . . . . . j. . . . . .
VEL
G 10
G 12
G 11
G 19
G . . . . . j. . . . . .
HDi
........... G 0
G 1
G 2
G . . . . . j. . . . . .
G 9
ROT2
ROT1
(a) The grid cell system schematic.
G 0
G . . . . . j. . . . . .
(b) 2D array of grid cells
2.1
Structure
Figure 1a shows a schematic of the neuron sets implemented in the model. The model consists of two fully connected attractors, one for the head direction cell set and another for the grid cell set. The head direction cell set has also been implemented on the ROS platform and is detailed in another, unpublished work1 . Each head direction cell, HDi is connected to every other head direction cell HD HDj via recurrent synapses wij . Similarly, every grid cell, Gk is connected to G every other grid cell Gl via recurrent synapses wkl . In this way, each synapse possesses a pre-synaptic and post-synaptic neuron necessary for our Hebbian learning based organization rule. The head direction cell set takes input from a set of rotation cells, ROTm , which project angular velocity information on to HD ROT them via synapses, wijm . Similarly, a set of velocity cells, V ELn , project G V EL . It is noteworthy speed information on to the grid cell set via synapses wkln that these synapses are effective synapses in that they represent the synaptic weight between a pre-synaptic head/grid cell, the rotation/velocity cell, and the post-synaptic head/grid cell. A set of visual feature cells, V ISo projects on to HD V IS G V IS both the head direction and grid cell sets via synapses wio and wko respectively. The visual feature cells are used for initial training of the synapses only in this model and represent an abstraction of the visual processing system.
1
Paper submitted to IJCNN 2014 for consideration
4
2.2
Bio-mimetic path integration using a self organizing population of grid cells
Dynamics
We employ a model similar to the one proposed by Stringer et al. for place cells[16]. The activation of each grid cell, Gk is given by: dhG φ0 X G k (t) (wkl − wIN H )rlG (t) = −hG k (t) + dt CG l X φ1 G V EL G + G×HD×V EL rl (t)rjHD (t)rnV EL (t)) (wkjn C τ
(1)
ljn
+
φ2
X
C G×V IS
G V IS V IS (wko ro (t))
o
Here, τ is the time constant while φ0 , φ1 , φ2 , C G , C G×HD×V EL , C G×V IS and wIN H are tunable parameters. These parameters control the effect the respective inputs have on the grid cell attractor. wIN H represents global inhibition that the GABAergic interneurons exert on the system. The combination of local excitation of grid cells and the global inhibition gives the system continuous attractor characteristics. The firing rate of each grid neuron is a sigmoid function of its activation: rkG (t) = f (hG k (t)) =
1 G 1 + e−2β(hk (t)−α)
(2)
where α and β are constants. Figure 3a shows the firing rate profile exhibited by the grid cell lattice after stabilization during a test run . Due to the regular learning employed in this implementation, the firing rates of all grid cells are similar. This isn’t the case in biology, where the firing rates of grid cells vary from one another and from layer to layer. The synapses between all neuron sets are set up using Hebbian learning: ∆w = k.(rpost ∗ rpre )
(3)
Here, ∆w is the change in synaptic weight. k is the learning rate of the synapse. rpre and rpost are the firing rates of the pre-synaptic and post-synaptic neurons respectively. This learning rule does not, however, include synaptic depression, or bounding of synaptic weights. We use a competition based normalization rule to bound our synapses: w ˆ=
w |w|
(4)
Here |w| is the norm of the w matrix and w ˆ is the normalized synaptic weight. It is worth noting that the above normalization departs from the Hebbian learning requirement of locality[4],i.e., the synapse between two neurons should only be modified by their behaviour. Various formulations of Hebbian learning have been proposed to overcome this shortcoming[3]. However, we use the normalization rule for sake of simplicity.
Bio-mimetic path integration using a self organizing population of grid cells
5
0.04 0.035 0.03 0.025 W 0.02 0.015 0.01 0.005 0
100
100 90
90 80
80 70
70 60
Grid cells
60 50
50 40
40 30
30 20
Grid cells
20 10
0 0
10
Fig. 2: Recurrent synaptic weight in grid cells
3
Experimental procedure and results
We implemented the model based on the ROS(Robot operating system)[14] platform which provides support for a number of robots, including the PR2. For development and testing, we collected data bags from the IMU sensors of the PR2 robot to run our simulations. We extended our earlier work on head direction cells to also include grid cells. We used a hundred grid cells to form a 10 × 10 mesh, and a hundred head direction cells to cover the 360°direction space. We used two rotation cells, one each for clockwise and anti clockwise rotation, a single speed cell, and a single visual cell (Figure 1a). The grid cell lattice merits some discussion. Conforming to biological observations, we designed a regular triangular lattice as shown in Figure 1b. Each cell is assigned a preferred location such that it coincides with the vertices of the equilateral triangles of the lattice. The distance between any two adjacent neurons is, therefore, one unit. Figure 1b shows how our implementation expands out to a two dimensional lattice. The system runs in three phases:
3.1
Setting up of synaptic weights to appropriate values
During this phase, we set up the synaptic weights in the network to their appropriate values. The network is initialized with all synaptic weights as zero, implying that no learning or association has taken place between the sets of G neurons. In order to set up both the internal grid cell synapses wkl and the efG V EL fective velocity synapses wkjn , we simulate movement in the system in the four major directions: forward, backward, left and right; neuron by neuron by projections from the visual feature cell. The firing rate of a grid cell is calculated as a function of the distance between it’s preferred location and the preferred
6
Bio-mimetic path integration using a self organizing population of grid cells
9
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 7
0
1
2
3 4 5 Grid cells (x)
6
7
8
9 0
1
2
8
1
8
0.9
0.8 0.6 0.4 0.2
7 6
0.8 0.7 0.6
5
0.5
Y
r
1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
4
0.4
3
9
6 5 4 Grid cells (y) 3
0.3
2
0.2
1
0.1
0
0 0
1
2
3
4
5
6
7
8
9
X
(a) 3D view
(b) Top view
Fig. 3: Stabilized activity packet
direction for the respective training iteration, ∆S: rlG = exp(−
1 + ∆S 2 2σ G
2
)
where ∆S is given by: q √ 2 2 ∆S = (min(|x|, |10 − x|)) + (min(|y|, |(10 ∗ ( 3/2)) − y|))
(5)
(6)
erred where, for each neuron Gk with preferred location xpref , ykpref erred for a k location X, Y erred pref erred (x, y) = (X, Y ) − (xpref , yk ) (7) k
σ G controls the width of the Gaussian profile. The above formulation of ∆S ensures the formation of a regular toroidal continuous attractor neural network. In order to calibrate the synapses of the network, we simulate activity in the input neuron sets: visual feature cell, velocity cell and head direction cells. Projections from the visual feature cell force a firing profile in the grid cell network which acts as the post synaptic neuron set. Since the velocity and head direction cells are simultaneously simulated to fire in accordance, they function as pre-synaptic neurons in our Hebbian style learning rule. G Figure 2 shows the recurrent synaptic weights of the grid cell network, wkl . As expected, the synaptic weights are maximum for neurons with similar preferred directions and decrease as the difference in preferred directions increase. 3.2
Initializing the network with an initial packet of activity
Once the synapses are trained, the network is initialized with a packet of activity which is the initial location or reference location of the system. An initial packet of activity is forced on to the grid cell attractor via projections from the visual cell and permitted to stabilize in the absence of output. Figure 3a shows the
Bio-mimetic path integration using a self organizing population of grid cells
7
two dimensional attractor with a packet of activity after initialization. Note that in order for the packet of activity to be stable in the absence of input, as is the characteristic of an attractor network, both the recurrent synaptic weights between the grid cells and the constants, φ0 and wIN H , that control the behaviour of the network must be set up correctly. (If this is not the case, the packet of activity will not be maintained in the absence of inputs and will instead flatten out.) 3.3
Running the system with velocity information
The firing rate of the rotation cells and the velocity cell are linear functions of the angular velocity and forward velocity respectively. The angular velocity is first integrated in the head direction cell attractor which in turn projects on to the grid cell set attractor. The rate at which the activity packet translates depends on the strength of projections on to the network: projections from the velocity set and the head direction cell set.
4
Discussion
A combination of head direction cells, border cells and grid cells can be thought to provide a complete path integrator system in biology. In spite of successful modelling of head direction and grid cells, our system is not yet optimal enough for deployment as a bio-mimetic system. The first issue to be tackled is the calibration of constants in the system that map movement in world co-ordinates. A method based on evolutionary algorithms has been employed by Kyriacou for a one dimensional head direction attractor network[7] that could be extended to our two dimensional attractor. Even when the system has been calibrated to exhibit considerable accuracy, it still suffers from the issue of drift, like any other path integrator. In order to correct drift, these neuron sets anchor to visual landmarks in biology. Implementing such a visual feature system that will project on to our attractors is a task we consider important for progress towards a complete bio-mimetic navigation system. In order to complete the bio-mimetic navigation system which the robot could use to navigate to goal locations safely, the implementation of other neuron sets such as place cells, border cells, and a reward system are also required. We set these as our future work.
5
Conclusion
In this paper, we detail the implementation of a grid cell network for bio-mimetic navigation. Our system uses self organized Hebbian synapses to train a two dimensional attractor network, the activity packet in which responds to angular velocity and speed inputs from head direction and velocity cells to encode the location of the agent. The navigation stack in the ROS platform is based on
8
Bio-mimetic path integration using a self organizing population of grid cells
classical robotic techniques due to the robustness of such systems, which has been tested and improved over time. On the other hand, computational modelling with bio-mimetic navigation as a goal is still a field in its infancy. The aim of our work is to bridge the gap between robotics and bio-mimetic navigation at a lower, neural level, rather than just at a high, behavioural one. In the process, we hope to contribute to both neuroscience and robotics with our interdisciplinary research.
References 1. Barshan, B., Durrant-Whyte, H.F.: Inertial navigation systems for mobile robots. Robotics and Automation, IEEE Transactions on 11(3), 328–342 (1995) 2. Derdikman, D., Moser, E.I.: A manifold of spatial maps in the brain. Trends in Cognitive Sciences 14(12), 561–569 (2010) 3. Gerstner, W., Kistler, W.M.: Mathematical formulations of hebbian learning. Biological Cybernetics 87(5-6), 404–415 (2002) 4. Gerstner, W., Kistler, W.M.: Spiking neuron models: Single neurons, populations, plasticity. Cambridge university press (2002) 5. Giocomo, L.M., Moser, M.B., Moser, E.I.: Computational models of grid cells. Neuron 71(4), 589–603 (2011) 6. Hafting, T., Fyhn, M., Molden, S., Moser, M.B., Moser, E.I.: Microstructure of a spatial map in the entorhinal cortex. Nature 436(7052), 801–806 (2005) 7. Kyriacou, T.: Using an evolutionary algorithm to determine the parameters of a biologically inspired model of head direction cells. Journal of computational neuroscience 32(2), 281–295 (2012) 8. McNaughton, B.L., Battaglia, F.P., Jensen, O., Moser, E.I., Moser, M.B.: Path integration and the neural basis of the’cognitive map’. Nature Reviews Neuroscience 7(8), 663–678 (2006) 9. Muller, R.U., Ranck, J.B., Taube, J.S.: Head direction cells: properties and functional significance. Current opinion in neurobiology 6(2), 196–206 (1996) 10. Muller, R.U., Stead, M., Pach, J.: The hippocampus as a cognitive graph. The Journal of general physiology 107(6), 663–694 (1996) 11. Nadel, L., MacDonald, L.: Hippocampus: Cognitive map or working memory? Behavioral and neural biology 29(3), 405–409 (1980) 12. O’Keefe, J.: Place units in the hippocampus of the freely moving rat. Experimental neurology 51(1), 78–109 (1976) 13. O’Keefe, J.: The hippocampal cognitive map and navigational strategies. (1991) 14. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA workshop on open source software. vol. 3 (2009) 15. Ranck Jr, J.: Head direction cells in the deep cell layer of dorsal presubiculum in freely moving rats. In: Society for Neuroscience Abstracts. vol. 10 (1984) 16. Stringer, S., Rolls, E., Trappenberg, T., De Araujo, I.: Self-organizing continuous attractor networks and path integration: two-dimensional models of place cells. Network: Computation in Neural Systems 13(4), 429–446 (2002) 17. Stringer, S., Trappenberg, T., Rolls, E., Araujo, I.E.T.: Self-organizing continuous attractor networks and path integration: one-dimensional models of head direction cells. Network: Computation in Neural Systems 13(2), 217–242 (2002) 18. Trullier, O., Meyer, J.A.: Animat navigation using a cognitive graph. Biological Cybernetics 83(3), 271–285 (2000)
