A model of egocentric to allocentric understanding in mammalian brains

In the mammalian brain, allocentric representations support efficient self-location and flexible navigation. A number of distinct populations of these spatial responses have been identified but no unified function has been shown to account for their emergence. Here we developed a network, trained with a simple predictive objective, that was capable of mapping egocentric information into an allocentric spatial reference frame. The prediction of visual inputs was sufficient to drive the appearance of spatial representations resembling those observed in rodents: head direction, boundary vector, and place cells, along with the recently discovered egocentric boundary cells, suggesting predictive coding as a principle for their emergence in animals. The network learned a solution for head direction tracking convergent with known biological connectivity, while suggesting a possible mechanism of boundary cell remapping. Moreover, like mammalian representations, responses were robust to environmental manipulations, including exposure to novel settings, and could be replayed in the absence of perceptual input, providing the means for offline learning. In contrast to existing reinforcement learning approaches, agents equipped with this network were able to flexibly reuse learnt behaviours - adapting rapidly to unfamiliar environments. Thus, our results indicate that these representations, derived from a simple egocentric predictive framework, form an efficient basis-set for cognitive mapping.

[1]  E. Tolman Cognitive maps in rats and men. , 1948, Psychological review.

[2]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[3]  R. Morris Spatial Localization Does Not Require the Presence of Local Cues , 1981 .

[4]  D. Zipser A computational model of hippocampal place fields. , 1985, Behavioral neuroscience.

[5]  R. Muller,et al.  The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[6]  Richard S. Sutton,et al.  Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming , 1990, ML.

[7]  R U Muller,et al.  Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  Patricia E. Sharp,et al.  Computer simulation of hippocampal place cells , 1991, Psychobiology.

[9]  M. Kramer Nonlinear principal component analysis using autoassociative neural networks , 1991 .

[10]  J O'Keefe,et al.  An allocentric spatial model for the hippocampal cognitive map , 1991, Hippocampus.

[11]  K M Gothard,et al.  Binding of hippocampal CA1 neural activity to multiple reference frames in a landmark-based navigation task , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[12]  L. F. Abbott,et al.  A Model of Spatial Map Formation in the Hippocampus of the Rat , 1999, Neural Computation.

[13]  J. O’Keefe,et al.  Geometric determinants of the place fields of hippocampal neurons , 1996, Nature.

[14]  A David Redishyx,et al.  A coupled attractor model of the rodent head direction system , 1996 .

[15]  K. Zhang,et al.  Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[17]  David S. Touretzky,et al.  Synaptic learning models of map separation in the hippocampus , 2000, Neurocomputing.

[18]  N. Kanwisher,et al.  Mental Imagery of Faces and Places Activates Corresponding Stimulus-Specific Brain Regions , 2000, Journal of Cognitive Neuroscience.

[19]  J. O’Keefe,et al.  Modeling place fields in terms of the cortical inputs to the hippocampus , 2000, Hippocampus.

[20]  H. T. Blair,et al.  The anatomical and computational basis of the rat head-direction cell signal , 2001, Trends in Neurosciences.

[21]  Gilles Dowek,et al.  What Is a Theory? , 2002, STACS.

[22]  Albert K. Lee,et al.  Memory of Sequential Experience in the Hippocampus during Slow Wave Sleep , 2002, Neuron.

[23]  K. Jeffery,et al.  Context‐specific acquisition of location discrimination by hippocampal place cells , 2003, The European journal of neuroscience.

[24]  R. Muller,et al.  Representation of Objects in Space by Two Classes of Hippocampal Pyramidal Cells , 2004, The Journal of general physiology.

[25]  H. Eichenbaum Hippocampus Cognitive Processes and Neural Representations that Underlie Declarative Memory , 2004, Neuron.

[26]  B. McNaughton,et al.  Independent Codes for Spatial and Episodic Memory in Hippocampal Neuronal Ensembles , 2005, Science.

[27]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[28]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[29]  D. Kumaran,et al.  An Unexpected Sequence of Events: Mismatch Detection in the Human Hippocampus , 2006, PLoS biology.

[30]  G. Einevoll,et al.  From grid cells to place cells: A mathematical model , 2006, Hippocampus.

[31]  K. Jeffery,et al.  The Boundary Vector Cell Model of Place Cell Firing and Spatial Memory , 2006, Reviews in the neurosciences.

[32]  K. Jeffery,et al.  Experience-dependent rescaling of entorhinal grids , 2007, Nature Neuroscience.

[33]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[34]  Laurenz Wiskott,et al.  Slowness and Sparseness Lead to Place, Head-Direction, and Spatial-View Cells , 2007, PLoS Comput. Biol..

[35]  S. Becker,et al.  Remembering the past and imagining the future: a neural model of spatial memory and imagery. , 2007, Psychological review.

[36]  M. Moser,et al.  Representation of Geometric Borders in the Entorhinal Cortex , 2008, Science.

[37]  A D Redish,et al.  Prediction, sequences and the hippocampus , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[38]  J. O’Keefe,et al.  Boundary Vector Cells in the Subiculum of the Hippocampal Formation , 2009, The Journal of Neuroscience.

[39]  Karl J. Friston,et al.  Predictive coding under the free-energy principle , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[40]  Demis Hassabis,et al.  The construction system of the brain , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[41]  Michael E. Hasselmo,et al.  Modeling Boundary Vector Cell Firing Given Optic Flow as a Cue , 2012, PLoS Comput. Biol..

[42]  Brad E. Pfeiffer,et al.  Hippocampal place cell sequences depict future paths to remembered goals , 2013, Nature.

[43]  David S. Greenberg,et al.  Rats maintain an overhead binocular field at the expense of constant fusion , 2013, Nature.

[44]  Mark P. Brandon,et al.  New and Distinct Hippocampal Place Codes Are Generated in a New Environment during Septal Inactivation , 2014, Neuron.

[45]  K. Jeffery,et al.  Weighted cue integration in the rodent head direction system , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[46]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[47]  David C Rowland,et al.  Place cells, grid cells, and memory. , 2015, Cold Spring Harbor perspectives in biology.

[48]  Johannes D. Seelig,et al.  Neural dynamics for landmark orientation and angular path integration , 2015, Nature.

[49]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[50]  David J. Foster,et al.  Memory and Space: Towards an Understanding of the Cognitive Map , 2015, The Journal of Neuroscience.

[51]  Ron Meir,et al.  Extracting grid cell characteristics from place cell inputs using non-negative principal component analysis , 2016, eLife.

[52]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[53]  Kimberly L. Stachenfeld,et al.  The hippocampus as a predictive map , 2017, Nature Neuroscience.

[54]  Francesca Cacucci,et al.  The development of spatial and memory circuits in the rat. , 2017, Wiley interdisciplinary reviews. Cognitive science.

[55]  V. Jayaraman,et al.  Ring attractor dynamics in the Drosophila central brain , 2017, Science.

[56]  Neil Burgess,et al.  A neural-level model of spatial memory and imagery , 2018, eLife.

[57]  Shane Legg,et al.  IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures , 2018, ICML.

[58]  Timothy Edward John Behrens,et al.  Generalisation of structural knowledge in the Hippocampal-Entorhinal system , 2018, NeurIPS.

[59]  Xue-Xin Wei,et al.  Emergence of grid-like representations by training recurrent neural networks to perform spatial localization , 2018, ICLR.

[60]  Sachin S. Deshmukh,et al.  Egocentric coding of external items in the lateral entorhinal cortex , 2018, Science.

[61]  Razvan Pascanu,et al.  Vector-based navigation using grid-like representations in artificial agents , 2018, Nature.

[62]  Sang Ah Lee,et al.  Vector Trace cells in the Subiculum of the Hippocampal formation , 2019, Nature Neuroscience.

[63]  M. Hasselmo,et al.  Neuronal representation of environmental boundaries in egocentric coordinates , 2019, Nature Communications.

[64]  Michael E. Hasselmo,et al.  Egocentric boundary vector tuning of the retrosplenial cortex , 2019, Science Advances.

[65]  Rachel I. Wilson,et al.  Sensorimotor experience remaps visual input to a heading-direction network , 2019, Nature.

[66]  Michael E Hasselmo,et al.  Egocentric boundary vector tuning of the retrosplenial cortex , 2020, Science Advances.

[67]  Christopher J. Cueva,et al.  Emergence of functional and structural properties of the head direction system by optimization of recurrent neural networks , 2019, ICLR.

[68]  Michael C. Frank,et al.  Unsupervised neural network models of the ventral visual stream , 2020, Proceedings of the National Academy of Sciences.

[69]  J. S. Guntupalli,et al.  Clone-structured graph representations enable flexible learning and vicarious evaluation of cognitive maps , 2021, Nature Communications.