A computational model for spatial cognition combining dorsal and ventral hippocampal place field maps: multiscale navigation

Classic studies have shown that place cells are organized along the dorsoventral axis of the hippocampus according to their field size, with dorsal hippocampal place cells having smaller field sizes than ventral place cells. Studies have also suggested that dorsal place cells are primarily involved in spatial navigation, while ventral place cells are primarily involved in context and emotional encoding. Additionally, recent work has shown that the entire longitudinal axis of the hippocampus may be involved in navigation. Based on the latter, in this paper we present a spatial cognition reinforcement learning model inspired by the multiscale organization of the dorsal–ventral axis of the hippocampus. The model analyzes possible benefits of a multiscale architecture in terms of the learning speed, the path optimality, and the number of cells in the context of spatial navigation. The model is evaluated in a goal-oriented task where simulated rats need to learn a path to the goal from multiple starting locations in various open-field maze configurations. The results show that smaller scales of representation are useful for improving path optimality, whereas larger scales are useful for reducing learning time and the number of cells required. The results also show that combining scales can enhance the performance of the multiscale model, with a trade-off between path optimality and learning time.

[1]  G. Kempermann,et al.  An old test for new neurons: refining the Morris water maze to study the functional relevance of adult hippocampal neurogenesis , 2013, Front. Neurosci..

[2]  Richard G M Morris,et al.  Longitudinal axis of the hippocampus: Both septal and temporal poles of the hippocampus support water maze spatial learning depending on the training protocol , 2003, Hippocampus.

[3]  Jean-Marc Fellous,et al.  A Role for the Longitudinal Axis of the Hippocampus in Multiscale Representations of Large and Complex Spatial Environments and Mnemonic Hierarchies , 2017, The Hippocampus - Plasticity and Functions.

[4]  Naomi S. Altman,et al.  Points of Significance: Visualizing samples with box plots , 2014, Nature Methods.

[5]  J. O’Keefe,et al.  Geometric determinants of the place fields of hippocampal neurons , 1996, Nature.

[6]  Martin Llofriu,et al.  The ventral hippocampus is involved in multi‐goal obstacle‐rich spatial navigation , 2018, Hippocampus.

[7]  R. Muller,et al.  The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  Uğur M Erdem,et al.  A goal‐directed spatial navigation model using forward trajectory planning based on grid cells , 2012, The European journal of neuroscience.

[9]  W E Skaggs,et al.  Deciphering the hippocampal polyglot: the hippocampus as a path integration system. , 1996, The Journal of experimental biology.

[10]  L. Frank,et al.  Awake Hippocampal Sharp-Wave Ripples Support Spatial Memory , 2012, Science.

[11]  J. Knierim,et al.  Influence of boundary removal on the spatial representations of the medial entorhinal cortex , 2008, Hippocampus.

[12]  B. McNaughton,et al.  Comparison of spatial firing characteristics of units in dorsal and ventral hippocampus of the rat , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[13]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[14]  C. Vorhees,et al.  Morris water maze: procedures for assessing spatial and related forms of learning and memory , 2006, Nature Protocols.

[15]  Hong-wei Dong,et al.  Are the Dorsal and Ventral Hippocampus Functionally Distinct Structures? , 2010, Neuron.

[16]  E. Lein,et al.  Functional organization of the hippocampal longitudinal axis , 2014, Nature Reviews Neuroscience.

[17]  Martin Llofriu,et al.  A Computational Model for a Multi-Goal Spatial Navigation Task inspired by Rodent Studies , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[18]  Vincenzo Piuri,et al.  Automatic multiscale meshing through HRBF networks , 2005, IEEE Transactions on Instrumentation and Measurement.

[19]  Adam Johnson,et al.  Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point , 2007, The Journal of Neuroscience.

[20]  B. McNaughton,et al.  Self‐motion and the origin of differential spatial scaling along the septo‐temporal axis of the hippocampus , 2005, Hippocampus.

[21]  Charles W. Anderson,et al.  Comparison of CMACs and radial basis functions for local function approximators in reinforcement learning , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[22]  Joachim Pouderoux,et al.  Adaptive hierarchical RBF interpolation for creating smooth digital elevation models , 2004, GIS '04.

[23]  L. Nadel,et al.  The Hippocampus as a Cognitive Map , 1978 .

[24]  Tomás Lozano-Pérez,et al.  An algorithm for planning collision-free paths among polyhedral obstacles , 1979, CACM.

[25]  I S Curthoys,et al.  Head impulse test in unilateral vestibular loss , 2008, Neurology.

[26]  R. Passingham The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[27]  M. Fyhn,et al.  Spatial Representation in the Entorhinal Cortex , 2004, Science.

[28]  R U Muller,et al.  Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  Hallvard Røe Evensmoen,et al.  Long-axis specialization of the human hippocampus , 2013, Trends in Cognitive Sciences.

[30]  Brad E. Pfeiffer,et al.  Hippocampal place cell sequences depict future paths to remembered goals , 2013, Nature.

[31]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[32]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[33]  T. Hafting,et al.  Finite Scale of Spatial Representation in the Hippocampus , 2008, Science.

[34]  Edvard I. Moser,et al.  Object-vector coding in the medial entorhinal cortex , 2018, bioRxiv.

[35]  J. O’Keefe,et al.  Boundary Vector Cells in the Subiculum of the Hippocampal Formation , 2009, The Journal of Neuroscience.

[36]  Michael Milford,et al.  Biologically-inspired visual place recognition with adaptive multiple scales , 2017, Robotics Auton. Syst..

[37]  Thilo Womelsdorf,et al.  Sharp Wave Ripples during Visual Exploration in the Primate Hippocampus , 2015, The Journal of Neuroscience.

[38]  Martin Llofriu,et al.  Goal-oriented robot navigation learning using a multi-scale space representation , 2015, Neural Networks.

[39]  Vincenzo Piuri,et al.  Online training of Hierarchical RBF , 2007, 2007 International Joint Conference on Neural Networks.

[40]  M. Moser,et al.  Representation of Geometric Borders in the Entorhinal Cortex , 2008, Science.

[41]  R. Muller,et al.  Representation of Objects in Space by Two Classes of Hippocampal Pyramidal Cells , 2004, The Journal of general physiology.

[42]  M. Hasselmo,et al.  A biologically inspired hierarchical goal directed navigation model , 2014, Journal of Physiology-Paris.

[43]  David N. Lyttle,et al.  Spatial scale and place field stability in a grid‐to‐place cell model of the dorsoventral axis of the hippocampus , 2013, Hippocampus.

[44]  Michael A. Arbib,et al.  Affordances. Motivations, and the World Graph Theory , 1998, Adapt. Behav..

[45]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[46]  J. Lisman,et al.  The Input–Output Transformation of the Hippocampal Granule Cells: From Grid Cells to Place Fields , 2009, The Journal of Neuroscience.

[47]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[48]  E. J. Green,et al.  Head-direction cells in the rat posterior cortex , 1994, Experimental Brain Research.