A Computational Model for a Multi-Goal Spatial Navigation Task inspired by Rodent Studies

We present a biologically-inspired computational model of the rodent hippocampus based on recent studies of the hippocampus showing that its longitudinal axis is involved in complex spatial navigation. While both poles of the hippocampus, i.e. septal (dorsal) and temporal (ventral), encode spatial information; the septal area has traditionally been attributed more to navigation and action selection; whereas the temporal pole has been more involved with learning and motivation. In this work we hypothesize that the septal-temporal organization of the hippocampus axis also provides a multi-scale spatial representation that may be exploited during complex rodent navigation. To test this hypothesis, we developed a multi-scale model of the hippocampus evaluated it with a simulated rat on a multi-goal task, initially in a simplified environment, and then on a more complex environment where multiple obstacles are introduced. In addition to the hippocampus providing a spatial representation of the environment, the model includes an actor-critic framework for the motivated learning of the different tasks.

[1]  J. O’Keefe,et al.  Boundary Vector Cells in the Subiculum of the Hippocampal Formation , 2009, The Journal of Neuroscience.

[2]  R. Passingham The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[3]  Martin Llofriu,et al.  Goal-oriented robot navigation learning using a multi-scale space representation , 2015, Neural Networks.

[4]  M. Fyhn,et al.  Progressive increase in grid scale from dorsal to ventral medial entorhinal cortex , 2008, Hippocampus.

[5]  Ricardo Chavarriaga,et al.  Path planning versus cue responding: a bio-inspired model of switching between navigation strategies , 2010, Biological Cybernetics.

[6]  Angelo Arleo,et al.  Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity , 2000, Biological Cybernetics.

[7]  Jean-Marc Fellous,et al.  Remaking memories: reconsolidation updates positively motivated spatial memory in rats. , 2012, Learning & memory.

[8]  Philippe Gaussier,et al.  From view cells and place cells to cognitive map learning: processing stages of the hippocampal system , 2002, Biological Cybernetics.

[9]  Alejandra Barrera,et al.  Biologically-inspired robot spatial cognition based on rat neurophysiological studies , 2008, Auton. Robots.

[10]  B. McNaughton,et al.  Comparison of spatial firing characteristics of units in dorsal and ventral hippocampus of the rat , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[11]  R. Muller,et al.  Representation of Objects in Space by Two Classes of Hippocampal Pyramidal Cells , 2004, The Journal of general physiology.

[12]  Naomi S. Altman,et al.  Points of Significance: Visualizing samples with box plots , 2014, Nature Methods.

[13]  Gordon Wyeth,et al.  Persistent Navigation and Mapping using a Biologically Inspired SLAM System , 2010, Int. J. Robotics Res..

[14]  Kae Nakamura,et al.  Predictive Reward Signal of Dopamine Neurons , 2015 .

[15]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[16]  B. Balleine,et al.  Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action , 2010, Neuropsychopharmacology.

[17]  R. Muller,et al.  The effects of changes in the environment on the spatial firing of hippocampal complex-spike cells , 1987, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[18]  Uğur M Erdem,et al.  A goal‐directed spatial navigation model using forward trajectory planning based on grid cells , 2012, The European journal of neuroscience.

[19]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[20]  E. Tolman Cognitive maps in rats and men. , 1948, Psychological review.

[21]  R U Muller,et al.  Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[23]  K Caluwaerts,et al.  A biologically inspired meta-control navigation system for the Psikharpax rat robot , 2012, Bioinspiration & biomimetics.

[24]  Jean-Marc Fellous,et al.  A Role for the Longitudinal Axis of the Hippocampus in Multiscale Representations of Large and Complex Spatial Environments and Mnemonic Hierarchies , 2017, The Hippocampus - Plasticity and Functions.

[25]  David N. Lyttle,et al.  Spatial scale and place field stability in a grid‐to‐place cell model of the dorsoventral axis of the hippocampus , 2013, Hippocampus.

[26]  Michael I. Jordan,et al.  Reinforcement Learning with Soft State Aggregation , 1994, NIPS.

[27]  Michael A. Arbib,et al.  Affordances. Motivations, and the World Graph Theory , 1998, Adapt. Behav..

[28]  Martin Llofriu,et al.  The ventral hippocampus is involved in multi‐goal obstacle‐rich spatial navigation , 2018, Hippocampus.

[29]  Jean-Arcady Meyer,et al.  Global localization and topological map-learning for robot navigation , 2002 .

[30]  A. Redish Beyond the Cognitive Map: From Place Cells to Episodic Memory , 1999 .

[31]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.