A hippocampo-cerebellar centred network for the learning and execution of sequence-based navigation

How do we translate self-motion into goal-directed actions? Here we investigate the cognitive architecture underlying self-motion processing during exploration and goal-directed behaviour. The task, performed in an environment with limited and ambiguous external landmarks, constrained mice to use self-motion based information for sequence-based navigation. The post-behavioural analysis combined brain network characterization based on c-Fos imaging and graph theory analysis as well as computational modelling of the learning process. The study revealed a widespread network centred around the cerebral cortex and basal ganglia during the exploration phase, while a network dominated by hippocampal and cerebellar activity appeared to sustain sequence-based navigation. The learning process could be modelled by an algorithm combining memory of past actions and model-free reinforcement learning, which parameters pointed toward a central role of hippocampal and cerebellar structures for learning to translate self-motion into a sequence of goal-directed actions.

[1]  R. Heath,et al.  Ascending projections of the cerebellar fastigial nucleus to the hippocampus, amygdala, and other temporal lobe sites: evoked potential and histological studies in monkeys and cats. , 1974, Experimental neurology.

[2]  D. Durstewitz,et al.  Abrupt Transitions between Prefrontal Neural Ensemble States Accompany Behavioral Transitions during Rule Learning , 2010, Neuron.

[3]  Makoto Ito,et al.  Model-based action planning involves cortico-cerebellar and basal ganglia networks , 2016, Scientific Reports.

[4]  Edward J. Sondik,et al.  The optimal control of par-tially observable Markov processes , 1971 .

[5]  Anton J. Enright,et al.  An efficient algorithm for large-scale detection of protein families. , 2002, Nucleic acids research.

[6]  Jacqueline N. Crawley,et al.  What's Wrong With My Mouse?: Behavioral Phenotyping of Transgenic and Knockout Mice , 2000 .

[7]  James J Knierim,et al.  Conflicts between Local and Global Spatial Frameworks Dissociate Neural Representations of the Lateral and Medial Entorhinal Cortex , 2013, The Journal of Neuroscience.

[8]  B. Bontempi,et al.  Complementary Roles of the Hippocampus and the Dorsomedial Striatum during Spatial and Sequence-Based Navigation Behavior , 2013, PloS one.

[9]  E. Koechlin,et al.  The Importance of Falsification in Computational Cognitive Modeling , 2017, Trends in Cognitive Sciences.

[10]  M. Petrides,et al.  Cognitive Strategies Dependent on the Hippocampus and Caudate Nucleus in Human Navigation: Variability and Change with Practice , 2003, The Journal of Neuroscience.

[11]  J. Wallis.  The Dynamics of Learning and Behavioral Flexibility , 2011, Neuron.

[12]  K. Doya.  Complementary roles of basal ganglia and cerebellum in learning and motor control , 2000, Current Opinion in Neurobiology.

[13]  E. Save,et al.  Coding for spatial goals in the prelimbic/infralimbic area of the rat frontal cortex. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[14]  M. Quirk,et al.  Requirement for Hippocampal CA3 NMDA Receptors in Associative Memory Recall , 2002, Science.

[15]  P. Dayan,et al.  Goals and Habits in the Brain , 2013, Neuron.

[16]  George Paxinos,et al.  The Mouse Brain in Stereotaxic Coordinates , 2001 .

[17]  Jean-Arcady Meyer,et al.  BIOLOGICALLY BASED ARTIFICIAL NAVIGATION SYSTEMS: REVIEW AND PROSPECTS , 1997, Progress in Neurobiology.

[18]  Matthijs A. A. van der Meer,et al.  Information Processing in Decision-Making Systems , 2012, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[19]  B. J. Clark,et al.  Interaction of Egocentric and World-Centered Reference Frames in the Rat Posterior Parietal Cortex , 2014, The Journal of Neuroscience.

[20]  M. Shapiro,et al.  Representing episodes in the mammalian brain , 2006, Current Opinion in Neurobiology.

[21]  L. Rondi-Reig,et al.  The cerebellum: a new key structure in the navigation system , 2013, Front. Neural Circuits.

[22]  S. Heinrichs.  :What’s Wrong With My Mouse?: Behavioral Phenotyping of Transgenic and Knockout Mice. Second Edition. , 2007 .

[23]  M. Roesch,et al.  Ventral Striatal Neurons Encode the Value of the Chosen Action in Rats Deciding between Differently Delayed or Sized Rewards , 2009, The Journal of Neuroscience.

[24]  Eric A. Zilli,et al.  Modeling the role of working memory and episodic memory in behavioral tasks , 2008, Hippocampus.

[25]  P. Strick,et al.  Cerebellum and nonmotor function. , 2009, Annual review of neuroscience.

[26]  J. Aggleton,et al.  Qualitatively Different Hippocampal Subfield Engagement Emerges with Mastery of a Spatial Memory Task by Rats , 2008, The Journal of Neuroscience.

[27]  W. Tischmeyer,et al.  Activation of immediate early genes and memory formation , 1999, Cellular and Molecular Life Sciences CMLS.

[28]  Dora E. Angelaki,et al.  Computation of Egomotion in the Macaque Cerebellar Vermis , 2010, The Cerebellum.

[29]  M. Khamassi,et al.  Integrating cortico-limbic-basal ganglia architectures for learning model-based and model-free navigation strategies , 2012, Front. Behav. Neurosci..

[30]  E. Maguire,et al.  What does the retrosplenial cortex do? , 2009, Nature Reviews Neuroscience.

[31]  E. J. Green,et al.  Head-direction cells in the rat posterior cortex , 1994, Experimental Brain Research.

[32]  Douglas A Nitz,et al.  Anterior cingulate neurons in the rat map anticipated effort and reward to their associated action sequences. , 2012, Journal of neurophysiology.

[33]  Benjamin J. Kraus,et al.  Hippocampal “Time Cells”: Time versus Path Integration , 2013, Neuron.

[34]  Julien Doyon,et al.  The multifaceted nature of the relationship between performance and brain activity in motor sequence learning , 2010, NeuroImage.

[35]  Angelo Arleo,et al.  A Navigation Analysis Tool (NAT) to assess spatial behavior in open-field and structured mazes , 2013, Journal of Neuroscience Methods.

[36]  L. Rondi-Reig,et al.  A new approach for modeling episodic memory from rodents to humans: The temporal order memory , 2010, Behavioural Brain Research.

[37]  Christian F. Doeller,et al.  Interaction Between Hippocampus and Cerebellum Crus I in Sequence-Based but not Place-Based Navigation. , 2015, Cerebral cortex.

[38]  Nathaniel D. Daw,et al.  Trial-by-trial data analysis using computational models , 2011 .

[39]  D. Marr.  A theory of cerebellar cortex , 1969, The Journal of physiology.

[40]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[41]  S. Romani,et al.  Theta sequences are essential for internally generated hippocampal firing fields , 2014, Nature Neuroscience.

[42]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[43]  John A. King,et al.  Memory for events and their spatial context: models and experiments. , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[44]  N. Mons,et al.  Activating a memory system focuses connectivity toward its central structure , 2009, Behavioural Brain Research.

[45]  Adam Johnson,et al.  Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point , 2007, The Journal of Neuroscience.

[46]  Stéphane Doncieux,et al.  Encouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study , 2012, Evolutionary Computation.

[47]  R. Passingham.  The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[48]  Laure Rondi-Reig,et al.  How the cerebellum may monitor sensory information for spatial representation , 2014, Front. Syst. Neurosci..

[49]  D. Nitz.  Tracking Route Progression in the Posterior Parietal Cortex , 2006, Neuron.

[50]  Raymond P. Kesner,et al.  A process analysis of the CA3 subregion of the hippocampus , 2013, Front. Cell. Neurosci..

[51]  Christian F. Doeller,et al.  Lateralized human hippocampal activity predicts navigation based on sequence or place memory , 2010, Proceedings of the National Academy of Sciences.

[52]  David J. Foster,et al.  A model of hippocampally dependent navigation, using the temporal difference learning rule , 2000, Hippocampus.

[53]  E. Maguire,et al.  The Well-Worn Route and the Path Less Traveled Distinct Neural Bases of Route Following and Wayfinding in Humans , 2003, Neuron.

[54]  L. Luo,et al.  Cerebellar granule cells encode the expectation of reward , 2017, Nature.

[55]  Alain Berthoz,et al.  Impaired Sequential Egocentric and Allocentric Memories in Forebrain-Specific–NMDA Receptor Knock-Out Mice during a New Task Dissociating Strategies of Navigation , 2006, The Journal of Neuroscience.

[56]  Guillén Fernández,et al.  Interaction between the Human Hippocampus and the Caudate Nucleus during Route Recognition , 2004, Neuron.

[57]  L. Rondi-Reig,et al.  Early detection of age-related memory deficits in individual mice , 2011, Neurobiology of Aging.

[58]  Gerit Pfuhl,et al.  Functional Split between Parietal and Entorhinal Cortices in the Rat , 2012, Neuron.

[59]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[60]  Richard Apps,et al.  Cerebellar cortical organization: a one-map hypothesis , 2009, Nature Reviews Neuroscience.

[61]  Rosa Cossart,et al.  Internally Recurring Hippocampal Sequences as a Population Template of Spatiotemporal Information , 2015, Neuron.

[62]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[63]  Natasa Kovacevic,et al.  Identification of a Functional Connectome for Long-Term Fear Memory in Mice , 2013, PLoS Comput. Biol..

[64]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[65]  D. R. Euston,et al.  The Role of Medial Prefrontal Cortex in Memory and Decision Making , 2012, Neuron.

[66]  G. Anastasi,et al.  Constrained spherical deconvolution analysis of the limbic network in human, with emphasis on a direct cerebello-limbic pathway , 2014, Front. Hum. Neurosci..

[67]  Matthijs A. A. van der Meer,et al.  Internally generated sequences in learning and executing goal-directed behavior , 2014, Trends in Cognitive Sciences.

[68]  Brad E. Pfeiffer,et al.  Hippocampal place cell sequences depict future paths to remembered goals , 2013, Nature.

[69]  T. Robbins,et al.  The hippocampal–striatal axis in learning, prediction and goal-directed behavior , 2011, Trends in Neurosciences.

[70]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[71]  A. Berthoz,et al.  Reference Frames for Spatial Cognition: Different Brain Areas are Involved in Viewer-, Object-, and Landmark-Centered Judgments About Object Location , 2004, Journal of Cognitive Neuroscience.

[72]  G. Buzsáki,et al.  Forward and reverse hippocampal place-cell sequences during ripples , 2007, Nature Neuroscience.

[73]  George Paxinos,et al.  The Mouse Brain in Stereotaxic Coordinates, The coronal plates and diagrams Compact, 3rd Edition , 2008 .

[74]  Angelo Arleo,et al.  Multimodal sensory integration and concurrent navigation strategies for spatial cognition in real and artificial organisms. , 2007, Journal of integrative neuroscience.

[75]  Lucia M. Talamini,et al.  Recall of memory sequences by interaction of the dentate and CA3: A revised model of the phase precession , 2005, Neural Networks.

[76]  H. Eichenbaum,et al.  Hippocampal “Time Cells” Bridge the Gap in Memory for Discontiguous Events , 2011, Neuron.

[77]  Stéphane Doncieux,et al.  Sferesv2: Evolvin' in the multi-core world , 2010, IEEE Congress on Evolutionary Computation.

[78]  Laure Rondi-Reig,et al.  Temporal Order Memory Assessed during Spatiotemporal Navigation As a Behavioral Cognitive Marker for Differential Alzheimer's Disease Diagnosis , 2012, The Journal of Neuroscience.

[79]  A. Redish.  Beyond the Cognitive Map: From Place Cells to Episodic Memory , 1999 .

[80]  S. Becker,et al.  Remembering the past and imagining the future: a neural model of spatial memory and imagery. , 2007, Psychological review.

[81]  Olaf Sporns,et al.  Complex network measures of brain connectivity: Uses and interpretations , 2010, NeuroImage.

[82]  D. Angelaki,et al.  Purkinje Cells in Posterior Cerebellar Vermis Encode Motion in an Inertial Reference Frame , 2007, Neuron.

[83]  Douglas A Nitz,et al.  Spaces within spaces: rat parietal cortex neurons register position across three reference frames , 2012, Nature Neuroscience.

[84]  Martin Vinck,et al.  Oscillatory Dynamics and Place Field Maps Reflect Hippocampal Ensemble Processing of Sequence and Place Memory under NMDA Receptor Control , 2014, Neuron.

[85]  Shogo Ohmae,et al.  Climbing fibers encode a temporal-difference prediction error during cerebellar learning in mice , 2015, Nature Neuroscience.

[86]  Bruce L. McNaughton,et al.  Path integration and the neural basis of the 'cognitive map' , 2006, Nature Reviews Neuroscience.

[87]  Seralynne D Vann,et al.  Granular and dysgranular retrosplenial cortices provide qualitatively different contributions to spatial working memory: evidence from immediate‐early gene imaging in rats , 2009, The European journal of neuroscience.

[88]  T. Robbins,et al.  Putting a spin on the dorsal–ventral divide of the striatum , 2004, Trends in Neurosciences.