Predictive Place-Cell Sequences for Goal-Finding Emerge from Goal Memory and the Cognitive Map: A Computational Model

Hippocampal place-cell sequences observed during awake immobility often represent previous experience, suggesting a role in memory processes. However, recent reports of goals being overrepresented in sequential activity suggest a role in short-term planning, although a detailed understanding of the origins of hippocampal sequential activity and of its functional role is still lacking. In particular, it is unknown which mechanism could support efficient planning by generating place-cell sequences biased toward known goal locations, in an adaptive and constructive fashion. To address these questions, we propose a model of spatial learning and sequence generation as interdependent processes, integrating cortical contextual coding, synaptic plasticity and neuromodulatory mechanisms into a map-based approach. Following goal learning, sequential activity emerges from continuous attractor network dynamics biased by goal memory inputs. We apply Bayesian decoding on the resulting spike trains, allowing a direct comparison with experimental data. Simulations show that this model (1) explains the generation of never-experienced sequence trajectories in familiar environments, without requiring virtual self-motion signals, (2) accounts for the bias in place-cell sequences toward goal locations, (3) highlights their utility in flexible route planning, and (4) provides specific testable predictions.

[1]  M. Wilson,et al.  Dentate Gyrus NMDA Receptors Mediate Rapid Pattern Separation in the Hippocampal Network , 2007, Science.

[2]  D. Hassabis,et al.  Hippocampal place cells construct reward related sequences through unexplored space , 2015, eLife.

[3]  Sean M Montgomery,et al.  Relationships between Hippocampal Sharp Waves, Ripples, and Fast Gamma Oscillation: Influence of Dentate and Entorhinal Cortical Activity , 2011, The Journal of Neuroscience.

[4]  B. McNaughton,et al.  Reactivation of Hippocampal Cell Assemblies: Effects of Behavioral State, Experience, and EEG Dynamics , 1999, The Journal of Neuroscience.

[5]  H. Yamahachi,et al.  Hippocampal Remapping after Partial Inactivation of the Medial Entorhinal Cortex , 2015, Neuron.

[6]  Julien Vitay,et al.  Frontiers in Computational Neuroscience Computational Neuroscience , 2022 .

[7]  H. Eichenbaum,et al.  Memory for the Order of Events in Specific Sequences: Contributions of the Hippocampus and Medial Prefrontal Cortex , 2011, The Journal of Neuroscience.

[8]  Anoopum S. Gupta Behavioral Correlates of Hippocampal Neural Sequences , 2011 .

[9]  J. Knierim,et al.  Attractor dynamics of spatially correlated neural activity in the limbic system. , 2012, Annual review of neuroscience.

[10]  Gyorgy Csizmadia,et al.  Storage of the Distance between Place Cell Firing Fields in the Strength of Plastic Synapses with a Novel Learning Rule , 2008 .

[11]  R. Morris,et al.  Delay‐dependent impairment of a matching‐to‐place task with chronic and intrahippocampal infusion of the NMDA‐antagonist D‐AP5 , 1999, Hippocampus.

[12]  M. Quirk,et al.  Experience-Dependent Asymmetric Shape of Hippocampal Receptive Fields , 2000, Neuron.

[13]  B. McNaughton,et al.  Spatial selectivity of unit activity in the hippocampal granular layer , 1993, Hippocampus.

[14]  H. Eichenbaum,et al.  Interplay of Hippocampus and Prefrontal Cortex in Memory , 2013, Current Biology.

[15]  Uğur M Erdem,et al.  A goal‐directed spatial navigation model using forward trajectory planning based on grid cells , 2012, The European journal of neuroscience.

[16]  M. Moser,et al.  A prefrontal–thalamo–hippocampal circuit for goal-directed spatial navigation , 2015, Nature.

[17]  R. Morris,et al.  Locus coeruleus and dopaminergic consolidation of everyday memory , 2016, Nature.

[18]  Brad E. Pfeiffer,et al.  Autoassociative dynamics in the generation of sequences of hippocampal place cells , 2015, Science.

[19]  D. Manahan‐Vaughan,et al.  Hippocampal long‐term potentiation that is elicited by perforant path stimulation or that occurs in conjunction with spatial learning is tightly controlled by beta‐adrenoreceptors and the locus coeruleus , 2015, Hippocampus.

[20]  W. Gerstner,et al.  Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity , 2006, The Journal of Neuroscience.

[21]  Laurenz Wiskott,et al.  A computational model for preplay in the hippocampus , 2013, Front. Comput. Neurosci..

[22]  Alexander Mathis,et al.  Connecting multiple spatial scales to decode the population activity of grid cells , 2015, Science Advances.

[23]  B L McNaughton,et al.  Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model , 1997, The Journal of Neuroscience.

[24]  David J. Foster,et al.  Reverse replay of behavioural sequences in hippocampal place cells during the awake state , 2006, Nature.

[25]  M. Kahana Associative retrieval processes in free recall , 1996, Memory & cognition.

[26]  Neil Burgess,et al.  Forward and Backward Inference in Spatial Cognition , 2013, PLoS Comput. Biol..

[27]  G. Buzsáki Two-stage model of memory trace formation: A role for “noisy” brain states , 1989, Neuroscience.

[28]  R. S. Jones The NMDA receptor Edited by J. C. Watkins and G. L. Collinridge. Oxford University Press, New York (1990) 242 pp. £45.00 , 1991, Neuroscience.

[29]  B. McNaughton,et al.  Replay of Neuronal Firing Sequences in Rat Hippocampus During Sleep Following Spatial Experience , 1996, Science.

[30]  D. R. Euston,et al.  The Role of Medial Prefrontal Cortex in Memory and Decision Making , 2012, Neuron.

[31]  Michael L. Waskom,et al.  Frontoparietal Representations of Task Context Support the Flexible Control of Goal-Directed Cognition , 2014, The Journal of Neuroscience.

[32]  Eran Stark,et al.  Excitation and Inhibition Compete to Control Spiking during Hippocampal Ripples: Intracellular Study in Behaving Mice , 2014, The Journal of Neuroscience.

[33]  M. Moser,et al.  Pattern Separation in the Dentate Gyrus and CA3 of the Hippocampus , 2007, Science.

[34]  W B Levy,et al.  A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal‐like tasks , 1996, Hippocampus.

[35]  S. Becker,et al.  Remembering the past and imagining the future: a neural model of spatial memory and imagery. , 2007, Psychological review.

[36]  W. Gerstner,et al.  Connectivity reflects coding: a model of voltage-based STDP with homeostasis , 2010, Nature Neuroscience.

[37]  C. Leibold A Trick for Computing Expected Values in High-Dimensional Probabilistic Models , 2011, Network.

[38]  Margaret F. Carr,et al.  Transient Slow Gamma Synchrony Underlies Hippocampal Memory Replay , 2012, Neuron.

[39]  Carol A Barnes,et al.  Impaired hippocampal rate coding after lesions of the lateral entorhinal cortex , 2013, Nature Neuroscience.

[40]  J. O’Neill,et al.  The reorganization and reactivation of hippocampal maps predict spatial memory performance , 2010, Nature Neuroscience.

[41]  M. Cammarota,et al.  Inactivation of the dorsal hippocampus or the medial prefrontal cortex impairs retrieval but has differential effect on spatial memory reconsolidation , 2015, Neurobiology of Learning and Memory.

[42]  Joshua P. Neunuebel,et al.  Spatial Firing Correlates of Physiologically Distinct Cell Types of the Rat Dentate Gyrus , 2012, The Journal of Neuroscience.

[43]  Michael Recce,et al.  A model of hippocampal function , 1994, Neural Networks.

[44]  S. Romani,et al.  Short‐term plasticity based network model of place cells dynamics , 2015, Hippocampus.

[45]  D. Paré,et al.  Ultrastructural organization of medial prefrontal inputs to the rhinal cortices , 2006, The European journal of neuroscience.

[46]  David J. Foster,et al.  A model of hippocampally dependent navigation, using the temporal difference learning rule , 2000, Hippocampus.

[47]  Menno P. Witter,et al.  Place Cells and Place Recognition Maintained by Direct Entorhinal-Hippocampal Circuitry , 2002, Science.

[48]  Richard Kempter,et al.  Memory replay in balanced recurrent networks , 2016, bioRxiv.

[49]  P. J. Sjöström,et al.  Rate, Timing, and Cooperativity Jointly Determine Cortical Synaptic Plasticity , 2001, Neuron.

[50]  J. A. Dani,et al.  Dopamine D1 and D5 Receptors Modulate Spike Timing-Dependent Plasticity at Medial Perforant Path to Dentate Granule Cell Synapses , 2014, The Journal of Neuroscience.

[51]  G. Buzsáki,et al.  Selective suppression of hippocampal ripples impairs spatial memory , 2009, Nature Neuroscience.

[52]  Xiao-Jing Wang,et al.  Angular Path Integration by Moving “Hill of Activity”: A Spiking Neuron Model without Recurrent Excitation of the Head-Direction System , 2005, The Journal of Neuroscience.

[53]  A. Sirota,et al.  The hippocampus: hub of brain network communication for memory , 2011, Trends in Cognitive Sciences.

[54]  Thomas Straube,et al.  Requirement of β‐adrenergic receptor activation and protein synthesis for LTP‐reinforcement by novelty in rat dentate gyrus , 2003 .

[55]  Laura A. Atherton,et al.  Memory trace replay: the shaping of memory consolidation by neuromodulation , 2015, Trends in Neurosciences.

[56]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Margaret F. Carr,et al.  Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval , 2011, Nature Neuroscience.

[58]  RU Muller,et al.  The hippocampus as a cognitive graph , 1996, The Journal of general physiology.

[59]  Daniel L Schacter,et al.  Ventromedial prefrontal cortex supports affective future simulation by integrating distributed knowledge , 2014, Proceedings of the National Academy of Sciences.

[60]  Julien Vitay,et al.  ANNarchy: a code generation approach to neural simulations on parallel hardware , 2015, Front. Neuroinform..

[61]  G. Buzsáki,et al.  Forward and reverse hippocampal place-cell sequences during ripples , 2007, Nature Neuroscience.

[62]  P. Caroni,et al.  Goal-oriented searching mediated by ventral hippocampus early in trial-and-error learning , 2012, Nature Neuroscience.

[63]  Matthew A. Wilson,et al.  Impaired Hippocampal Ripple-Associated Replay in a Mouse Model of Schizophrenia , 2013, Neuron.

[64]  Richard Kempter,et al.  Modeling Inheritance of Phase Precession in the Hippocampal Formation , 2014, The Journal of Neuroscience.

[65]  Si Wu,et al.  Tracking Changing Stimuli in Continuous Attractor Neural Networks , 2008, NIPS.

[66]  Conor Liston,et al.  Projections from neocortex mediate top-down control of memory retrieval , 2015, Nature.

[67]  Hongkui Zeng,et al.  Forebrain-Specific Calcineurin Knockout Selectively Impairs Bidirectional Synaptic Plasticity and Working/Episodic-like Memory , 2001, Cell.

[68]  Inah Lee,et al.  A Double Dissociation between Hippocampal Subfields Differential Time Course of CA3 and CA1 Place Cells for Processing Changed Environments , 2004, Neuron.

[69]  Martin Vinck,et al.  Perirhinal firing patterns are sustained across large spatial segments of the task environment , 2017, Nature Communications.

[70]  M. Larkum,et al.  Dopamine modulates synaptic plasticity in dendrites of rat and human dentate granule cells , 2010, Proceedings of the National Academy of Sciences.

[71]  J. Leiden Rapid response , 1998, Nature.

[72]  Matthijs A. A. van der Meer,et al.  Internally generated sequences in learning and executing goal-directed behavior , 2014, Trends in Cognitive Sciences.

[73]  D. Manahan‐Vaughan,et al.  Regulation of depotentiation and long-term potentiation in the dentate gyrus of freely moving rats by dopamine D2-like receptors. , 2003, Cerebral cortex.

[74]  Neil Burgess,et al.  Using Grid Cells for Navigation , 2015, Neuron.

[75]  Romain Brette,et al.  The Brian Simulator , 2009, Front. Neurosci..

[76]  G. Buzsáki,et al.  Cell Assembly Sequences Arising from Spike Threshold Adaptation Keep Track of Time in the Hippocampus , 2011, The Journal of Neuroscience.

[77]  Xiao-Jing Wang,et al.  Robust Spatial Working Memory through Homeostatic Synaptic Scaling in Heterogeneous Cortical Networks , 2003, Neuron.

[78]  Caswell Barry,et al.  Coordinated grid and place cell replay during rest , 2016, Nature Neuroscience.

[79]  Matthew A. Wilson,et al.  Hippocampal Replay of Extended Experience , 2009, Neuron.

[80]  K M Gothard,et al.  Dentate Gyrus and CA1 Ensemble Activity during Spatial Reference Frame Shifts in the Presence and Absence of Visual Input , 2001, The Journal of Neuroscience.

[81]  Matthijs A. A. van der Meer,et al.  Information Processing in Decision-Making Systems , 2012, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[82]  David S. Touretzky,et al.  The Role of the Hippocampus in Solving the Morris Water Maze , 1998, Neural Computation.

[83]  Colin Molter,et al.  Reactivation of behavioral activity during sharp waves: A computational model for two stage hippocampal dynamics , 2007, Hippocampus.

[84]  G. Buzsáki,et al.  Pyramidal Cell-Interneuron Interactions Underlie Hippocampal Ripple Oscillations , 2014, Neuron.

[85]  D. Durstewitz,et al.  A Quantitative Analysis of Context-Dependent Remapping of Medial Frontal Cortex Neurons and Ensembles , 2016, The Journal of Neuroscience.

[86]  Wulfram Gerstner,et al.  Attractor Network Dynamics Enable Preplay and Rapid Path Planning in Maze-like Environments , 2015, NIPS.

[87]  P. Dayan,et al.  Goals and Habits in the Brain , 2013, Neuron.

[88]  Sven Jahnke,et al.  A Unified Dynamic Model for Learning, Replay, and Sharp-Wave/Ripples , 2015, The Journal of Neuroscience.

[89]  Wulfram Gerstner,et al.  Spike-Based Reinforcement Learning in Continuous State and Action Space: When Policy Gradient Methods Fail , 2009, PLoS Comput. Biol..

[90]  J. Lisman,et al.  Storage, recall, and novelty detection of sequences by the hippocampus: Elaborating on the SOCRATIC model to account for normal and aberrant effects of dopamine , 2001, Hippocampus.

[91]  R. Passingham The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[92]  K. Reymann,et al.  A post-tetanic time window for the reinforcement of long-term potentiation by appetitive and aversive stimuli. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[93]  John J Hopfield,et al.  Neurodynamics of mental exploration , 2009, Proceedings of the National Academy of Sciences.

[94]  L. Frank,et al.  Awake Hippocampal Sharp-Wave Ripples Support Spatial Memory , 2012, Science.

[95]  Matthijs A. A. van der Meer,et al.  Hippocampal Replay Is Not a Simple Function of Experience , 2010, Neuron.

[96]  Antal Berényi,et al.  Role of Hippocampal CA2 Region in Triggering Sharp-Wave Ripples , 2016, Neuron.

[97]  O Jensen,et al.  Theta/gamma networks with slow NMDA channels learn sequences and encode episodic memory: role of NMDA channels in recall. , 1996, Learning & memory.

[98]  N. Burgess The 2014 Nobel Prize in Physiology or Medicine: A Spatial Model for Cognitive Neuroscience , 2014, Neuron.

[99]  H. Sompolinsky,et al.  Theory of orientation tuning in visual cortex. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[100]  L. Colgin,et al.  Slow gamma rhythms in CA3 are entrained by slow gamma activity in the dentate gyrus. , 2016, Journal of neurophysiology.

[101]  Arne D. Ekstrom,et al.  NMDA Receptor Antagonism Blocks Experience-Dependent Expansion of Hippocampal “Place Fields” , 2001, Neuron.

[102]  Javier Baladron,et al.  Fast and Slow Learning in a Neuro-Computational Model of Category Acquisition , 2016, ICANN.

[103]  Margaret F. Carr,et al.  Hippocampal SWR Activity Predicts Correct Decisions during the Initial Learning of an Alternation Task , 2013, Neuron.

[104]  Nicole M. Long,et al.  Successful memory formation is driven by contextual encoding in the core memory network , 2015, NeuroImage.

[105]  Charlotte N. Boccara,et al.  Superficial layers of the medial entorhinal cortex replay independently of the hippocampus , 2017, Science.

[106]  Andrew Philippides,et al.  Dual Coding with STDP in a Spiking Recurrent Neural Network Model of the Hippocampus , 2010, PLoS Comput. Biol..

[107]  Marco Idiart,et al.  Grid Cells and Place Cells: An Integrated View of their Navigational and Memory Function , 2015, Trends in Neurosciences.

[108]  Brad E. Pfeiffer,et al.  Hippocampal place cell sequences depict future paths to remembered goals , 2013, Nature.

[109]  Szabolcs Káli,et al.  Mechanisms of Sharp Wave Initiation and Ripple Generation , 2014, The Journal of Neuroscience.

[110]  D Marr,et al.  Simple memory: a theory for archicortex. , 1971, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[111]  David K Bilkey,et al.  Place, space, and taste: Combining context and spatial information in a hippocampal navigation system , 2012, Hippocampus.

[112]  D. Durstewitz,et al.  Contextual encoding by ensembles of medial prefrontal cortex neurons , 2012, Proceedings of the National Academy of Sciences.

[113]  Mattias P. Karlsson,et al.  A hippocampal network for spatial coding during immobility and sleep , 2016, Nature.

[114]  Angelo Arleo,et al.  Rapid response of head direction cells to reorienting visual cues: a computational model , 2004, Neurocomputing.

[115]  J. Rawlins,et al.  Dissecting Spatial Knowledge from Spatial Choice by Hippocampal NMDA Receptor Deletion , 2012, Nature Neuroscience.

[116]  J. O’Neill,et al.  Place-selective firing contributes to the reverse-order reactivation of CA1 pyramidal cells during sharp waves in open-field exploration , 2007, The European journal of neuroscience.

[117]  Wulfram Gerstner,et al.  Learning Navigational Maps Through Potentiation and Modulation of Hippocampal Place Cells , 2004, Journal of Computational Neuroscience.

[118]  Wulfram Gerstner,et al.  Adaptive exponential integrate-and-fire model as an effective description of neuronal activity. , 2005, Journal of neurophysiology.