Spatial Learning and Action Planning in a Prefrontal Cortical Network Model

The interplay between hippocampus and prefrontal cortex (PFC) is fundamental to spatial cognition. Complementing hippocampal place coding, prefrontal representations provide more abstract and hierarchically organized memories suitable for decision making. We model a prefrontal network mediating distributed information processing for spatial learning and action planning. Specific connectivity and synaptic adaptation principles shape the recurrent dynamics of the network arranged in cortical minicolumns. We show how the PFC columnar organization is suitable for learning sparse topological-metrical representations from redundant hippocampal inputs. The recurrent nature of the network supports multilevel spatial processing, allowing structural features of the environment to be encoded. An activation diffusion mechanism spreads the neural activity through the column population leading to trajectory planning. The model provides a functional framework for interpreting the activity of PFC neurons recorded during navigation tasks. We illustrate the link from single unit activity to behavioral responses. The results suggest plausible neural mechanisms subserving the cognitive “insight” capability originally attributed to rodents by Tolman & Honzik. Our time course analysis of neural responses shows how the interaction between hippocampus and PFC can yield the encoding of manifold information pertinent to spatial planning, including prospective coding and distance-to-goal correlates.

[1]  M. Khamassi,et al.  Replay of rule-learning related neural patterns in the prefrontal cortex during sleep , 2009, Nature Neuroscience.

[2]  Adam Johnson,et al.  Neural Ensembles in CA3 Transiently Encode Paths Forward of the Animal at a Decision Point , 2007, The Journal of Neuroscience.

[3]  Y Burnod,et al.  Organizational levels of the cerebral cortex: An integrated model , 1991, Acta biotheoretica.

[4]  S. Otani Prefrontal cortex function, quasi-physiological stimuli, and synaptic plasticity , 2003, Journal of Physiology-Paris.

[5]  William A. Roberts,et al.  Spatial memory in the rat on a hierarchical maze , 1979 .

[6]  M. Quirk,et al.  Experience-Dependent Asymmetric Shape of Hippocampal Receptive Fields , 2000, Neuron.

[7]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[8]  R. Passingham The prefrontal cortex: Anatomy physiology and neurophysiology of the frontal lobe J. M. Fuster. Raven Press, New York (1980), 222 pp., $29.92 , 1981, Neuroscience.

[9]  Jonathan D. Cohen,et al.  Computational perspectives on dopamine function in prefrontal cortex , 2002, Current Opinion in Neurobiology.

[10]  M. Hasselmo,et al.  An integrate-and-fire model of prefrontal cortex neuronal activity during performance of goal-directed decision making. , 2005, Cerebral cortex.

[11]  M. Shapiro,et al.  Prospective and Retrospective Memory Coding in the Hippocampus , 2003, Neuron.

[12]  Hugo J Spiers,et al.  A navigational guidance system in the human brain , 2007, Hippocampus.

[13]  M. Botvinick,et al.  Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective , 2009, Cognition.

[14]  Michael E. Hasselmo,et al.  A Model of Prefrontal Cortical Mechanisms for Goal-directed Behavior , 2005, Journal of Cognitive Neuroscience.

[15]  M. Arbib,et al.  Multiple representations of space underlying behavior , 1982, Behavioral and Brain Sciences.

[16]  Alice Alvernhe,et al.  Different CA1 and CA3 Representations of Novel Routes in a Shortcut Situation , 2008, The Journal of Neuroscience.

[17]  R. Poldrack,et al.  Competition among multiple memory systems: converging evidence from animal and human brain studies , 2003, Neuropsychologia.

[18]  E. Miller,et al.  Prospective Coding for Objects in Primate Prefrontal Cortex , 1999, The Journal of Neuroscience.

[19]  G. Lei A neuron model with fluid properties for solving labyrinthian puzzle , 1990, Biological Cybernetics.

[20]  M. Wilson,et al.  Theta Rhythms Coordinate Hippocampal–Prefrontal Interactions in a Spatial Memory Task , 2005, PLoS biology.

[21]  Eleanor A. Maguire,et al.  Thoughts, behaviour, and brain dynamics during navigation in the real world , 2006, NeuroImage.

[22]  Jean-Arcady Meyer,et al.  BIOLOGICALLY BASED ARTIFICIAL NAVIGATION SYSTEMS: REVIEW AND PROSPECTS , 1997, Progress in Neurobiology.

[23]  R. Muller,et al.  Representation of Objects in Space by Two Classes of Hippocampal Pyramidal Cells , 2004, The Journal of general physiology.

[24]  Philippe Gaussier,et al.  A Hierarchy of Associations in Hippocampo-Cortical Systems: Cognitive Maps and Navigation Strategies , 2005, Neural Computation.

[25]  E. Tolman Cognitive maps in rats and men. , 1948, Psychological review.

[26]  W A Roberts,et al.  Further evidence for hierarchical chunking in rat spatial memory. , 1995, Journal of experimental psychology. Animal behavior processes.

[27]  T. Shallice,et al.  Deficits in strategy application following frontal lobe damage in man. , 1991, Brain : a journal of neurology.

[28]  D. Buxhoeveden,et al.  The minicolumn hypothesis in neuroscience. , 2002, Brain : a journal of neurology.

[29]  P. Rakic Confusing cortical columns , 2008, Proceedings of the National Academy of Sciences.

[30]  W. Gerstner,et al.  Is there a geometric module for spatial orientation? Insights from a rodent navigation model. , 2009, Psychological review.

[31]  Maja J. Mataric,et al.  Integration of representation into goal-driven behavior-based robots , 1992, IEEE Trans. Robotics Autom..

[32]  Nestor Schmajuk,et al.  Exploration and Navigation Using Hierarchical Cognitive Maps , 2006 .

[33]  N. Hatsopoulos,et al.  Propagating waves mediate information transfer in the motor cortex , 2006, Nature Neuroscience.

[34]  B Poucet,et al.  Medial prefrontal lesions in the rat and spatial navigation: evidence for impaired planning. , 1995, Behavioral neuroscience.

[35]  Angelo Arleo,et al.  Cognitive navigation based on nonuniform Gabor space sampling, unsupervised growing networks, and reinforcement learning , 2004, IEEE Transactions on Neural Networks.

[36]  S Dehaene,et al.  A hierarchical neuronal network for planning behavior. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[37]  E. Koechlin,et al.  The Architecture of Cognitive Control in the Human Prefrontal Cortex , 2003, Science.

[38]  J. Glowinski,et al.  Dopaminergic Terminals in the Rat Cortex , 1973, Science.

[39]  A. Graybiel Habits, rituals, and the evaluative brain. , 2008, Annual review of neuroscience.

[40]  W H Meck,et al.  Perinatal choline supplementation increases the threshold for chunking in spatial memory , 1997, Neuroreport.

[41]  Bernhard Schölkopf,et al.  Learning View Graphs for Robot Navigation , 1997, AGENTS '97.

[42]  A. Grace,et al.  Dopamine modulation of hippocampal-prefrontal cortical interaction drives memory-guided behavior. , 2008, Cerebral cortex.

[43]  D. Amaral,et al.  The three-dimensional organization of the hippocampal formation: A review of anatomical data , 1989, Neuroscience.

[44]  Bruce L. McNaughton,et al.  An Information-Theoretic Approach to Deciphering the Hippocampal Code , 1992, NIPS.

[45]  J. Eccles The modular operation of the cerebral neocortex considered as the material basis of mental events , 1981, Neuroscience.

[46]  R. Passingham The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[47]  Mehdi Khamassi,et al.  Coherent Theta Oscillations and Reorganization of Spike Timing in the Hippocampal- Prefrontal Network upon Learning , 2010, Neuron.

[48]  Ariane S Etienne,et al.  Path integration in mammals , 2004, Hippocampus.

[49]  Bruno B Averbeck,et al.  Parallel processing of serial movements in prefrontal cortex , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[50]  RU Muller,et al.  The hippocampus as a cognitive graph , 1996, The Journal of general physiology.

[51]  J. Szentágothai The ‘module-concept’ in cerebral cortex architecture , 1975, Brain Research.

[52]  Michele Bezzi,et al.  Measuring Information Spatial Densities , 2001, Neural Computation.

[53]  Christian F. Doeller,et al.  Differential hippocampal and prefrontal-striatal contributions to instance-based and rule-based learning , 2006, NeuroImage.

[54]  J. Fuster The Prefrontal Cortex—An Update Time Is of the Essence , 2001, Neuron.

[55]  G. Buzsáki,et al.  Temporal Encoding of Place Sequences by Hippocampal Cell Assemblies , 2006, Neuron.

[56]  Jeffrey S. Taube,et al.  Head direction cells and the neural mechanisms of spatial orientation , 2005 .

[57]  Robert P. Vertes,et al.  Interactions among the medial prefrontal cortex, hippocampus and midline thalamus in emotional and cognitive processing in the rat , 2006, Neuroscience.

[58]  S. Becker,et al.  A Computational Model of Prefrontal Control in Free Recall: Strategic Memory Use in the California Verbal Learning Task , 2003, Journal of Cognitive Neuroscience.

[59]  Michael Recce,et al.  A model of hippocampal function , 1994, Neural Networks.

[60]  G. Winocur,et al.  Memory consolidation or transformation: context manipulation and hippocampal representations of memory , 2007, Nature Neuroscience.

[61]  T. Hafting,et al.  Finite Scale of Spatial Representation in the Hippocampus , 2008, Science.

[62]  T. McNamara,et al.  Subjective hierarchies in spatial memory. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[63]  Jian-Young Wu,et al.  Propagating Waves of Activity in the Neocortex: What They Are, What They Do , 2008, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[64]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[65]  Yves Burnod,et al.  An adaptive neural network - the cerebral cortex , 1991 .

[66]  E. Save,et al.  Coding for spatial goals in the prelimbic/infralimbic area of the rat frontal cortex. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[67]  Christian F. Doeller,et al.  Prefrontal-hippocampal dynamics involved in learning regularities across episodes. , 2005, Cerebral cortex.

[68]  B. Poucet Spatial cognitive maps in animals: new hypotheses on their structure and neural mechanisms. , 1993, Psychological review.

[69]  J. O’Keefe,et al.  Phase relationship between hippocampal place units and the EEG theta rhythm , 1993, Hippocampus.

[70]  Alain Berthoz,et al.  Impaired Sequential Egocentric and Allocentric Memories in Forebrain-Specific–NMDA Receptor Knock-Out Mice during a New Task Dissociating Strategies of Navigation , 2006, The Journal of Neuroscience.

[71]  T. Jay,et al.  Distribution of hippocampal CA1 and subicular efferents in the prefrontal cortex of the rat studied by means of anterograde transport of Phaseolus vulgaris‐leucoagglutinin , 1991, The Journal of comparative neurology.

[72]  S. B. Fountain,et al.  Coding of hierarchical versus linear pattern structure in rats and humans. , 1995, Journal of experimental psychology. Animal behavior processes.

[73]  Hervé Frezza-Buet,et al.  Modeling prefrontal functions for robot navigation , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[74]  P. Goldman-Rakic,et al.  Isodirectional tuning of adjacent interneurons and pyramidal cells during working memory: evidence for microcolumnar organization in PFC. , 1999, Journal of neurophysiology.

[75]  L. Abbott,et al.  Neural network dynamics. , 2005, Annual review of neuroscience.

[76]  V. Mountcastle The columnar organization of the neocortex. , 1997, Brain : a journal of neurology.

[77]  Paul Leonard Gabbott,et al.  Prefrontal cortex in the rat: Projections to subcortical autonomic, motor, and limbic centers , 2005, The Journal of comparative neurology.

[78]  Andrzej Bieszczad Neurosolver: a step toward a neuromorphic general problem solver , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[79]  Daniel L Adams,et al.  The cortical column: a structure without a function , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[80]  Paul Leonard Gabbott,et al.  The organisation of dendritic bundles in the prelimbic cortex (area 32) of the rat , 1996, Brain Research.

[81]  S. Becker,et al.  Remembering the past and imagining the future: a neural model of spatial memory and imagery. , 2007, Psychological review.

[82]  Masataka Watanabe Reward expectancy in primate prefrental neurons , 1996, Nature.

[83]  Cyriel M. A. Pennartz,et al.  Learning-related changes in response patterns of prefrontal neurons during instrumental conditioning , 2003, Behavioural Brain Research.

[84]  J. Tanji,et al.  Activity in the Lateral Prefrontal Cortex Reflects Multiple Steps of Future Events in Action Plans , 2006, Neuron.

[85]  D. Nitz Tracking Route Progression in the Posterior Parietal Cortex , 2006, Neuron.

[86]  Hanspeter A. Mallot,et al.  Embodied spatial cognition: Biological and artificial systems , 2009, Image Vis. Comput..

[87]  B. Kolb,et al.  Do rats have a prefrontal cortex? , 2003, Behavioural Brain Research.

[88]  H. J. Spiers,et al.  The neuroscience of remote spatial memory: A tale of two cities , 2007, Neuroscience.

[89]  Angelo Arleo,et al.  Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity , 2000, Biological Cybernetics.

[90]  Nancy L. Dallal,et al.  Hierarchical structures: chunking by food type facilitates spatial memory. , 1990, Journal of experimental psychology. Animal behavior processes.

[91]  Bruce L. McNaughton,et al.  Path integration and the neural basis of the 'cognitive map' , 2006, Nature Reviews Neuroscience.

[92]  G. Buzsáki,et al.  Selective suppression of hippocampal ripples impairs spatial memory , 2009, Nature Neuroscience.

[93]  François Fleuret,et al.  DEA: An Architecture for Goal Planning and Classification , 2000, Neural Computation.

[94]  Angelo Arleo,et al.  Multimodal sensory integration and concurrent navigation strategies for spatial cognition in real and artificial organisms. , 2007, Journal of integrative neuroscience.

[95]  David Filliat,et al.  Map-based navigation in mobile robots: II. A review of map-learning and path-planning strategies , 2003, Cognitive Systems Research.

[96]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[97]  Horatiu Voicu,et al.  Hierarchical cognitive maps , 2003, Neural Networks.

[98]  B. McNaughton,et al.  Firing characteristics of deep layer neurons in prefrontal cortex in rats performing spatial working memory tasks. , 1998, Cerebral cortex.

[99]  David J. Foster,et al.  A model of hippocampally dependent navigation, using the temporal difference learning rule , 2000, Hippocampus.

[100]  H. Markram The Blue Brain Project , 2006, Nature Reviews Neuroscience.

[101]  Erin L. Rich,et al.  Rat Prefrontal Cortical Neurons Selectively Code Strategy Switches , 2009, The Journal of Neuroscience.

[102]  Ricardo Chavarriaga,et al.  Path planning versus cue responding: a bio-inspired model of switching between navigation strategies , 2010, Biological Cybernetics.

[103]  D. Schacter,et al.  Episodic Simulation of Future Events , 2008, Annals of the New York Academy of Sciences.

[104]  W. Schultz,et al.  Relative reward preference in primate orbitofrontal cortex , 1999, Nature.

[105]  M. Walton,et al.  Separate neural pathways process different decision costs , 2006, Nature Neuroscience.

[106]  James J Knierim,et al.  Neural representations of location outside the hippocampus. , 2006, Learning & memory.

[107]  H. Kita,et al.  Amygdaloid projections to the frontal cortex and the striatum in the rat , 1990, The Journal of comparative neurology.

[108]  J. Banquet,et al.  Spatial Navigation and Hippocampal Place Cell Firing: The Problem of Goal Encoding , 2004, Reviews in the neurosciences.

[109]  E. Miller,et al.  Neural Activity in the Primate Prefrontal Cortex during Associative Learning , 1998, Neuron.

[110]  Nestor A. Schmajuk,et al.  Purposive behavior and cognitive mapping: a neural network model , 1992, Biological Cybernetics.

[111]  B L McNaughton,et al.  Dynamics of the hippocampal ensemble code for space. , 1993, Science.

[112]  R. J. McDonald,et al.  Multiple Parallel Memory Systems in the Brain of the Rat , 2002, Neurobiology of Learning and Memory.

[113]  V. Mountcastle Modality and topographic properties of single neurons of cat's somatic sensory cortex. , 1957, Journal of neurophysiology.

[114]  Edward Chace Tolman,et al.  "Insight" in rats , 1930 .

[115]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[116]  David A. Lewis,et al.  Specificity in the functional architecture of primate prefrontal cortex , 2002, Journal of neurocytology.

[117]  D. Geldmacher,et al.  The Amygdala , 1992, Neurology.

[118]  J. D. McGaugh,et al.  Inactivation of Hippocampus or Caudate Nucleus with Lidocaine Differentially Affects Expression of Place and Response Learning , 1996, Neurobiology of Learning and Memory.