What Is a Cognitive Map? Organizing Knowledge for Flexible Behavior

It is proposed that a cognitive map encoding the relationships between entities in the world supports flexible behavior, but the majority of the neural evidence for such a system comes from studies of spatial navigation. Recent work describing neuronal parallels between spatial and non-spatial behaviors has rekindled the notion of a systematic organization of knowledge across multiple domains. We review experimental evidence and theoretical frameworks that point to principles unifying these apparently disparate functions. These principles describe how to learn and use abstract, generalizable knowledge and suggest that map-like representations observed in a spatial context may be an instance of general coding mechanisms capable of organizing knowledge of all kinds. We highlight how artificial agents endowed with such principles exhibit flexible behavior and learn map-like representations observed in the brain. Finally, we speculate on how these principles may offer insight into the extreme generalizations, abstractions, and inferences that characterize human cognition.

[1]  Anne E Carpenter,et al.  Neuron-type specific signals for reward and punishment in the ventral tegmental area , 2011, Nature.

[2]  J. O’Keefe,et al.  Neural Representations of Location Composed of Spatially Periodic Bands , 2012, Science.

[3]  Ron Meir,et al.  Extracting grid cell characteristics from place cell inputs using non-negative principal component analysis , 2016, eLife.

[4]  Nathaniel J. Killian,et al.  Grid cells map the visual world , 2017, Nature Neuroscience.

[5]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[6]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[7]  Timothy Edward John Behrens,et al.  Reward-Guided Learning with and without Causal Attribution , 2016, Neuron.

[8]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[9]  Joel Z. Leibo,et al.  Prefrontal cortex as a meta-reinforcement learning system , 2018, bioRxiv.

[10]  J. Delacour,et al.  Effects of selective lesions of Fimbria-Fornix on learning set in the rat , 1987, Physiology & Behavior.

[11]  Regina Paxton Gazes,et al.  Cognitive mechanisms for transitive inference performance in rhesus monkeys: measuring the influence of associative strength and inferred order. , 2012, Journal of experimental psychology. Animal behavior processes.

[12]  Marcin Andrychowicz,et al.  Learning to learn by gradient descent by gradient descent , 2016, NIPS.

[13]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[14]  Christian F. Doeller,et al.  Evidence for grid cells in a human memory network , 2010, Nature.

[15]  Yoram Burak,et al.  Accurate Path Integration in Continuous Attractor Network Models of Grid Cells , 2008, PLoS Comput. Biol..

[16]  Pablo E. Jercog,et al.  Neural ensemble dynamics underlying a long-term associative memory , 2017, Nature.

[17]  Dylan A. Simon,et al.  Model-based choices involve prospective neural activity , 2015, Nature Neuroscience.

[18]  G. Winocur,et al.  Higher-Order Conditioning Is Impaired by Hippocampal Lesions , 2014, Current Biology.

[19]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[20]  Transitive Inference Formation in Pigeons , 2019 .

[21]  Raymond J Dolan,et al.  A map of abstract relational knowledge in the human hippocampal–entorhinal cortex , 2017, eLife.

[22]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[23]  Thomas R. Zentall,et al.  Transitive inference in pigeons , 1991 .

[24]  Sachin S. Deshmukh,et al.  Influence of local objects on hippocampal representations: Landmark vectors and memory , 2013, Hippocampus.

[25]  Robert C. Wilson,et al.  Orbitofrontal Cortex as a Cognitive Map of Task Space , 2014, Neuron.

[26]  Neil Burgess,et al.  Using Grid Cells for Navigation , 2015, Neuron.

[27]  Nathaniel D. Daw,et al.  Grid Cells, Place Cells, and Geodesic Generalization for Spatial Reinforcement Learning , 2011, PLoS Comput. Biol..

[28]  H. Terrace,et al.  Transitive inference in humans and rhesus macaques after massed training of the last two list items , 2016, bioRxiv.

[29]  F. R. Treichler,et al.  Concurrent conditional discrimination tests of transitive inference by macaque monkeys: list linking. , 1996, Journal of experimental psychology. Animal behavior processes.

[30]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[31]  Dmitriy Aronov,et al.  Mapping of a non-spatial dimension by the hippocampal/entorhinal circuit , 2017, Nature.

[32]  Timothy E.J. Behrens,et al.  Intuitive planning: global navigation through cognitive maps based on grid-like codes , 2018 .

[33]  M. Andersson,et al.  Independent Codes for Spatial and Episodic Memory in Hippocampal Neuronal Ensembles , 2005 .

[34]  Yoram Burakyy,et al.  Accurate Path Integration in Continuous Attractor Network Models of Grid Cells , 2009 .

[35]  Timothy Edward John Behrens,et al.  Separable Learning Systems in the Macaque Brain and the Role of Orbitofrontal Cortex in Contingent Learning , 2010, Neuron.

[36]  E. Bostock,et al.  Experience‐dependent modifications of hippocampal place cell firing , 1991, Hippocampus.

[37]  R. Passingham.  The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[38]  Timothy Edward John Behrens,et al.  Two Anatomically and Computationally Distinct Learning Signals Predict Changes to Stimulus-Outcome Associations in Hippocampus , 2016, Neuron.

[39]  P. Dayan,et al.  Adaptive integration of habits into depth-limited planning defines a habitual-goal–directed spectrum , 2016, Proceedings of the National Academy of Sciences.

[40]  J. Lisman,et al.  D1/D5 Dopamine Receptor Activation Increases the Magnitude of Early Long-Term Potentiation at CA1 Hippocampal Synapses , 1996, The Journal of Neuroscience.

[41]  J. O’Keefe,et al.  Grid cell firing patterns signal environmental novelty by expansion , 2012, Proceedings of the National Academy of Sciences.

[42]  Fabian Grabenhorst,et al.  A dynamic code for economic object valuation in prefrontal cortex neurons , 2016, Nature Communications.

[43]  Treichler Fr,et al.  Concurrent conditional discrimination tests of transitive inference by macaque monkeys: list linking. , 1996 .

[44]  Timothy E. J. Behrens,et al.  Review Frontal Cortex and Reward-guided Learning and Decision-making Figure 1. Frontal Brain Regions in the Macaque Involved in Reward-guided Learning and Decision-making Finer Grained Anatomical Divisions with Frontal Cortical Systems for Reward-guided Behavior , 2022 .

[45]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[46]  Peter Dayan,et al.  Improving Generalization for Temporal Difference Learning: The Successor Representation , 1993, Neural Computation.

[47]  Timothy E. J. Behrens,et al.  Online evaluation of novel choices by simultaneous representation of multiple memories , 2013, Nature Neuroscience.

[48]  Russell A. Epstein,et al.  Human entorhinal cortex represents visual space using a boundary-anchored grid , 2017, Nature Neuroscience.

[49]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[50]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[51]  E. Lein,et al.  Functional organization of the hippocampal longitudinal axis , 2014, Nature Reviews Neuroscience.

[52]  Matthew T. Kaufman,et al.  A neural network that finds a naturalistic solution for the production of muscle activity , 2015, Nature Neuroscience.

[53]  T. Cornsweet,et al.  The staircrase-method in psychophysics. , 1962, The American journal of psychology.

[54]  Camillo Padoa-Schioppa,et al.  Neuronal Remapping and Circuit Persistence in Economic Decisions , 2016, Nature Neuroscience.

[55]  M. Fyhn,et al.  Progressive increase in grid scale from dorsal to ventral medial entorhinal cortex , 2008, Hippocampus.

[56]  W. Schultz,et al.  Retroactive modulation of spike timing-dependent plasticity by dopamine , 2015, eLife.

[57]  J. Gibson.  The Senses Considered As Perceptual Systems , 1967 .

[58]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[59]  Kevin J. Miller,et al.  Value representations in the rodent orbitofrontal cortex drive learning, not choice , 2018, bioRxiv.

[60]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[61]  D. Hassabis,et al.  Hippocampal place cells construct reward related sequences through unexplored space , 2015, eLife.

[62]  KiJung Yoon,et al.  Grid Cell Responses in 1D Environments Assessed as Slices through a 2D Lattice , 2016, Neuron.

[63]  T. Toyoizumi,et al.  Spatial representations of self and other in the hippocampus , 2018, Science.

[64]  David Gaffan,et al.  Frontal-temporal disconnection abolishes object discrimination learning set in macaque monkeys. , 2006, Cerebral cortex.

[65]  Transitive inference in humans and rhesus macaques after massed training of the last two list items , 2016 .

[66]  T. Robbins,et al.  Differential Contributions of the Primate Ventrolateral Prefrontal and Orbitofrontal Cortex to Serial Reversal Learning , 2010, The Journal of Neuroscience.

[67]  Sho Yagishita,et al.  A critical time window for dopamine actions on the structural plasticity of dendritic spines , 2014, Science.

[68]  H. Yamahachi,et al.  Hippocampal Remapping after Partial Inactivation of the Medial Entorhinal Cortex , 2015, Neuron.

[69]  R. Buckner,et al.  Opinion TRENDS in Cognitive Sciences Vol.11 No.2 Self-projection and the brain , 2022 .

[70]  F. Heider,et al.  An experimental study of apparent behavior , 1944 .

[71]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[72]  N. Ulanovsky,et al.  Social place-cells in the bat hippocampus , 2018, Science.

[73]  M. Botvinick,et al.  Neural representations of events arise from temporal community structure , 2013, Nature Neuroscience.

[74]  C. Koch,et al.  Invariant visual representation by single neurons in the human brain , 2005, Nature.

[75]  C. H. Honzik,et al.  Degrees of hunger, reward and non-reward, and maze learning in rats, and Introduction and removal of reward, and maze performance in rats , 1930 .

[76]  R U Muller,et al.  Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[77]  Kevin J. Miller,et al.  Dorsal hippocampus contributes to model-based planning , 2017, Nature Neuroscience.

[78]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[79]  E. Tolman.  Cognitive maps in rats and men. , 1948, Psychological review.

[80]  Surya Ganguli,et al.  Continual Learning Through Synaptic Intelligence , 2017, ICML.

[81]  Andrew M. Wikenheiser,et al.  Suppression of Ventral Hippocampal Output Impairs Integrated Orbitofrontal Encoding of Task Structure , 2017, Neuron.

[82]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[83]  Matthijs A. A. van der Meer,et al.  Information Processing in Decision-Making Systems , 2012, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[84]  Sepp Hochreiter,et al.  Learning to Learn Using Gradient Descent , 2001, ICANN.

[85]  J. O’Keefe,et al.  Boundary Vector Cells in the Subiculum of the Hippocampal Formation , 2009, The Journal of Neuroscience.

[86]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[87]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[88]  Andrew M. Wikenheiser,et al.  Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex , 2016, Nature Reviews Neuroscience.

[89]  H. Eichenbaum,et al.  Evolution of declarative memory , 2006, Hippocampus.

[90]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[91]  E. Koechlin,et al.  The Architecture of Cognitive Control in the Human Prefrontal Cortex , 2003, Science.

[92]  Razvan Pascanu,et al.  Vector-based navigation using grid-like representations in artificial agents , 2018, Nature.

[93]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[94]  Peter Dayan,et al.  Interplay of approximate planning strategies , 2015, Proceedings of the National Academy of Sciences.

[95]  E. Tolman,et al.  Studies in spatial learning: Orientation and the short-cut. , 1946, Journal of experimental psychology.

[96]  H. Eichenbaum,et al.  The hippocampus and memory for orderly stimulus relations. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[97]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[98]  Kenneth A. Norman,et al.  Offline Replay Supports Planning: fMRI Evidence from Reward Revaluation , 2017, bioRxiv.

[99]  Joshua L. Jones,et al.  Orbitofrontal Cortex Supports Behavior and Learning Using Inferred But Not Cached Values , 2012, Science.

[100]  M. Botvinick,et al.  The hippocampus as a predictive map , 2016 .

[101]  H. Harlow,et al.  The formation of learning sets. , 1949, Psychological review.

[102]  M. Shapiro,et al.  A Map for Social Navigation in the Human Brain , 2015, Neuron.

[103]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[104]  A. Treves,et al.  Hippocampal remapping and grid realignment in entorhinal cortex , 2007, Nature.

[105]  B. McGonigle,et al.  Are monkeys logical? , 1977, Nature.

[106]  Robert C. Wilson,et al.  Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex , 2011, Nature Neuroscience.

[107]  Vincent P. Ferrera,et al.  Implicit Value Updating Explains Transitive Inference Performance: The Betasort Model , 2015, PLoS Comput. Biol..

[108]  E. Wasserman,et al.  Transitive inference in pigeons: Measuring the associative values of Stimuli B and D , 2012, Behavioural Processes.

[109]  Roddy M. Grieves,et al.  The representation of space in the brain , 2017, Behavioural Processes.

[110]  Jeffrey L. Gauthier,et al.  A Dedicated Population for Reward Coding in the Hippocampus , 2018, Neuron.

[111]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[112]  E. Murray,et al.  The Orbitofrontal Oracle: Cortical Mechanisms for the Prediction and Evaluation of Specific Behavioral Outcomes , 2014, Neuron.

[113]  Timothy Edward John Behrens,et al.  Generalisation of structural knowledge in the Hippocampal-Entorhinal system , 2018, NeurIPS.

[114]  M. Botvinick,et al.  The successor representation in human reinforcement learning , 2016, Nature Human Behaviour.

[115]  D. Amaral,et al.  Entorhinal Cortex Lesions Disrupt the Relational Organization of Memory in Monkeys , 2004, The Journal of Neuroscience.

[116]  Timothy Edward John Behrens,et al.  The Neural Network Underlying Incentive-Based Learning: Implications for Interpreting Circuit Disruptions in Psychiatric Disorders , 2014, Neuron.

[117]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[118]  Xue-Xin Wei,et al.  Emergence of grid-like representations by training recurrent neural networks to perform spatial localization , 2018, ICLR.

[119]  H. Eichenbaum,et al.  The global record of memory in hippocampal neuronal activity , 1999, Nature.

[120]  J. Staddon,et al.  Transitive inference formation in pigeons. , 1991 .

[121]  Alexander Mathis,et al.  Connecting multiple spatial scales to decode the population activity of grid cells , 2015, Science Advances.

[122]  Daniel Tranel,et al.  The Human Ventromedial Prefrontal Cortex Is Critical for Transitive Inference , 2012, Journal of Cognitive Neuroscience.

[123]  W. Newsome,et al.  Context-dependent computation by recurrent dynamics in prefrontal cortex , 2013, Nature.

[124]  Charles Kemp,et al.  The discovery of structural form , 2008, Proceedings of the National Academy of Sciences.

[125]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[126]  Tobias Navarro Schröder,et al.  Hexadirectional coding of visual space in human entorhinal cortex , 2018, Nature Neuroscience.

[127]  Øyvind Arne Høydal,et al.  Object-vector coding in the medial entorhinal cortex , 2019, Nature.

[128]  E. Murray,et al.  Dissociable Effects of Subtotal Lesions within the Macaque Orbital Prefrontal Cortex on Reward-Guided Behavior , 2011, The Journal of Neuroscience.

[129]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[130]  D. Hassabis,et al.  Patients with hippocampal amnesia cannot imagine new experiences , 2007, Proceedings of the National Academy of Sciences.

[131]  Nachum Ulanovsky,et al.  Vectorial representation of spatial goals in the hippocampus of bats , 2017, Science.

[132]  I. Fried,et al.  Direct recordings of grid-like neuronal activity in human spatial navigation , 2013, Nature Neuroscience.

[133]  M. Moser,et al.  Representation of Geometric Borders in the Entorhinal Cortex , 2008, Science.

[134]  Adam Johnson,et al.  Cognitive Neural Ensembles in CA 3 Transiently Encode Paths Forward of the Animal at a Decision Point , 2007 .

[135]  H. Eichenbaum,et al.  Robust Conjunctive Item–Place Coding by Hippocampal Neurons Parallels Learning What Happens Where , 2009, The Journal of Neuroscience.

[136]  Surya Ganguli,et al.  A Multiplexed, Heterogeneous, and Adaptive Code for Navigation in Medial Entorhinal Cortex , 2017, Neuron.

[137]  Timothy E. J. Behrens,et al.  Organizing conceptual knowledge in humans with a gridlike code , 2016, Science.