Episodic, Semantic, Pavlovian, and Procedural Cognitive Maps

Current theories of planning associate the hippocampus with a cognitive map, a theoretical construct used to predict the consequences of actions. This formulation is problematic for two reasons: First, cognitive maps are traditionally conceptualized to generalize over individual episodes, which conflicts with evidence associating the hippocampus with episodic memory, and second, it fails to explain seemingly non-hippocampal forms of planning. Here we propose a novel theoretical framework that resolves these issues: each long-term memory system is a cognitive map, predicting consequences of actions based on its unique computational properties. It follows that hippocampal maps are episode-based and that semantic, procedural, and Pavlovian memories each implement a specialized map. We present evidence for each type of map from neuropsychology, neuroimaging and animal electrophysiology studies.

[1]  F. W. Irwin Purposive Behavior in Animals and Men , 1932, The Psychological Clinic.

[2]  E. Tolman Cognitive maps in rats and men. , 1948, Psychological review.

[3]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[4]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[5]  Allen Newell,et al.  Human Problem Solving. , 1973 .

[6]  N. Guttman,et al.  Robust-single-trial delayed backward conditioning. , 1975 .

[7]  E. Rosch Cognitive Representations of Semantic Categories. , 1975 .

[8]  H. Grill,et al.  Contribution of the rat's neocortex to ingestive control: I. Latent learning for the taste of sodium chloride. , 1982, Journal of comparative and physiological psychology.

[9]  Kristian J. Hammond,et al.  CHEF: A Model of Case-Based Planning , 1986, AAAI.

[10]  E. Balint Memory and consciousness. , 1987, The International journal of psycho-analysis.

[11]  Joseph E LeDoux Emotion: clues from the brain. , 1995, Annual review of psychology.

[12]  A. Roth,et al.  On the Role of Reinforcement Learning in Experimental Games: The Cognitive Game Theory Approach , 1999 .

[13]  N. Charness,et al.  The perceptual aspect of skilled performance in chess: Evidence from eye movements , 2001, Memory & cognition.

[14]  G. Aschersleben,et al.  The Theory of Event Coding (TEC): a framework for perception and action planning. , 2001, The Behavioral and brain sciences.

[15]  H. Lipp,et al.  Conditioned taste aversion as a learning and memory paradigm , 2001, Behavioural Brain Research.

[16]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[17]  Jonathan D. Cohen,et al.  Prefrontal cortex and dynamic categorization tasks: representational organization and neuromodulatory control. , 2002, Cerebral cortex.

[18]  J. Kihlstrom,et al.  THE EFFECTS OF EPISODIC MEMORY LOSS ON AN AMNESIC PATIENT ’ S ABILITY TO REMEMBER THE PAST AND IMAGINE THE FUTURE , 2003 .

[19]  E. Tulving Episodic memory: from mind to brain. , 2002, Annual review of psychology.

[20]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[21]  R. Cabeza,et al.  Functional neuroimaging of autobiographical memory , 2007, Trends in Cognitive Sciences.

[22]  Afdc Hamilton,et al.  The motor hierarchy: from kinematics to goals and intentions , 2007 .

[23]  D. Hassabis,et al.  Patients with hippocampal amnesia cannot imagine new experiences , 2007, Proceedings of the National Academy of Sciences.

[24]  T. Sun SEEING THE FUTURE WITH , 2007 .

[25]  E. Maguire,et al.  The dynamic nature of cognition during wayfinding , 2008, Journal of environmental psychology.

[26]  P. Dayan,et al.  Human Pavlovian–Instrumental Transfer , 2008, The Journal of Neuroscience.

[27]  John J Hopfield,et al.  Neurodynamics of mental exploration , 2009, Proceedings of the National Academy of Sciences.

[28]  R. Nathan Spreng,et al.  The Common Neural Basis of Autobiographical Memory, Prospection, Navigation, Theory of Mind, and the Default Mode: A Quantitative Meta-analysis , 2009, Journal of Cognitive Neuroscience.

[29]  Dirk P. Kroese,et al.  Statistical Learning , 2001, Science.

[30]  H. Eichenbaum,et al.  The Episodic Memory System: Neurocircuitry and Disorders , 2010, Neuropsychopharmacology.

[31]  B. Balleine,et al.  Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action , 2010, Neuropsychopharmacology.

[32]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[33]  M. Hasselmo How We Remember: Brain Mechanisms of Episodic Memory , 2011 .

[34]  D. Schacter,et al.  The Hippocampus and Imagining the Future: Where Do We Stand? , 2012, Front. Hum. Neurosci..

[35]  Z. Kurth-Nelson,et al.  A theoretical account of cognitive effects in delay discounting , 2012, The European journal of neuroscience.

[36]  D. Addis,et al.  Considering the role of semantic memory in episodic future thinking: evidence from semantic dementia. , 2012, Brain : a journal of neurology.

[37]  Thomas T. Hills,et al.  Cognitive search : evolution, algorithms, and the brain , 2012 .

[38]  William Bart,et al.  Moves in Mind: The Psychology of Board Games , 2012, Int. J. Gaming Comput. Mediat. Simulations.

[39]  G. Schoenbaum,et al.  Model‐based learning and the contribution of the orbitofrontal cortex to the model‐free world , 2012, The European journal of neuroscience.

[40]  Pascale Piolino,et al.  Factors affecting medial temporal lobe engagement for past and future episodic events: An ALE meta-analysis of neuroimaging studies , 2012, Brain and Cognition.

[41]  D. Schacter,et al.  The Hippocampus and Imagining the Future: Where Do We Stand? , 2011, Front. Hum. Neurosci..

[42]  Uğur M Erdem,et al.  A goal‐directed spatial navigation model using forward trajectory planning based on grid cells , 2012, The European journal of neuroscience.

[43]  Matthew J. C. Crump,et al.  Keeping an Eye on Guitar Skill: Visual Representations of Guitar Chords , 2012 .

[44]  Nathaniel D. Daw,et al.  Cortical and Hippocampal Correlates of Deliberation during Model-Based Decisions for Rewards in Humans , 2013, PLoS Comput. Biol..

[45]  Bernard W. Balleine,et al.  Actions, Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically Organized , 2013, PLoS Comput. Biol..

[46]  Peter Bossaerts,et al.  Evidence for Model-based Computations in the Human Amygdala during Pavlovian Conditioning , 2013, PLoS Comput. Biol..

[47]  K. Berridge,et al.  Instant Transformation of Learned Repulsion into Motivational “Wanting” , 2013, Current Biology.

[48]  Blai Bonet,et al.  A Concise Introduction to Models and Methods for Automated Planning , 2013, A Concise Introduction to Models and Methods for Automated Planning.

[49]  A. Redish,et al.  The Mind within the Brain: How We Make Decisions and How those Decisions Go Wrong , 2013 .

[50]  P. Dayan,et al.  Goals and Habits in the Brain , 2013, Neuron.

[51]  Hallvard Røe Evensmoen,et al.  Long-axis specialization of the human hippocampus , 2013, Trends in Cognitive Sciences.

[52]  Arnaud D'Argembeau,et al.  Phenomenology of future-oriented mind-wandering episodes , 2013, Front. Psychol..

[53]  Thomas H. B. FitzGerald,et al.  Disruption of Dorsolateral Prefrontal Cortex Decreases Model-Based in Favor of Model-free Control in Humans , 2013, Neuron.

[54]  Ari Weinstein,et al.  Model-based hierarchical reinforcement learning and human action control , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[55]  P. Dayan,et al.  Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation , 2014, Cognitive, affective & behavioral neuroscience.

[56]  Jörn Diedrichsen,et al.  Human premotor areas parse sequences into their spatial and temporal features , 2014, eLife.

[57]  Morris Moscovitch,et al.  The perceptual richness of complex memory episodes is compromised by medial temporal lobe damage , 2014, Hippocampus.

[58]  Robert C. Wilson,et al.  Orbitofrontal Cortex as a Cognitive Map of Task Space , 2014, Neuron.

[59]  Craig A. Knoblock,et al.  A Survey of Digital Map Processing Techniques , 2014, ACM Comput. Surv..

[60]  Ivan Serina,et al.  Progress in Case-Based Planning , 2015, ACM Comput. Surv..

[61]  R. N. Spreng,et al.  A taxonomy of prospection: Introducing an organizational framework for future-oriented cognition , 2014, Proceedings of the National Academy of Sciences.

[62]  Vanessa E. Ghosh,et al.  What is a memory schema? A historical perspective on current neuroscience literature , 2014, Neuropsychologia.

[63]  Alec Solway,et al.  Optimal Behavioral Hierarchy , 2014, PLoS Comput. Biol..

[64]  P. Dayan,et al.  The algorithmic anatomy of model-based evaluation , 2014, Philosophical Transactions of the Royal Society B: Biological Sciences.

[65]  S. Becker,et al.  One spatial map or many? Spatial coding of connected environments. , 2014, Journal of experimental psychology. Learning, memory, and cognition.

[66]  M. Verfaellie,et al.  The medial temporal lobes are critical for reward‐based decision making under conditions that promote episodic future thinking , 2015, Hippocampus.

[67]  Dylan A. Simon,et al.  Model-based choices involve prospective neural activity , 2015, Nature Neuroscience.

[68]  Peter Dayan,et al.  Interplay of approximate planning strategies , 2015, Proceedings of the National Academy of Sciences.

[69]  Andrew M. Wikenheiser,et al.  Decoding the cognitive map: ensemble hippocampal sequences and decision making , 2015, Current Opinion in Neurobiology.

[70]  R. Dolan,et al.  Ventral striatal dopamine reflects behavioral and neural signatures of model-based control during sequential decision making , 2015, Proceedings of the National Academy of Sciences.

[71]  David J. Foster,et al.  Memory and Space: Towards an Understanding of the Cognitive Map , 2015, The Journal of Neuroscience.

[72]  Karl J. Friston,et al.  Active Inference, homeostatic regulation and adaptive behavioural control , 2015, Progress in Neurobiology.

[73]  M. Tsagareli I.S. Beritashvili and Psychoneural Integration of Behavior , 2015 .

[74]  D. Bendor,et al.  Does the Hippocampus Map Out the Future? , 2016, Trends in Cognitive Sciences.

[75]  J. Hodges,et al.  Neural Substrates of Semantic Prospection – Evidence from the Dementias , 2016, Front. Behav. Neurosci..

[76]  M. Botvinick,et al.  Complementary learning systems within the hippocampus: A neural network modeling approach to reconciling episodic memory with statistical learning , 2016, bioRxiv.

[77]  Nicolas W. Schuck,et al.  Human Orbitofrontal Cortex Represents a Cognitive Map of State Space , 2016, Neuron.

[78]  Andrew M. Wikenheiser,et al.  Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex , 2016, Nature Reviews Neuroscience.

[79]  G. Winocur,et al.  Episodic Memory and Beyond: The Hippocampus and Neocortex in Transformation. , 2016, Annual review of psychology.

[80]  R. N. Spreng,et al.  Seeing the Future : Theoretical Perspectives on Future-Oriented Mental Time Travel , 2017 .

[81]  D. Hassabis,et al.  Neural Mechanisms of Hierarchical Planning in a Virtual Subway Network , 2016, Neuron.

[82]  Nicholas B. Turk-Browne,et al.  Complementary learning systems within the hippocampus: A neural network modeling approach to reconciling episodic memory with statistical learning , 2016, bioRxiv.

[83]  Clay B. Holroyd,et al.  Running head : EPISODIC , SEMANTIC , PAVLOVIAN , AND PROCEDURAL COGNITIVE MAPS 1 Episodic , Semantic , Pavlovian , and Procedural Cognitive Maps , 2017 .

[84]  N. Daw,et al.  Reinforcement Learning and Episodic Memory in Humans and Animals: An Integrative Framework , 2017, Annual review of psychology.

[85]  Ana Bazac,et al.  Anticipation: learning from the past. The Russian/Soviet contributions to the science of anticipation , 2018, Int. J. Gen. Syst..