Computational cognitive models of spatial memory in navigation space: A review

Spatial memory refers to the part of the memory system that encodes, stores, recognizes and recalls spatial information about the environment and the agent's orientation within it. Such information is required to be able to navigate to goal locations, and is vitally important for any embodied agent, or model thereof, for reaching goals in a spatially extended environment. In this paper, a number of computationally implemented cognitive models of spatial memory are reviewed and compared. Three categories of models are considered: symbolic models, neural network models, and models that are part of a systems-level cognitive architecture. Representative models from each category are described and compared in a number of dimensions along which simulation models can differ (level of modeling, types of representation, structural accuracy, generality and abstraction, environment complexity), including their possible mapping to the underlying neural substrate. Neural mappings are rarely explicated in the context of behaviorally validated models, but they could be useful to cognitive modeling research by providing a new approach for investigating a model's plausibility. Finally, suggested experimental neuroscience methods are described for verifying the biological plausibility of computational cognitive models of spatial memory, and open questions for the field of spatial memory modeling are outlined.

[1]  Benjamin Kuipers,et al.  An Intellectual History of the Spatial Semantic Hierarchy , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[2]  Horatiu Voicu,et al.  Hierarchical cognitive maps , 2003, Neural Networks.

[3]  Gary L. Allen,et al.  Human spatial memory : remembering where , 2004 .

[4]  F. E. H. N. Wijermans The Cambridge Handbook of Computational Psychology , 2009 .

[5]  J. O’Keefe,et al.  Boundary Vector Cells in the Subiculum of the Hippocampal Formation , 2009, The Journal of Neuroscience.

[6]  David Waller,et al.  Transient and enduring spatial representations under disorientation and self-rotation. , 2006, Journal of experimental psychology. Learning, memory, and cognition.

[7]  Yoram Burak,et al.  Accurate Path Integration in Continuous Attractor Network Models of Grid Cells , 2008, PLoS Comput. Biol..

[8]  E. Maguire,et al.  The Well-Worn Route and the Path Less Traveled Distinct Neural Bases of Route Following and Wayfinding in Humans , 2003, Neuron.

[9]  C. Galletti,et al.  Reaching activity in parietal area V6A of macaque: eye influence on arm activity or retinocentric coding of reaching movements? , 2008, The European journal of neuroscience.

[10]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[11]  A S Etienne,et al.  Path integration in mammals and its interaction with visual landmarks. , 1996, The Journal of experimental biology.

[12]  Hojjat Adeli,et al.  Spiking Neural Networks , 2009, Int. J. Neural Syst..

[13]  Benjamin Kuipers,et al.  Factoring the Mapping Problem: Mobile Robot Map-building in the Hybrid Spatial Semantic Hierarchy , 2010, Int. J. Robotics Res..

[14]  R. Beer Dynamical approaches to cognitive science , 2000, Trends in Cognitive Sciences.

[15]  M. Rugg,et al.  Separating the Brain Regions Involved in Recollection and Familiarity in Recognition Memory , 2005, The Journal of Neuroscience.

[16]  Peter Stone,et al.  Reinforcement learning , 2019, Scholarpedia.

[17]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[18]  Hanspeter A. Mallot,et al.  Biomimetic robot navigation , 2000, Robotics Auton. Syst..

[19]  Alex Martin,et al.  Semantic memory and the brain: structure and processes , 2001, Current Opinion in Neurobiology.

[20]  Yoram Burakyy,et al.  Accurate Path Integration in Continuous Attractor Network Models of Grid Cells , 2009 .

[21]  E. Moser,et al.  A manifold of spatial maps in the brain , 2010, Trends in Cognitive Sciences.

[22]  Arne D. Ekstrom,et al.  Cellular networks underlying human spatial navigation , 2003, Nature.

[23]  J. Rieser,et al.  Bayesian integration of spatial information. , 2007, Psychological bulletin.

[24]  Pete R. Jones,et al.  Development of Cue Integration in Human Navigation , 2008, Current Biology.

[25]  R. Andersen,et al.  Dorsal Premotor Neurons Encode the Relative Position of the Hand, Eye, and Goal during Reach Planning , 2006, Neuron.

[26]  Matthew A. Wilson,et al.  Uncovering spatial topology represented by rat hippocampal population neuronal codes , 2012, Journal of Computational Neuroscience.

[27]  Jason B. Mattingley,et al.  Medial Parietal Cortex Encodes Perceived Heading Direction in Humans , 2010, The Journal of Neuroscience.

[28]  Angelo Arleo,et al.  Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity , 2000, Biological Cybernetics.

[29]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[30]  B L McNaughton,et al.  Path Integration and Cognitive Mapping in a Continuous Attractor Neural Network Model , 1997, The Journal of Neuroscience.

[31]  Alejandra Barrera,et al.  Biologically-inspired robot spatial cognition based on rat neurophysiological studies , 2008, Auton. Robots.

[32]  Ke Chen,et al.  Spatial Working Memory in the LIDA Cognitive Architecture , 2013, ICCM 2013.

[33]  James L. McClelland,et al.  Connectionist models of cognition. , 2008 .

[34]  K. Zilles,et al.  Neural Correlates of First-Person Perspective as One Constituent of Human Self-Consciousness , 2004, Journal of Cognitive Neuroscience.

[35]  E. Tolman Cognitive maps in rats and men. , 1948, Psychological review.

[36]  Bruce L. McNaughton,et al.  Path integration and the neural basis of the 'cognitive map' , 2006, Nature Reviews Neuroscience.

[37]  Neil Burgess,et al.  Predictions derived from modelling the hippocampal role in navigation , 2000, Biological Cybernetics.

[38]  Alejandra Barrera,et al.  Comparative Experimental Studies on Spatial Memory and Learning in Rats and Robots , 2011, J. Intell. Robotic Syst..

[39]  M. Moser,et al.  Representation of Geometric Borders in the Entorhinal Cortex , 2008, Science.

[40]  Russell A. Epstein Parahippocampal and retrosplenial contributions to human spatial navigation , 2008, Trends in Cognitive Sciences.

[41]  Daiel Darabos Simultaneous Localization andMapping withParticle Swarm Localization , 2005 .

[42]  J. Taube The head direction signal: origins and sensory-motor integration. , 2007, Annual review of neuroscience.

[43]  J. O’Keefe,et al.  Geometric determinants of the place fields of hippocampal neurons , 1996, Nature.

[44]  John R. Anderson,et al.  ACT-R Meets fMRI , 2006, WImBI.

[45]  P. Godfrey‐Smith Theory and Reality , 2003 .

[46]  Holger Schultheis,et al.  Eye Movements Reflect Reasoning with Mental Images but Not with Mental Models in Orientation Knowledge Tasks , 2010, Spatial Cognition.

[47]  R. Sun Introduction to Computational Cognitive Modeling , 2007 .

[48]  Jean-Arcady Meyer,et al.  BIOLOGICALLY BASED ARTIFICIAL NAVIGATION SYSTEMS: REVIEW AND PROSPECTS , 1997, Progress in Neurobiology.

[49]  Barbara Tversky,et al.  The Cambridge Handbook of Visuospatial Thinking: Functional Significance of Visuospatial Representations , 2005 .

[50]  Holger Schultheis,et al.  Extending ACT-R ’ s Memory Capabilities , 2007 .

[51]  Ning Zhong,et al.  EEG/ERP Meets ACT-R: A Case Study for Investigating Human Computation Mechanism , 2009, Brain Informatics.

[52]  Nestor A. Schmajuk,et al.  Exploration, Navigation and Cognitive Mapping , 2000, Adapt. Behav..

[53]  Bruno Laeng,et al.  Psychology of spatial cognition. , 2012, Wiley interdisciplinary reviews. Cognitive science.

[54]  Robert Trappl,et al.  Bayesian Integration of Information in Hippocampal Place Cells , 2014, PloS one.

[55]  Samuel Kaski,et al.  Winner-take-all networks for physiological models of competitive learning , 1994, Neural Networks.

[56]  C. Malsburg,et al.  How patterned neural connections can be set up by self-organization , 1976, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[57]  J. Lisman,et al.  Position reconstruction from an ensemble of hippocampal place cells: contribution of theta phase coding. , 2000, Journal of neurophysiology.

[58]  Benjamin Kuipers,et al.  The Spatial Semantic Hierarchy , 2000, Artif. Intell..

[59]  Pat Langley,et al.  Ability, Breadth, and Parsimony in Computational Models of Higher-Order Cognition , 2008, Cogn. Sci..

[60]  C. Barry,et al.  Specific evidence of low-dimensional continuous attractor dynamics in grid cells , 2013, Nature Neuroscience.

[61]  David Zipser,et al.  Feature Discovery by Competive Learning , 1986, Cogn. Sci..

[62]  Bernard J. Baars,et al.  CONSCIOUSNESS IS COMPUTATIONAL: THE LIDA MODEL OF GLOBAL WORKSPACE THEORY , 2009 .

[63]  M. Goldberg,et al.  Ventral intraparietal area of the macaque: congruent visual and somatic response properties. , 1998, Journal of neurophysiology.

[64]  B. Webb,et al.  Can robots make good models of biological behaviour? , 2001, Behavioral and Brain Sciences.

[65]  Ron Sun,et al.  The importance of cognitive architectures: an analysis based on CLARION , 2007, J. Exp. Theor. Artif. Intell..

[66]  Margaret E. Jefferies,et al.  Robot Cognitive Mapping - A Role for a Global Metric Map in a Cognitive Mapping Process , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[67]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[68]  W E Skaggs,et al.  Deciphering the hippocampal polyglot: the hippocampus as a path integration system. , 1996, The Journal of experimental biology.

[69]  Holger Schultheis,et al.  Casimir: An Architecture for Mental Spatial Knowledge Processing , 2011, Top. Cogn. Sci..

[70]  Bernard J. Baars,et al.  Global Workspace Dynamics: Cortical “Binding and Propagation” Enables Conscious Contents , 2013, Front. Psychol..

[71]  J. Taube,et al.  Where am I and how will I get there from here? A role for posterior parietal cortex in the integration of spatial information and route planning , 2009, Neurobiology of Learning and Memory.

[72]  Wai-Kiang Yeap,et al.  Using a Mobile Robot to Test a Theory of Cognitive Mapping , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[73]  Theodosios Pavlidis,et al.  Segmentation of Plane Curves , 1974, IEEE Transactions on Computers.

[74]  S. Becker,et al.  Remembering the past and imagining the future: a neural model of spatial memory and imagery. , 2007, Psychological review.

[75]  E. Maguire,et al.  What does the retrosplenial cortex do? , 2009, Nature Reviews Neuroscience.

[76]  Bernhard Schölkopf,et al.  View-Based Cognitive Mapping and Path Planning , 1995, Adapt. Behav..

[77]  Terence R. Smith,et al.  Human Way-Finding in an Urban Environment: A Performance Analysis of a Computational Process Model , 1990 .

[78]  Mark A. Pitt,et al.  Model Evaluation, Testing and Selection , 2005 .

[79]  Jonathan W. Kelly,et al.  Multiple systems of spatial memory and action , 2008, Cognitive Processing.

[80]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[81]  R. Klatzky,et al.  Navigator: A psychologically based model of environmental learning through navigation , 1989 .

[82]  D. Schacter,et al.  Remembering the Past and Imagining the Future in the Elderly , 2012, Gerontology.

[83]  J. D. McGaugh,et al.  Inactivation of Hippocampus or Caudate Nucleus with Lidocaine Differentially Affects Expression of Place and Response Learning , 1996, Neurobiology of Learning and Memory.

[84]  John R. Anderson,et al.  A central circuit of the mind , 2008, Trends in Cognitive Sciences.

[85]  P. Johnson-Laird,et al.  The mental representation of spatial descriptions , 1982, Memory & cognition.

[86]  Ben Goertzel,et al.  A world survey of artificial brain projects, Part II: Biologically inspired cognitive architectures , 2010, Neurocomputing.

[87]  Ben J. A. Kröse,et al.  Navigation using an appearance based topological map , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[88]  Gordon Wyeth,et al.  Persistent Navigation and Mapping using a Biologically Inspired SLAM System , 2010, Int. J. Robotics Res..

[89]  J. Jonides,et al.  Evidence of hierarchies in cognitive maps , 1985, Memory & cognition.

[90]  Emilio Kropff,et al.  Place cells, grid cells, and the brain's spatial representation system. , 2008, Annual review of neuroscience.

[91]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[92]  John R. Anderson,et al.  ACT-R: A Theory of Higher Level Cognition and Its Relation to Visual Attention , 1997, Hum. Comput. Interact..

[93]  Hugh Durrant-Whyte,et al.  Simultaneous localization and mapping (SLAM): part II , 2006 .

[94]  Anil K. Jain,et al.  Artificial Neural Networks: A Tutorial , 1996, Computer.

[95]  T. Maia Reinforcement learning, conditioning, and the brain: Successes and challenges , 2009, Cognitive, affective & behavioral neuroscience.

[96]  N. Burgess Spatial Cognition and the Brain , 2008, Annals of the New York Academy of Sciences.

[97]  K. Jeffery,et al.  The Boundary Vector Cell Model of Place Cell Firing and Spatial Memory , 2006, Reviews in the neurosciences.

[98]  R. O’Reilly Six principles for biologically based computational models of cortical cognition , 1998, Trends in Cognitive Sciences.

[99]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[100]  David M. Mark,et al.  Cognitive models of geographical space , 1999, Int. J. Geogr. Inf. Sci..

[101]  R. Sun The Cambridge Handbook of Computational Psychology: Introduction to Computational Cognitive Modeling , 2008 .

[102]  David Ball,et al.  Maintaining a Cognitive Map in Darkness: The Need to Fuse Boundary Knowledge with Path Integration , 2012, PLoS Comput. Biol..

[103]  A. G. Cohn,et al.  Topological Coding in the Hippocampus , 2007, q-bio/0702052.

[104]  Hanspeter A. Mallot,et al.  A Robot System for Biomimetic Navigation - From Snapshots to Metric Embeddings of View Graphs , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[105]  Selmer Bringsjord,et al.  The Cambridge Handbook of Computational Psychology: Declarative/Logic-Based Cognitive Modeling , 2008 .

[106]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[107]  Joaquin M. Fuster,et al.  Physiology of executive functions: The perception-action cycle. , 2002 .

[108]  Malcolm W. Brown,et al.  Recognition memory: What are the roles of the perirhinal cortex and hippocampus? , 2001, Nature Reviews Neuroscience.

[109]  Charles R. Gallistel,et al.  Dead Reckoning, Cognitive Maps, Animal Navigation and the Representation of Space: An Introduction , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[110]  Richard A. Andersen,et al.  Separate body- and world-referenced representations of visual space in parietal cortex , 1998, Nature.

[111]  Tamas Madl,et al.  LIDA: A Systems-level Architecture for Cognition, Emotion, and Learning , 2014, IEEE Transactions on Autonomous Mental Development.

[112]  James L. McClelland The Place of Modeling in Cognitive Science , 2009, Top. Cogn. Sci..

[113]  T. Hafting,et al.  Microstructure of a spatial map in the entorhinal cortex , 2005, Nature.

[114]  Paul Smolensky,et al.  Connectionist AI, symbolic AI, and the brain , 1987, Artificial Intelligence Review.

[115]  Ron Sun,et al.  From implicit skills to explicit knowledge: a bottom-up model of skill learning , 2001, Cogn. Sci..

[116]  E. Rolls,et al.  Spatial view cells in the primate hippocampus and memory recall. , 2008, Reviews in the neurosciences.

[117]  Jan Frederik Sima The Nature of Mental Images - An Integrative Computational Theory , 2011, CogSci.

[118]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[119]  Allen Newell,et al.  Computer science as empirical inquiry: symbols and search , 1976, CACM.

[120]  B. Webb What does robotics offer animal behaviour? , 2000, Animal Behaviour.

[121]  Chris Eliasmith,et al.  Attractor network , 2007, Scholarpedia.

[122]  J. O’Keefe,et al.  Modeling place fields in terms of the cortical inputs to the hippocampus , 2000, Hippocampus.

[123]  Barbara Tversky 1 Functional Significance of Visuospatial Representations , 2005 .

[124]  Dwight J. Kravitz,et al.  A new neural framework for visuospatial processing , 2011, Nature Reviews Neuroscience.

[125]  G. Einevoll,et al.  From grid cells to place cells: A mathematical model , 2006, Hippocampus.

[126]  M. Sholl,et al.  Allocentric coding of object-to-object relations in overlearned and novel environments. , 2005, Journal of Experimental Psychology. Learning, Memory and Cognition.

[127]  G. Vallortigara,et al.  From natural geometry to spatial cognition , 2012, Neuroscience & Biobehavioral Reviews.

[128]  M. Dawson,et al.  Connectionism, Confusion and Cognitive Science , 1994 .

[129]  E. Reed The Ecological Approach to Visual Perception , 1989 .

[130]  Alexei V. Samsonovich,et al.  Toward a Unified Catalog of Implemented Cognitive Architectures , 2010, BICA.

[131]  T. Hafting,et al.  Finite Scale of Spatial Representation in the Hippocampus , 2008, Science.

[132]  J. O'Doherty,et al.  Reward representations and reward-related learning in the human brain: insights from neuroimaging , 2004, Current Opinion in Neurobiology.

[133]  I. J. Myung,et al.  Toward a method of selecting among computational models of cognition. , 2002, Psychological review.

[134]  Howard Eichenbaum,et al.  A cognitive map for object memory in the hippocampus. , 2009, Learning & memory.

[135]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[136]  Aaron Sloman,et al.  What Sort of Architecture is Required for a Human-Like Agent? , 1999 .

[137]  Inah Lee,et al.  Neural Correlates of Object-in-Place Learning in Hippocampus and Prefrontal Cortex , 2011, The Journal of Neuroscience.

[138]  Wai-Kiang Yeap,et al.  Robotics and Cognitive Approaches to Spatial Mapping , 2010, Springer Tracts in Advanced Robotics.

[139]  Richard A Andersen,et al.  Parietal reach region encodes reach depth using retinal disparity and vergence angle signals. , 2009, Journal of neurophysiology.

[140]  Fred W. Mast,et al.  The neural basis of the egocentric and allocentric spatial frame of reference , 2007, Brain Research.

[141]  M. E. Hasselmo,et al.  Modeling of grid cell activity demonstrates in vivo entorhinal ‘look-ahead’ properties , 2013, Neuroscience.

[142]  M. D’Esposito Working memory. , 2008, Handbook of clinical neurology.

[143]  Bruno B Averbeck,et al.  Neural Ensemble Decoding Reveals a Correlate of Viewer- to Object-Centered Spatial Transformation in Monkey Parietal Cortex , 2008, The Journal of Neuroscience.

[144]  T. McNamara Mental representations of spatial relations , 1986, Cognitive Psychology.

[145]  Rudolf Kadlec,et al.  A computational model of the allocentric and egocentric spatial memory by means of virtual agents, or how simple virtual agents can help to build complex computational models , 2012, Cognitive Systems Research.

[146]  Wai-Kiang Yeap Towards a Computational Theory of Cognitive Maps , 1988, Artif. Intell..

[147]  E. Save,et al.  Coding for spatial goals in the prelimbic/infralimbic area of the rat frontal cortex. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[148]  Tony J. Prescott,et al.  Hippocampus as unitary coherent particle filter , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[149]  Michael E. Hasselmo,et al.  A biophysical implementation of a bidirectional graph search algorithm to solve multiple goal navigation tasks , 2005, Connect. Sci..

[150]  B. Poucet Spatial cognitive maps in animals: new hypotheses on their structure and neural mechanisms. , 1993, Psychological review.

[151]  M. Hasselmo Grid cell mechanisms and function: Contributions of entorhinal persistent spiking and phase resetting , 2008, Hippocampus.

[152]  Ricardo Chavarriaga,et al.  Robust self-localisation and navigation based on hippocampal place cells , 2005, Neural Networks.

[153]  Uğur M Erdem,et al.  A goal‐directed spatial navigation model using forward trajectory planning based on grid cells , 2012, The European journal of neuroscience.

[154]  M. Raubal UMAN WAYFINDING IN UNFAMILIAR BUILDINGS : A SIMULATION WITH A COGNIZING AGENT , 2001 .

[155]  Paul Graham,et al.  View-based navigation in insects: how wood ants (Formica rufa L.) look at and are guided by extended landmarks. , 2002, The Journal of experimental biology.

[156]  Xi Zhang,et al.  Top-down versus bottom-up learning in cognitive skill acquisition , 2004, Cognitive Systems Research.

[157]  Sebastian Thrun,et al.  Simultaneous Localization and Mapping , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[158]  Wlodzislaw Duch,et al.  Cognitive Architectures: Where do we go from here? , 2008, AGI.

[159]  Markus Plank,et al.  Behavioral, Electrocortical and Neuroanatomical Correlates of Egocentric and Allocentric Reference Frames during Visual Path Integration , 2009 .

[160]  E N Brown,et al.  A Statistical Paradigm for Neural Spike Train Decoding Applied to Position Prediction from Ensemble Firing Patterns of Rat Hippocampal Place Cells , 1998, The Journal of Neuroscience.

[161]  Stan Franklin,et al.  IDA, a Conscious Artifact? , 2006 .