Multiple levels of spatial organization: World Graphs and spatial difference learning

It is often suggested that the place cells of the hippocampus (and more recently, the grid cells of the entorhinal cortex) furnish a cognitive map. However, this can only be part of the story: (1) the “hippocampal chart” provided by place cells and grid cells differs radically when a rat is placed in different environments and so a higher level organization is needed to link these charts into an overall cognitive map of the rat’s world; and (2) even without a hippocampus a rat can exploit much of the spatial structure of its world. The World Graph (WG) model addressed the former problem, whereas the Taxon Affordance Model (TAM) was developed to address the latter, with the two models being integrated to form the TAM-WG model. Here we provide a new version strengthened in three ways: (1) we relate the TAM model to explicit ideas of executability and desirability; (2) we show how temporal difference learning elegantly supplies “spatial difference learning” to resolve the debate between the local hypothesis and the non-local hypothesis for node selection in the original WG model; and (3) we analyze an explicit example of how the “locometric map” provided by grid cells and place cells can complement the high-level cognitive map given by the WG, demonstrating the importance of navigation algorithms that integrate across multiple levels of spatial organization.

[1]  A. Bonci,et al.  Orexin B/hypocretin 2 increases glutamatergic transmission to ventral tegmental area neurons , 2008, The European journal of neuroscience.

[2]  Bruno Poucet,et al.  Functional interaction between the associative parietal cortex and hippocampal place cell firing in the rat , 2005, The European journal of neuroscience.

[3]  Janet Wiles,et al.  Learning spatial concepts from RatSLAM representations , 2006, Robotics Auton. Syst..

[4]  L. Abbott,et al.  A model of multiplicative neural responses in parietal cortex. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[5]  M A Arbib,et al.  From visual affordances in monkey parietal cortex to hippocampo-parietal interactions underlying rat navigation. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[6]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[7]  L. Swanson,et al.  Anatomy of the soul as reflected in the cerebral hemispheres: Neural circuits underlying voluntary control of basic motivated behaviors , 2005, The Journal of comparative neurology.

[8]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[9]  J. Gibson.  The optical expansion-pattern in aerial locomotion. , 1955, The American journal of psychology.

[10]  G. Schöner,et al.  Dynamic Field Theory of Movement Preparation , 2022 .

[11]  Gordon Wyeth,et al.  Persistent Navigation and Mapping using a Biologically Inspired SLAM System , 2010, Int. J. Robotics Res..

[12]  Amy Coplan.  Simulating Minds: The Philosophy, Psychology, and Neuroscience of Mindreading by goldman, alvin , 2008 .

[13]  L. Swanson.  Cerebral hemisphere regulation of motivated behavior 1 1 Published on the World Wide Web on 2 November 2000. , 2000, Brain Research.

[14]  M. A. Arbib,et al.  A Model of the Effects of Speed, Accuracy, and Perturbation on Visually Guided Reaching , 1992 .

[15]  Michael A. Arbib,et al.  The Neural Simulation Language: A System for Brain Modeling , 2002 .

[16]  E. J. Green,et al.  Cortical representation of motion during unrestrained spatial navigation in the rat. , 1994, Cerebral cortex.

[17]  Nestor A. Schmajuk,et al.  Exploration, Navigation and Cognitive Mapping , 2000, Adapt. Behav..

[18]  Richard S. Sutton,et al.  Reinforcement learning with replacing eligibility traces , 2004, Machine Learning.

[19]  C. Colby.  Action-Oriented Spatial Reference Frames in Cortex , 1998, Neuron.

[20]  M. A. Arbib,et al.  Models of Trajectory Formation and Temporal Interaction of Reach and Grasp. , 1993, Journal of motor behavior.

[21]  泰義 横小路,et al.  IEEE International Conference on Robotics and Automation , 1992 .

[22]  L. W. Swanson,et al.  Neural mechanisms for the functional coupling of autonomic, endocrine and somatomotor responses in adaptive behavior , 1981, Brain Research Reviews.

[23]  J. Taube,et al.  Where am I and how will I get there from here? A role for posterior parietal cortex in the integration of spatial information and route planning , 2009, Neurobiology of Learning and Memory.

[24]  Douglas Nitz,et al.  Parietal cortex, navigation, and the construction of arbitrary reference frames for spatial information , 2009, Neurobiology of Learning and Memory.

[25]  B. McNaughton,et al.  Cortical-hippocampal interactions and cognitive mapping: A hypothesis based on reintegration of the parietal and inferotemporal pathways for visual processing , 1989 .

[26]  M A Arbib,et al.  Competitive Hebbian learning and the hippocampal place cell system: Modeling the interaction of visual and path integration cues , 2001, Hippocampus.

[27]  W. Seifert.  Neurobiology of the hippocampus , 1983 .

[28]  M V Srinivasan,et al.  Honeybee navigation: nature and calibration of the "odometer". , 2000, Science.

[29]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[30]  Bartlett W. Mel.  Synaptic integration in an excitable dendritic tree. , 1993, Journal of neurophysiology.

[31]  Michael A. Arbib,et al.  Affordances. Motivations, and the World Graph Theory , 1998, Adapt. Behav..

[32]  M. Arbib,et al.  Motivational Learning of Spatial Behavior , 1977 .

[33]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[34]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[35]  Peter Ford Dominey,et al.  A cortico-subcortical model for generation of spatially accurate sequential saccades. , 1992, Cerebral cortex.

[36]  R. Passingham.  The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[37]  T. Sakurai.  Roles of orexins and orexin receptors in central regulation of feeding behavior and energy homeostasis. , 2006, CNS & neurological disorders drug targets.

[38]  Si Wu,et al.  Population Coding and Decoding in a Neural Field: A Computational Study , 2002, Neural Computation.

[39]  M. Arbib,et al.  Multiple representations of space underlying behavior , 1982, Behavioral and Brain Sciences.

[40]  S. Amari,et al.  Competition and Cooperation in Neural Nets , 1982 .

[41]  C. Berridge,et al.  Hypocretin/Orexin Selectively Increases Dopamine Efflux within the Prefrontal Cortex: Involvement of the Ventral Tegmental Area , 2006, Neuropsychopharmacology.

[42]  Michael A. Arbib,et al.  Extending the mirror neuron system model, II: what did I just do? A new role for mirror neurons , 2010, Biological Cybernetics.