Comparative Experimental Studies on Spatial Memory and Learning in Rats and Robots

The study of behavioral and neurophysiological mechanisms involved in rat spatial cognition provides a basis for the development of computational models and robotic experimentation of goal-oriented learning tasks. These models and robotics architectures offer neurobiologists and neuroethologists alternative platforms to study, analyze and predict spatial cognition based behaviors. In this paper we present a comparative analysis of spatial cognition in rats and robots by contrasting similar goal-oriented tasks in a cyclical maze, where studies in rat spatial cognition are used to develop computational system-level models of hippocampus and striatum integrating kinesthetic and visual information to produce a cognitive map of the environment and drive robot experimentation. During training, Hebbian learning and reinforcement learning, in the form of Actor-Critic architecture, enable robots to learn the optimal route leading to a goal from a designated fixed location in the maze. During testing, robots exploit maximum expectations of reward stored within the previously acquired cognitive map to reach the goal from different starting positions. A detailed discussion of comparative experiments in rats and robots is presented contrasting learning latency while characterizing behavioral procedures during navigation such as errors associated with the selection of a non-optimal route, body rotations, normalized length of the traveled path, and hesitations. Additionally, we present results from evaluating neural activity in rats through detection of the immediate early gene Arc to verify the engagement of hippocampus and striatum in information processing while solving the cyclical maze task, such as robots use our corresponding models of those neural structures.

[1]  Angelo Arleo,et al.  Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity , 2000, Biological Cybernetics.

[2]  Alejandra Barrera,et al.  Biologically-inspired robot spatial cognition based on rat neurophysiological studies , 2008, Auton. Robots.

[3]  L. Swanson,et al.  The structural organization of connections between hypothalamus and cerebral cortex 1 Published on the World Wide Web on 2 June 1997. 1 , 1997, Brain Research Reviews.

[4]  Bruce L. McNaughton,et al.  Environment-specific expression of the immediate-early gene Arc in hippocampal neuronal ensembles , 1999, Nature Neuroscience.

[5]  Jean-Arcady Meyer,et al.  Global localization and topological map-learning for robot navigation , 2002 .

[6]  W. Seifert Neurobiology of the hippocampus , 1983 .

[7]  A. Weitzenfeld,et al.  Rat-inspired model of robot target learning and place recognition , 2007, 2007 Mediterranean Conference on Control & Automation.

[8]  B. Webb,et al.  Can robots make good models of biological behaviour? , 2001, Behavioral and Brain Sciences.

[9]  David J. Foster,et al.  A model of hippocampally dependent navigation, using the temporal difference learning rule , 2000, Hippocampus.

[10]  Alfredo Weitzenfeld,et al.  A Prey Catching and Predator Avoidance Neural-Schema Architecture for Single and Multiple Robots , 2008, J. Intell. Robotic Syst..

[11]  Joel L. Davis,et al.  Adaptive Critics and the Basal Ganglia , 1995 .

[12]  Emilio Kropff,et al.  Place cells, grid cells, and the brain's spatial representation system. , 2008, Annual review of neuroscience.

[13]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[14]  J. Hollerman,et al.  Reward prediction in primate basal ganglia and frontal cortex , 1998, Neuropharmacology.

[15]  Philippe Gaussier,et al.  From view cells and place cells to cognitive map learning: processing stages of the hippocampal system , 2002, Biological Cybernetics.

[16]  P. Worley,et al.  Immediate-Early Genes and Synaptic Function , 1998, Neurobiology of Learning and Memory.

[17]  E. J. Green,et al.  Cortical representation of motion during unrestrained spatial navigation in the rat. , 1994, Cerebral cortex.

[18]  Bruno Poucet,et al.  Involvement of the rat prefrontal cortex in cognitive functions: A central role for the prelimbic area , 2000, Psychobiology.

[19]  Michael A. Arbib,et al.  Affordances. Motivations, and the World Graph Theory , 1998, Adapt. Behav..

[20]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[21]  C. L. Hull The goal-gradient hypothesis and maze learning. , 1932 .

[22]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[23]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[24]  A. Kelley Ventral striatal control of appetitive motivation: role in ingestive behavior and reward-related learning , 2004, Neuroscience & Biobehavioral Reviews.

[25]  Stefan Leutgeb,et al.  Pattern separation, pattern completion, and new neuronal codes within a continuous CA3 map. , 2007, Learning & memory.

[26]  William A. Roberts,et al.  Principles of Animal Cognition , 1997 .

[27]  B. Webb What does robotics offer animal behaviour? , 2000, Animal Behaviour.

[28]  E. Tolman Cognitive maps in rats and men. , 1948, Psychological review.

[29]  James J. Knierim,et al.  Ensemble Dynamics of Hippocampal Regions CA3 and CA1 , 2004, Neuron.

[30]  P. E. Sharp,et al.  Simulation of spatial learning in the Morris water maze by a neural network model of the hippocampal formation and nucleus accumbens , 1995, Hippocampus.

[31]  T. Collett,et al.  Animal Navigation: Path Integration, Visual Landmarks and Cognitive Maps , 2004, Current Biology.

[32]  Alfredo Weitzenfeld,et al.  From schemas to neural networks: A multi-level modelling approach to biologically-inspired autonomous robotic systems , 2008, Robotics Auton. Syst..

[33]  Carol A Barnes,et al.  Spatial Exploration-Induced Arc mRNA and Protein Expression: Evidence for Selective, Network-Specific Reactivation , 2005, The Journal of Neuroscience.

[34]  C. Barnes,et al.  Spatial exploration induces ARC, a plasticity‐related immediate‐early gene, only in calcium/calmodulin‐dependent protein kinase II‐positive principal excitatory and inhibitory neurons of the rat forebrain , 2006, The Journal of comparative neurology.

[35]  K. Jeffery,et al.  Learned interaction of visual and idiothetic cues in the control of place field orientation , 1999, Experimental Brain Research.

[36]  J. Hollerman,et al.  Reward processing in primate orbitofrontal cortex and basal ganglia. , 2000, Cerebral cortex.

[37]  N. Burgess,et al.  Complementary memory systems: competition, cooperation and compensation , 2005, Trends in Neurosciences.

[38]  Michael Recce,et al.  A model of hippocampal function , 1994, Neural Networks.

[39]  A. Barto,et al.  Adaptive Critics and the Basal Ganglia , 1994 .

[40]  R. Muller,et al.  The firing of hippocampal place cells in the dark depends on the rat's recent experience , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[41]  R. Passingham The hippocampus as a cognitive map J. O'Keefe & L. Nadel, Oxford University Press, Oxford (1978). 570 pp., £25.00 , 1979, Neuroscience.

[42]  P. E. Sharp,et al.  Head direction, place, and movement correlates for cells in the rat retrosplenial cortex. , 2001, Behavioral neuroscience.

[43]  Angelo Arleo,et al.  Cognitive navigation based on nonuniform Gabor space sampling, unsupervised growing networks, and reinforcement learning , 2004, IEEE Transactions on Neural Networks.

[44]  Michael A. Arbib,et al.  The Neural Simulation Language: A System for Brain Modeling , 2002 .

[45]  Gordon Wyeth,et al.  Spatial Mapping and Map Exploitation: A Bio-inspired Engineering Perspective , 2007, COSIT.

[46]  E. Save,et al.  Evidence for entorhinal and parietal cortices involvement in path integration in the rat , 2004, Experimental Brain Research.

[47]  J. O’Keefe,et al.  Hippocampal place units in the freely moving rat: Why they fire where they fire , 1978, Experimental Brain Research.

[48]  D. Touretzky,et al.  Cognitive maps beyond the hippocampus , 1997, Hippocampus.

[49]  Stephan Winter,et al.  Spatial Information Theory, 8th International Conference, COSIT 2007, Melbourne, Australia, September 19-23, 2007, Proceedings , 2007, COSIT.

[50]  R. Morris Spatial Localization Does Not Require the Presence of Local Cues , 1981 .

[51]  Carol A Barnes,et al.  Integration of New Neurons into Functional Neural Networks , 2006, The Journal of Neuroscience.

[52]  E. Bizzi,et al.  The Cognitive Neurosciences , 1996 .

[53]  B. Poucet Spatial cognitive maps in animals: new hypotheses on their structure and neural mechanisms. , 1993, Psychological review.