Download details: IP Address: 128.196.98.99

Rodent spatial navigation requires the dynamic evaluation of multiple sources of information, including visual cues, self-motion signals and reward signals. The nature of the evaluation, its dynamics and the relative weighting of the multiple information streams are largely unknown and have generated many hypotheses in the field of robotics. We use the framework of the traveling salesperson problem (TSP) to study how this evaluation may be achieved. The TSP is a classical artificial intelligence NP-hard problem that requires an agent to visit a fixed set of locations once, minimizing the total distance traveled. We show that after a few trials, rats converge on a short route between rewarded food cups. We propose that this route emerges from a series of local decisions that are derived from weighing information embedded in the context of the task. We study the relative weighting of spatial and reward information and establish that, in the conditions of this experiment, when the contingencies are not in conflict, rats choose the spatial or reward optimal solution. There was a trend toward a preference for space when the contingencies were in conflict. We also show that the spatial decision about which cup to go to next is biased by the orientation of the animal. Reward contingencies are also shown to significantly and dynamically modulate the decision-making process. This paradigm will allow for further neurophysiological studies aimed at understanding the synergistic role of brain areas involved in planning, reward processing and spatial navigation. These insights will in turn suggest new neural-like architectures for the control of mobile autonomous robots.

[1]  Camelia M. Kuhnen,et al.  Delays Conferred by Escalating Costs Modulate Dopamine Release to Rewards But Not Their Predictors , 2010, The Journal of Neuroscience.

[2]  Abraham P. Punnen,et al.  The traveling salesman problem and its variations , 2007 .

[3]  E. Menzel Chimpanzee Spatial Memory Organization , 1973, Science.

[4]  Bruno Poucet,et al.  Goal-Related Activity in Hippocampal Place Cells , 2007, The Journal of Neuroscience.

[5]  Douglas L Rosene,et al.  Asymmetry of neuron numbers in the hippocampal formation of prenatally malnourished and normally nourished rats: A stereological investigation , 2006, Hippocampus.

[6]  Stan B. Floresco,et al.  Cortico-limbic-striatal circuits subserving different forms of cost-benefit decision making , 2008, Cognitive, affective & behavioral neuroscience.

[7]  Richard M. Karp,et al.  Dynamic programming meets the principle of inclusion and exclusion , 1982, Oper. Res. Lett..

[8]  T G Bever,et al.  Peripheral and cerebral asymmetries in the rat. , 1997, Science.

[9]  Janet Wiles,et al.  Solving Navigational Uncertainty Using Grid Cells on Robots , 2010, PLoS Comput. Biol..

[10]  T. Ormerod,et al.  Convex hull or crossing avoidance? Solution heuristics in the traveling salesperson problem , 2004, Memory & cognition.

[11]  Richard W. Byrne,et al.  How do wild baboons (Papio ursinus) plan their routes? Travel among multiple high-quality food sources with inter-group competition , 2009, Animal Cognition.

[12]  Edward A Wasserman,et al.  Pigeons and people select efficient routes when solving a one-way "traveling salesperson" task. , 2007, Journal of experimental psychology. Animal behavior processes.

[13]  Alok Aggarwal,et al.  The angular-metric traveling salesman problem , 1997, SODA '97.

[14]  K M Gothard,et al.  Binding of hippocampal CA1 neural activity to multiple reference frames in a landmark-based navigation task , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[15]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[16]  Tim Kovacs,et al.  On optimal decision-making in brains and social insect colonies , 2009, Journal of The Royal Society Interface.

[17]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[18]  Noah D. Goodman,et al.  Optimal habits can develop spontaneously through sensitivity to local cost , 2010, Proceedings of the National Academy of Sciences.

[19]  John L Kubie,et al.  Heading‐vector navigation based on head‐direction cells and path integration , 2009, Hippocampus.

[20]  Audrey E. Cramer,et al.  Vervet monkeys as travelling salesmen , 1997, Nature.

[21]  Thora Tenbrink,et al.  The verbalization of multiple strategies in a variant of the traveling salesperson problem , 2009, Cognitive Processing.

[22]  Chelliah Sriskandarajah,et al.  A review of TSP based approaches for flowshop scheduling , 2006, Eur. J. Oper. Res..

[23]  M. Lee,et al.  The aesthetic appeal of minimal structures: Judging the attractiveness of solutions to traveling salesperson problems , 2006, Perception & psychophysics.

[24]  Sheri J. Y. Mizumori,et al.  Hippocampal Place Fields , 2008 .

[25]  C R Gallistel,et al.  Computations on metric maps in mammals: getting oriented and choosing a multi-destination route. , 1996, The Journal of experimental biology.

[26]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[27]  Lisa M. Giocomo,et al.  Phase precession and variable spatial scaling in a periodic attractor map model of medial entorhinal grid cells with realistic after‐spike dynamics , 2012, Hippocampus.

[28]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[29]  C. Jordan,et al.  Sex Differences, Laterality, and Hormonal Regulation of Androgen Receptor Immunoreactivity in Rat Hippocampus , 2002, Hormones and Behavior.

[30]  William J. Cook,et al.  The Traveling Salesman Problem: A Computational Study , 2007 .

[31]  L. Green,et al.  Dopamine modulates effort-based decision making in rats. , 2009, Behavioral neuroscience.

[32]  Sheri J Y Mizumori,et al.  Ventral tegmental area disruption selectively affects CA1/CA2 but not CA3 place fields during a differential reward working memory task , 2011, Hippocampus.

[33]  Kazuo Fujita,et al.  Route selection by pigeons (Columba livia) in "traveling salesperson" navigation tasks presented on an LCD screen. , 2010, Journal of comparative psychology.

[34]  C. Büchel,et al.  Differential Recruitment of the Hippocampus, Medial Prefrontal Cortex, and the Human Motion Complex during Path Integration in Humans , 2007, The Journal of Neuroscience.

[35]  W. Schultz Dopamine signals for reward value and risk: basic and recent data , 2010, Behavioral and Brain Functions.

[36]  M. Held,et al.  A dynamic programming approach to sequencing problems , 1962, ACM National Meeting.

[37]  J. O’Neill,et al.  The reorganization and reactivation of hippocampal maps predict spatial memory performance , 2010, Nature Neuroscience.

[38]  Jan Bureš,et al.  Can rats solve a simple version of the traveling salesman problem? , 1992, Behavioural Brain Research.

[39]  David R. Karger,et al.  Approximation algorithms for orienteering and discounted-reward TSP , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[40]  Thora Tenbrink,et al.  Conceptual layers and strategies in tour planning , 2011, Cognitive Processing.

[41]  J. C. Cardema,et al.  Optimal path planning of mobile robots for sample collection , 2004, J. Field Robotics.

[42]  Sheri J. Y. Mizumori,et al.  Hippocampal place fields : relevance to learning and memory , 2008 .

[43]  M. Tamosiunaite,et al.  Hippocampal CA1 Place Cells Encode Intended Destination on a Maze with Multiple Choice Points , 2007, The Journal of Neuroscience.

[44]  J. Salamone,et al.  Dopamine, Behavioral Economics, and Effort , 2009, Front. Behav. Neurosci..

[45]  K M Gothard,et al.  Dynamics of Mismatch Correction in the Hippocampal Ensemble Code for Space: Interaction between Path Integration and Environmental Cues , 1996, The Journal of Neuroscience.

[46]  Jeffrey S Taube,et al.  Head direction cell firing properties and behavioural performance in 3‐D space , 2011, The Journal of physiology.

[47]  Mark C. Fuhs,et al.  A Spin Glass Model of Path Integration in Rat Medial Entorhinal Cortex , 2006, The Journal of Neuroscience.

[48]  S. D. Glick Heritable differences in turning behavior of rats. , 1985, Life sciences.