Navigation of mobile robot using location map of place cells and reinforcement learning

It is known that the hippocampus of rats contains neural cells called “place cells.” Place cells are neural cells that respond selectively when the rat arrives at a particular place. This paper proposes a mobile robot navigation method that uses a place cell position map and reinforcement learning. First, the place cell position map is created by a neural gas using image data and position data from observation points. Next, routes between the place cells are established, and the path to the goal is learned using the actor–critic method, which is a type of reinforcement learning method. Numerical simulations demonstrate that the goal can be reached even if there are motion errors by moving the robot along the route. © 2007 Wiley Periodicals, Inc. Syst Comp Jpn, 38(7): 65–75, 2007; Published online in Wiley InterScience (www.interscience. wiley.com). DOI 10.1002/scj.20632

[1]  Yoshiaki Shirai,et al.  Mobile robot localization based on eigenspace analysis , 1997, Systems and Computers in Japan.

[2]  Takio Kurita,et al.  Constructing a map of place cells for mobile robot navigation , 2004 .

[3]  Kenji Nishida,et al.  Navigation of Mobile Robot using Neural-Gas and Reinforcement Learning , 2002 .

[4]  David J. Foster,et al.  A model of hippocampally dependent navigation, using the temporal difference learning rule , 2000, Hippocampus.

[5]  Yoshiaki Shirai,et al.  Selection of efficient landmarks for an autonomous vehicle , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[6]  Jean-Claude Latombe,et al.  Reliable navigation using landmarks , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[7]  R. Morris,et al.  Place navigation impaired in rats with hippocampal lesions , 1982, Nature.

[8]  Naokazu Yokoya,et al.  Memory-based self-localization using omnidirectional images , 2003, Systems and Computers in Japan.

[9]  Naokazu Yokoya,et al.  Memory-based self-localization using omnidirectional images , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[10]  J. O’Keefe,et al.  Hippocampal place units in the freely moving rat: Why they fire where they fire , 1978, Experimental Brain Research.

[11]  J. O’Keefe,et al.  Geometric determinants of the place fields of hippocampal neurons , 1996, Nature.

[12]  J. O'Keefe,et al.  The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. , 1971, Brain research.

[13]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.