A Novel Hierarchical Adaptive Controller with Dynamic Memory for an Autonomous Robot

This work concerns practical issues surrounding the application of learning and memory in a robot toward optimal navigation in dynamic environments. A novel hierarchical adaptive controller that contains two-level units was developed and trained in a physical mobile robot. In the low-level unit, the robot holds N numbers of biologically inspired Aplysia-like spiking neural networks that have the property of spike timedependent plasticity. Each of these networks is trained to become an expert in a particular local environment(s). All the trained networks are stored in a tree-type memory structure in the highlevel unit. These stored networks are used as experiences for the robot to enhance its navigation ability in the upcoming environments. Forgetting and dynamic clustering techniques are also developed to control the memory size. Experimental results show that the proposed model can produce a robot with learning and memorizing capabilities that enable it to survive in highly dynamic environments.

[1]  Jacob Nielsen,et al.  Spiking neural building block robot with Hebbian learning , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[2]  Stefano Nolfi,et al.  Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems , 1998, Neural Networks.

[3]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[4]  Michael J. Carey,et al.  A Study of Index Structures for a Main Memory Database Management System , 1986, HPTS.

[5]  Andrew W. Moore,et al.  Efficient memory-based learning for robot control , 1990 .

[6]  Kazuyuki Murase,et al.  An Autonomous Mobile Robot Controlled by a Spike Neuron Network with one Hidden-Layer Neuron having Spike Timing-Dependent Plasticity , 2006 .

[7]  Michael V. Johnston,et al.  Fundamental neuroscience, 2nd edition: Edited by Duane E. Haines, PhD. 582 pp., illustrated. New York: Churchill Livingstone, 2002. $52.00. ISBN 0-443-066035. , 2002 .

[8]  Andrew McCallum,et al.  Reinforcement learning with selective perception and hidden state , 1996 .

[9]  Ming Tan,et al.  Cost-Sensitive Reinforcement Learning for Adaptive Classification and Control , 1991, AAAI.

[10]  Zsolt Kira,et al.  Forgetting Bad Behavior: Memory Management for Case-Based Navigation , 2004 .

[11]  Zbigniew Michalewicz,et al.  Adaptive evolutionary planner/navigator for mobile robots , 1997, IEEE Trans. Evol. Comput..

[12]  Hugh F. Durrant-Whyte,et al.  A solution to the simultaneous localization and map building (SLAM) problem , 2001, IEEE Trans. Robotics Autom..

[13]  Kazuyuki Murase,et al.  Self-organization of Spiking Neural Network that Generates Autonomous Behavior in a Real Mobile Robot , 2006, Int. J. Neural Syst..

[14]  F. Craik,et al.  Levels of Pro-cessing: A Framework for Memory Research , 1975 .

[15]  K. Ganesan,et al.  Case-based path planning for autonomous underwater vehicles , 1994, Auton. Robots.

[16]  Kazuyuki Murase,et al.  Sensor-fusion in spiking neural network that generates autonomous behavior in real mobile robot , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[17]  Ronald C. Arkin,et al.  Learning behavioral parameterization using spatio-temporal case-based reasoning , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[18]  Longxin Lin Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching , 2004, Machine Learning.

[19]  Andreas Zimmermann,et al.  Context-Awareness in User Modelling: Requirements Analysis for a Case-Based Reasoning Application , 2003, ICCBR.

[20]  Genhong Cheng,et al.  The Art of War: Innate and adaptive immune responses , 2003, Cellular and Molecular Life Sciences CMLS.

[21]  Jürgen Branke,et al.  Evolutionary Optimization in Dynamic Environments , 2001, Genetic Algorithms and Evolutionary Computation.