Real-time adaptation technique to real robots: an experiment with a humanoid robot

We introduce a technique that allows a real robot to execute a real-time learning, in which GP and RL are integrated. In our former research, we showed the result of an experiment with a real robot "AIBO" and proved the technique performed better than the traditional Q-learning method. Based on the proposed technique, we can acquire the common programs using a GP, applicable to various types of robots. We execute reinforcement learning with the acquired program in a real robot. In this way, the robot can adapt to its own operational characteristics and learn effective actions. In this paper, we show the experimental results in which a humanoid robot "HOAP-1" has been evolved to perform effectively to solve the box-moving task.

[1]  Björn Andersson,et al.  On-Line Evolution of Control for a Four-Legged Robot Using Genetic Programming , 2000, EvoWorkshops.

[2]  Hitoshi Iba,et al.  Multi-agent Robot Learning by Means of Genetic Programming: Solving an Escape Problem , 2001, ICES.

[3]  Shigenobu Kobayashi,et al.  Reinforcement learning of walking behavior for a four-legged robot , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[4]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[5]  Hitoshi Iba,et al.  Integration of Genetic Programming and Reinforcement Learning for Real Robots , 2003, GECCO.

[6]  Mats G. Nordahl,et al.  Sound Localization for a Humanoid Robot by Means of Genetic Programming , 2000, EvoWorkshops.

[7]  Jordi Madrenas,et al.  Evolvable Systems: From Biology to Hardware , 1996, Lecture Notes in Computer Science.

[8]  John J. Grefenstette,et al.  Simulation-Assisted Learning by Competition: Effects of Noise Differences Between Training Model and Target Environment , 1990, ML.

[9]  William B. Langdon,et al.  Evolving Hand-Eye Coordination for a Humanoid Robot with Machine Code Genetic Programming , 2001, EuroGP.

[10]  Francesco Mondada,et al.  Evolution of homing navigation in a real mobile robot , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[11]  Stefano Nolfi,et al.  Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines , 2000 .

[12]  Mats G. Nordahl,et al.  An evolutionary architecture for a humanoid robot , 1999 .

[13]  K. Downing.  Adaptive genetic programs via reinforcement learning , 2001 .

[14]  Diego Calvanese,et al.  Unifying Class-Based Representation Formalisms , 2011, J. Artif. Intell. Res..

[15]  John J. Grefenstette,et al.  Evolutionary Algorithms for Reinforcement Learning , 1999, J. Artif. Intell. Res..

[16]  Marco Colombetti,et al.  Robot Shaping: An Experiment in Behavior Engineering , 1997 .

[17]  M. Asada,et al.  Purposive Behavior Acquisitionfor a Real Rob ot by Vi si on-Based Rei nforcement Learning , .

[18]  Mats G. Nordahl,et al.  Stereoscopic Vision for a Humanoid Robot Using Genetic Programming , 2000, EvoWorkshops.

[19]  Minoru Asada,et al.  Purposive Behavior Acquisition for a Real Robot by Vision-Based Reinforcement Learning , 2005, Machine Learning.

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Minoru Asada,et al.  Reasonable performance in less learning time by real robot based on incremental state space segmentation , 1996, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS '96.