A Humanoid Approaches to the Goal - Reinforcement Learning Based on Rhythmic Walking Parameters

This paper presents a method for generating vision-based humanoid behaviors by reinforcement learning with rhythmic walking parameters. The walking is stabilized by a rhythmic motion controller such as CPG or neural oscillator. The learning process consists of two stages: the first one is building an action space with two parameters (a forward step length and a turning angle) so that infeasible combinations of them are inhibited. The second one is reinforcement learning with the constructed action space and the state space consisting of visual features and posture parameters to find feasible action. The method is applied to a situation of the RoboCupSoccer Humanoid league, that is, to reach the ball and to shoot it into the goal. Instructions by human are given to start up the learning process and the rest is completely self-learning in real situations.

[1]  Shuuji Kajita,et al.  Adaptive Gait Control of a Biped Robot Based on Realtime Sensing of the Ground Profile , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[2]  Katsuyoshi Tsujita,et al.  Locomotion Control of a Quadruped Locomotion Robot based on Central Pattern Generator Model , 2002 .

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Atsuo Takanishi,et al.  Development of a dynamic biped walking system for humanoid - development of a biped walking robot adapting to the humans' living floor , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[5]  Hiroshi Kimura,et al.  Biologically-inspired adaptive dynamic walking of the quadruped on irregular terrain , 1999, Proceedings of the 1999 IEEE International Conference on Control Applications (Cat. No.99CH36328).

[6]  S. Grillner.  Neurobiological bases of rhythmic motor acts in vertebrates. , 1985, Science.

[7]  T. Takenaka,et al.  The development of Honda humanoid robot , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[8]  Hiroshi Shimizu,et al.  Self-organized control of bipedal locomotion by neural oscillators in unpredictable environment , 1991, Biological Cybernetics.

[9]  Gentaro Taga,et al.  A model of the neuro-musculo-skeletal system for anticipatory adjustment of human locomotion during obstacle avoidance , 1998, Biological Cybernetics.

[10]  J. Thomson,et al.  The role of visual information in control of a constrained locomotor task. , 1988, Journal of motor behavior.

[11]  Yasuo Kuniyoshi,et al.  Three dimensional bipedal stepping motion using neural oscillators-towards humanoid motion in the real world , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[12]  Hiroaki Kitano,et al.  The RoboCup humanoid challenge as the millennium challenge for advanced robotics , 1998, Adv. Robotics.