Learning Capture Points for humanoid push recovery

We present a method for learning capture points for humanoid push recovery. A capture point is a point on the ground to which the biped can step and stop without requiring another step. Being able to predict the location of such points is very useful for recovery from significant disturbances, such as after being pushed. While dynamic models can be used to compute capture points, model assumptions and modeling errors can lead to stepping in the wrong place, which can result in large velocity errors after stepping.We present a method for computing capture points by learning offsets to the capture points predicted by the linear inverted pendulum model, which assumes a point mass biped with constant center of Mass height. We validate our method on a three dimensional humanoid robot simulation with 12 actuated lower body degrees of freedom, distributed mass, and articulated limbs. Using our learning approach, robustness to pushes is significantly improved as compared to using the linear inverted pendulum model without learning.

[1]  Jun Morimoto,et al.  A simple reinforcement learning algorithm for biped walking , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[2]  W. T. Miller.  Learning dynamic balance of a biped walking robot , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[3]  Masaki Ogino,et al.  Vision-based reinforcement learning for humanoid behavior generation with rhythmic walking parameters , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[4]  Toshio Fukuda,et al.  Stabilization control of biped locomotion robot based learning with GAs having self-adaptive mutation and recurrent neural networks , 1997, Proceedings of International Conference on Robotics and Automation.

[5]  Jaewon Kho,et al.  A learning controller for repetitive gait control of biped walking robot , 2004, SICE 2004 Annual Conference.

[6]  Taku Komura,et al.  A Feedback Controller for Biped Humanoids that Can Counteract Large Perturbations During Gait , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[7]  H. Sebastian Seung,et al.  Stochastic policy gradient reinforcement learning on a simple 3D biped , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[8]  T. Y. Kuc,et al.  Fourier series learning of biped walking motion , 2002, Proceedings of the IEEE Internatinal Symposium on Intelligent Control.

[9]  Gordon Cheng,et al.  Disturbance Rejection for Biped Humanoids , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[10]  Robert Babuska,et al.  Reinforcement Learning Control for Biped Robot Walking on Uneven Surfaces , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[11]  C. Atkeson,et al.  Minimax differential dynamic programming: application to a biped walking robot , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[12]  Jun Morimoto,et al.  Poincaré-Map-Based Reinforcement Learning For Biped Walking , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[13]  Sergey V. Drakunov,et al.  Capture Point: A Step toward Humanoid Push Recovery , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[14]  Jae Won Kho,et al.  Implementation of an Intelligent Controller for Biped Walking Robot using Genetic Algorithm , 2006, 2006 IEEE International Symposium on Industrial Electronics.

[15]  Shin Ishii,et al.  Reinforcement Learning for CPG-Driven Biped Robot , 2004, AAAI.

[16]  Ambarish Goswami,et al.  A Biomechanically Motivated Two-Phase Strategy for Biped Upright Balance Control , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[17]  Atsuo Takanishi,et al.  Learning of robot biped walking with the cooperation of a human , 1993, Proceedings of 1993 2nd IEEE International Workshop on Robot and Human Communication.

[18]  Marko B. Popovic,et al.  A sliding controller for bipedal balancing using integrated movement of contact and non-contact limbs , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[19]  Alan F. Murray,et al.  IEEE International Conference on Neural Networks , 1997 .

[20]  Gordon Cheng,et al.  Gravity Compensation and Full-Body Balancing for Humanoid Robots , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[21]  Andreas G. Hofmann.  Robust execution of bipedal walking tasks from biomechanical principles , 2006 .

[22]  Shin Ishii,et al.  On-line learning of a feedback controller for quasi-passive-dynamic walking by a stochastic policy gradient method , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Jerry Pratt,et al.  Velocity-Based Stability Margins for Fast Bipedal Walking , 2006 .

[24]  Atsuo Takanishi,et al.  Learning of Walking Stabilization with the Cooperation of a Human for a Biped Walking Robot , 1995 .

[25]  Christopher G. Atkeson,et al.  Controlling Velocity In Bipedal Walking: A Dynamic Programming Approach , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[26]  Qinghua Li,et al.  Learning control for a biped walking robot with a trunk , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[27]  Chee-Meng Chew,et al.  Dynamic bipedal walking assisted by learning , 2002, Robotica.

[28]  Chee-Meng Chew,et al.  Frontal plane algorithms for dynamic bipedal walking , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[29]  Kazuhito Yokoi,et al.  The 3D linear inverted pendulum mode: a simple modeling for a biped walking pattern generation , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[30]  Sergey V. Drakunov,et al.  Derivation and Application of a Conserved Orbital Energy for the Inverted Pendulum Bipedal Walking Model , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[31]  Jiaxin Wang,et al.  A Reinforcement Learning Based Dynamic Walking Control , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[32]  Jerry E. Pratt,et al.  Exploiting inherent robustness and natural dynamics in the control of bipedal walking robots , 2000 .

[33]  Fumihiko Asano,et al.  Virtual coupling control for dynamic bipedal walking , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[34]  Changjiu Zhou,et al.  Reinforcement learning with fuzzy evaluative feedback for a biped robot , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[35]  Jun Morimoto,et al.  Learning Biped Locomotion , 2007, IEEE Robotics & Automation Magazine.

[36]  Martijn Wisse,et al.  Using a controller based on reinforcement learning for a passive dynamic walking robot , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..