Learning, planning, and control for quadruped locomotion over challenging terrain

We present a control architecture for fast quadruped locomotion over rough terrain. We approach the problem by decomposing it into many sub-systems, in which we apply state-of-the-art learning, planning, optimization, and control techniques to achieve robust, fast locomotion. Unique features of our control strategy include: (1) a system that learns optimal foothold choices from expert demonstration using terrain templates, (2) a body trajectory optimizer based on the Zero-Moment Point (ZMP) stability criterion, and (3) a floating-base inverse dynamics controller that, in conjunction with force control, allows for robust, compliant locomotion over unperceived obstacles. We evaluate the performance of our controller by testing it on the LittleDog quadruped robot, over a wide variety of rough terrains of varying difficulty levels. The terrain that the robot was tested on includes rocks, logs, steps, barriers, and gaps, with obstacle sizes up to the leg length of the robot. We demonstrate the generalization ability of this controller by presenting results from testing performed by an independent external test team on terrain that has never been shown to us.

[1]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[2]  Robert B. McGhee,et al.  Finite state control of quadruped locomotion , 1967 .

[3]  R. McGhee,et al.  On the stability properties of quadruped creeping gaits , 1968 .

[4]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[5]  S. Hirose,et al.  Titan III, A quadruped walking vehicle , 1985 .

[6]  Marc H. Raibert,et al.  Legged Robots That Balance , 1986, IEEE Expert.

[7]  Jessica K. Hodgins,et al.  Adjusting step length for rough terrain locomotion , 1991, IEEE Trans. Robotics Autom..

[8]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[9]  Yoram Singer,et al.  Learning to Order Things , 1997, NIPS.

[10]  Steven Dubowsky,et al.  Coordinated Motion and Force Control of Multi-Limbed Robotic Systems , 1999, Auton. Robots.

[11]  Bruno Siciliano,et al.  Modelling and Control of Robot Manipulators , 1997, Advanced Textbooks in Control and Signal Processing.

[12]  L. Siciliano.  Modelling and Control of Robot Manipulators , 2000 .

[13]  Daniel E. Koditschek,et al.  RHex: A Simple and Highly Mobile Hexapod Robot , 2001, Int. J. Robotics Res..

[14]  Hisashi Tamaki,et al.  Walking pattern acquisition for quadruped robot by using modular reinforcement learning , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[15]  Kazuhito Yokoi,et al.  A realtime pattern generator for biped walking , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[16]  Bernhard Schölkopf,et al.  A Short Introduction to Learning with Kernels , 2002, Machine Learning Summer School.

[17]  Hirochika Inoue,et al.  Real-time humanoid motion generation through ZMP manipulation based on inverted pendulum control , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[18]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[19]  Kazuhito Yokoi,et al.  Biped walking pattern generation by using preview control of zero-moment point , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[20]  Yasuhiro Fukuoka,et al.  Adaptive Dynamic Walking of a Quadruped Robot on Irregular Terrain Based on Biological Concepts , 2003, Int. J. Robotics Res..

[21]  Sebastian Thrun,et al.  ARA*: Anytime A* with Provable Bounds on Sub-Optimality , 2003, NIPS.

[22]  Mark R. Cutkosky,et al.  Stride Period Adaptation of a Biomimetic Running Hexapod , 2004, Int. J. Robotics Res..

[23]  Miomir Vukobratovic,et al.  Zero-Moment Point - Thirty Five Years of its Life , 2004, Int. J. Humanoid Robotics.

[24]  Peter Stone,et al.  Policy gradient reinforcement learning for fast quadrupedal locomotion , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[25]  Ioannis Poulakakis,et al.  Rotary gallop in the untethered quadrupedal robot scout II , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[26]  A. Ng.  Feature selection, L1 vs. L2 regularization, and rotational invariance , 2004, Twenty-first international conference on Machine learning - ICML '04.

[27]  Farhad Aghili,et al.  A unified approach for inverse and direct dynamics of constrained multibody systems based on linear projection operator: applications to control and simulation , 2005, IEEE Transactions on Robotics.

[28]  M. Sniedovich.  Dijkstra's algorithm revisited: the dynamic programming connexion , 2006 .

[29]  Jong-Hwan Kim,et al.  ZMP Analysis for Realisation of Humanoid Motion on Complex Topologies , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[30]  Oussama Khatib,et al.  Synthesis and control of whole-body behaviors in humanoid systems , 2007 .

[31]  Stefan Schaal,et al.  Automatic Outlier Detection: A Bayesian Approach , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[32]  Pieter Abbeel,et al.  Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion , 2007, NIPS.

[33]  Roy Featherstone,et al.  Rigid Body Dynamics Algorithms , 2007 .

[34]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale l1-Regularized Logistic Regression , 2007, J. Mach. Learn. Res..

[35]  Jerry E. Pratt,et al.  A Controller for the LittleDog Quadruped Walking on Rough Terrain , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[36]  Auke Jan Ijspeert,et al.  Central pattern generators for locomotion control in animals and robots: A review , 2008, Neural Networks.

[37]  Jun Nakanishi,et al.  Operational Space Control: A Theoretical and Empirical Comparison , 2008, Int. J. Robotics Res..

[38]  Andrew Y. Ng,et al.  A control architecture for quadruped locomotion over rough terrain , 2008, 2008 IEEE International Conference on Robotics and Automation.

[39]  Kevin Blankespoor,et al.  BigDog, the Rough-Terrain Quadruped Robot , 2008 .

[40]  Nicholas Roy,et al.  Reliable Dynamic Motions for a Stiff Quadruped , 2009, ISER.

[41]  Jun Nakanishi,et al.  Inverse kinematics with floating base and constraints for full body humanoid robot control , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[42]  Michael Nalin Mistry.  The representation, learning, and control of dexterous motor skills in humans and humanoid robots , 2009 .

[43]  Siddhartha S. Srinivasa,et al.  CHOMP: Gradient optimization techniques for efficient motion planning , 2009, 2009 IEEE International Conference on Robotics and Automation.

[44]  Stefan Schaal,et al.  Compliant quadruped locomotion over rough terrain , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[45]  Stefan Schaal,et al.  Learning locomotion over rough terrain using terrain templates , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[46]  Stefan Schaal,et al.  Fast, robust quadruped locomotion over challenging terrain , 2010, 2010 IEEE International Conference on Robotics and Automation.

[47]  Christopher G. Atkeson,et al.  An optimization approach to rough terrain locomotion , 2010, 2010 IEEE International Conference on Robotics and Automation.

[48]  Stefan Schaal,et al.  Inverse dynamics control of floating base systems using orthogonal decomposition , 2010, 2010 IEEE International Conference on Robotics and Automation.