Nonholonomic virtual constraints and gait optimization for robust walking control

A key challenge in robotic bipedal locomotion is the design of feedback controllers that function well in the presence of uncertainty, in both the robot and its environment. This paper addresses the design of feedback controllers and periodic gaits that function well in the presence of modest terrain variation, without over-reliance on perception and a priori knowledge of the environment. Model-based design methods are introduced and subsequently validated in simulation and experiment on MARLO, an underactuated three-dimensional bipedal robot that is of roughly human size and is equipped with an inertial measurement unit and joint encoders. Innovations include an optimization method that accounts for multiple types of disturbances and a feedback control design that enables continuous velocity-based posture regulation via nonholonomic virtual constraints. Using a single continuously defined controller taken directly from optimization, MARLO traverses sloped sidewalks and parking lots, terrain covered with randomly thrown boards, and grass fields, all while maintaining average walking speeds between 0.9 and 0.98 m/s and setting a new precedent for walking efficiency in realistic environments.

[1]  Koushil Sreenath,et al.  A Compliant Hybrid Zero Dynamics Controller for Stable, Efficient and Fast Bipedal Walking on MABEL , 2011, Int. J. Robotics Res..

[2]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[3]  Jessy W. Grizzle,et al.  Experimental Validation of a Framework for the Design of Controllers that Induce Stable Walking in Planar Bipeds , 2004, Int. J. Robotics Res..

[4]  Russ Tedrake,et al.  L2-gain optimization for robust bipedal walking on unknown terrain , 2013, 2013 IEEE International Conference on Robotics and Automation.

[5]  Christine Chevallereau,et al.  Asymptotically Stable Walking of a Five-Link Underactuated 3-D Bipedal Robot , 2009, IEEE Transactions on Robotics.

[6]  Andrew Y. Ng,et al.  Policy search via the signed derivative , 2009, Robotics: Science and Systems.

[7]  Katie Byl,et al.  Robust Policies via Meshing for Metastable Rough Terrain Walking , 2014, Robotics: Science and Systems.

[8]  Ian R. Manchester,et al.  Stable dynamic walking over uneven terrain , 2011, Int. J. Robotics Res..

[9]  Jessy W. Grizzle,et al.  The Spring Loaded Inverted Pendulum as the Hybrid Zero Dynamics of an Asymmetric Hopper , 2009, IEEE Transactions on Automatic Control.

[10]  Kikuo Fujimura,et al.  The intelligent ASIMO: system overview and integration , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  A. Bloch,et al.  Nonholonomic Mechanics and Control , 2004, IEEE Transactions on Automatic Control.

[12]  Martijn Wisse,et al.  Ankle Actuation for Limit Cycle Walkers , 2008, Int. J. Robotics Res..

[13]  Jessy W. Grizzle,et al.  A Finite-State Machine for Accommodating Unexpected Large Ground-Height Variations in Bipedal Robot Walking , 2013, IEEE Transactions on Robotics.

[14]  Aaron D. Ames,et al.  Human-inspired multi-contact locomotion with AMBER2 , 2014, 2014 ACM/IEEE International Conference on Cyber-Physical Systems (ICCPS).

[15]  Aaron D. Ames,et al.  Achieving bipedal locomotion on rough terrain through human-inspired control , 2012, 2012 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR).

[16]  Twan Koolen,et al.  Capturability-based analysis and control of legged locomotion, Part 1: Theory and application to three simple gait models , 2011, Int. J. Robotics Res..

[17]  Elliott J. Rouse,et al.  Evidence for a Time-Invariant Phase Variable in Human Ankle Control , 2014, PloS one.

[18]  Jessy W. Grizzle,et al.  Exponentially stabilizing continuous-time controllers for periodic orbits of hybrid systems: Application to bipedal locomotion with ground height variations , 2016, Int. J. Robotics Res..

[19]  Kazuhito Yokoi,et al.  Biped walking stabilization based on linear inverted pendulum tracking , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Jerry Pratt,et al.  Velocity-Based Stability Margins for Fast Bipedal Walking , 2006 .

[21]  Meng Joo Er,et al.  Intelligent Fuzzy Q-Learning Control of Humanoid Robots , 2005, ISNN.

[22]  Marc H. Raibert,et al.  Legged Robots That Balance , 1986, IEEE Expert.

[23]  John J. Craig,et al.  Introduction to Robotics Mechanics and Control , 1986 .

[24]  Bernard Espiau,et al.  Limit Cycles in a Passive Compass Gait Biped and Passivity-Mimicking Control Laws , 1997, Auton. Robots.

[25]  Russ Tedrake,et al.  Optimizing robust limit cycles for legged locomotion on unknown terrain , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[26]  E. Westervelt,et al.  Feedback Control of Dynamic Bipedal Robot Locomotion , 2007 .

[27]  Christine Chevallereau,et al.  Models, feedback control, and open problems of 3D bipedal robotic walking , 2014, Autom..

[28]  Joe Engelberger Economics should be main basis for using robots , 1983 .

[29]  David C. Post,et al.  Velocity disturbance rejection for planar bipeds walking with HZD-based control , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Hartmut Geyer,et al.  Swing-leg retraction: a simple control model for stable running , 2003, Journal of Experimental Biology.

[31]  A. Tornambe,et al.  Reduced-order observers for the velocity estimation of non-linear mechanical systems subject to non-smooth impacts , 2002, Proceedings of the 2002 American Control Conference (IEEE Cat. No.CH37301).

[32]  Jessy W. Grizzle,et al.  Performance Analysis and Feedback Control of ATRIAS, A Three-Dimensional Bipedal Robot , 2014 .

[33]  Tad McGeer,et al.  Passive Dynamic Walking , 1990, Int. J. Robotics Res..

[34]  Changjiu Zhou,et al.  Robot learning with GA-based fuzzy reinforcement learning agents , 2002, Inf. Sci..

[35]  Russ Tedrake,et al.  Convex optimization of nonlinear feedback controllers via occupation measures , 2013, Int. J. Robotics Res..

[36]  Franck Plestan,et al.  Asymptotically stable walking for biped robots: analysis via systems with impulse effects , 2001, IEEE Trans. Autom. Control..

[37]  Frans C. T. van der Helm,et al.  How to keep from falling forward: elementary swing leg action for passive dynamic walkers , 2005, IEEE Transactions on Robotics.

[38]  W. Art Chaovalitwongse,et al.  Machine Learning Algorithms in Bipedal Robot Control , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[39]  Dan B. Marghitu,et al.  Rigid Body Collisions of Planar Kinematic Chains With Multiple Contact Points , 1994, Int. J. Robotics Res..

[40]  Aaron D. Ames,et al.  3D dynamic walking with underactuated humanoid robots: A direct collocation framework for optimizing hybrid zero dynamics , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Jessy W. Grizzle,et al.  Continuous-time controllers for stabilizing periodic orbits of hybrid systems: Application to an underactuated 3D bipedal robot , 2014, 53rd IEEE Conference on Decision and Control.

[42]  Jerry Pratt,et al.  Toward humanoid robots for operations in complex urban environments , 2010, Defense + Commercial Sensing.

[43]  Nathan van de Wouw,et al.  Sensitivity analysis of hybrid systems with state jumps with application to trajectory tracking , 2014, 53rd IEEE Conference on Decision and Control.

[44]  Peter Stone,et al.  Policy gradient reinforcement learning for fast quadrupedal locomotion , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[45]  Judy A. Franklin,et al.  Biped dynamic walking using reinforcement learning , 1997, Robotics Auton. Syst..

[46]  A. Kuo,et al.  Mechanical and energetic consequences of rolling foot shape in human walking , 2013, Journal of Experimental Biology.

[47]  Jessy W. Grizzle,et al.  Experimental results for 3D bipedal robot walking based on systematic optimization of virtual constraints , 2016, 2016 American Control Conference (ACC).

[48]  Ian R. Manchester,et al.  Real-time planning with primitives for dynamic walking over uneven terrain , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[49]  Russell L. Tedrake,et al.  Applied optimal control for dynamically stable legged locomotion , 2004 .

[50]  Koushil Sreenath,et al.  MABEL, a new robotic bipedal walker and runner , 2009, 2009 American Control Conference.

[51]  Shuuji Kajita,et al.  Dynamic walking control of a biped robot along a potential energy conserving orbit , 1992, IEEE Trans. Robotics Autom..

[52]  Andrew Y. Ng,et al.  Shaping and policy search in reinforcement learning , 2003 .

[53]  James P. Schmiedeler,et al.  A framework for the control of stable aperiodic walking in underactuated planar bipeds , 2009, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[54]  Jerry E. Pratt,et al.  Learning Capture Points for humanoid push recovery , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[55]  Twan Koolen,et al.  Capturability-based analysis and control of legged locomotion, Part 2: Application to M2V2, a lower-body humanoid , 2012, Int. J. Robotics Res..

[56]  Leonid B. Freidovich,et al.  Controlled Invariants and Trajectory Planning for Underactuated Mechanical Systems , 2014, IEEE Trans. Autom. Control..

[57]  Christine Chevallereau,et al.  Restricted discrete invariance and self-synchronization for stable walking of bipedal robots , 2015, 2015 American Control Conference (ACC).

[58]  M.T. Rosenstein,et al.  Reinforcement learning with supervision by a stable controller , 2004, Proceedings of the 2004 American Control Conference.

[59]  Kazuhito Yokoi,et al.  Biped walking pattern generation by using preview control of zero-moment point , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[60]  D. Luenberger Observers for multivariable systems , 1966 .

[61]  Albert Wang,et al.  Design principles for highly efficient quadrupeds and implementation on the MIT Cheetah robot , 2013, 2013 IEEE International Conference on Robotics and Automation.

[62]  Chandana Paul,et al.  Low-bandwidth reflex-based control for lower power walking: 65 km on a single battery charge , 2014, Int. J. Robotics Res..

[63]  Katie Byl,et al.  Quantifying the trade-offs between stability versus energy use for underactuated biped walking , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[64]  Christine Chevallereau,et al.  From stable walking to steering of a 3D bipedal robot with passive point feet , 2012, Robotica.

[65]  Reinhard Blickhan,et al.  Spring-Legged Locomotion on uneven Ground: A Control Approach to keep the running Speed constant , 2009 .

[66]  Jun Morimoto,et al.  Learning Biped Locomotion , 2007, IEEE Robotics & Automation Magazine.

[67]  T. Takenaka,et al.  The development of Honda humanoid robot , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[68]  Ian R. Manchester,et al.  Phase-indexed ILC for control of underactuated walking robots , 2015, 2015 IEEE Conference on Control Applications (CCA).

[69]  Daniel E. Koditschek,et al.  Hybrid zero dynamics of planar biped walkers , 2003, IEEE Trans. Autom. Control..

[70]  Katie Byl,et al.  Switching policies for metastable walking , 2013, 52nd IEEE Conference on Decision and Control.

[71]  Richard S. Sutton,et al.  Reinforcement Learning is Direct Adaptive Optimal Control , 1992, 1991 American Control Conference.

[72]  David C. Post,et al.  The effects of foot geometric properties on the gait of planar bipeds walking under HZD-based control , 2014, Int. J. Robotics Res..

[73]  M. Coleman,et al.  The simplest walking model: stability, complexity, and scaling. , 1998, Journal of biomechanical engineering.

[74]  Katie Byl,et al.  Metastable Walking Machines , 2009, Int. J. Robotics Res..

[75]  Kazuhito Yokoi,et al.  The 3D linear inverted pendulum mode: a simple modeling for a biped walking pattern generation , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[76]  Jonathan W. Hurst,et al.  THE DESIGN OF ATRIAS 1.0 A UNIQUE MONOPOD, HOPPING ROBOT ∗ , 2012 .

[77]  Roland Siegwart,et al.  SLIP running with an articulated robotic leg , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[78]  Alin Albu-Schäffer,et al.  Three-Dimensional Bipedal Walking Control Based on Divergent Component of Motion , 2015, IEEE Transactions on Robotics.

[79]  Chih-Han Yu,et al.  Quadruped robot obstacle negotiation via reinforcement learning , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[80]  Arthur D. Kuo,et al.  Stabilization of Lateral Motion in Passive Dynamic Walking , 1999, Int. J. Robotics Res..

[81]  Martijn Wisse,et al.  Swing-Leg Retraction for Limit Cycle Walkers Improves Disturbance Rejection , 2008, IEEE Transactions on Robotics.

[82]  최준호,et al.  On observer-based feedback stabilization of periodic orbits in bipedal locomotion , 2007 .

[83]  Yevgeniy Yesilevskiy,et al.  Selecting gaits for economical locomotion of legged robots , 2016, Int. J. Robotics Res..

[84]  Jessy W. Grizzle,et al.  Walking gait optimization for accommodation of unknown terrain height variations , 2015, 2015 American Control Conference (ACC).

[85]  Jun-Ho Oh,et al.  Mechanical design of humanoid robot platform KHR-3 (KAIST Humanoid Robot 3: HUBO) , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[86]  Shuuji Kajita,et al.  Study of dynamic biped locomotion on rugged terrain-derivation and application of the linear inverted pendulum mode , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[87]  Martijn Wisse,et al.  Passive-Based Walking Robot , 2007, IEEE Robotics & Automation Magazine.

[88]  Aaron D. Ames,et al.  Hybrid zero dynamics based multiple shooting optimization with applications to robotic walking , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[89]  Stefan Schaal,et al.  Policy Gradient Methods for Robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[90]  R J Full,et al.  Templates and anchors: neuromechanical hypotheses of legged locomotion on land. , 1999, The Journal of experimental biology.

[91]  Martijn Wisse,et al.  A Disturbance Rejection Measure for Limit Cycle Walkers: The Gait Sensitivity Norm , 2007, IEEE Transactions on Robotics.

[92]  Dong Jin Hyun,et al.  High speed trot-running: Implementation of a hierarchical controller using proprioceptive impedance control on the MIT Cheetah , 2014, Int. J. Robotics Res..

[93]  Jessica K. Hodgins,et al.  Adjusting step length for rough terrain locomotion , 1991, IEEE Trans. Robotics Autom..

[94]  Gill A. Pratt,et al.  Legged robots at MIT: what's new since Raibert? , 2000, IEEE Robotics Autom. Mag..

[95]  David C. Post,et al.  Design and experimental implementation of a hybrid zero dynamics-based controller for planar bipeds with curved feet , 2014, Int. J. Robotics Res..

[96]  A. Isidori Nonlinear Control Systems: An Introduction , 1986 .

[97]  Russ Tedrake,et al.  Efficient Bipedal Robots Based on Passive-Dynamic Walkers , 2005, Science.

[98]  Nikolaos G. Tsagarakis,et al.  Bipedal walking energy minimization by reinforcement learning with evolving policy parameterization , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[99]  Andrew Y. Ng,et al.  Task-space trajectories via cubic spline optimization , 2009, 2009 IEEE International Conference on Robotics and Automation.

[100]  S. Shankar Sastry,et al.  Dimension reduction near periodic orbits of hybrid systems , 2011, IEEE Conference on Decision and Control and European Control Conference.

[101]  Aaron D. Ames,et al.  Dynamic multi-domain bipedal walking with atrias through SLIP based human-inspired control , 2014, HSCC.

[102]  Jessy W. Grizzle,et al.  Nonholonomic virtual constraints for dynamic walking , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[103]  Chee-Meng Chew,et al.  Blind walking of a planar bipedal robot on sloped terrain , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[104]  Chee-Meng Chew,et al.  Virtual Model Control: An Intuitive Approach for Bipedal Locomotion , 2001, Int. J. Robotics Res..

[105]  Alin Albu-Schäffer,et al.  Walking control of fully actuated robots based on the Bipedal SLIP model , 2012, 2012 IEEE International Conference on Robotics and Automation.

[106]  Jessy W. Grizzle,et al.  From 2D Design of Underactuated Bipedal Gaits to 3D Implementation: Walking With Speed Tracking , 2016, IEEE Access.

[107]  Patrick MacAlpine,et al.  Humanoid robots learning to walk faster: from the real world to simulation and back , 2013, AAMAS.

[108]  Hae-Won Park,et al.  Control of a Bipedal Robot Walker on Rough Terrain , 2012 .

[109]  Jessy W. Grizzle,et al.  Event-Based Stabilization of Periodic Orbits for Underactuated 3-D Bipedal Robots With Left-Right Symmetry , 2014, IEEE Transactions on Robotics.

[110]  Jonathon W. Sensinger,et al.  Virtual Constraint Control of a Powered Prosthetic Leg: From Simulation to Experiments With Transfemoral Amputees , 2014, IEEE Transactions on Robotics.

[111]  Peter J Seiler,et al.  Help on SOS [Ask the Experts] , 2010 .

[112]  R. Blickhan The spring-mass model for running and hopping. , 1989, Journal of biomechanics.

[113]  Jessy W. Grizzle,et al.  Preliminary walking experiments with underactuated 3D bipedal robot MARLO , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[114]  George Council,et al.  Deadbeat control with (almost) no sensing in a hybrid model of legged locomotion , 2014, Proceedings of the 2014 International Conference on Advanced Mechatronic Systems.

[115]  Prahlad Vadakkepat,et al.  An Evolutionary Algorithm for Trajectory Based Gait Generation of Biped Robot , 2003 .

[116]  Kevin Blankespoor,et al.  BigDog, the Rough-Terrain Quadruped Robot , 2008 .