Robust walking based on MPC with viability-based feasibility guarantees

Model predictive control (MPC) has shown great success for controlling complex systems such as legged robots. However, when closing the loop, the performance and feasibility of the finite horizon optimal control problem solved at each control cycle is not guaranteed anymore. This is due to model discrepancies, the effect of low-level controllers, uncertainties and sensor noise. To address these issues, we propose a modified version of a standard MPC approach used in legged locomotion with viability (weak forward invariance) guarantees that ensures the feasibility of the optimal control problem. Moreover, we use past experimental data to find the best cost weights, which measure a combination of performance, constraint satisfaction robustness, or stability (invariance). These interpretable costs measure the trade off between robustness and performance. For this purpose, we use Bayesian optimization (BO) to systematically design experiments that help efficiently collect data to learn a cost function leading to robust performance. Our simulation results with different realistic disturbances (i.e. external pushes, unmodeled actuator dynamics and computational delay) show the effectiveness of our approach to create robust controllers for humanoid robots.

[1]  Ludovic Righetti,et al.  Stochastic and Robust MPC for Bipedal Locomotion: A Comparative Study on Robustness and Performance , 2020, 2020 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids).

[2]  Nicolas Mansard,et al.  Multicontact Locomotion of Legged Robots , 2018, IEEE Transactions on Robotics.

[3]  Nicolas Mansard,et al.  Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[4]  Scott Kuindersma,et al.  Robust direct trajectory optimization using approximate invariant funnels , 2018, Auton. Robots.

[5]  Sylvain Calinon,et al.  A Survey on Policy Search Algorithms for Learning Robot Controllers in a Handful of Trials , 2018, IEEE Transactions on Robotics.

[6]  Johan Löfberg,et al.  Oops! I cannot do it again: Testing for recursive feasibility in MPC , 2012, Autom..

[7]  Kazuhito Yokoi,et al.  Biped walking pattern generation by using preview control of zero-moment point , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[8]  Alexander Herzog,et al.  Step timing adjustment: A step toward generating robust gaits , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[9]  Andrei Herdt,et al.  Online Walking Motion Generation with Automatic Footstep Placement , 2010, Adv. Robotics.

[10]  Abderrahmane Kheddar,et al.  Dynamic walking over rough terrains by nonlinear predictive control of the floating-base inverted pendulum , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Nando de Freitas,et al.  Portfolio Allocation for Bayesian Optimization , 2010, UAI.

[12]  Jan Peters,et al.  An experimental comparison of Bayesian optimization for bipedal locomotion , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Eiichi Yoshida,et al.  Model preview control in multi-contact motion-application to a humanoid robot , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  David Q. Mayne,et al.  Robust model predictive control of constrained linear systems with bounded disturbances , 2005, Autom..

[15]  Stefan Schaal,et al.  Automatic LQR tuning based on Gaussian process global optimization , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Pierre-Brice Wieber,et al.  Model predictive control of biped walking with bounded uncertainties , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[17]  Olivier Stasse,et al.  Whole-body model-predictive control applied to the HRP-2 humanoid , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Takashi Matsumoto,et al.  Real time motion generation and control for biped robot -1st report: Walking gait pattern generation- , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Ludovic Righetti,et al.  Efficient Multicontact Pattern Generation With Sequential Convex Approximations of the Centroidal Dynamics , 2020, IEEE Transactions on Robotics.

[20]  Nicolas Mansard,et al.  Robustness to Joint-Torque-Tracking Errors in Task-Space Inverse Dynamics , 2016, IEEE Transactions on Robotics.

[21]  Ruben Grandia,et al.  Feedback MPC for Torque-Controlled Legged Robots , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  David Q. Mayne,et al.  Constrained model predictive control: Stability and optimality , 2000, Autom..

[23]  Pierre-Brice Wieber,et al.  Viability and predictive control for safe locomotion , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Alexander Herzog,et al.  On Time Optimization of Centroidal Momentum Dynamics , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Zhibin Li,et al.  Bayesian Optimization for Whole-Body Control of High-Degree-of-Freedom Robots Through Reduction of Dimensionality , 2019, IEEE Robotics and Automation Letters.

[26]  Pierre-Brice Wieber,et al.  Trajectory Free Linear Model Predictive Control for Stable Walking in the Presence of Strong Perturbations , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[27]  Christopher G. Atkeson,et al.  Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Alexander Herzog,et al.  Walking Control Based on Step Timing Adaptation , 2017, IEEE Transactions on Robotics.

[29]  Alexander Herzog,et al.  Pattern Generation for Walking on Slippery Terrains , 2017, 2017 5th RSI International Conference on Robotics and Mechatronics (ICRoM).

[30]  Yuval Tassa,et al.  Synthesis and stabilization of complex behaviors through online trajectory optimization , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[31]  Lukas Hewing,et al.  Stochastic Model Predictive Control for Linear Systems Using Probabilistic Reachable Sets , 2018, 2018 IEEE Conference on Decision and Control (CDC).

[32]  Ricardo G. Sanfelice,et al.  Forward Invariance of Sets for Hybrid Dynamical Systems (Part I) , 2018, IEEE Transactions on Automatic Control.

[33]  Ludovic Righetti,et al.  Robust Humanoid Locomotion Using Trajectory Optimization and Sample-Efficient Learning* , 2019, 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids).

[34]  Jan Peters,et al.  Bayesian Gait Optimization for Bipedal Locomotion , 2014, LION.

[35]  Ruben Grandia,et al.  Locomotion Planning through a Hybrid Bayesian Trajectory Optimization , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[36]  Twan Koolen,et al.  Capturability-based analysis and control of legged locomotion, Part 1: Theory and application to three simple gait models , 2011, Int. J. Robotics Res..

[37]  Jan Peters,et al.  Learning to Control Highly Accelerated Ballistic Movements on Muscular Robots , 2019, Robotics Auton. Syst..

[38]  Russ Tedrake,et al.  Planning robust walking motion on uneven terrain via convex optimization , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[39]  Yuval Tassa,et al.  Control-limited differential dynamic programming , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[40]  Franco Blanchini,et al.  Set invariance in control , 1999, Autom..

[41]  Jun-ichi Imura,et al.  Controlled invariant feasibility - A general approach to enforcing strong feasibility in MPC applied to move-blocking , 2009, Autom..

[42]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[43]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[44]  Christopher G. Atkeson,et al.  Sample efficient optimization for learning controllers for bipedal locomotion , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[45]  Giuseppe Oriolo,et al.  Learning Robust Task Priorities of QP-Based Whole-Body Torque-Controllers , 2018, 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids).

[46]  Scott Kuindersma,et al.  Modeling and Control of Legged Robots , 2016, Springer Handbook of Robotics, 2nd Ed..