Learning Certified Control using Contraction Metric

In this paper, we solve the problem of finding a certified control policy that drives a robot from any given initial state and under any bounded disturbance to the desired reference trajectory, with guarantees on the convergence or bounds on the tracking error. Such a controller is crucial in safe motion planning. We leverage the advanced theory in Control Contraction Metric and design a learning framework based on neural networks to co-synthesize the contraction metric and the controller for control-affine systems. We further provide methods to validate the convergence and bounded error guarantees. We demonstrate the performance of our method using a suite of challenging robotic models, including models with learned dynamics as neural networks. We compare our approach with leading methods using sum-of-squares programming, reinforcement learning, and model predictive control. Results show that our methods indeed can handle a broader class of systems with less tracking error and faster execution speed. Code is available at https://github.com/sundw2014/C3M.

[1]  Paulo Tabuada,et al.  Control Barrier Functions: Theory and Applications , 2019, 2019 18th European Control Conference (ECC).

[2]  Rolf Findeisen,et al.  Discrete-time robust model predictive control for continuous-time nonlinear systems , 2015, 2015 American Control Conference (ACC).

[3]  George J. Pappas,et al.  A Framework for Worst-Case and Stochastic Safety Verification Using Barrier Certificates , 2007, IEEE Transactions on Automatic Control.

[4]  Natarajan Shankar,et al.  Duality-Based Nested Controller Synthesis from STL Specifications for Stochastic Linear Systems , 2018, FORMATS.

[5]  Sterling J. Anderson,et al.  An optimal-control-based framework for trajectory planning, threat assessment, and semi-autonomous control of passenger vehicles in hazard avoidance scenarios , 2010 .

[6]  Stewart Worrall,et al.  Towards Provably Not-at-Fault Control of Autonomous Robots in Arbitrary Dynamic Environments , 2019, Robotics: Science and Systems.

[7]  Frank Allgöwer,et al.  Tube MPC scheme based on robust control invariant set with application to Lipschitz nonlinear systems , 2011, CDC/ECC.

[8]  Emilio Frazzoli,et al.  Anytime Motion Planning using the RRT* , 2011, 2011 IEEE International Conference on Robotics and Automation.

[9]  Manfred Morari,et al.  Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks , 2019, NeurIPS.

[10]  Ian R. Manchester,et al.  Nonlinear stabilization via Control Contraction Metrics: A pseudospectral approach for computing geodesics , 2016, 2017 American Control Conference (ACC).

[11]  Russ Tedrake,et al.  Safety verification of reactive controllers for UAV flight in cluttered environments using barrier certificates , 2012, 2012 IEEE International Conference on Robotics and Automation.

[12]  Dimos V. Dimarogonas,et al.  Learning Control Barrier Functions from Expert Demonstrations , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[13]  Marco Pavone,et al.  Learning stabilizable nonlinear dynamics with contraction-based regularization , 2019, Int. J. Robotics Res..

[14]  Sriram Sankaranarayanan,et al.  Robust controller synthesis of switched systems using counterexample guided framework , 2016, 2016 International Conference on Embedded Software (EMSOFT).

[15]  Russ Tedrake,et al.  Funnel libraries for real-time robust feedback motion planning , 2016, Int. J. Robotics Res..

[16]  Chuchu Fan,et al.  Bounded Verification with On-the-Fly Discrepancy Computation , 2015, ATVA.

[17]  Aaron D. Ames,et al.  Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems* , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Yuval Tassa,et al.  Control-limited differential dynamic programming , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[19]  Paulo Tabuada,et al.  Control barrier function based quadratic programs with application to adaptive cruise control , 2014, 53rd IEEE Conference on Decision and Control.

[20]  Sanjit A. Seshia,et al.  Safe Autonomy Under Perception Uncertainty Using Chance-Constrained Temporal Logic , 2017, Journal of Automated Reasoning.

[21]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[22]  Ashish Tiwari,et al.  TeLEx: Passive STL Learning Using Only Positive Examples , 2017, RV.

[23]  Alessandro Rinaldo,et al.  Distribution-Free Predictive Inference for Regression , 2016, Journal of the American Statistical Association.

[24]  Russ Tedrake,et al.  LQR-trees: Feedback motion planning on sparse randomized trees , 2009, Robotics: Science and Systems.

[25]  Marco Pavone,et al.  Fast marching tree: A fast marching sampling-based method for optimal motion planning in many dimensions , 2013, ISRR.

[26]  Alexandre M. Bayen,et al.  Computational techniques for the verification of hybrid systems , 2003, Proc. IEEE.

[27]  P. Olver Nonlinear Systems , 2013 .

[28]  Lydia E. Kavraki,et al.  Analysis of probabilistic roadmaps for path planning , 1998, IEEE Trans. Robotics Autom..

[29]  Li Wang,et al.  Barrier-Certified Adaptive Reinforcement Learning With Applications to Brushbot Navigation , 2018, IEEE Transactions on Robotics.

[30]  Marin Kobilarov,et al.  Cross-entropy motion planning , 2012, Int. J. Robotics Res..

[31]  Pablo A. Parrilo,et al.  Stability and robustness analysis of nonlinear systems via contraction metrics and SOS programming , 2006, at - Automatisierungstechnik.

[32]  Byron Boots,et al.  Differentiable MPC for End-to-end Planning and Control , 2018, NeurIPS.

[33]  Marko Bacic,et al.  Model predictive control , 2003 .

[34]  Jean-Jacques E. Slotine,et al.  On Contraction Analysis for Non-linear Systems , 1998, Autom..

[35]  Claire J. Tomlin,et al.  A differential game approach to planning in adversarial scenarios: A case study on capture-the-flag , 2011, 2011 IEEE International Conference on Robotics and Automation.

[36]  Sriram Sankaranarayanan,et al.  Simulation-guided lyapunov analysis for hybrid dynamical systems , 2014, HSCC.

[37]  Soon-Jo Chung,et al.  Robust Regression for Safe Exploration in Control , 2019, L4DC.

[38]  Sergey Levine,et al.  Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models , 2018, NeurIPS.

[39]  S. LaValle Rapidly-exploring random trees : a new tool for path planning , 1998 .

[40]  Qinmin Yang,et al.  Reinforcement Learning Controller Design for Affine Nonlinear Discrete-Time Systems using Online Approximators , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[41]  Chuchu Fan,et al.  Fast and Guaranteed Safe Controller Synthesis for Nonlinear Vehicle Models , 2020, CAV.

[42]  Sicun Gao,et al.  Neural Lyapunov Control , 2020, NeurIPS.

[43]  J. Slotine,et al.  Robust feedback motion planning via contraction theory , 2023, Int. J. Robotics Res..

[44]  Aude Billard,et al.  Learning control Lyapunov function to ensure stability of dynamical system-based robot reaching motions , 2014, Robotics Auton. Syst..

[45]  David Q. Mayne,et al.  Robust model predictive control of constrained linear systems with bounded disturbances , 2005, Autom..

[46]  Ian R. Manchester,et al.  An Amendment to "Control Contraction Metrics: Convex and Intrinsic Criteria for Nonlinear Feedback Design" , 2017, ArXiv.

[47]  Koushil Sreenath,et al.  Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions , 2020, Robotics: Science and Systems.

[48]  Mo Chen,et al.  Decomposition of Reachable Sets and Tubes for a Class of Nonlinear Systems , 2016, IEEE Transactions on Automatic Control.

[49]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[50]  Soon-Jo Chung,et al.  Neural Contraction Metrics for Robust Estimation and Control: A Convex Optimization Approach , 2020, IEEE Control Systems Letters.

[51]  Charles Richter,et al.  Polynomial Trajectory Planning for Aggressive Quadrotor Flight in Dense Indoor Environments , 2016, ISRR.

[52]  Jaime F. Fisac,et al.  Planning, Fast and Slow: A Framework for Adaptive Real-Time Safe Trajectory Planning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[53]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[54]  V. Kaul,et al.  Planning , 2012 .

[55]  Susmit Jhal,et al.  Data Efficient Learning of Robust Control Policies , 2018, 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[56]  Russ Tedrake,et al.  Robust Online Motion Planning with Regions of Finite Time Invariance , 2012, WAFR.

[57]  Mo Chen,et al.  FaSTrack: A modular framework for fast and guaranteed safe motion planning , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[58]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[59]  Marcello Farina,et al.  Tube-based robust sampled-data MPC for linear continuous-time systems , 2012, Autom..

[60]  Mo Chen,et al.  Safe Sequential Path Planning of Multi-Vehicle Systems Under Presence of Disturbances and Imperfect Information , 2016, ArXiv.

[61]  W. Gasarch,et al.  The Book Review Column 1 Coverage Untyped Systems Simple Types Recursive Types Higher-order Systems General Impression 3 Organization, and Contents of the Book , 2022 .

[62]  Claire J. Tomlin,et al.  Design of guaranteed safe maneuvers using reachable sets: Autonomous quadrotor aerobatics in theory and practice , 2010, 2010 IEEE International Conference on Robotics and Automation.

[63]  Carlos Bordons Alba,et al.  Model Predictive Control , 2012 .

[64]  David Q. Mayne,et al.  Tube‐based robust nonlinear model predictive control , 2011 .

[65]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[66]  Yisong Yue,et al.  Learning for Safety-Critical Control with Control Barrier Functions , 2019, L4DC.

[67]  Athanasios S. Polydoros,et al.  Survey of Model-Based Reinforcement Learning: Applications on Robotics , 2017, J. Intell. Robotic Syst..

[68]  David Q. Mayne,et al.  Robust model predictive control using tubes , 2004, Autom..