Neural Lyapunov Control

We propose new methods for learning control policies and neural network Lyapunov functions for nonlinear control problems, with provable guarantee of stability. The framework consists of a learner that attempts to find the control and Lyapunov functions, and a falsifier that finds counterexamples to quickly guide the learner towards solutions. The procedure terminates when no counterexample is found by the falsifier, in which case the controlled nonlinear system is provably stable. The approach significantly simplifies the process of Lyapunov control design, provides end-to-end correctness guarantee, and can obtain much larger regions of attraction than existing methods such as LQR and SOS/SDP. We show experiments on how the new methods obtain high-quality solutions for challenging robot control problems such as path tracking for wheeled vehicles and humanoid robot balancing.

[1]  Amir Ali Ahmadi.  On the difficulty of deciding asymptotic stability of cubic homogeneous vector fields , 2011, 2012 American Control Conference (ACC).

[2]  Ofir Nachum,et al.  A Lyapunov-based Approach to Safe Reinforcement Learning , 2018, NeurIPS.

[3]  Edmund M. Clarke,et al.  δ-Complete Decision Procedures for Satisfiability over the Reals , 2012, IJCAR.

[4]  A. Garulli,et al.  Positive Polynomials in Control , 2005 .

[5]  Amir Ali Ahmadi,et al.  Lower Bounds on Complexity of Lyapunov Functions for Switched Linear Systems , 2015, ArXiv.

[6]  Andreas Krause,et al.  Safe learning of regions of attraction for uncertain, nonlinear systems with Gaussian processes , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[7]  Johan Löfberg,et al.  Pre- and Post-Processing Sum-of-Squares Programs in Practice , 2009, IEEE Transactions on Automatic Control.

[8]  Sriram Sankaranarayanan,et al.  Simulation-guided lyapunov analysis for hybrid dynamical systems , 2014, HSCC.

[9]  Huibert Kwakernaak,et al.  Linear Optimal Control Systems , 1972 .

[10]  Ryan Feeley,et al.  Some controls applications of sum of squares programming , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[11]  Andreas Krause,et al.  The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems , 2018, CoRL.

[12]  Benjamin Recht,et al.  Simple random search of static linear policies is competitive for reinforcement learning , 2018, NeurIPS.

[13]  D. Henrion,et al.  Guest Editorial: Special Issue on Positive Polynomials in Control , 2009 .

[14]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[15]  Andreas Krause,et al.  Safe Model-based Reinforcement Learning with Stability Guarantees , 2017, NIPS.

[16]  P. Parrilo.  Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization , 2000 .

[17]  Sriram Sankaranarayanan,et al.  Learning control lyapunov functions from counterexamples and demonstrations , 2018, Autonomous Robots.

[18]  Xin Xin,et al.  Controllability and Observability of an $n$-Link Planar Robot with a Single Actuator Having Different Actuator–Sensor Configurations , 2016, IEEE Transactions on Automatic Control.

[19]  Edmund M. Clarke,et al.  dReal: An SMT Solver for Nonlinear Theories over the Reals , 2013, CADE.

[20]  Amir Ali Ahmadi,et al.  Stability of Polynomial Differential Equations: Complexity and Converse Lyapunov Questions , 2013, ArXiv.

[21]  W. Haddad,et al.  Nonlinear Dynamical Systems and Control: A Lyapunov-Based Approach , 2008 .

[22]  Russ Tedrake,et al.  Funnel libraries for real-time robust feedback motion planning , 2016, Int. J. Robotics Res..

[23]  Miroslav Krstic,et al.  A globally asymptotically stable polynomial vector field with no polynomial Lyapunov function , 2011, IEEE Conference on Decision and Control and European Control Conference.