Stability Analysis using Quadratic Constraints for Systems with Neural Network Controllers

A method is presented to analyze the stability of feedback systems with neural network controllers. Two stability theorems are given to prove asymptotic stability and to compute an ellipsoidal inner-approximation to the region of attraction. The first theorem addresses linear time-invariant plant dynamics, and merges Lyapunov theory with local (sector) quadratic constraints to bound the nonlinear activation functions in the neural network. The second theorem allows the plant dynamics to include perturbations such as unmodeled dynamics, slope-restricted nonlinearities, and time delay, using integral quadratic constraint (IQCs) to capture their input/output behavior. Both results rely on semidefinite programming to compute Lyapunov functions and inner-estimates of the region of attraction. The method is illustrated on systems with neural networks trained to stabilize a nonlinear inverted pendulum as well as vehicle lateral dynamics with actuator uncertainty.

[1]  Sergey Levine,et al.  Trust Region Policy Optimization , 2015, ICML.

[2]  Manfred Morari,et al.  Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks , 2019, NeurIPS.

[3]  George J. Pappas,et al.  Reach-SDP: Reachability Analysis of Closed-Loop Systems with Neural Network Controllers via Semidefinite Programming , 2020, 2020 59th IEEE Conference on Decision and Control (CDC).

[4]  Manfred Morari,et al.  Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming , 2019, ArXiv.

[5]  Carsten W. Scherer,et al.  Robust stability and performance analysis based on integral quadratic constraints , 2016, Eur. J. Control.

[6]  Andreas Krause,et al.  Safe learning of regions of attraction for uncertain, nonlinear systems with Gaussian processes , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[7]  Jiameng Fan,et al.  ReachNN , 2019, ACM Trans. Embed. Comput. Syst..

[8]  Murat Arcak,et al.  Reachability Analysis Using Dissipation Inequalities For Uncertain Nonlinear Systems , 2018 .

[9]  A. Rantzer,et al.  System analysis via integral quadratic constraints , 1997, IEEE Trans. Autom. Control..

[10]  Peter Seiler,et al.  Region of attraction analysis with Integral Quadratic Constraints , 2019, Autom..

[11]  Adrian Wills,et al.  Zames-Falb Multipliers for Quadratic Programming , 2007, IEEE Transactions on Automatic Control.

[12]  Benjamin Recht,et al.  Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints , 2014, SIAM J. Optim..

[13]  Sriram Sankaranarayanan,et al.  Reachability analysis for neural feedback systems using regressive polynomial rule inference , 2019, HSCC.

[14]  Andrew G. Alleyne.  A Comparison of Alternative Intervention Strategies for Unintended Roadway Departure (URD) Control , 1997 .

[15]  Stefan Schaal,et al.  2008 Special Issue: Reinforcement learning of motor skills with policy gradients , 2008 .

[16]  Murat Arcak,et al.  Backward Reachability Using Integral Quadratic Constraints for Uncertain Nonlinear Systems , 2021, IEEE Control Systems Letters.

[17]  Stephen P. Boyd,et al.  Analysis of linear systems with saturation using convex optimization , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[18]  Frank Allgöwer,et al.  Training robust neural networks using Lipschitz bounds , 2020, ArXiv.

[19]  Richard D. Braatz,et al.  Standard representation and unified stability analysis for dynamic artificial neural network models , 2018, Neural Networks.

[20]  Kumpati S. Narendra,et al.  Control of nonlinear dynamical systems using neural networks: controllability and stabilization , 1993, IEEE Trans. Neural Networks.

[21]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[22]  Javad Lavaei,et al.  Stability-Certified Reinforcement Learning: A Control-Theoretic Perspective , 2018, IEEE Access.

[23]  P. Falb,et al.  Stability Conditions for Systems with Monotone and Slope-Restricted Nonlinearities , 1968 .

[24]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[25]  Sham M. Kakade,et al.  A Natural Policy Gradient , 2001, NIPS.

[26]  Insup Lee,et al.  Verisig: verifying safety properties of hybrid systems with neural network controllers , 2018, HSCC.