Learning safe neural network controllers with barrier certificates

We provide a novel approach to synthesize controllers for nonlinear continuous dynamical systems with control against safety properties. The controllers are based on neural networks (NNs). To certify the safety property we utilize barrier functions, which are represented by NNs as well. We train the controller-NN and barrier-NN simultaneously, achieving a verification-in-the-loop synthesis. We provide a prototype tool nncontroller with a number of case studies. The experiment results confirm the feasibility and efficacy of our approach.

[1]  Alexandros G. Dimakis,et al.  Exactly Computing the Local Lipschitz Constant of ReLU Networks , 2020, NeurIPS.

[2]  Jyotirmoy V. Deshmukh,et al.  Reasoning about Safety of Learning-Enabled Components in Autonomous Cyber-physical Systems , 2018 .

[3]  Paulo Tabuada,et al.  Control Barrier Functions: Theory and Applications , 2019, 2019 18th European Control Conference (ECC).

[4]  Andreas Krause,et al.  The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems , 2018, CoRL.

[5]  Sicun Gao,et al.  Neural Lyapunov Control , 2020, NeurIPS.

[6]  Stefan Ratschan,et al.  Converse Theorems for Safety and Barrier Certificates , 2017, IEEE Transactions on Automatic Control.

[7]  Yasser Shoukry,et al.  Formal verification of neural network controlled autonomous systems , 2018, HSCC.

[8]  Insup Lee,et al.  Verisig: verifying safety properties of hybrid systems with neural network controllers , 2018, HSCC.

[9]  Liyun Dai,et al.  Barrier certificates revisited , 2013, J. Symb. Comput..

[10]  Alexander S. Poznyak,et al.  Differential Neural Networks for Robust Nonlinear Control , 2004, IEEE Transactions on Neural Networks.

[11]  Sriram Sankaranarayanan,et al.  Training Neural Network Controllers Using Control Barrier Functions in the Presence of Disturbances , 2020, 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).

[12]  Nathan Fulton,et al.  Safe Reinforcement Learning via Formal Methods: Toward Safe Control Through Proof and Learning , 2018, AAAI.

[13]  Mayank Mittal,et al.  Neural Lyapunov Model Predictive Control , 2020, ArXiv.

[14]  J. Zico Kolter,et al.  Learning Stable Deep Dynamics Models , 2020, NeurIPS.

[15]  André Platzer,et al.  Vector Barrier Certificates and Comparison Systems , 2018, FM.

[16]  Luca Pulina,et al.  An Abstraction-Refinement Approach to Verification of Artificial Neural Networks , 2010, CAV.

[17]  Weiming Xiang,et al.  Output Reachable Set Estimation and Verification for Multilayer Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[18]  James Kapinski,et al.  INVITED: Reasoning about Safety of Learning-Enabled Components in Autonomous Cyber-physical Systems , 2018, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[19]  Inderjit S. Dhillon,et al.  Towards Fast Computation of Certified Robustness for ReLU Networks , 2018, ICML.

[20]  George J. Pappas,et al.  A Framework for Worst-Case and Stochastic Safety Verification Using Barrier Certificates , 2007, IEEE Transactions on Automatic Control.

[21]  Aaron D. Ames,et al.  Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems* , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Liqian Chen,et al.  Analyzing Deep Neural Networks with Symbolic Propagation: Towards Higher Precision and Faster Verification , 2019, SAS.

[23]  Nathan Coppedge,et al.  Systems Theory , 2016 .

[24]  Rafael Wisniewski,et al.  Compositional safety analysis using barrier certificates , 2012, HSCC '12.

[25]  Gábor Orosz,et al.  End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks , 2019, AAAI.

[26]  Dario Amodei,et al.  Benchmarking Safe Exploration in Deep Reinforcement Learning , 2019 .

[27]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[28]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[29]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[30]  João Pedro Hespanha,et al.  Linear Systems Theory , 2009 .

[31]  Andreas Krause,et al.  Safe Model-based Reinforcement Learning with Stability Guarantees , 2017, NIPS.

[32]  Jyotirmoy V. Deshmukh,et al.  Learning Deep Neural Network Controllers for Dynamical Systems with Safety Guarantees: Invited Paper , 2019, 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[33]  Sriram Sankaranarayanan,et al.  Learning control lyapunov functions from counterexamples and demonstrations , 2018, Autonomous Robots.

[34]  Ashish Tiwari,et al.  Output Range Analysis for Deep Feedforward Neural Networks , 2018, NFM.

[35]  Weiming Xiang,et al.  NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems , 2020, CAV.

[36]  Koushil Sreenath,et al.  Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions , 2020, Robotics: Science and Systems.

[37]  Joel W. Burdick,et al.  Safe Policy Synthesis in Multi-Agent POMDPs via Discrete-Time Barrier Functions , 2019, 2019 IEEE 58th Conference on Decision and Control (CDC).

[38]  Taolue Chen,et al.  Synthesizing barrier certificates using neural networks , 2020, HSCC.

[39]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[40]  Suresh Jagannathan,et al.  An inductive synthesis framework for verifiable reinforcement learning , 2019, PLDI.

[41]  Sriram Sankaranarayanan,et al.  Reachability analysis for neural feedback systems using regressive polynomial rule inference , 2019, HSCC.

[42]  Insup Lee,et al.  Case study: verifying the safety of an autonomous racing car with a neural network controller , 2019, HSCC.

[43]  Yisong Yue,et al.  Learning for Safety-Critical Control with Control Barrier Functions , 2019, L4DC.

[44]  Ashish Tiwari,et al.  Learning and Verification of Feedback Control Systems using Feedforward Neural Networks , 2018, ADHS.

[45]  Sanjit A. Seshia,et al.  VerifAI: A Toolkit for the Formal Design and Analysis of Artificial Intelligence-Based Systems , 2019, CAV.

[46]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[47]  Hui Kong,et al.  Exponential-Condition-Based Barrier Certificate Generation for Safety Verification of Hybrid Systems , 2013, CAV.