Reachability Analysis and Safety Verification for Neural Network Control Systems

Autonomous cyber-physical systems (CPS) rely on the correct operation of numerous components, with state-of-the-art methods relying on machine learning (ML) and artificial intelligence (AI) components in various stages of sensing and control. This paper develops methods for estimating the reachable set and verifying safety properties of dynamical systems under control of neural network-based controllers that may be implemented in embedded software. The neural network controllers we consider are feedforward neural networks called multilayer perceptrons (MLP) with general activation functions. As such feedforward networks are memoryless, they may be abstractly represented as mathematical functions, and the reachability analysis of the network amounts to range (image) estimation of this function provided a set of inputs. By discretizing the input set of the MLP into a finite number of hyper-rectangular cells, our approach develops a linear programming (LP) based algorithm for over-approximating the output set of the MLP with its input set as a union of hyper-rectangular cells. Combining the over-approximation for the output set of an MLP based controller and reachable set computation routines for ordinary difference/differential equation (ODE) models, an algorithm is developed to estimate the reachable set of the closed-loop system. Finally, safety verification for neural network control systems can be performed by checking the existence of intersections between the estimated reachable set and unsafe regions. The approach is implemented in a computational software prototype and evaluated on numerical examples.

[1]  Weiming Xiang,et al.  Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations , 2017, ArXiv.

[2]  Ashish Tiwari,et al.  Output Range Analysis for Deep Neural Networks , 2017, ArXiv.

[3]  Mahesh Viswanathan,et al.  C2E2: A Verification Tool for Stateflow Models , 2015, TACAS.

[4]  James Kapinski,et al.  Locally optimal reach set over-approximation for nonlinear systems , 2016, 2016 International Conference on Embedded Software (EMSOFT).

[5]  Mykel J. Kochenderfer,et al.  Policy compression for aircraft collision avoidance systems , 2016, 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC).

[6]  Xin Chen,et al.  Flow*: An Analyzer for Non-linear Hybrid Systems , 2013, CAV.

[7]  Sergiy Bogomolov,et al.  HYST: a source transformation and translation tool for hybrid automaton models , 2015, HSCC.

[8]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[9]  Peng Shi,et al.  Exponential Stabilization for Sampled-Data Neural-Network-Based Control Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[11]  Mykel J. Kochenderfer,et al.  Neural Network Guidance for UAVs , 2017 .

[12]  Anuradha M. Annaswamy,et al.  Stable Neural Controllers for Nonlinear Dynamic Systems , 1998, Autom..

[13]  Weiming Xiang,et al.  Reachable Set Estimation and Safety Verification for Piecewise Linear Systems with Neural Network Controllers , 2018, 2018 Annual American Control Conference (ACC).

[14]  Luca Pulina,et al.  Challenging SMT solvers to verify neural networks , 2012, AI Commun..

[15]  Weiming Xiang,et al.  Output Reachable Set Estimation and Verification for Multilayer Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Manfred Morari,et al.  Multi-Parametric Toolbox 3.0 , 2013, 2013 European Control Conference (ECC).

[17]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[18]  Luca Pulina,et al.  An Abstraction-Refinement Approach to Verification of Artificial Neural Networks , 2010, CAV.

[19]  Tao Zhang,et al.  Adaptive neural network control of nonlinear systems by state and output feedback , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[20]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[21]  Antoine Girard,et al.  SpaceEx: Scalable Verification of Hybrid Systems , 2011, CAV.

[22]  Mahesh Viswanathan,et al.  Automatic Reachability Analysis for Nonlinear Hybrid Models with C2E2 , 2016, CAV.

[23]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[24]  Ashish Tiwari,et al.  Output Range Analysis for Deep Feedforward Neural Networks , 2018, NFM.

[25]  Alessio Lomuscio,et al.  An approach to reachability analysis for feed-forward ReLU neural networks , 2017, ArXiv.

[26]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[27]  Stanley Bak,et al.  HyLAA: A Tool for Computing Simulation-Equivalent Reachability for Linear Systems , 2017, HSCC.

[28]  Xin Chen,et al.  Decomposed Reachability Analysis for Nonlinear Systems , 2016, 2016 IEEE Real-Time Systems Symposium (RTSS).