INVITED: Reasoning about Safety of Learning-Enabled Components in Autonomous Cyber-physical Systems

We present a simulation-based approach for generating barrier certificate functions for safety verification of cyber-physical systems (CPS) that contain neural network-based controllers. A linear programming solver is utilized to find a candidate generator function from a set of simulation traces obtained by randomly selecting initial states for the CPS model. A level set of the generator function is then selected to act as a barrier certificate for the system, meaning it demonstrates that no unsafe system states are reachable from a given set of initial states. The barrier certificate properties are verified with an SMT solver. This approach is demonstrated on a case study in which a Dubins car model of an autonomous vehicle is controlled by a neural network to follow a given path.

[1]  Edmund M. Clarke,et al.  δ-Complete Decision Procedures for Satisfiability over the Reals , 2012, IJCAR.

[2]  Christian Igel,et al.  Neuroevolution for reinforcement learning using evolution strategies , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[3]  Ali Jadbabaie,et al.  Safety Verification of Hybrid Systems Using Barrier Certificates , 2004, HSCC.

[4]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[5]  J. Doyle,et al.  Optimization-based methods for nonlinear and hybrid systems verification , 2005 .

[6]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[7]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[8]  Sriram Sankaranarayanan,et al.  Simulation-guided lyapunov analysis for hybrid dynamical systems , 2014, HSCC.

[9]  Jyotirmoy V. Deshmukh,et al.  Simulation-guided Contraction Analysis , 2015 .

[10]  Ashish Tiwari,et al.  Output Range Analysis for Deep Neural Networks , 2017, ArXiv.

[11]  Rüdiger Ehlers,et al.  Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks , 2017, ATVA.

[12]  Sanjit A. Seshia,et al.  Compositional Falsification of Cyber-Physical Systems with Machine Learning Components , 2017, NFM.

[13]  Paulo Tabuada,et al.  Underminer , 2017, ACM Trans. Embed. Comput. Syst..

[14]  Edmund M. Clarke,et al.  Delta-Decidability over the Reals , 2012, 2012 27th Annual IEEE Symposium on Logic in Computer Science.

[15]  Edmund M. Clarke,et al.  dReal: An SMT Solver for Nonlinear Theories over the Reals , 2013, CADE.