Verification Approaches for Learning-Enabled Autonomous Cyber–Physical Systems

<italic>Editor’s notes:</italic> Neural network control systems are often at the heart of autonomous systems. The authors classify existing verification methods for these systems and advocate the necessity of integrating verification techniques in the training process to enhance robustness. —<italic>Selma Saidi, TU Dortmund</italic>

[1]  Taylor T. Johnson,et al.  Improved Geometric Path Enumeration for Verifying ReLU Neural Networks , 2020, CAV.

[2]  Weiming Xiang,et al.  Reachable Set Estimation for Neural Network Control Systems: A Simulation-Guided Approach , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Weiming Xiang,et al.  NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems , 2020, CAV.

[4]  Taylor T. Johnson,et al.  Verification of Deep Convolutional Neural Networks Using ImageStars , 2020, CAV.

[5]  Xenofon D. Koutsoukos,et al.  Safety Verification of Cyber-Physical Systems with Reinforcement Learning Control , 2019, ACM Trans. Embed. Comput. Syst..

[6]  Weiming Xiang,et al.  Star-Based Reachability Analysis of Deep Neural Networks , 2019, FM.

[7]  Jiameng Fan,et al.  ReachNN , 2019, ACM Trans. Embed. Comput. Syst..

[8]  Mark Harman,et al.  Machine Learning Testing: Survey, Landscapes and Horizons , 2019, IEEE Transactions on Software Engineering.

[9]  Weiming Xiang,et al.  Parallelizable Reachability Analysis Algorithms for Feed-Forward Neural Networks , 2019, 2019 IEEE/ACM 7th International Conference on Formal Methods in Software Engineering (FormaliSE).

[10]  Taylor T. Johnson,et al.  ARCH-COMP19 Category Report: Artificial Intelligence and Neural Network Control Systems (AINNCS) for Continuous and Hybrid Systems Plants , 2019, ARCH@CPSIoTWeek.

[11]  Ashish Tiwari,et al.  Sherlock - A tool for verification of neural network feedback systems: demo abstract , 2019, HSCC.

[12]  Sriram Sankaranarayanan,et al.  Reachability analysis for neural feedback systems using regressive polynomial rule inference , 2019, HSCC.

[13]  Alberto L. Sangiovanni-Vincentelli,et al.  A Formalization of Robustness for Deep Neural Networks , 2019, ArXiv.

[14]  Mykel J. Kochenderfer,et al.  Algorithms for Verifying Deep Neural Networks , 2019, Found. Trends Optim..

[15]  Timon Gehr,et al.  An abstract domain for certifying neural networks , 2019, Proc. ACM Program. Lang..

[16]  Weiming Xiang,et al.  Specification-Guided Safety Verification for Feedforward Neural Networks , 2018, ArXiv.

[17]  Insup Lee,et al.  Verisig: verifying safety properties of hybrid systems with neural network controllers , 2018, HSCC.

[18]  Yasser Shoukry,et al.  Formal verification of neural network controlled autonomous systems , 2018, HSCC.

[19]  Sanjit A. Seshia,et al.  Formal Specification for Deep Neural Networks , 2018, ATVA.

[20]  Weiming Xiang,et al.  Verification for Machine Learning, Autonomy, and Neural Networks Survey , 2018, ArXiv.

[21]  Alberto L. Sangiovanni-Vincentelli,et al.  Scenic: a language for scenario specification and scene generation , 2018, PLDI.

[22]  Weiming Xiang,et al.  Reachability Analysis and Safety Verification for Neural Network Control Systems , 2018, ArXiv.

[23]  Luca Pulina,et al.  Automated Verification of Neural Networks: Advances, Challenges and Perspectives , 2018, ArXiv.

[24]  Swarat Chaudhuri,et al.  AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[25]  Junfeng Yang,et al.  Formal Security Analysis of Neural Networks using Symbolic Intervals , 2018, USENIX Security Symposium.

[26]  Somesh Jha,et al.  Semantic Adversarial Deep Learning , 2018, IEEE Design & Test.

[27]  Ashish Tiwari,et al.  Output Range Analysis for Deep Feedforward Neural Networks , 2018, NFM.

[28]  Daniel Kroening,et al.  Global Robustness Evaluation of Deep Neural Networks with Provable Guarantees for L0 Norm , 2018, ArXiv.

[29]  James Kapinski,et al.  INVITED: Reasoning about Safety of Learning-Enabled Components in Autonomous Cyber-physical Systems , 2018, 2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC).

[30]  Pushmeet Kohli,et al.  A Dual Approach to Scalable Verification of Deep Networks , 2018, UAI.

[31]  Weiming Xiang,et al.  Reachable Set Estimation and Safety Verification for Piecewise Linear Systems with Neural Network Controllers , 2018, 2018 Annual American Control Conference (ACC).

[32]  Weiming Xiang,et al.  Reachable Set Estimation and Verification for Neural Network Models of Nonlinear Dynamic Systems , 2018, Safe, Autonomous and Intelligent Vehicles.

[33]  Jinfeng Yi,et al.  Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach , 2018, ICLR.

[34]  Weiming Xiang,et al.  Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations , 2017, ArXiv.

[35]  Russ Tedrake,et al.  Verifying Neural Networks with Mixed Integer Programming , 2017, ArXiv.

[36]  Pushmeet Kohli,et al.  Piecewise Linear Neural Network verification: A comparative study , 2017, ArXiv.

[37]  Chih-Hong Cheng,et al.  Verification of Binarized Neural Networks , 2017, ArXiv.

[38]  Luan Viet Nguyen,et al.  Hyperproperties of real-valued signals , 2017, MEMOCODE.

[39]  Ashish Tiwari,et al.  Output Range Analysis for Deep Neural Networks , 2017, ArXiv.

[40]  Leonid Ryzhyk,et al.  Verifying Properties of Binarized Deep Neural Networks , 2017, AAAI.

[41]  Suman Jana,et al.  DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[42]  Weiming Xiang,et al.  Output Reachable Set Estimation and Verification for Multilayer Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Alessio Lomuscio,et al.  An approach to reachability analysis for feed-forward ReLU neural networks , 2017, ArXiv.

[44]  Junfeng Yang,et al.  DeepXplore: Automated Whitebox Testing of Deep Learning Systems , 2017, SOSP.

[45]  Rüdiger Ehlers,et al.  Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks , 2017, ATVA.

[46]  Sanjit A. Seshia,et al.  Compositional Falsification of Cyber-Physical Systems with Machine Learning Components , 2017, Journal of Automated Reasoning.

[47]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[48]  Mykel J. Kochenderfer,et al.  Neural Network Guidance for UAVs , 2017 .

[49]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[50]  Mykel J. Kochenderfer,et al.  Policy compression for aircraft collision avoidance systems , 2016, 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC).

[51]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[52]  Sergiy Bogomolov,et al.  HYST: a source transformation and translation tool for hybrid automaton models , 2015, HSCC.

[53]  Lui Sha,et al.  Real-Time Reachability for Verified Simplex Design , 2014, 2014 IEEE Real-Time Systems Symposium.

[54]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[55]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[56]  Manfred Morari,et al.  Multi-Parametric Toolbox 3.0 , 2013, 2013 European Control Conference (ECC).

[57]  Xin Chen,et al.  Flow*: An Analyzer for Non-linear Hybrid Systems , 2013, CAV.

[58]  Luca Pulina,et al.  Challenging SMT solvers to verify neural networks , 2012, AI Commun..

[59]  Luca Pulina,et al.  NeVer: a tool for artificial neural networks verification , 2011, Annals of Mathematics and Artificial Intelligence.

[60]  Luca Pulina,et al.  An Abstraction-Refinement Approach to Verification of Artificial Neural Networks , 2010, CAV.

[61]  Martin Fränzle,et al.  HySAT: An efficient proof engine for bounded model checking of hybrid systems , 2007, Formal Methods Syst. Des..

[62]  R.R. Zakrzewski,et al.  Randomized approach to verification of neural networks , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[63]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[64]  Matthew Mirman,et al.  Fast and Effective Robustness Certification , 2018, NeurIPS.

[65]  S. Seshia.  Compositional Verification without Compositional Specification for Learning-Based Systems , 2017 .

[66]  Bernd Becker,et al.  Towards Verification of Artificial Neural Networks , 2015, MBMV.

[67]  Clark W. Barrett,et al.  The SMT-LIB Standard Version 2.0 , 2010 .