ARCH-COMP21 Category Report: Artificial Intelligence and Neural Network Control Systems (AINNCS) for Continuous and Hybrid Systems Plants

This report presents the results of a friendly competition for formal verification of continuous and hybrid systems with artificial intelligence (AI) components. Specifically, machine learning (ML) components in cyber-physical systems (CPS), such as feedforward neural networks used as feedback controllers in closed-loop systems are considered, which is a class of systems classically known as intelligent control systems, or in more modern and specific terms, neural network control systems (NNCS). We more broadly refer to this category as AI and NNCS (AINNCS). The friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in 2021. In the third edition of this AINNCS category at ARCH-COMP, three tools have been applied to solve seven different benchmark problems, (in alphabetical order): JuliaReach, NNV, and Verisig. JuliaReach is a new participant in this category, Verisig participated previously in 2019 and NNV has participated in all previous competitions. This report is a snapshot of the current landscape of tools and the types of benchmarks for which these tools are suited. Due to the diversity of problems, lack of a shared hardware platform, and the early stage of the competition, we are not ranking tools in terms of performance, yet the presented results combined with 2020 results probably provide the most complete assessment of current tools for safety verification of NNCS.

[1]  Matthias Althoff,et al.  Implementation of Taylor models in CORA 2018 , 2018, ARCH@ADHS.

[2]  Weiming Xiang,et al.  Reachable Set Estimation for Neural Network Control Systems: A Simulation-Guided Approach , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Weiming Xiang,et al.  Verification for Machine Learning, Autonomy, and Neural Networks Survey , 2018, ArXiv.

[4]  Samy Bengio,et al.  Adversarial examples in the physical world , 2016, ICLR.

[5]  Weiming Xiang,et al.  Verification of Deep Convolutional Neural Networks Using ImageStars , 2020, CAV.

[6]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[7]  George J. Pappas,et al.  Verisig 2.0: Verification of Neural Network Controllers Using Taylor Model Preconditioning , 2021, CAV.

[8]  Pushmeet Kohli,et al.  A Dual Approach to Scalable Verification of Deep Networks , 2018, UAI.

[9]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[10]  Ashish Tiwari,et al.  Learning and Verification of Feedback Control Systems using Feedforward Neural Networks , 2018, ADHS.

[11]  Weiming Xiang,et al.  Specification-Guided Safety Verification for Feedforward Neural Networks , 2018, ArXiv.

[12]  Weiming Xiang,et al.  Output Reachable Set Estimation and Verification for Multilayer Neural Networks , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[13]  S. Joe Qin,et al.  An Overview of Nonlinear Model Predictive Control Applications , 2000 .

[14]  Insup Lee,et al.  Verisig: verifying safety properties of hybrid systems with neural network controllers , 2018, HSCC.

[15]  David P. Sanders,et al.  TaylorSeries.jl: Taylor expansions in one and several variables in Julia , 2019, J. Open Source Softw..

[16]  Qing Nie,et al.  DifferentialEquations.jl – A Performant and Feature-Rich Ecosystem for Solving Differential Equations in Julia , 2017, Journal of Open Research Software.

[17]  Matthias Althoff,et al.  Implementation of Interval Arithmetic in CORA 2016 , 2016, ARCH@CPSWeek.

[18]  Weiming Xiang,et al.  NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systems , 2020, CAV.

[19]  Mykel J. Kochenderfer,et al.  Algorithms for Verifying Deep Neural Networks , 2019, Found. Trends Optim..

[20]  Matthias Althoff,et al.  An Introduction to CORA 2015 , 2015, ARCH@CPSWeek.

[21]  Mykel J. Kochenderfer,et al.  A Reachability Method for Verifying Dynamical Systems with Deep Neural Network Controllers , 2019, ArXiv.

[22]  Thomas A. Henzinger,et al.  Scalable Static Hybridization Methods for Analysis of Nonlinear Systems , 2016, HSCC.

[23]  Mrdjan Jankovic,et al.  TORA example: cascade- and passivity-based control designs , 1996, IEEE Trans. Control. Syst. Technol..

[24]  Insup Lee,et al.  Case study: verifying the safety of an autonomous racing car with a neural network controller , 2019, HSCC.

[25]  Xin Chen,et al.  Flow*: An Analyzer for Non-linear Hybrid Systems , 2013, CAV.

[26]  Sergiy Bogomolov,et al.  HYST: a source transformation and translation tool for hybrid automaton models , 2015, HSCC.

[27]  Sriram Sankaranarayanan,et al.  Reachability analysis for neural feedback systems using regressive polynomial rule inference , 2019, HSCC.

[28]  Weiming Xiang,et al.  Star-Based Reachability Analysis of Deep Neural Networks , 2019, FM.

[29]  Weiming Xiang,et al.  Reachability Analysis and Safety Verification for Neural Network Control Systems , 2018, ArXiv.

[30]  Sergiy Bogomolov,et al.  JuliaReach: a toolbox for set-based reachability , 2019, HSCC.

[31]  ARCH-COMP20 Category Report: Artificial Intelligence and Neural Network Control Systems (AINNCS) for Continuous and Hybrid Systems Plants , 2020, ARCH.

[32]  Stuart J. Russell,et al.  Research Priorities for Robust and Beneficial Artificial Intelligence , 2015, AI Mag..

[33]  Taylor T. Johnson,et al.  ARCH-COMP19 Category Report: Artificial Intelligence and Neural Network Control Systems (AINNCS) for Continuous and Hybrid Systems Plants , 2019, ARCH@CPSIoTWeek.

[34]  Xenofon D. Koutsoukos,et al.  Safety Verification of Cyber-Physical Systems with Reinforcement Learning Control , 2019, ACM Trans. Embed. Comput. Syst..

[35]  George J. Pappas,et al.  Verifying the Safety of Autonomous Systems with Neural Network Controllers , 2020, ACM Trans. Embed. Comput. Syst..

[36]  Weiming Xiang,et al.  Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations , 2017, ArXiv.

[37]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[38]  Taylor Johnson,et al.  The Second International Verification of Neural Networks Competition (VNN-COMP 2021): Summary and Results , 2021, ArXiv.

[39]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[40]  Rajeev Alur,et al.  Formal verification of hybrid systems , 2011, 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT).

[41]  Matthew Mirman,et al.  Fast and Effective Robustness Certification , 2018, NeurIPS.