Gray-box adversarial testing for control systems with machine learning components

Neural Networks (NN) have been proposed in the past as an effective means for both modeling and control of systems with very complex dynamics. However, despite the extensive research, NN-based controllers have not been adopted by the industry for safety critical systems. The primary reason is that systems with learning based controllers are notoriously hard to test and verify. Even harder is the analysis of such systems against system-level specifications. In this paper, we provide a gradient based method for searching the input space of a closed-loop control system in order to find adversarial samples against some system-level requirements. Our experimental results show that combined with randomized search, our method outperforms Simulated Annealing optimization.

[1]  Houssam Abbas,et al.  Safe At Any Speed: A Simulation-Based Test Harness for Autonomous Vehicles , 2017, CyPhy.

[2]  Ashish Tiwari,et al.  Output Range Analysis for Deep Neural Networks , 2017, ArXiv.

[3]  Georgios Fainekos,et al.  Falsification of Temporal Logic Requirements Using Gradient Based Local Search in Space and Time , 2018, ADHS.

[4]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[5]  Sanjit A. Seshia,et al.  Towards Verified Artificial Intelligence , 2016, ArXiv.

[6]  Alberto L. Sangiovanni-Vincentelli,et al.  Systematic Testing of Convolutional Neural Networks for Autonomous Driving , 2017, ArXiv.

[7]  Georgios Fainekos,et al.  Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[8]  Houssam Abbas,et al.  Smooth operator: Control using the smooth robustness of temporal logic , 2017, 2017 IEEE Conference on Control Technology and Applications (CCTA).

[9]  Alberto L. Sangiovanni-Vincentelli,et al.  Counterexample-Guided Data Augmentation , 2018, IJCAI.

[10]  David A. Wagner,et al.  Towards Evaluating the Robustness of Neural Networks , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[11]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[12]  Nhan T. Nguyen,et al.  Stability, Convergence, and Verification and Validation Challenges of Neural Net Adaptive Flight Control , 2010, Applications of Neural Networks in High Assurance Systems.

[13]  Houssam Abbas,et al.  Functional gradient descent method for Metric Temporal Logic specifications , 2014, 2014 American Control Conference.

[14]  Weiming Xiang,et al.  Reachability Analysis and Safety Verification for Neural Network Control Systems , 2018, ArXiv.

[15]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[16]  Ashish Tiwari,et al.  Learning and Verification of Feedback Control Systems using Feedforward Neural Networks , 2018, ADHS.

[17]  Anouck Girard,et al.  Optimal Control Based Falsification of Unknown Systems with Time Delays: A Gasoline Engine A/F Ratio Control Case Study , 2018 .

[18]  Dejan Nickovic,et al.  Specification-Based Monitoring of Cyber-Physical Systems: A Survey on Theory, Tools and Applications , 2018, Lectures on Runtime Verification.

[19]  Georgios E. Fainekos,et al.  Local Descent For Temporal Logic Falsification of Cyber-Physical Systems (Extended Technical Report) , 2017, CyPhy.

[20]  George J. Pappas,et al.  Robustness of temporal logic specifications for continuous-time signals , 2009, Theor. Comput. Sci..

[21]  Sriram Sankaranarayanan,et al.  S-TaLiRo: A Tool for Temporal Logic Falsification for Hybrid Systems , 2011, TACAS.

[22]  Aviral Shrivastava,et al.  Timestamp Temporal Logic (TTL) for Testing the Timing of Cyber-Physical Systems , 2017, ACM Trans. Embed. Comput. Syst..

[23]  Sriram Sankaranarayanan,et al.  Multiple shooting, CEGAR-based falsification for hybrid systems , 2014, EMSOFT '14.

[24]  Sanjit A. Seshia,et al.  Compositional Falsification of Cyber-Physical Systems with Machine Learning Components , 2017, NFM.

[25]  Georgios E. Fainekos,et al.  Hybrid approximate gradient and stochastic descent for falsification of nonlinear systems , 2017, 2017 American Control Conference (ACC).

[26]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.