Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components

Many organizations are developing autonomous driving systems, which are expected to be deployed at a large scale in the near future. Despite this, there is a lack of agreement on appropriate methods to test, debug, and certify the performance of these systems. One of the main challenges is that many autonomous driving systems have machine learning (ML) components, such as deep neural networks, for which formal properties are difficult to characterize. We present a testing framework that is compatible with test case generation and automatic falsification methods, which are used to evaluate cyber-physical systems. We demonstrate how the framework can be used to evaluate closed-loop properties of an autonomous driving system model that includes the ML components, all within a virtual environment. We demonstrate how to use test case generation methods, such as covering arrays, as well as requirement falsification methods to automatically identify problematic test scenarios. The resulting framework can be used to increase the reliability of autonomous driving systems.

[1]  Forrest N. Iandola,et al.  SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[2]  Dejan Nickovic,et al.  Specification-Based Monitoring of Cyber-Physical Systems: A Survey on Theory, Tools and Applications , 2018, Lectures on Runtime Verification.

[3]  Junfeng Yang,et al.  DeepXplore: Automated Whitebox Testing of Deep Learning Systems , 2017, SOSP.

[4]  BaekGyu Kim,et al.  The SMT-based automatic road network generation in vehicle simulation environment , 2016, 2016 International Conference on Embedded Software (EMSOFT).

[5]  Georgios Fainekos,et al.  Towards Formal Specification Visualization for Testing and Monitoring of Cyber-Physical Systems , 2014 .

[6]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[7]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[8]  Ken Butts,et al.  Simulation-Based Approaches for Verification of Embedded Control Systems: An Overview of Traditional and Advanced Modeling, Testing, and Verification Techniques , 2016, IEEE Control Systems.

[9]  Amnon Shashua,et al.  On a Formal Model of Safe and Scalable Self-driving Cars , 2017, ArXiv.

[10]  Houssam Abbas,et al.  Computer-aided design for safe autonomous vehicles , 2017, 2017 Resilience Week (RWS).

[11]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[12]  Alberto L. Sangiovanni-Vincentelli,et al.  Systematic Testing of Convolutional Neural Networks for Autonomous Driving , 2017, ArXiv.

[13]  Luca Pulina,et al.  An Abstraction-Refinement Approach to Verification of Artificial Neural Networks , 2010, CAV.

[14]  Sriram Sankaranarayanan,et al.  Verification of automotive control applications using S-TaLiRo , 2012, 2012 American Control Conference (ACC).

[15]  Georgios E. Fainekos,et al.  Utilizing S-TaLiRo as an automatic test generation framework for autonomous vehicles , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[16]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[17]  Tapani Raiko,et al.  International Conference on Learning Representations (ICLR) , 2016 .

[18]  Sanjit A. Seshia,et al.  Compositional Falsification of Cyber-Physical Systems with Machine Learning Components , 2017, NFM.

[19]  Ananthram Swami,et al.  The Limitations of Deep Learning in Adversarial Settings , 2015, 2016 IEEE European Symposium on Security and Privacy (EuroS&P).

[20]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[21]  Ian J. Goodfellow,et al.  NIPS 2016 Tutorial: Generative Adversarial Networks , 2016, ArXiv.

[22]  A. Hartman.  Software and Hardware Testing Using Combinatorial Covering Suites , 2005 .

[23]  Sriram Sankaranarayanan,et al.  Probabilistic Temporal Logic Falsification of Cyber-Physical Systems , 2013, TECS.

[24]  Yu Lei,et al.  Introduction to Combinatorial Testing , 2013 .

[25]  Suman Jana,et al.  DeepTest: Automated Testing of Deep-Neural-Network-Driven Autonomous Cars , 2017, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[26]  Jiri Matas,et al.  Forward-Backward Error: Automatic Detection of Tracking Failures , 2010, 2010 20th International Conference on Pattern Recognition.

[27]  Oded Maler,et al.  Robust Satisfaction of Temporal Logic over Real-Valued Signals , 2010, FORMATS.

[28]  Somesh Jha,et al.  Semantic Adversarial Deep Learning , 2018, IEEE Design & Test.

[29]  Jinfeng Yi,et al.  ZOO: Zeroth Order Optimization Based Black-box Attacks to Deep Neural Networks without Training Substitute Models , 2017, AISec@CCS.

[30]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[31]  Andreas Geiger,et al.  Are we ready for autonomous driving? The KITTI vision benchmark suite , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Ananthram Swami,et al.  Practical Black-Box Attacks against Machine Learning , 2016, AsiaCCS.

[33]  Ashish Tiwari,et al.  Output Range Analysis for Deep Neural Networks , 2017, ArXiv.

[34]  George J. Pappas,et al.  Robustness of Temporal Logic Specifications , 2006, FATES/RV.

[35]  Heni Ben Amor,et al.  Deep Predictive Models for Collision Risk Assessment in Autonomous Driving , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Olivier Michel,et al.  Cyberbotics Ltd. Webots™: Professional Mobile Robot Simulation , 2004 .

[37]  Shinichi Shiraishi,et al.  Testing Autonomous Vehicle Software in the Virtual Prototyping Environment , 2017, IEEE Embedded Systems Letters.