Maximum Resilience of Artificial Neural Networks

The deployment of Artificial Neural Networks (ANNs) in safety-critical applications poses a number of new verification and certification challenges. In particular, for ANN-enabled self-driving vehicles it is important to establish properties about the resilience of ANNs to noisy or even maliciously manipulated sensory input. We are addressing these challenges by defining resilience properties of ANN-based classifiers as the maximal amount of input or sensor perturbation which is still tolerated. This problem of computing maximal perturbation bounds for ANNs is then reduced to solving mixed integer optimization problems (MIP). A number of MIP encoding heuristics are developed for drastically reducing MIP-solver runtimes, and using parallelization of MIP-solvers results in an almost linear speed-up in the number (up to a certain limit) of computing cores in our experiments. We demonstrate the effectiveness and scalability of our approach by means of computing maximal resilience bounds for a number of ANN benchmark sets ranging from typical image recognition scenarios to the autonomous maneuvering of robots.

[1]  Bernd Becker,et al.  Towards Verification of Artificial Neural Networks , 2015, MBMV.

[2]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[3]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[4]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Min Wu,et al.  Safety Verification of Deep Neural Networks , 2016, CAV.

[6]  I. Grossmann.  Review of Nonlinear Mixed-Integer and Disjunctive Programming Techniques , 2002 .

[7]  Ananthram Swami,et al.  Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples , 2016, ArXiv.

[8]  Luca Pulina,et al.  Challenging SMT solvers to verify neural networks , 2012, AI Commun..

[9]  Grant Potter,et al.  ConvNetJS: Deep Learning in your browser , 2017 .

[10]  Xin Zhang,et al.  End to End Learning for Self-Driving Cars , 2016, ArXiv.

[11]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[12]  Richard G. Lyons,et al.  Efficient Approximations for the Arctangent Function , 2007 .

[13]  Abhisek Ukil,et al.  Fast computation of arctangent functions for embedded applications: A comparative analysis , 2011, 2011 IEEE International Symposium on Industrial Electronics.

[14]  Antonio Criminisi,et al.  Measuring Neural Net Robustness with Constraints , 2016, NIPS.

[15]  Nikolaj Bjørner,et al.  νZ - An Optimizing SMT Solver , 2015, TACAS.

[16]  Ananthram Swami,et al.  Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks , 2015, 2016 IEEE Symposium on Security and Privacy (SP).

[17]  Samy Bengio,et al.  Adversarial examples in the physical world , 2016, ICLR.

[18]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[19]  Hsuan-Tien Lin,et al.  Learning From Data , 2012 .

[20]  Matthew J. Saltzman,et al.  Computational Experience with a Software Framework for Parallel Integer Programming , 2009, INFORMS J. Comput..

[21]  S. Bhattacharyya,et al.  Certification considerations for adaptive systems , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[22]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[23]  Ananthram Swami,et al.  Practical Black-Box Attacks against Machine Learning , 2016, AsiaCCS.

[24]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[25]  Luca Pulina,et al.  An Abstraction-Refinement Approach to Verification of Artificial Neural Networks , 2010, CAV.