Weighted Average Precision: Adversarial Example Detection in the Visual Perception of Autonomous Vehicles

Recent works have shown that neural networks are vulnerable to carefully crafted adversarial examples (AE). By adding small perturbations to input images, AEs are able to make the victim model predicts incorrect outputs. Several research work in adversarial machine learning started to focus on the detection of AEs in autonomous driving. However, the existing studies either use preliminary assumption on outputs of detections or ignore the tracking system in the perception pipeline. In this paper, we firstly propose a novel distance metric for practical autonomous driving object detection outputs. Then, we bridge the gap between the current AE detection research and the real-world autonomous systems by providing a temporal detection algorithm, which takes the impact of tracking system into consideration. We perform evaluation on Berkeley Deep Drive (BDD) and CityScapes datasets to show how our approach outperforms existing single-frame-mAP based AE detections by increasing 17.76% accuracy of performance.

[1]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[2]  Lawrence Carin,et al.  Second-Order Adversarial Attack and Certifiable Robustness , 2018, ArXiv.

[3]  J. Zico Kolter,et al.  Scaling provable adversarial defenses , 2018, NeurIPS.

[4]  Ruigang Yang,et al.  The ApolloScape Open Dataset for Autonomous Driving and Its Application , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  David A. Wagner,et al.  Towards Evaluating the Robustness of Neural Networks , 2016, 2017 IEEE Symposium on Security and Privacy (SP).

[6]  Dan Boneh,et al.  Ensemble Adversarial Training: Attacks and Defenses , 2017, ICLR.

[7]  Qinru Qiu,et al.  Learning Topics Using Semantic Locality , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[8]  Hao Chen,et al.  MagNet: A Two-Pronged Defense against Adversarial Examples , 2017, CCS.

[9]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jin Zhao,et al.  A Simulation Framework For Fast Design Space Exploration Of Unmanned Air System Traffic Management Policies , 2019, 2019 Integrated Communications, Navigation and Surveillance Conference (ICNS).

[11]  Qi Zhao,et al.  Foveation-based Mechanisms Alleviate Adversarial Examples , 2015, ArXiv.

[12]  Senem Velipasalar,et al.  HUMAN ACTIVITY CLASSIFICATION INCORPORATING EGOCENTRIC VIDEO AND INERTIAL MEASUREMENT UNIT DATA , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[13]  Senem Velipasalar,et al.  Autonomous Choice of Deep Neural Network Parameters by a Modified Generative Adversarial Network , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[14]  Matthew Mirman,et al.  Differentiable Abstract Interpretation for Provably Robust Neural Networks , 2018, ICML.

[15]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[16]  Senem Velipasalar,et al.  Human activity classification from wearable devices with cameras , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[17]  Qinru Qiu,et al.  Simulation of Real-time Routing for UAS traffic Management with Communication and Airspace Safety Considerations , 2019, 2019 IEEE/AIAA 38th Digital Avionics Systems Conference (DASC).

[18]  Robert Grover Brown,et al.  Introduction to random signals and applied Kalman filtering : with MATLAB exercises and solutions , 1996 .

[19]  Tao Wei,et al.  Fooling Detection Alone is Not Enough: First Adversarial Attack against Multiple Object Tracking , 2019, ArXiv.

[20]  James Bailey,et al.  Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality , 2018, ICLR.

[21]  Ruigang Yang,et al.  The ApolloScape Dataset for Autonomous Driving , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[22]  Chia-Mu Yu,et al.  On the Limitation of Local Intrinsic Dimensionality for Characterizing the Subspaces of Adversarial Examples , 2018, ICLR.

[23]  David Wagner,et al.  Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods , 2017, AISec@CCS.

[24]  Senem Velipasalar,et al.  Autonomous Footstep Counting and Traveled Distance Calculation by Mobile Devices Incorporating Camera and Accelerometer Data , 2017, IEEE Sensors Journal.

[25]  Senem Velipasalar,et al.  Autonomous Human Activity Classification from Ego-vision Camera and Accelerometer Data , 2019, ArXiv.

[26]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Senem Velipasalar,et al.  Enhancing Cross-task Transferability of Adversarial Examples with Dispersion Reduction , 2019, ArXiv.

[28]  Xiangyu Zhang,et al.  Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples , 2018, NeurIPS.

[29]  Tao Wei,et al.  Fooling Detection Alone is Not Enough: Adversarial Attack against Multiple Object Tracking , 2020, ICLR.

[30]  Aleksander Madry,et al.  Towards Deep Learning Models Resistant to Adversarial Attacks , 2017, ICLR.

[31]  Mingyan Liu,et al.  Characterizing Adversarial Examples Based on Spatial Consistency Information for Semantic Segmentation , 2018, ECCV.

[32]  Senem Velipasalar,et al.  Wearable Sensor Applications: Processing of Egocentric Videos and Inertial Measurement Unit Data , 2019, Embedded, Cyber-Physical, and IoT Systems.

[33]  Moustapha Cissé,et al.  Countering Adversarial Images using Input Transformations , 2018, ICLR.

[34]  Kamyar Azizzadenesheli,et al.  Stochastic Activation Pruning for Robust Adversarial Defense , 2018, ICLR.

[35]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Samy Bengio,et al.  Adversarial examples in the physical world , 2016, ICLR.

[37]  Qinru Qiu,et al.  Temporal and Spatial Routing for Large Scale Safe and Connected UAS Traffic Management in Urban Areas , 2019, 2019 IEEE 25th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA).

[38]  Yilan Li,et al.  Efficient Human Activity Classification from Egocentric Videos Incorporating Actor-Critic Reinforcement Learning , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[39]  Yanjun Qi,et al.  Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks , 2017, NDSS.

[40]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[41]  Ryan R. Curtin,et al.  Detecting Adversarial Samples from Artifacts , 2017, ArXiv.

[42]  Lawrence Carin,et al.  Enhancing Cross-Task Black-Box Transferability of Adversarial Examples With Dispersion Reduction , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Senem Velipasalar,et al.  Autonomous Human Activity Classification From Wearable Multi-Modal Sensors , 2019, IEEE Sensors Journal.

[44]  Senem Velipasalar,et al.  Autonomously and Simultaneously Refining Deep Neural Network Parameters by a Bi-Generative Adversarial Network Aided Genetic Algorithm , 2018, ArXiv.

[45]  Luca Rigazio,et al.  Towards Deep Neural Network Architectures Robust to Adversarial Examples , 2014, ICLR.

[46]  J. Zico Kolter,et al.  Provable defenses against adversarial examples via the convex outer adversarial polytope , 2017, ICML.

[47]  Shu-Li Sun,et al.  Multi-sensor optimal information fusion Kalman filter , 2004, Autom..

[48]  Senem Velipasalar,et al.  Robust footstep counting and traveled distance calculation by mobile phones incorporating camera geometry , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[49]  Michael P. Wellman,et al.  Towards the Science of Security and Privacy in Machine Learning , 2016, ArXiv.

[50]  Shinpei Kato,et al.  An Open Approach to Autonomous Vehicles , 2015, IEEE Micro.