Generating Natural Adversarial Remote Sensing Images

Over the last years, Remote Sensing Images (RSI) analysis have started resorting to using deep neural networks to solve most of the commonly faced problems, such as detection, land cover classification or segmentation. As far as critical decision making can be based upon the results of RSI analysis, it is important to clearly identify and understand potential security threats occurring in those machine learning algorithms. Notably, it has recently been found that neural networks are particularly sensitive to carefully designed attacks, generally crafted given the full knowledge of the considered deep network. In this paper, we consider the more realistic but challenging case where one wants to generate such attacks in the case of a black-box neural network. In this case, only the prediction score of the network is accessible, given a specific input. Examples that lure away the network's prediction, while being perceptually similar to real images, are called natural or unrestricted adversarial examples. We present an original method to generate such examples, based on a variant of the Wasserstein Generative Adversarial Network. We demonstrate its effectiveness on natural adversarial hyper-spectral image generation and image modification for fooling a state-of-the-art detector. Among others, we also conduct a perceptual evaluation with human annotators to better assess the effectiveness of the proposed method.

[1]  Gabriel Peyré,et al.  Learning Generative Models with Sinkhorn Divergences , 2017, AISTATS.

[2]  Olac Fuentes,et al.  On the Defense Against Adversarial Examples Beyond the Visible Spectrum , 2018, MILCOM 2018 - 2018 IEEE Military Communications Conference (MILCOM).

[3]  Xiao Xiang Zhu,et al.  Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources , 2017, IEEE Geoscience and Remote Sensing Magazine.

[4]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[5]  Nicolas Audebert,et al.  Deep Learning for Classification of Hyperspectral Data: A Comparative Review , 2019, IEEE Geoscience and Remote Sensing Magazine.

[6]  Olac Fuentes,et al.  Integrated Learning and Feature Selection for Deep Neural Networks in Multispectral Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  Youngjoo Jo,et al.  SC-FEGAN: Face Editing Generative Adversarial Network With User’s Sketch and Color , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Ryan R. Curtin,et al.  Detecting Adversarial Samples from Artifacts , 2017, ArXiv.

[9]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[10]  Ting-Chun Wang,et al.  Image Inpainting for Irregular Holes Using Partial Convolutions , 2018, ECCV.

[11]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[12]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[13]  Rui Shu.  AC-GAN Learns a Biased Distribution , 2017 .

[14]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[15]  Taesung Park,et al.  Semantic Image Synthesis With Spatially-Adaptive Normalization , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Asja Fischer,et al.  On the regularization of Wasserstein GANs , 2017, ICLR.

[17]  Sameer Singh,et al.  Generating Natural Adversarial Examples , 2017, ICLR.

[18]  Yiming Yang,et al.  MMD GAN: Towards Deeper Understanding of Moment Matching Network , 2017, NIPS.

[19]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[20]  R. Gribonval,et al.  Learning with minibatch Wasserstein : asymptotic and gradient properties , 2019, AISTATS.

[21]  Jun Zhu,et al.  Towards Robust Detection of Adversarial Examples , 2017, NeurIPS.

[22]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Gabriel Peyré,et al.  Computational Optimal Transport , 2018, Found. Trends Mach. Learn..

[24]  Nicholas Carlini,et al.  Unrestricted Adversarial Examples , 2018, ArXiv.

[25]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Nicolas Courty,et al.  A Cycle Gan Approach for Heterogeneous Domain Adaptation in Land Use Classification , 2020, IGARSS 2020 - 2020 IEEE International Geoscience and Remote Sensing Symposium.

[27]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[28]  Yang Wang,et al.  MARTA GANs: Unsupervised Representation Learning for Remote Sensing Image Classification , 2016, IEEE Geoscience and Remote Sensing Letters.

[29]  Han Zhang,et al.  Improving GANs Using Optimal Transport , 2018, ICLR.

[30]  Junjun Jiang,et al.  Edge-Enhanced GAN for Remote Sensing Image Superresolution , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[32]  Wojciech Czaja,et al.  Adversarial examples in remote sensing , 2018, SIGSPATIAL/GIS.

[33]  Yang Song,et al.  Constructing Unrestricted Adversarial Examples with Generative Models , 2018, NeurIPS.

[34]  Fan Zhang,et al.  Deep Convolutional Neural Networks for Hyperspectral Image Classification , 2015, J. Sensors.

[35]  Sebastian Nowozin,et al.  f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization , 2016, NIPS.

[36]  Haifeng Li,et al.  Adversarial Example in Remote Sensing Image Recognition , 2019, ArXiv.

[37]  Seyed-Mohsen Moosavi-Dezfooli,et al.  DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Vladlen Koltun,et al.  Multi-Scale Context Aggregation by Dilated Convolutions , 2015, ICLR.

[39]  Bertrand Le Saux,et al.  Generative Adversarial Networks for Realistic Synthesis of Hyperspectral Samples , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[40]  Jianhua Lu,et al.  GAN-NL: Unsupervised Representation Learning for Remote Sensing Image Classification , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).