Using Local Convolutional Units to Defend Against Adversarial Examples

Deep neural networks are known to be sensitive to adversarial examples – inputs that are created in such a way that they are similar (if viewed by people) to clean inputs, but the neural network has high confidence that they belong to another class.In this paper, we study a new type of neural network unit similar to the convolutional units, but with a more local behavior. The unit is based on the Gaussian radial basis function. We show that if we replace the first convolutional layer in a convolutional network by the new layer (called RBFolutional), we obtain better robustness towards adversarial samples on the MNIST and CIFAR10 datasets, without sacrificing the performance on the clean examples.

[1]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Kouichi Sakurai,et al.  One Pixel Attack for Fooling Deep Neural Networks , 2017, IEEE Transactions on Evolutionary Computation.

[3]  Roman Neruda,et al.  Evolutionary generation of adversarial examples for deep and shallow machine learning models , 2016, MISNC, SI, DS 2016.

[4]  Luca Rigazio,et al.  Towards Deep Neural Network Architectures Robust to Adversarial Examples , 2014, ICLR.

[5]  Suvrit Sra,et al.  Deep-RBF Networks Revisited: Robust Classification with Rejection , 2018, ArXiv.

[6]  Samy Bengio,et al.  Adversarial examples in the physical world , 2016, ICLR.

[7]  David A. Wagner,et al.  Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples , 2018, ICML.

[8]  Aleksander Madry,et al.  Towards Deep Learning Models Resistant to Adversarial Attacks , 2017, ICLR.

[9]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[12]  Ananthram Swami,et al.  The Limitations of Deep Learning in Adversarial Settings , 2015, 2016 IEEE European Symposium on Security and Privacy (EuroS&P).

[13]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[14]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Lujo Bauer,et al.  Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition , 2016, CCS.

[17]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[18]  Ananthram Swami,et al.  Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples , 2016, ArXiv.

[19]  Mohammad Rouhani,et al.  Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures , 2016, ArXiv.

[20]  Ousmane Amadou Dia,et al.  Adversarial Examples in Modern Machine Learning: A Review , 2019, ArXiv.

[21]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.