A Neuromorphic Sparse Coding Defense to Adversarial Images

Adversarial images are a class of images that have been slightly altered by very specific noise to change the way a deep learning neural network classifies the image. In many cases, this particular noise is imperceptible to the human vision system and thus presents a vulnerability of significant concern to the machine learning and artificial intelligence community. Research towards mitigating this type of attack has taken many forms, one of which is to filter or post process the image before classifying the image with a deep neural network. Techniques such as smoothing, filtering, and compression have been used with varying levels of success. In our work, we explored the use of a neuromorphic software and hardware approach as a protection against adversarial image attack. The algorithm governing our neuromorphic approach is based upon sparse coding. Our sparse coding approach is solved using a dynamic system of equations that models biological low level vision. Our quantitative and qualitative results show that a sparse coding reconstruction is remarkably invariant to changes in sparsity and reconstruction error with respect to classification accuracy. Furthermore, our approach is able to maintain low reconstruction errors without sacrificing classification performance.

[1]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Ananthram Swami,et al.  The Limitations of Deep Learning in Adversarial Settings , 2015, 2016 IEEE European Symposium on Security and Privacy (EuroS&P).

[4]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[5]  Edward Kim,et al.  Classifiers Based on Deep Sparse Coding Architectures are Robust to Deep Learning Transferable Examples , 2018, ArXiv.

[6]  Dan Boneh,et al.  Ensemble Adversarial Training: Attacks and Defenses , 2017, ICLR.

[7]  Kouichi Sakurai,et al.  One Pixel Attack for Fooling Deep Neural Networks , 2017, IEEE Transactions on Evolutionary Computation.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yanjun Qi,et al.  Feature Squeezing Mitigates and Detects Carlini/Wagner Adversarial Examples , 2017, ArXiv.

[10]  Ilya Kostrikov,et al.  PlaNet - Photo Geolocation with Convolutional Neural Networks , 2016, ECCV.

[11]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[12]  E. B. Baum,et al.  Internal representations for associative memory , 1988, Biological Cybernetics.

[13]  Edward Kim,et al.  Deep Sparse Coding for Invariant Multimodal Halle Berry Neurons , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[14]  Richard G. Baraniuk,et al.  Locally Competitive Algorithms for Sparse Approximation , 2007, 2007 IEEE International Conference on Image Processing.

[15]  Zoubin Ghahramani,et al.  A study of the effect of JPG compression on adversarial images , 2016, ArXiv.

[16]  Wei Lu,et al.  Replicating Kernels with a Short Stride Allows Sparse Reconstructions with Fewer Independent Kernels , 2014, ArXiv.

[17]  R. Baddeley.  Visual perception. An efficient code in V1? , 1996, Nature.

[18]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[19]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[20]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[21]  Yoshua Bengio,et al.  Measuring the tendency of CNNs to Learn Surface Statistical Regularities , 2017, ArXiv.

[22]  Ping Tak Peter Tang,et al.  Sparse Coding by Spiking Neural Networks: Convergence Theory and Computational Results , 2017, ArXiv.

[23]  Peter Földiák,et al.  SPARSE CODING IN THE PRIMATE CORTEX , 2002 .

[24]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[25]  Moustapha Cissé,et al.  Countering Adversarial Images using Input Transformations , 2018, ICLR.

[26]  Seyed-Mohsen Moosavi-Dezfooli,et al.  DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[28]  Steven David Prestwich,et al.  Denoising Dictionary Learning Against Adversarial Perturbations , 2018, AAAI Workshops.

[29]  Upamanyu Madhow,et al.  Sparsity-based Defense Against Adversarial Attacks on Linear Classifiers , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[30]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[31]  Roland Baddeley,et al.  An efficient code in V1? , 1996, Nature.