Convolutional sparse coding on neurosynaptic cognitive system

Image features can be learned and subsequently used for reconstruction and classification tasks in the fields of machine learning and computer vision. In this work, we propose image reconstruction using Convolutional Sparse Coding (CSC) on IBM's TrueNorth Neuromorphic computing system. CSC explicitly models local interactions through the convolution operations. Convolutional kernels define a dictionary and Sparse Feature Maps (SFMs) that are generated through a training process. The images are reconstructed with convolutional operations on SFMs and respective kernels. In this paper, we report on experimental results demonstrating promising sparse reconstructions on the IBM Neuromorphic TrueNorth hardware for two different benchmarks: MNIST and CIFAR-10. It is noted that this is the first ever important step towards convolutional sparse coding on neuromorphic hardware.

[1]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[2]  Qinru Qiu,et al.  Designing reconfigurable large-scale deep learning systems using stochastic computing , 2016, 2016 IEEE International Conference on Rebooting Computing (ICRC).

[3]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[4]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[5]  Yann LeCun,et al.  Convolutional Matching Pursuit and Dictionary Training , 2010, ArXiv.

[6]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[7]  Bahram Parvin,et al.  Classification of Histology Sections via Multispectral Convolutional Sparse Coding , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Brendt Wohlberg,et al.  Efficient convolutional sparse coding , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[9]  David B. Dunson,et al.  Deep Learning with Hierarchical Convolutional Factor Analysis , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[11]  Andrew S. Cassidy,et al.  A million spiking-neuron integrated circuit with a scalable communication network and interface , 2014, Science.

[12]  Anders P. Eriksson,et al.  Fast Convolutional Sparse Coding , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Paulo Martins Engel,et al.  Convolutional Sparse Feature Descriptor for Object Recognition in CIFAR-10 , 2013, 2013 Brazilian Conference on Intelligent Systems.

[14]  Andrew S. Cassidy,et al.  Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[15]  Andrew S. Cassidy,et al.  Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[16]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[17]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[18]  Andrew S. Cassidy,et al.  Cognitive computing programming paradigm: A Corelet Language for composing networks of neurosynaptic cores , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[19]  Volkan Cevher,et al.  Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analytics , 2014, IEEE Signal Processing Magazine.

[20]  Marc'Aurelio Ranzato,et al.  Sparse Feature Learning for Deep Belief Networks , 2007, NIPS.

[21]  Andrew S. Cassidy,et al.  Convolutional networks for fast, energy-efficient neuromorphic computing , 2016, Proceedings of the National Academy of Sciences.

[22]  Jason Cong,et al.  Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks , 2015, FPGA.