Combining STDP and Reward-Modulated STDP in Deep Convolutional Spiking Neural Networks for Digit Recognition

The primate visual system has inspired the development of deep artificial neural networks, which have revolutionized the computer vision domain. Yet these networks are much less energy-efficient than their biological counterparts, and they are typically trained with backpropagation, which is extremely data-hungry. To address these limitations, we used a deep convolutional spiking neural network (DCSNN) and a latency-coding scheme. We trained it using a combination of spike-timing-dependent plasticity (STDP) for the lowest layers and reward-modulated STDP (R-STDP) for the highest ones. In short, with R-STDP a correct (resp. incorrect) decision leads to STDP (resp. anti-STDP). This approach led to an accuracy of 97.2% on MNIST, without requiring an external classifier. In addition, we demonstrated that R-STDP extracts features that are diagnostic for the task at hand, and discards the other ones, whereas STDP extracts any feature that repeats. Finally, our approach is biologically plausible, hardware friendly, and energy-efficient.

[1]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[2]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[3]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[4]  Matthieu Gilson,et al.  STDP Allows Fast Rate-Modulated Coding with Poisson-Like Spike Trains , 2011, PLoS Comput. Biol..

[5]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[6]  Deepak Khosla,et al.  Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition , 2014, International Journal of Computer Vision.

[7]  Bernabé Linares-Barranco,et al.  Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Pierre Tirilly,et al.  Multi-layered Spiking Neural Network with Target Timestamp Threshold Adaptation and STDP , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[9]  K. Doya,et al.  The computational neurobiology of learning and reward , 2006, Current Opinion in Neurobiology.

[10]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[11]  Somnath Paul,et al.  Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines , 2016, Front. Neurosci..

[12]  D. Querlioz,et al.  Immunity to Device Variations in a Spiking Neural Network With Memristive Nanodevices , 2013, IEEE Transactions on Nanotechnology.

[13]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[14]  Zenghui Wang,et al.  Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review , 2017, Neural Computation.

[15]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..

[16]  J. Mink,et al.  Ratio of central nervous system to body metabolism in vertebrates: its constancy and functional basis. , 1981, The American journal of physiology.

[17]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[18]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[19]  Shaista Hussain,et al.  Improved margin multi-class classification using dendritic neurons with morphological learning , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[20]  S. Thorpe,et al.  STDP-based spiking deep convolutional neural networks for object recognition , 2018 .

[21]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[22]  Shih-Chii Liu,et al.  Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification , 2017, Front. Neurosci..

[23]  Bernabé Linares-Barranco,et al.  Hardware implementation of convolutional STDP for on-line visual feature learning , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[24]  Walter Senn,et al.  Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics , 2007, Neural Computation.

[25]  Timothée Masquelier,et al.  STDP Allows Close-to-Optimal Spatiotemporal Spike Pattern Detection by Single Coincidence Detector Neurons , 2016, Neuroscience.

[27]  Lingfeng Wang,et al.  Deep unsupervised learning with consistent inference of latent representations , 2017, Pattern Recognit..

[28]  Matthew Cook,et al.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[29]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[30]  Steve Furber,et al.  Large-scale neuromorphic computing systems , 2016, Journal of neural engineering.

[31]  Wolfram Schultz,et al.  Reward , 2019, HR for Creative Companies.

[32]  Y. Niv.  Reinforcement learning in the brain , 2009 .

[33]  Bipin Rajendran,et al.  Spiking neural networks for handwritten digit recognition - Supervised learning and network optimization , 2018, Neural Networks.

[34]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[35]  J Gautrais,et al.  Rate coding versus temporal order coding: a theoretical approach. , 1998, Bio Systems.

[36]  Matthew Cook,et al.  A wake-sleep algorithm for recurrent, spiking neural networks , 2017, ArXiv.

[37]  Steven C. H. Hoi,et al.  Online Learning: A Comprehensive Survey , 2018, Neurocomputing.

[38]  Luiz Eduardo Soares de Oliveira,et al.  Handwritten digit segmentation: Is it still necessary? , 2018, Pattern Recognit..

[39]  Max Welling,et al.  Deep Spiking Networks , 2016, ArXiv.

[40]  W. Gerstner,et al.  Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules , 2016, Front. Neural Circuits.

[41]  Nikil D. Dutt,et al.  Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule , 2013, Neural Networks.

[42]  Andrzej J. Kasinski,et al.  Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting , 2010, Neural Computation.

[43]  H. Seo,et al.  Neural basis of reinforcement learning and decision making. , 2012, Annual review of neuroscience.

[44]  Anthony S. Maida,et al.  Bio-Inspired Spiking Convolutional Neural Network using Layer-wise Sparse Coding and STDP Learning , 2016, ArXiv.

[45]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[46]  Timothée Masquelier,et al.  Deep Learning in Spiking Neural Networks , 2018, Neural Networks.

[47]  Tao Liu,et al.  MT-spike: A multilayer time-based spiking neuromorphic architecture with temporal error backpropagation , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[48]  Haizhou Li,et al.  Rapid Feedforward Computation by Temporal Encoding and Learning With Spiking Neurons , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[49]  W. Schultz.  Neuronal Reward and Decision Signals: From Theories to Data. , 2015, Physiological reviews.

[50]  Timothée Masquelier,et al.  First-Spike-Based Visual Categorization Using Reward-Modulated STDP , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[51]  Tobi Delbruck,et al.  Real-time classification and sensor fusion with a spiking deep belief network , 2013, Front. Neurosci..

[52]  Josiah R. Boivin,et al.  A Causal Link Between Prediction Errors, Dopamine Neurons and Learning , 2013, Nature Neuroscience.

[53]  Vishal Saxena,et al.  Deep Convolutional Spiking Neural Networks for Image Classification , 2019, ArXiv.

[54]  Lei Deng,et al.  Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks , 2017, Front. Neurosci..

[55]  W. Schultz,et al.  Sequential neuromodulation of Hebbian plasticity offers mechanism for effective reward-based navigation , 2017, eLife.

[56]  Timothée Masquelier,et al.  Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity , 2007, PLoS Comput. Biol..

[57]  Gert Cauwenberghs,et al.  Event-driven contrastive divergence for spiking neuromorphic systems , 2013, Front. Neurosci..

[58]  Abbas Nowzari-Dalini,et al.  SpykeTorch: Efficient Simulation of Convolutional Spiking Neural Networks With at Most One Spike per Neuron , 2019, Front. Neurosci..

[59]  Timothée Masquelier,et al.  Acquisition of visual features through probabilistic spike-timing-dependent plasticity , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).