Bio-inspired digit recognition using reward-modulated spike-timing-dependent plasticity in deep convolutional networks

The primate visual system has inspired the development of deep artificial neural networks, which have revolutionized the computer vision domain. Yet these networks are much less energy-efficient than their biological counterparts, and they are typically trained with backpropagation, which is extremely data-hungry. To address these limitations, we used a deep convolutional spiking neural network (DCSNN) and a latency-coding scheme. We trained it using a combination of spike-timingdependent plasticity (STDP) for the lower layers and reward-modulated STDP (R-STDP) for the higher ones. In short, with R-STDP a correct (resp. incorrect) decision leads to STDP (resp. antiSTDP). This approach led to an accuracy of 97.2% on MNIST, without requiring an external classifier. In addition, we demonstrated that R-STDP extracts features that are diagnostic for the task at hand, and discards the other ones, whereas STDP ∗Corresponding author. Email addresses: milad.mozafari@ut.ac.ir (MM), mgtabesh@ut.ac.ir (MG) nowzari@ut.ac.ir (AND) simon.thorpe@cnrs.fr (SJT) timothee.masquelier@cnrs.fr (TM). extracts any feature that repeats. Finally, our approach is biologically plausible, hardware friendly, and energy-efficient.

[1]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[2]  Tobi Delbrück,et al.  Training Deep Spiking Neural Networks Using Backpropagation , 2016, Front. Neurosci..

[3]  Wolfram Schultz,et al.  Reward , 2019, HR for Creative Companies.

[4]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[5]  J Gautrais,et al.  Rate coding versus temporal order coding: a theoretical approach. , 1998, Bio Systems.

[6]  Matthew Cook,et al.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity , 2015, Front. Comput. Neurosci..

[7]  H. Seo,et al.  Neural basis of reinforcement learning and decision making. , 2012, Annual review of neuroscience.

[8]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[9]  Deepak Khosla,et al.  Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition , 2014, International Journal of Computer Vision.

[10]  Lei Deng,et al.  Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks , 2017, Front. Neurosci..

[11]  K. Doya,et al.  The computational neurobiology of learning and reward , 2006, Current Opinion in Neurobiology.

[12]  Tao Liu,et al.  MT-spike: A multilayer time-based spiking neuromorphic architecture with temporal error backpropagation , 2017, 2017 IEEE/ACM International Conference on Computer-Aided Design (ICCAD).

[13]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[14]  Somnath Paul,et al.  Event-Driven Random Back-Propagation: Enabling Neuromorphic Deep Learning Machines , 2016, Front. Neurosci..

[15]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[16]  Y. Niv.  Reinforcement learning in the brain , 2009 .

[17]  Bipin Rajendran,et al.  Spiking neural networks for handwritten digit recognition - Supervised learning and network optimization , 2018, Neural Networks.

[18]  Zenghui Wang,et al.  Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review , 2017, Neural Computation.

[19]  Hong Wang,et al.  Loihi: A Neuromorphic Manycore Processor with On-Chip Learning , 2018, IEEE Micro.

[20]  Steve Furber,et al.  Large-scale neuromorphic computing systems , 2016, Journal of neural engineering.

[21]  Matthieu Gilson,et al.  STDP Allows Fast Rate-Modulated Coding with Poisson-Like Spike Trains , 2011, PLoS Comput. Biol..

[22]  Wulfram Gerstner,et al.  A neuronal learning rule for sub-millisecond temporal coding , 1996, Nature.

[23]  D. Querlioz,et al.  Immunity to Device Variations in a Spiking Neural Network With Memristive Nanodevices , 2013, IEEE Transactions on Nanotechnology.

[24]  W. Schultz.  Neuronal Reward and Decision Signals: From Theories to Data. , 2015, Physiological reviews.

[25]  S. Thorpe,et al.  Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike Trains , 2008, PloS one.

[26]  Dharmendra S. Modha,et al.  A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm , 2011, 2011 IEEE Custom Integrated Circuits Conference (CICC).

[27]  Shaista Hussain,et al.  Improved margin multi-class classification using dendritic neurons with morphological learning , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[28]  S. Thorpe,et al.  STDP-based spiking deep convolutional neural networks for object recognition , 2018 .

[30]  Timothée Masquelier,et al.  Optimal Localist and Distributed Coding of Spatiotemporal Spike Patterns Through STDP and Coincidence Detection , 2018, Front. Comput. Neurosci..

[31]  Timothée Masquelier,et al.  Acquisition of visual features through probabilistic spike-timing-dependent plasticity , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[32]  Lingfeng Wang,et al.  Deep unsupervised learning with consistent inference of latent representations , 2017, Pattern Recognit..

[33]  Luiz Eduardo Soares de Oliveira,et al.  Handwritten digit segmentation: Is it still necessary? , 2018, Pattern Recognit..

[34]  Bernabé Linares-Barranco,et al.  Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[35]  G. Bi,et al.  Synaptic Modifications in Cultured Hippocampal Neurons: Dependence on Spike Timing, Synaptic Strength, and Postsynaptic Cell Type , 1998, The Journal of Neuroscience.

[36]  Romain Brette,et al.  Computing with Neural Synchrony , 2012, PLoS Comput. Biol..

[37]  J. Mink,et al.  Ratio of central nervous system to body metabolism in vertebrates: its constancy and functional basis. , 1981, The American journal of physiology.

[38]  Walter Senn,et al.  Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics , 2007, Neural Computation.

[39]  Timothée Masquelier,et al.  STDP Allows Close-to-Optimal Spatiotemporal Spike Pattern Detection by Single Coincidence Detector Neurons , 2016, Neuroscience.

[40]  W. Gerstner,et al.  Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules , 2016, Front. Neural Circuits.

[41]  Timothée Masquelier,et al.  First-Spike-Based Visual Categorization Using Reward-Modulated STDP , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[42]  Tobi Delbruck,et al.  Real-time classification and sensor fusion with a spiking deep belief network , 2013, Front. Neurosci..

[43]  Josiah R. Boivin,et al.  A Causal Link Between Prediction Errors, Dopamine Neurons and Learning , 2013, Nature Neuroscience.

[44]  W. Schultz,et al.  Sequential neuromodulation of Hebbian plasticity offers mechanism for effective reward-based navigation , 2017, eLife.

[45]  Timothée Masquelier,et al.  Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity , 2007, PLoS Comput. Biol..

[46]  Gert Cauwenberghs,et al.  Event-driven contrastive divergence for spiking neuromorphic systems , 2013, Front. Neurosci..

[47]  Haizhou Li,et al.  Rapid Feedforward Computation by Temporal Encoding and Learning With Spiking Neurons , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[48]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[49]  Shih-Chii Liu,et al.  Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification , 2017, Front. Neurosci..

[50]  Bernabé Linares-Barranco,et al.  Hardware implementation of convolutional STDP for on-line visual feature learning , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[51]  Nikil D. Dutt,et al.  Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule , 2013, Neural Networks.

[52]  Andrzej J. Kasinski,et al.  Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting , 2010, Neural Computation.