Efficient training of supervised spiking neural networks via the normalized perceptron based learning rule

Abstract The spiking neural networks (SNNs) are the third generation of artificial neural networks, which have made great achievements in the field of pattern recognition. However, the existing supervised training methods of SNNs are not efficient enough to meet the real-time requirement in most cases. To address this issue, the normalized perceptron based learning rule (NPBLR) is proposed in this paper for the supervised training of the multi-layer SNNs. Different from traditional methods, our algorithm only trains the selected misclassified time points and the target ones, employing the perceptron based neuron. Furthermore, the weight modification in our algorithm is normalized by a voltage based function, which is more efficient than the traditional time based method because the firing time is calculated by the voltage value. Superior to the traditional multi-layer algorithm ignoring the time accumulation of spikes, our algorithm defines the spiking activity of the postsynaptic neuron as the rate accumulation function of all presynaptic neurons in a specific time-frame. By these strategies, our algorithm overcomes some difficulties in the training of SNNs, e.g., the inefficient and no-fire problems. Comprehensive simulations are conducted both in single and multi-layer networks to investigate the learning performance of our algorithm, whose results demonstrate that our algorithm possesses higher learning efficiency and stronger parameter robustness than traditional algorithms.

[1]  Paul H. E. Tiesinga,et al.  A New Correlation-Based Measure of Spike Timing Reliability , 2002, Neurocomputing.

[2]  Linda Bushnell,et al.  Fast Modifications of the SpikeProp Algorithm , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[3]  Sundaram Suresh,et al.  Development of a Self-Regulating Evolving Spiking Neural Network for classification problem , 2016, Neurocomputing.

[4]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[5]  M. R. Mehta,et al.  Role of experience and oscillations in transforming a rate code into a temporal code , 2002, Nature.

[6]  Narasimhan Sundararajan,et al.  A two stage learning algorithm for a Growing-Pruning Spiking Neural Network for pattern classification problems , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[7]  Bernabé Linares-Barranco,et al.  Feedforward Categorization on AER Motion Events Using Cortex-Like Features in a Spiking Neural Network , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Zhang Yi,et al.  Efficient Training of Supervised Spiking Neural Network via Accurate Synaptic-Efficiency Adjustment Method , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Ammar Belatreche,et al.  An online supervised learning method for spiking neural networks with adaptive structure , 2014, Neurocomputing.

[10]  Andrzej J. Kasinski,et al.  Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting , 2010, Neural Computation.

[11]  Héctor M. Pérez Meana,et al.  An efficient hardware implementation of a novel unary Spiking Neural Network multiplier with variable dendritic delays , 2016, Neurocomputing.

[12]  Haim Sompolinsky,et al.  Learning Input Correlations through Nonlinear Temporally Asymmetric Hebbian Plasticity , 2003, The Journal of Neuroscience.

[13]  Qingxiang Wu,et al.  Wavelet transform and texture recognition based on spiking neural network for visual images , 2015, Neurocomputing.

[14]  Ioana Sporea,et al.  Supervised Learning in Multilayer Spiking Neural Networks , 2012, Neural Computation.

[15]  Stefan Schliebs,et al.  Span: Spike Pattern Association Neuron for Learning Spatio-Temporal Spike Patterns , 2012, Int. J. Neural Syst..

[16]  T. Martin McGinnity,et al.  A biologically inspired spiking model of visual processing for image feature detection , 2015, Neurocomputing.

[17]  Qingxiang Wu,et al.  Evolutionary design of spiking neural networks. , 2006 .

[18]  Hojjat Adeli,et al.  A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection , 2009, Neural Networks.

[19]  Sander M. Bohte,et al.  The evidence for neural information processing with precise spike-times: A survey , 2004, Natural Computing.

[20]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[21]  Mehdi Khamassi,et al.  Coherent Theta Oscillations and Reorganization of Spike Timing in the Hippocampal- Prefrontal Network upon Learning , 2010, Neuron.

[22]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[23]  Qingxiang Wu,et al.  A visual attention model based on hierarchical spiking neural networks , 2013, Neurocomputing.

[24]  Mark C. W. van Rossum,et al.  A Novel Spike Distance , 2001, Neural Computation.

[25]  Arash Ahmadi,et al.  Effect of spike-timing-dependent plasticity on neural assembly computing , 2016, Neurocomputing.

[26]  Liam McDaid,et al.  SWAT: A Spiking Neural Network Training Algorithm for Classification Problems , 2010, IEEE Transactions on Neural Networks.

[27]  Qingxiang Wu,et al.  Processing visual stimuli using hierarchical spiking neural networks , 2008, Neurocomputing.

[28]  Jonathan D. Victor,et al.  Metric-space analysis of spike trains: theory, algorithms and application , 1998, q-bio/0309031.

[29]  Xiaoqin Zeng,et al.  A New Supervised Learning Algorithm for Spiking Neurons , 2013, Neural Computation.

[30]  S. Thorpe,et al.  Spike times make sense , 2005, Trends in Neurosciences.

[31]  J. Koenderink.  Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[32]  John P. Miller,et al.  Temporal encoding in nervous systems: A rigorous definition , 1995, Journal of Computational Neuroscience.

[33]  Mancia Anguita,et al.  A Spiking Neural Simulator Integrating Event-Driven and Time-Driven Computation Schemes Using Parallel CPU-GPU Co-Processing: A Case Study , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[34]  P. Lennie.  Receptive fields , 2003, Current Biology.

[35]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[36]  Răzvan V. Florian.  The chronotron: a neuron that learns to fire temporally-precise spike patterns , 2010 .

[37]  Jing Yang,et al.  A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks , 2013, Neural Networks.

[38]  Linqiang Pan,et al.  Spiking neural P systems with request rules , 2016, Neurocomputing.

[39]  Ammar Belatreche,et al.  DL-ReSuMe: A Delay Learning-Based Remote Supervised Method for Spiking Neurons , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[40]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[41]  César Torres-Huitzil,et al.  A CPG system based on spiking neurons for hexapod robot locomotion , 2015, Neurocomputing.

[42]  Malu Zhang,et al.  Improved perception-based spiking neuron learning rule for real-time user authentication , 2015, Neurocomputing.

[43]  Timothée Masquelier,et al.  Competitive STDP-Based Spike Pattern Learning , 2009, Neural Computation.

[44]  Qingxiang Wu,et al.  Learning under weight constraints in networks of temporal encoding spiking neurons , 2006, Neurocomputing.

[45]  Narayan Srinivasa,et al.  HRLSim: A High Performance Spiking Neural Network Simulator for GPGPU Clusters , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Herman P. Snippe,et al.  Parameter Extraction from Population Codes: A Critical Assessment , 1996, Neural Computation.