SPAN : spike pattern association neuron for learning spatio-temporal sequences

Spiking Neural Networks (SNN) were shown to be suitable tools for the processing of spatiotemporal information. However, due to their inherent complexity, the formulation of efficient supervised learning algorithms for SNN is difficult and remains an important problem in the research area. This article presents SPAN — a spiking neuron that is able to learn associations of arbitrary spike trains in a supervised fashion allowing the processing of spatio-temporal information encoded in the precise timing of spikes. The idea of the proposed algorithm is to transform spike trains during the learning phase into analog signals so that common mathematical operations can be performed on them. Using this conversion, it is possible to apply the well-known Widrow–Hoff rule directly to the transformed spike trains in order to adjust the synaptic weights and to achieve a desired input/output spike behavior of the neuron. In the presented experimental analysis, the proposed learning algorithm is evaluated regarding its learning capabilities, its memory capacity, its robustness to noisy stimuli and its classification performance. Differences and similarities of SPAN regarding two related algorithms, ReSuMe and Chronotron, are discussed. DOI: https://doi.org/10.1142/S0129065712500128 Posted at the Zurich Open Repository and Archive, University of Zurich ZORA URL: https://doi.org/10.5167/uzh-75343 Accepted Version Originally published at: Mohemmed, A; Schliebs, S; Matsuda, S; Kasabov, N (2012). SPAN: spike pattern association neuron for learning spatio-temporal sequences. International journal of neural systems, 22(4):1250012. DOI: https://doi.org/10.1142/S0129065712500128 SPAN: Spike Pattern Association Neuron for Learning Spatio-Temporal Spike Patterns AMMAR MOHEMMED and STEFAN SCHLIEBS Knowledge Engineering and Discovery Research Institute Auckland University of Technology, New Zealand E-mail: amohemme@aut.ac.nz, sschlieb@aut.ac.nz

[1]  Sander M. Bohte,et al.  The evidence for neural information processing with precise spike-times: A survey , 2004, Natural Computing.

[2]  László Bakó,et al.  Real-time classification of datasets with hardware embedded neuromorphic neural networks , 2010, Briefings Bioinform..

[3]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[4]  Hojjat Adeli,et al.  A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection , 2009, Neural Networks.

[5]  Alexander K. Vidybida,et al.  Testing of information condensation in a model reverberating spiking neural network , 2010, Int. J. Neural Syst..

[6]  Wolfgang Maass,et al.  Computing with spiking neurons , 1999 .

[7]  J. J. Hopfield,et al.  Pattern recognition computation using action potential timing for stimulus representation , 1995, Nature.

[8]  Stefan Schliebs,et al.  Optimization of Spiking Neural Networks with dynamic synapses for spike sequence generation using PSO , 2011, The 2011 International Joint Conference on Neural Networks.

[9]  Giacomo Indiveri,et al.  A VLSI network of spiking neurons with an asynchronous static random access memory , 2011, 2011 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[10]  Chun-I Yeh,et al.  Temporal precision in the neural code and the timescales of natural vision , 2007, Nature.

[11]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[12]  T. Martin McGinnity,et al.  An FPGA Hardware/Software Co-Design towards Evolvable Spiking Neural Networks for Robotics Application , 2010, Int. J. Neural Syst..

[13]  F. Ponulak.  ReSuMe-New Supervised Learning Method for Spiking Neural Networks , 2005 .

[14]  Hojjat Adeli,et al.  Spiking Neural Networks , 2009, Int. J. Neural Syst..

[15]  T. Sejnowski,et al.  Regulation of spike timing in visual cortical circuits , 2008, Nature Reviews Neuroscience.

[16]  Wolfgang Maass,et al.  Dynamic Stochastic Synapses as Computational Units , 1997, Neural Computation.

[17]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[18]  Alessandro E. P. Villa,et al.  Emergence of Preferred Firing Sequences in Large Spiking Neural Networks during Simulated Neuronal Development , 2008, Int. J. Neural Syst..

[19]  Răzvan V. Florian.  The chronotron: a neuron that learns to fire temporally-precise spike patterns , 2010 .

[20]  Bernabé Linares-Barranco,et al.  Spike-Based Convolutional Network for Real-Time Processing , 2010, 2010 20th International Conference on Pattern Recognition.

[21]  Henry Markram,et al.  Neural Networks with Dynamic Synapses , 1998, Neural Computation.

[22]  Giacomo Indiveri,et al.  Online spatio-temporal pattern recognition with evolving spiking neural networks utilising address event representation, rank order, and temporal spike learning , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[23]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[24]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[25]  Mark C. W. van Rossum,et al.  A Novel Spike Distance , 2001, Neural Computation.

[26]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[27]  N. Kasabov.  Evolving , Probabilistic Spiking Neural Networks and Neurogenetic Systems for Spatio-and Spectro-Temporal Data Modelling and Pattern Recognition , 2012 .

[28]  Eric Nichols,et al.  Case Study on a Self-Organizing Spiking Neural Network for Robot Navigation , 2010, Int. J. Neural Syst..

[29]  Snjezana Soltic,et al.  Knowledge Extraction from Evolving Spiking Neural Networks with Rank Order Population Coding , 2010, Int. J. Neural Syst..

[30]  Benjamin Schrauwen,et al.  Improving SpikeProp: Enhancements to An Error-Backpropagation Rule for Spiking Neural Networks , 2004 .

[31]  John G. Harris,et al.  Towards Spike-Based Speech Processing: A Biologically Plausible Approach to Simple Acoustic Classification , 2008, Int. J. Appl. Math. Comput. Sci..

[32]  Silvia Tolu,et al.  Adaptive cerebellar Spiking Model Embedded in the Control Loop: Context Switching and Robustness against noise , 2011, Int. J. Neural Syst..

[33]  Sander M. Bohte,et al.  SpikeProp: backpropagation for networks of spiking neurons , 2000, ESANN.

[34]  S. Thorpe,et al.  Spike times make sense , 2005, Trends in Neurosciences.

[35]  Filip Ponulak,et al.  Analysis of the ReSuMe Learning Process For Spiking Neural Networks , 2008, Int. J. Appl. Math. Comput. Sci..

[36]  Nikola K. Kasabov,et al.  Incremental learning algorithm for spatio-temporal spike pattern classification , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[37]  T. Martin McGinnity,et al.  An STDP Training Algorithm for a Spiking Neural Network with Dynamic Threshold Neurons , 2010, Int. J. Neural Syst..

[38]  F. Ponulak,et al.  ReSuMe learning method for Spiking Neural Networks dedicated to neuroprostheses control , 2006 .

[39]  Rufin van Rullen,et al.  SpikeNet: real-time visual processing with one spike per neuron , 2004, Neurocomputing.

[40]  Marc-Oliver Gewaltig,et al.  NEST (NEural Simulation Tool) , 2007, Scholarpedia.

[41]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[42]  Simei Gomes Wysoski,et al.  Evolving spiking neural networks for audiovisual information processing , 2010, Neural Networks.

[43]  Jonathan D. Victor,et al.  Metric-space analysis of spike trains: theory, algorithms and application , 1998, q-bio/0309031.

[44]  Stefan Schliebs,et al.  Method for Training a Spiking Neuron to Associate Input-Output Spike Trains , 2011, EANN/AIAI.

[45]  Hojjat Adeli,et al.  Improved spiking neural networks for EEG classification and epilepsy and seizure detection , 2007, Integr. Comput. Aided Eng..

[46]  Robert A. Legenstein,et al.  What Can a Neuron Learn with Spike-Timing-Dependent Plasticity? , 2005, Neural Computation.

[47]  Anila Jahangiri,et al.  Phase resetting Analysis of High potassium epileptiform Activity in CA3 Region of the rat Hippocampus , 2011, Int. J. Neural Syst..

[48]  Jianguo Xin,et al.  Supervised learning with spiking neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[49]  Long Cheng,et al.  A behavior controller based on spiking neural networks for mobile robots , 2008, Neurocomputing.

[50]  Nikola K. Kasabov,et al.  To spike or not to spike: A probabilistic spiking neuron model , 2010, Neural Networks.

[51]  Razvan V. Florian,et al.  Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity , 2007, Neural Computation.

[52]  Sophie Denève,et al.  Bayesian Spiking Neurons I: Inference , 2008, Neural Computation.

[53]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[54]  Benjamin Schrauwen,et al.  Linking non-binned spike train kernels to several existing spike train metrics , 2006, ESANN.

[55]  Andrzej J. Kasinski,et al.  Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting , 2010, Neural Computation.

[56]  M. O’Halloran,et al.  Spiking Neural Networks for Breast Cancer Classification in a Dielectrically Heterogeneous Breast , 2011 .

[57]  U. Rajendra Acharya,et al.  Analysis and Automatic Identification of Sleep Stages Using Higher Order Spectra , 2010, Int. J. Neural Syst..

[58]  Marc-Oliver Gewaltig,et al.  Towards Reproducible Descriptions of Neuronal Network Models , 2009, PLoS Comput. Biol..