A brain-inspired spiking neural network model with temporal encoding and learning

Abstract Neural coding and learning are important components in cognitive memory system, by processing the sensory inputs and distinguishing different patterns to allow for higher level brain functions such as memory storage and retrieval. Benefitting from biological relevance, this paper presents a spiking neural network of leaky integrate-and-fire (LIF) neurons for pattern recognition. A biologically plausible supervised synaptic learning rule is used so that neurons can efficiently make a decision. The whole system contains encoding, learning and readout. Utilizing the temporal coding and learning, networks of spiking neurons can effectively and efficiently perform various classification tasks. It can classify complex patterns of activities stored in a vector, as well as the real-world stimuli. Our approach is also benchmarked on the nonlinearly separable Iris dataset. The proposed approach achieves a good generalization, with a classification accuracy of 99.63% for training and 92.55% for testing. In addition, the trained networks demonstrate that the temporal coding is a viable means for fast neural information processing.

[1]  Simei Gomes Wysoski,et al.  Fast and adaptive network of spiking neurons for multi-view visual pattern recognition , 2008, Neurocomputing.

[2]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[3]  Xiaochuan Pan,et al.  The spatiotemporal learning rule and its efficiency in separating spatiotemporal patterns , 2005, Biological Cybernetics.

[4]  Hojjat Adeli,et al.  Improved spiking neural networks for EEG classification and epilepsy and seizure detection , 2007, Integr. Comput. Aided Eng..

[5]  J. J. Hopfield,et al.  Pattern recognition computation using action potential timing for stimulus representation , 1995, Nature.

[6]  Sander M. Bohte,et al.  Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks , 2002, IEEE Trans. Neural Networks.

[7]  Masao Ito Mechanisms of motor learning in the cerebellum 1 1 Published on the World Wide Web on 24 November 2000. , 2000, Brain Research.

[8]  Christof Koch,et al.  Temporal Precision of Spike Trains in Extrastriate Cortex of the Behaving Macaque Monkey , 1999, Neural Computation.

[9]  Eugene M. Izhikevich,et al.  Resonate-and-fire neurons , 2001, Neural Networks.

[10]  L. Abbott,et al.  Competitive Hebbian learning through spike-timing-dependent synaptic plasticity , 2000, Nature Neuroscience.

[11]  Jacques Gautrais,et al.  SpikeNET: A simulator for modeling large networks of integrate and fire neurons , 1999, Neurocomputing.

[12]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[13]  Kay Chen Tan,et al.  A hybrid evolutionary algorithm for attribute selection in data mining , 2009, Expert Syst. Appl..

[14]  M. Randić,et al.  Long-term potentiation and long-term depression of primary afferent neurotransmission in the rat spinal cord , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[15]  E. Knudsen Supervised learning in the brain , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  Nicolas Brunel,et al.  Sensory neural codes using multiplexed temporal scales , 2010, Trends in Neurosciences.

[17]  J. Seamans,et al.  The principal features and mechanisms of dopamine modulation in the prefrontal cortex , 2004, Progress in Neurobiology.

[18]  Mohammad Hassan Moradi,et al.  A Hybrid Higher Order Neural Classifier for handling classification problems , 2011, Expert Syst. Appl..

[19]  Wulfram Gerstner,et al.  Spike-Based Compared to Rate-Based Hebbian Learning , 1998, NIPS.

[20]  E. Chichilnisky,et al.  Precision of spike trains in primate retinal ganglion cells. , 2004, Journal of neurophysiology.

[21]  Kay Chen Tan,et al.  Learning real-world stimuli by single-spike coding and tempotron rule , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[22]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[23]  W. Singer,et al.  Relation Between Dendritic Ca2+ Levels and the Polarity of Synaptic Long‐term Modifications in Rat Visual Cortex Neurons , 1997, The European journal of neuroscience.

[24]  Kay Chen Tan,et al.  Pattern recognition computation in a spiking neural network with temporal encoding and learning , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[25]  Hojjat Adeli,et al.  Machine Learning: Neural Networks, Genetic Algorithms, and Fuzzy Systems , 1994 .

[26]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1990 .

[27]  Tim Gollisch,et al.  Rapid Neural Coding in the Retina with Relative Spike Latencies , 2008, Science.

[28]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[29]  Hojjat Adeli,et al.  Machine Learning: Neural Networks , 1994 .

[30]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[31]  Edmund T. Rolls,et al.  What determines the capacity of autoassociative memories in the brain? Network , 1991 .

[32]  Walter Senn,et al.  Learning Real-World Stimuli in a Neural Network with Spike-Driven Synaptic Dynamics , 2007, Neural Computation.

[33]  Liam McDaid,et al.  SWAT: A Spiking Neural Network Training Algorithm for Classification Problems , 2010, IEEE Transactions on Neural Networks.

[34]  Alexander Borst,et al.  Information theory and neural coding , 1999, Nature Neuroscience.

[35]  W. T. Thach Motor Learning and Synaptic Plasticity in the Cerebellum: On the specific role of the cerebellum in motor learning and cognition: Clues from PET activation and lesion studies in man , 1997 .

[36]  MaassWolfgang,et al.  Real-time computing without stable states , 2002 .

[37]  Henry Markram,et al.  Real-Time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbations , 2002, Neural Computation.

[38]  Andrzej J. Kasinski,et al.  Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting , 2010, Neural Computation.

[39]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[40]  Cameron Johnson,et al.  Encoding real values into polychronous spiking networks , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[41]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[42]  A. Brown,et al.  Neuromodulation , 1992, Current Opinion in Neurobiology.

[43]  Robert A. Legenstein,et al.  What Can a Neuron Learn with Spike-Timing-Dependent Plasticity? , 2005, Neural Computation.

[44]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[45]  Timothée Masquelier,et al.  Competitive STDP-Based Spike Pattern Learning , 2009, Neural Computation.

[46]  R. Reid,et al.  Temporal Coding of Visual Information in the Thalamus , 2000, The Journal of Neuroscience.

[47]  Haizhou Li,et al.  Memory Dynamics in Attractor Networks with Saliency Weights , 2010, Neural Computation.

[48]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[49]  Treves,et al.  Graded-response neurons and information encodings in autoassociative memories. , 1990, Physical review. A, Atomic, molecular, and optical physics.

[50]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[51]  Christian W. Eurich,et al.  Multidimensional Encoding Strategy of Spiking Neurons , 2000, Neural Computation.

[52]  R. Kempter,et al.  Hebbian learning and spiking neurons , 1999 .

[53]  Giacomo Indiveri,et al.  Real-Time Classification of Complex Patterns Using Spike-Based Learning in Neuromorphic VLSI , 2009, IEEE Transactions on Biomedical Circuits and Systems.

[54]  Megan R. Carey,et al.  Instructive signals for motor learning from visual cortical area MT , 2005, Nature Neuroscience.

[55]  J. Knott The organization of behavior: A neuropsychological theory , 1951 .

[56]  Chun-I Yeh,et al.  Temporal precision in the neural code and the timescales of natural vision , 2007, Nature.

[57]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[58]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[59]  Wolfgang Maass,et al.  Lower Bounds for the Computational Power of Networks of Spiking Neurons , 1996, Neural Computation.

[60]  R. Foehring,et al.  Neuromodulation, development and synaptic plasticity. , 1999, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[61]  Y. Dan,et al.  Spike-timing-dependent synaptic modification induced by natural spike trains , 2002, Nature.

[62]  PonulakFilip,et al.  Supervised learning in spiking neural networks with resume , 2010 .