A Minimal Spiking Neural Network to Rapidly Train and Classify Handwritten Digits in Binary and 10-Digit Tasks

This paper reports the results of experiments to develop a minimal neural network for pattern classification. The network uses biologically plausible neural and learning mechanisms and is applied to a subset of the MNIST dataset of handwritten digits. The research goal is to assess the classification power of a very simple biologically motivated mechanism. The network architecture is primarily a feedforward spiking neural network (SNN) composed of Izhikevich regular spiking (RS) neurons and conductance-based synapses. The weights are trained with the spike timing-dependent plasticity (STDP) learning rule. The proposed SNN architecture contains three neuron layers which are connected by both static and adaptive synapses. Visual input signals are processed by the first layer to generate input spike trains. The second and third layers contribute to spike train segmentation and STDP learning, respectively. The network is evaluated by classification accuracy on the handwritten digit images from the MNIST dataset. The simulation results show that although the proposed SNN is trained quickly without error-feedbacks in a few number of iterations, it results in desirable performance (97.6%) in the binary classification (0 and 1). In addition, the proposed SNN gives acceptable recognition accuracy in 10-digit (0-9) classification in comparison with statistical methods such as support vector machine (SVM) and multi-perceptron neural network.

[1]  Roberto Antonio Vázquez Pattern Recognition Using Spiking Neurons and Firing Rates , 2010, IBERAMIA.

[2]  Stefan Fritsch,et al.  neuralnet: Training of Neural Networks , 2010, R J..

[3]  Theodore W. Berger,et al.  Synaptic dynamics: Linear model and adaptation algorithm , 2014, Neural Networks.

[4]  Liam McDaid,et al.  SWAT: A Spiking Neural Network Training Algorithm for Classification Problems , 2010, IEEE Transactions on Neural Networks.

[5]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[6]  Simei Gomes Wysoski,et al.  Fast and adaptive network of spiking neurons for multi-view visual pattern recognition , 2008, Neurocomputing.

[7]  Robert F. Stengel,et al.  An introduction to neural networks , 2018 .

[8]  Guillermo Ricardo Simari,et al.  Advances in Artificial Intelligence – IBERAMIA 2010 , 2010, Lecture Notes in Computer Science.

[9]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[10]  David Kappel,et al.  STDP Installs in Winner-Take-All Circuits an Online Approximation to Hidden Markov Model Learning , 2014, PLoS Comput. Biol..

[11]  Timothée Masquelier,et al.  Unsupervised Learning of Visual Features through Spike Timing Dependent Plasticity , 2007, PLoS Comput. Biol..

[12]  Wolfgang Maass,et al.  On the Computational Power of Winner-Take-All , 2000, Neural Computation.

[13]  Simei Gomes Wysoski,et al.  Evolving spiking neural networks for audiovisual information processing , 2010, Neural Networks.

[14]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[15]  Grzegorz Rozenberg,et al.  Handbook of Natural Computing , 2011, Springer Berlin Heidelberg.

[16]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[17]  Y. Dan,et al.  Spike timing-dependent plasticity: a Hebbian learning rule. , 2008, Annual review of neuroscience.

[18]  Ammar Belatreche,et al.  An online supervised learning method for spiking neural networks with adaptive structure , 2014, Neurocomputing.

[19]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[20]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[21]  Yaochu Jin,et al.  Computational modeling of neural plasticity for self-organization of neural networks , 2014, Biosyst..

[22]  Wolfgang Maass,et al.  Bayesian Computation Emerges in Generic Cortical Microcircuits through Spike-Timing-Dependent Plasticity , 2013, PLoS Comput. Biol..

[23]  Nikil D. Dutt,et al.  Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule , 2013, Neural Networks.

[24]  Kurt Hornik,et al.  Support Vector Machines in R , 2006 .

[25]  Nikola Kasabov,et al.  Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. , 2013, Neural networks : the official journal of the International Neural Network Society.

[26]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[27]  Wolfgang Maass,et al.  STDP enables spiking neurons to detect hidden causes of their inputs , 2009, NIPS.

[28]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .