Development of a Self-Regulating Evolving Spiking Neural Network for classification problem

This paper presents a new spiking neural network for pattern classification problems, referred to as the Self-Regulating Evolving Spiking Neural (SRESN) classifier, that regulates the learning process of the network. It uses a two layered spiking neural network and the input layer consists of receptive field neurons, which convert a real valued input to spikes using the population coding scheme without any delays. The output layer consists of leaky integrate-and-fire neurons. Since SRESN does not use any delays, the number of network parameters for SRESN is significantly lower than that used by other spiking neural networks, used in this study. During training, the learning algorithm for SRESN, automatically evolves neurons in the output layer based on the training data stream and the current knowledge stored in the network. Depending on the knowledge in the sample and the class specific knowledge stored in the network, it can choose to either add a neuron or update the network parameters or skip learning the sample resulting in self-regulation of the learning process. In case of neuron addition, the weights for the newly added neuron are initialized using a modified rank order scheme which facilitates SRESN for use in online/sequential as well as batch learning modes. The parameter update strategy in SRESN ensures that connections with non-zero postsynaptic potential at the time of the spike are alone updated which helps prevent over training. While evaluating the performance of SRESN, first a study is conducted to assess the impact of various parameters on its performance and establish guidelines to choose suitable values for these parameters. Next, the performance of SRESN, operating in batch mode, is compared with other spiking neural classifiers, including SpikeProp and MuSpiNN, for the UCI benchmark problems of Iris flower classification and Wisconsin breast cancer. Subsequently, the performance of SRESN in online and batch learning mode is compared with an evolving spiking neural classifier for five benchmark data sets from the UCI machine learning repository. Finally, SRESN is applied to solve the practical problem of Epilepsy detection. The performance comparison clearly indicates that SRESN provides a higher generalization accuracy using fewer network parameters.

[1]  Ammar Belatreche,et al.  An online supervised learning method for spiking neural networks with adaptive structure , 2014, Neurocomputing.

[2]  Linda Bushnell,et al.  Fast Modifications of the SpikeProp Algorithm , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[3]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[4]  Simei Gomes Wysoski,et al.  Fast and adaptive network of spiking neurons for multi-view visual pattern recognition , 2008, Neurocomputing.

[5]  Hojjat Adeli,et al.  A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection , 2009, Neural Networks.

[6]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[7]  Huajin Tang,et al.  Precise-Spike-Driven Synaptic Plasticity: Learning Hetero-Association of Spatiotemporal Spike Patterns , 2013, PloS one.

[8]  Rufin van Rullen,et al.  Rate Coding Versus Temporal Order Coding: What the Retinal Ganglion Cells Tell the Visual Cortex , 2001, Neural Computation.

[9]  K Lehnertz,et al.  Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: dependence on recording region and brain state. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[10]  Wolfgang Maass,et al.  Noisy Spiking Neurons with Temporal Coding have more Computational Power than Sigmoidal Neurons , 1996, NIPS.

[11]  Wolfgang Maass,et al.  Fast Sigmoidal Networks via Spiking Neurons , 1997, Neural Computation.

[12]  Sundaram Suresh,et al.  Meta-cognitive Neural Network for classification problems in a sequential learning framework , 2012, Neurocomputing.

[13]  Narasimhan Sundararajan,et al.  A sequential multi-category classifier using radial basis function networks , 2008, Neurocomputing.

[14]  Sundaram Suresh,et al.  A basis coupled evolving spiking neural network with afferent input neurons , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[15]  PonulakFilip,et al.  Supervised learning in spiking neural networks with resume , 2010 .

[16]  Chun-I Yeh,et al.  Temporal precision in the neural code and the timescales of natural vision , 2007, Nature.

[17]  S. Thorpe,et al.  Spike times make sense , 2005, Trends in Neurosciences.

[18]  H. Sompolinsky,et al.  The tempotron: a neuron that learns spike timing–based decisions , 2006, Nature Neuroscience.

[19]  Jacques Gautrais,et al.  Rank order coding , 1998 .

[20]  Hojjat Adeli,et al.  Improved spiking neural networks for EEG classification and epilepsy and seizure detection , 2007, Integr. Comput. Aided Eng..

[21]  Nikola Kasabov,et al.  Dynamic evolving spiking neural networks for on-line spatio- and spectro-temporal pattern recognition. , 2013, Neural networks : the official journal of the International Neural Network Society.

[22]  H. J. Kim,et al.  A sequential learning algorithm for self-adaptive resource allocation network classifier , 2010, Neurocomputing.

[23]  Liam McDaid,et al.  SWAT: A Spiking Neural Network Training Algorithm for Classification Problems , 2010, IEEE Transactions on Neural Networks.

[24]  Jianguo Xin,et al.  Supervised learning with spiking neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[25]  Andrzej J. Kasinski,et al.  Supervised Learning in Spiking Neural Networks with ReSuMe: Sequence Learning, Classification, and Spike Shifting , 2010, Neural Computation.

[26]  J. Victor,et al.  Nature and precision of temporal coding in visual cortex: a metric-space analysis. , 1996, Journal of neurophysiology.

[27]  Mark C. W. van Rossum,et al.  A Novel Spike Distance , 2001, Neural Computation.

[28]  Nikola Kasabov,et al.  Evolving Connectionist Systems: The Knowledge Engineering Approach , 2007 .

[29]  Hojjat Adeli,et al.  A Wavelet-Chaos Methodology for Analysis of EEGs and EEG Subbands to Detect Seizure and Epilepsy , 2007, IEEE Transactions on Biomedical Engineering.

[30]  Razvan V. Florian,et al.  The Chronotron: A Neuron That Learns to Fire Temporally Precise Spike Patterns , 2010, PloS one.

[31]  Sundaram Suresh,et al.  A Metacognitive Complex-Valued Interval Type-2 Fuzzy Inference System , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Narasimhan Sundararajan,et al.  A Metacognitive Neuro-Fuzzy Inference System (McFIS) for Sequential Classification Problems , 2013, IEEE Transactions on Fuzzy Systems.

[33]  Hojjat Adeli,et al.  Mixed-Band Wavelet-Chaos-Neural Network Methodology for Epilepsy and Epileptic Seizure Detection , 2007, IEEE Transactions on Biomedical Engineering.

[34]  Qing Song,et al.  Adaptive learning rate of SpikeProp based on weight convergence analysis , 2015, Neural Networks.

[35]  Sundaram Suresh,et al.  A meta-cognitive learning algorithm for a Fully Complex-valued Relaxation Network , 2012, Neural Networks.

[36]  Kay Chen Tan,et al.  A brain-inspired spiking neural network model with temporal encoding and learning , 2014, Neurocomputing.

[37]  Nikola Kasabov,et al.  Evolving connectionist systems , 2002 .

[38]  Sundaram Suresh,et al.  A sequential learning algorithm for a Minimal Spiking Neural Network (MSNN) classifier , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[39]  L. F Abbott,et al.  Lapicque’s introduction of the integrate-and-fire model neuron (1907) , 1999, Brain Research Bulletin.

[40]  Jing Yang,et al.  A supervised multi-spike learning algorithm based on gradient descent for spiking neural networks , 2013, Neural Networks.