A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection

A new Multi-Spiking Neural Network (MuSpiNN) model is presented in which information from one neuron is transmitted to the next in the form of multiple spikes via multiple synapses. A new supervised learning algorithm, dubbed Multi-SpikeProp, is developed for training MuSpiNN. The model and learning algorithm employ the heuristic rules and optimum parameter values presented by the authors in a recent paper that improved the efficiency of the original single-spiking Spiking Neural Network (SNN) model by two orders of magnitude. The classification accuracies of MuSpiNN and Multi-SpikeProp are evaluated using three increasingly more complicated problems: the XOR problem, the Fisher iris classification problem, and the epilepsy and seizure detection (EEG classification) problem. It is observed that MuSpiNN learns the XOR problem in twice the number of epochs compared with the single-spiking SNN model but requires only one-fourth the number of synapses. For the iris and EEG classification problems, a modular architecture is employed to reduce each 3-class classification problem to three 2-class classification problems and improve the classification accuracy. For the complicated EEG classification problem a classification accuracy in the range of 90.7%-94.8% was achieved, which is significantly higher than the 82% classification accuracy obtained using the single-spiking SNN with SpikeProp.

[1]  Hans-Georg Zimmermann,et al.  Recurrent Neural Networks Are Universal Approximators , 2006, ICANN.

[2]  J. Feldman,et al.  Connectionist models and their implications: readings from cognitive science , 1988 .

[3]  Bard Ermentrout,et al.  Type I Membranes, Phase Resetting Curves, and Synchrony , 1996, Neural Computation.

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  Hieu Tat Nguyen,et al.  A gradient descent rule for spiking neurons emitting multiple spikes , 2005, Inf. Process. Lett..

[6]  Hans-Georg Zimmermann,et al.  Recurrent Neural Networks are Universal approximators , 2007, Int. J. Neural Syst..

[7]  Hojjat Adeli,et al.  An adaptive conjugate gradient learning algorithm for efficient training of neural networks , 1994 .

[8]  N. K. Bose,et al.  Neural Network Fundamentals with Graphs, Algorithms and Applications , 1995 .

[9]  E. Izhikevich,et al.  Weakly connected neural networks , 1997 .

[10]  Robert Hecht-Nielsen,et al.  Applications of counterpropagation networks , 1988, Neural Networks.

[11]  Shengyuan Xu,et al.  Delay-Dependent Robust Exponential Stability for Uncertain Recurrent Neural Networks with Time-Varying Delays , 2007, Int. J. Neural Syst..

[12]  Hojjat Adeli,et al.  Object-oriented backpropagation and its application to structural design , 1994, Neurocomputing.

[13]  H. Sullivan.  Ionic Channels of Excitable Membranes, 2nd Ed. , 1992, Neurology.

[14]  G. Ermentrout,et al.  Parabolic bursting in an excitable system coupled with a slow oscillation , 1986 .

[15]  Hojjat Adeli,et al.  Dynamic fuzzy wavelet neuroemulator for non‐linear control of irregular building structures , 2008 .

[16]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[17]  D. G. Stork,et al.  Is backpropagation biologically plausible? , 1989, International 1989 Joint Conference on Neural Networks.

[18]  G. Ermentrout,et al.  Analysis of neural excitability and oscillations , 1989 .

[19]  T Natschläger,et al.  Spatial and temporal pattern analysis via spiking neurons. , 1998, Network.

[20]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[21]  L. Abbott,et al.  Model neurons: From Hodgkin-Huxley to hopfield , 1990 .

[22]  Hojjat Adeli,et al.  Improved spiking neural networks for EEG classification and epilepsy and seizure detection , 2007, Integr. Comput. Aided Eng..

[23]  Terrence J. Sejnowski,et al.  Open questions about computation in cerebral cortex , 1986 .

[24]  René V. Mayorga,et al.  A Radial Basis Function Network Approach for the Computation of Inverse Continuous Time Variant Functions , 2007, Int. J. Neural Syst..

[25]  Hojjat Adeli,et al.  Wavelet-Chaos Methodology for Analysis of EEGs and EEG Sub-Bands , 2010 .

[26]  Hojjat Adeli,et al.  Neural network model for rapid forecasting of freeway link travel time , 2003 .

[27]  Stephen Grossberg,et al.  Studies of mind and brain , 1982 .

[28]  Hyo Seon Park,et al.  COUNTERPROPAGATION NEURAL NETWORKS IN STRUCTURAL ENGINEERING , 1995 .

[29]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[30]  Hojjat Adeli,et al.  RADIAL BASIS FUNCTION NEURAL NETWORK FOR WORK ZONE CAPACITY AND QUEUE ESTIMATION , 2003 .

[31]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[32]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[33]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[34]  S. Grossberg,et al.  Spikes, synchrony, and attentive learning by laminar thalamocortical circuits , 2006, Brain Research.

[35]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[36]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[37]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[38]  Hojjat Adeli,et al.  NEURO-FUZZY LOGIC MODEL FOR FREEWAY WORK ZONE CAPACITY ESTIMATION , 2003 .

[39]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[40]  Witold Pedrycz,et al.  Experience-Consistent Modeling for Radial Basis Function Neural Networks , 2008, Int. J. Neural Syst..

[41]  S. Schultz.  Principles of Neural Science, 4th ed. , 2001 .

[42]  Wolfgang Maass,et al.  Lower Bounds for the Computational Power of Networks of Spiking Neurons , 1996, Neural Computation.

[43]  Andrzej Kasiński,et al.  Comparison of supervised learning methods for spike time coding in spiking neural networks , 2006 .

[44]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[45]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[46]  Hojjat Adeli,et al.  Parallel backpropagation learning algorithms on CRAY Y-MP8/864 supercomputer , 1993, Neurocomputing.

[47]  Wulfram Gerstner,et al.  Spiking Neuron Models , 2002 .

[48]  E. Izhikevich.  Solving the distal reward problem through linkage of STDP and dopamine signaling , 2007, BMC Neuroscience.

[49]  Hojjat Adeli,et al.  Neural Network Model for Uplift Load Capacity of Metal Roof Panels , 2001 .

[50]  W. Gerstner,et al.  Time structure of the activity in neural network models. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[51]  K Lehnertz,et al.  Indications of nonlinear deterministic and finite-dimensional structures in time series of brain electrical activity: dependence on recording region and brain state. , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[52]  Wulfram Gerstner,et al.  Reduction of the Hodgkin-Huxley Equations to a Single-Variable Threshold Model , 1997, Neural Computation.

[53]  Wolfgang Maass,et al.  Noisy Spiking Neurons with Temporal Coding have more Computational Power than Sigmoidal Neurons , 1996, NIPS.

[54]  Wolfgang Maass,et al.  Fast Sigmoidal Networks via Spiking Neurons , 1997, Neural Computation.

[55]  B. Hille.  Ionic channels of excitable membranes , 2001 .

[56]  Hojjat Adeli,et al.  Mixed-Band Wavelet-Chaos-Neural Network Methodology for Epilepsy and Epileptic Seizure Detection , 2007, IEEE Transactions on Biomedical Engineering.

[57]  Stephen Grossberg,et al.  Competitive Learning: From Interactive Activation to Adaptive Resonance , 1987, Cogn. Sci..

[58]  ASHIF PANAKKAT,et al.  Neural Network Models for Earthquake Magnitude Prediction Using Multiple seismicity Indicators , 2007, Int. J. Neural Syst..

[59]  Philipp Slusallek,et al.  Introduction to real-time ray tracing , 2005, SIGGRAPH Courses.

[60]  Hojjat Adeli,et al.  Machine Learning: Neural Networks, Genetic Algorithms, and Fuzzy Systems , 1994 .

[61]  Gerasimos G. Rigatos.  Adaptive Fuzzy Control with Output Feedback for H∞ Tracking of SISO Nonlinear Systems , 2008, Int. J. Neural Syst..

[62]  Michael I. Jordan,et al.  A more biologically plausible learning rule than backpropagation applied to a network model of cortical area 7a. , 1991, Cerebral cortex.

[63]  Kurosh Madani,et al.  Autonomous biped gait pattern based on Fuzzy-CMAC neural networks , 2007, Integr. Comput. Aided Eng..

[64]  Hojjat Adeli,et al.  TOWARD INTELLIGENT VARIABLE MESSAGE SIGNS IN FREEWAY WORK ZONES: NEURAL NETWORK MODEL , 2004 .

[65]  Hojjat Adeli,et al.  Comparison of fuzzy-wavelet radial basis function neural network freeway incident detection model with California algorithm , 2002 .

[66]  B. Schrauwen,et al.  Extending SpikeProp , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[67]  Xin Wang,et al.  A Fast Method for Implicit Surface Reconstruction Based on Radial Basis Functions Network from 3D Scattered Points , 2007, Int. J. Neural Syst..

[68]  Idan Segev,et al.  Methods in Neuronal Modeling , 1988 .

[69]  Jianguo Xin,et al.  Supervised learning with spiking neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[70]  Tao Xiong,et al.  A combined SVM and LDA approach for classification , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[71]  Eve Marder,et al.  Reduction of conductance-based neuron models , 1992, Biological Cybernetics.

[72]  Hojjat Adeli,et al.  Neural Network-Wavelet Microsimulation Model for Delay and Queue Length Estimation at Freeway Work Zones , 2006 .

[73]  Hojjat Adeli,et al.  Principal Component Analysis-Enhanced Cosine Radial Basis Function Neural Network for Robust Epilepsy and Seizure Detection , 2008, IEEE Transactions on Biomedical Engineering.

[74]  Eugene M. Izhikevich,et al.  Resonate-and-fire neurons , 2001, Neural Networks.

[75]  Linda Bushnell,et al.  Fast Modifications of the SpikeProp Algorithm , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[76]  H. Adeli,et al.  Dynamic Fuzzy Wavelet Neural Network Model for Structural System Identification , 2006 .

[77]  Hojjat Adeli,et al.  FUZZY-WAVELET RBFNN MODEL FOR FREEWAY INCIDENT DETECTION , 2000 .

[78]  Haim Sompolinsky,et al.  STATISTICAL MECHANICS OF NEURAL NETWORKS , 1988 .

[79]  Nancy Kopell,et al.  Subcellular oscillations and bursting , 1986 .

[80]  Alessandro E. P. Villa,et al.  Emergence of Preferred Firing Sequences in Large Spiking Neural Networks during Simulated Neuronal Development , 2008, Int. J. Neural Syst..