Evolutionary features and parameter optimization of spiking neural networks for unsupervised learning

This paper introduces two new hybrid models for clustering problems in which the input features and parameters of a spiking neural network (SNN) are optimized using evolutionary algorithms. We used two novel evolutionary approaches, the quantum-inspired evolutionary algorithm (QIEA) and the optimization by genetic programming (OGP) methods, to develop the quantum binary-real evolving SNN (QbrSNN) and the SNN optimized by genetic programming (SNN-OGP) neuro-evolutionary models, respectively. The proposed models are applied to 8 benchmark datasets, and a significantly higher clustering accuracy compared to a standard SNN without feature and parameter optimization is achieved with fewer iterations. When comparing QbrSNN and SNN-OGP, the former performed slightly better but at the expense of increased computational effort.

[1]  Wolfgang Maass,et al.  Networks of Spiking Neurons: The Third Generation of Neural Network Models , 1996, Electron. Colloquium Comput. Complex..

[2]  Michael Defoin-Platel,et al.  Integrated Feature and Parameter Optimization for an Evolving Spiking Neural Network , 2008, ICONIP.

[3]  Michael Defoin-Platel,et al.  Integrated feature and parameter optimization for an evolving spiking neural network: Exploring heterogeneous probabilistic models , 2009, Neural Networks.

[4]  Benjamin Schrauwen,et al.  Isolated word recognition using a Liquid State Machine , 2005, ESANN.

[5]  Andreas Knoblauch,et al.  Neural associative memory for brain modeling and information retrieval , 2005, Inf. Process. Lett..

[6]  P. Sopp.  Cluster analysis. , 1996, Veterinary immunology and immunopathology.

[7]  Christopher J. Bishop,et al.  Pulsed Neural Networks , 1998 .

[8]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[9]  Mengjie Zhang,et al.  Cooperative coevolution of Elman recurrent neural networks for chaotic time series prediction , 2012, Neurocomputing.

[10]  Ludmila I. Kuncheva,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2004 .

[11]  Marco Aurélio Cavalcanti Pacheco,et al.  Numerical optimization by multi-gene genetic programming , 2013, GECCO '13 Companion.

[12]  Michael Schmitt,et al.  Self-organization of spiking neurons using action potential timing , 1998, IEEE Trans. Neural Networks.

[13]  Jiye Liang,et al.  The $K$-Means-Type Algorithms Versus Imbalanced Data Distributions , 2012, IEEE Transactions on Fuzzy Systems.

[14]  Sander M. Bohte,et al.  SpikeProp: backpropagation for networks of spiking neurons , 2000, ESANN.

[15]  Lars Kindermann,et al.  Finding iterative roots with a spiking neural network , 2005, Inf. Process. Lett..

[16]  Nikola Kasabov,et al.  Evolving Connectionist Systems: The Knowledge Engineering Approach , 2007 .

[17]  Zhen Ma,et al.  A review of algorithms for medical image segmentation and their applications to the female pelvic cavity , 2010, Computer methods in biomechanics and biomedical engineering.

[18]  Wofgang Maas,et al.  Networks of spiking neurons: the third generation of neural network models , 1997 .

[19]  J. J. Hopfield,et al.  Pattern recognition computation using action potential timing for stimulus representation , 1995, Nature.

[20]  Sander M. Bohte,et al.  Unsupervised clustering with spiking neurons by sparse temporal coding and multilayer RBF networks , 2002, IEEE Trans. Neural Networks.

[21]  Simei Gomes Wysoski,et al.  Evolving spiking neural networks for taste recognition , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[22]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[23]  T Natschläger,et al.  Spatial and temporal pattern analysis via spiking neurons. , 1998, Network.

[24]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[25]  Stefan Schliebs,et al.  Reservoir-Based Evolving Spiking Neural Network for Spatio-temporal Pattern Recognition , 2011, ICONIP.

[26]  Bart Baesens,et al.  A new SOM-based method for profile generation: Theory and an application in direct marketing , 2012, Eur. J. Oper. Res..

[27]  Michael Defoin-Platel,et al.  A versatile quantum-inspired evolutionary algorithm , 2007, 2007 IEEE Congress on Evolutionary Computation.

[28]  W. Gerstner,et al.  Time structure of the activity in neural network models. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[29]  Simei Gomes Wysoski,et al.  Evolving spiking neural networks for audiovisual information processing , 2010, Neural Networks.

[30]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[31]  Michael Defoin-Platel,et al.  Quantum-inspired feature and parameter optimisation of evolving spiking neural networks with a case study from ecological modeling , 2009, 2009 International Joint Conference on Neural Networks.

[32]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[33]  Nikola K. Kasabov,et al.  Integrated Feature Selection and Parameter Optimization for Evolving Spiking Neural Networks Using Quantum Inspired Particle Swarm Optimization , 2009, 2009 International Conference of Soft Computing and Pattern Recognition.

[34]  M. Pacheco,et al.  Quantum-Inspired Evolutionary Algorithm for Numerical Optimization , 2006 .

[35]  Subhash C. Bagui,et al.  Combining Pattern Classifiers: Methods and Algorithms , 2005, Technometrics.

[36]  Siti Mariyam Shamsuddin,et al.  Quantum-inspired particle swarm optimization for feature selection and parameter optimization in evolving spiking neural networks for classification tasks , 2011 .

[37]  Marley M. B. R. Vellasco,et al.  A new model for credit approval problems: A quantum-inspired neuro-evolutionary algorithm with binary-real representation , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).