Integrated feature and parameter optimization for an evolving spiking neural network: Exploring heterogeneous probabilistic models

This study introduces a quantum-inspired spiking neural network (QiSNN) as an integrated connectionist system, in which the features and parameters of an evolving spiking neural network are optimized together with the use of a quantum-inspired evolutionary algorithm. We propose here a novel optimization method that uses different representations to explore the two search spaces: A binary representation for optimizing feature subsets and a continuous representation for evolving appropriate real-valued configurations of the spiking network. The properties and characteristics of the improved framework are studied on two different synthetic benchmark datasets. Results are compared to traditional methods, namely a multi-layer-perceptron and a naïve Bayesian classifier (NBC). A previously used real world ecological dataset on invasive species establishment prediction is revisited and new results are obtained and analyzed by an ecological expert. The proposed method results in a much faster convergence to an optimal solution (or a close to it), in a better accuracy, and in a more informative set of features selected.

[1]  Michael J. Watts,et al.  Using MLP to Determine Abiotic Factors In uencing the Establishment of Insect Pest Species , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[2]  Lars Kindermann,et al.  Finding iterative roots with a spiking neural network , 2005, Inf. Process. Lett..

[3]  Simei Gomes Wysoski,et al.  Evolving spiking neural networks for adaptive audiovisual pattern recognition , 2008 .

[4]  Michael Defoin-Platel,et al.  A versatile quantum-inspired evolutionary algorithm , 2007, 2007 IEEE Congress on Evolutionary Computation.

[5]  Jong-Hwan Kim,et al.  Quantum-inspired evolutionary algorithm for a class of combinatorial optimization , 2002, IEEE Trans. Evol. Comput..

[6]  Nikola Kasabov,et al.  Computational Neurogenetic Modeling , 2007 .

[7]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.

[8]  D.T. Pham,et al.  Evolutionary feature selection for artificial neural network pattern classifiers , 2005, 2009 7th IEEE International Conference on Industrial Informatics.

[9]  David Naso,et al.  Real-Valued Compact Genetic Algorithms for Embedded Microcontroller Optimization , 2008, IEEE Transactions on Evolutionary Computation.

[10]  R. Sutherst,et al.  Potential Geographical Distribution of the Mediterranean Fruit Fly, Ceratitis capitata (Diptera: Tephritidae), with Emphasis on Argentina and Australia , 2002 .

[11]  John Geweke,et al.  Efficient Simulation from the Multivariate Normal and Student-t Distributions Subject to Linear Constraints and the Evaluation of Constraint Probabilities , 1991 .

[12]  N. Kasabov,et al.  Evolving Intelligence in Humans and Machines: Integrative Evolving Connectionist Systems Approach , 2008, IEEE Computational Intelligence Magazine.

[13]  Michael Defoin-Platel,et al.  Integrated Feature and Parameter Optimization for an Evolving Spiking Neural Network , 2008, ICONIP.

[14]  Dirk Thierens,et al.  Expanding from Discrete to Continuous Estimation of Distribution Algorithms: The IDEA , 2000, PPSN.

[15]  Nikola Kasabov,et al.  Evolving Connectionist Systems: The Knowledge Engineering Approach , 2007 .

[16]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[17]  Marc Schoenauer,et al.  Selected Papers from the Third European Conference on Artificial Evolution , 1995 .

[18]  Marcus Gallagher,et al.  Real-valued Evolutionary Optimization using a Flexible Probability Density Estimator , 1999, GECCO.

[19]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.

[20]  Louise Travé-Massuyès,et al.  Telephone Network Traffic Overloading Diagnosis and Evolutionary Computation Techniques , 1997, Artificial Evolution.

[21]  Simei Gomes Wysoski,et al.  On-Line Learning with Structural Adaptation in a Network of Spiking Neurons for Visual Pattern Recognition , 2006, ICANN.

[22]  Wolfgang Maass,et al.  Computing with spiking neurons , 1999 .

[23]  Marcus Gallagher,et al.  Playing in continuous spaces: some analysis and extension of population-based incremental learning , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[24]  Simei Gomes Wysoski,et al.  Fast and adaptive network of spiking neurons for multi-view visual pattern recognition , 2008, Neurocomputing.

[25]  Simei Gomes Wysoski,et al.  Brain-Like Evolving Spiking Neural Networks for Multimodal Information Processing , 2010, Brain-Inspired Information Technology.

[26]  J. A. Lozano,et al.  Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms (Studies in Fuzziness and Soft Computing) , 2006 .

[27]  Jacek M. Zurada,et al.  Normalized Mutual Information Feature Selection , 2009, IEEE Transactions on Neural Networks.

[28]  Nikola K. Kasabov,et al.  Integrative connectionist learning systems inspired by nature: current models, future trends and challenges , 2009, Natural Computing.

[29]  R VanRullen,et al.  Is it a Bird? Is it a Plane? Ultra-Rapid Visual Categorisation of Natural and Artifactual Objects , 2001, Perception.

[30]  Michèle Sebag,et al.  Extending Population-Based Incremental Learning to Continuous Search Spaces , 1998, PPSN.

[31]  Wulfram Gerstner,et al.  SPIKING NEURON MODELS Single Neurons , Populations , Plasticity , 2002 .

[32]  Michael Defoin-Platel,et al.  Quantum-inspired feature and parameter optimisation of evolving spiking neural networks with a case study from ecological modeling , 2009, 2009 International Joint Conference on Neural Networks.

[33]  Marcus Gallagher,et al.  Population-Based Continuous Optimization, Probabilistic Modelling and Mean Shift , 2005, Evolutionary Computation.

[34]  K. Lang,et al.  Learning to tell two spirals apart , 1988 .

[35]  Andreas Knoblauch,et al.  Neural associative memory for brain modeling and information retrieval , 2005, Inf. Process. Lett..

[36]  Michal Valko,et al.  Evolutionary Feature Selection for Spiking Neural Network Pattern Classifiers , 2005, 2005 portuguese conference on artificial intelligence.

[37]  Nikola Kasabov,et al.  Quantum-Inspired Evolutionary Algorithm: , 2009 .

[38]  Benjamin Schrauwen,et al.  Isolated word recognition using a Liquid State Machine , 2005, ESANN.

[39]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[40]  Simei Gomes Wysoski,et al.  Evolving spiking neural networks for taste recognition , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[41]  Simon J. Thorpe,et al.  How can the human visual system process a natural scene in under 150ms? Experiments and neural network models , 1997, ESANN.