Quantum-inspired particle swarm optimization for feature selection and parameter optimization in evolving spiking neural networks for classification tasks

Particle Swarm Optimization (PSO) was introduced in 1995 by Russell Eberhart and James Kennedy (Eberhart & Kennedy, 1995). PSO is a biologically-inspired technique based around the study of collective behaviour in decentralized and self-organized animal society systems. The systems are typically made up from a population of candidates (particles) interacting with one another within their environment (swarm) to solve a given problem. Because of its efficiency and simplicity, PSO has been successfully applied as an optimizer in many applications such as function optimization, artificial neural network training, fuzzy system control. However, despite recent research and development, there is an opportunity to find the most effective methods for parameter optimization and feature selection tasks. This chapter deals with the problem of feature (variable) and parameter optimization for neural network models, utilising a proposed Quantum–inspired PSO (QiPSO) method. In this method the features of the model are represented probabilistically as a quantum bit (qubit) vector and the model parameter values as real numbers. The principles of quantum superposition and quantum probability are used to accelerate the search for an optimal set of features, that combined through co-evolution with a set of optimised parameter values, will result in a more accurate computational neural network model. The method has been applied to the problem of feature and parameter optimization in Evolving Spiking Neural Network (ESNN) for classification. A swarm of particles is used to find the most accurate classification model for a given classification task. The QiPSO will be integrated within ESNN where features and parameters are simultaneously and more efficiently optimized. A hybrid particle structure is required for the qubit and real number data types. In addition, an improved search strategy has been introduced to find the most relevant and eliminate the irrelevant features on a synthetic dataset. The method is tested on a benchmark classification problem. The proposed method results in the design of faster and more accurate neural network classification models than the ones optimised through the use of standard evolutionary optimization algorithms.

[1]  Wenbo Xu,et al.  Particle swarm optimization with particles having quantum behavior , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[2]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[3]  Simei Gomes Wysoski,et al.  On-Line Learning with Structural Adaptation in a Network of Spiking Neurons for Visual Pattern Recognition , 2006, ICANN.

[4]  Jong-Hwan Kim,et al.  Genetic quantum algorithm and its application to combinatorial optimization problem , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[5]  D. Johnston,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997 .

[6]  Nikola Kasabov,et al.  Evolving connectionist systems , 2002 .

[7]  Nikola K. Kasabov,et al.  To spike or not to spike: A probabilistic spiking neuron model , 2010, Neural Networks.

[8]  H. Markram,et al.  Regulation of Synaptic Efficacy by Coincidence of Postsynaptic APs and EPSPs , 1997, Science.

[9]  J. Huguenard.  Reliability of axonal propagation: the spike doesn't stop here. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[10]  M. Pacheco,et al.  Quantum-Inspired Evolutionary Algorithm for Numerical Optimization , 2006 .

[11]  Mohamed Batouche,et al.  A Novel Quantum-Inspired Evaluation Algorithm for Multi-Source Affine Image Registration , 2006, ˜The œinternational Arab journal of information technology.

[12]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[13]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[14]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[15]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[16]  Wulfram Gerstner,et al.  Spiking Neuron Models: An Introduction , 2002 .

[17]  Peter W. Shor,et al.  Algorithms for Quantum Computation: Discrete Log and Factoring (Extended Abstract) , 1994, FOCS 1994.

[18]  Lov K. Grover.  A fast quantum mechanical algorithm for database search , 1996, STOC '96.

[19]  Jacek M. Zurada,et al.  Normalized Mutual Information Feature Selection , 2009, IEEE Transactions on Neural Networks.

[20]  Hiroshi Kojima,et al.  An Analysis of Synaptic Transmission and its Plasticity by Glutamate Receptor Channel Kinetics Models and 2-Photon Laser Photolysis , 2008, ICONIP.

[21]  Simon J. Thorpe,et al.  How can the human visual system process a natural scene in under 150ms? Experiments and neural network models , 1997, ESANN.

[22]  Jong-Hwan Kim,et al.  Face detection using quantum-inspired evolutionary algorithm , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[23]  Ajit Narayanan,et al.  Quantum computing for beginners , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[24]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[25]  Sander M. Bohte,et al.  Error-backpropagation in temporally encoded networks of spiking neurons , 2000, Neurocomputing.