Training algorithm for radial basis function neural network based on quantum-behaved particle swarm optimization

Radial basis function (RBF) networks are widely applied in function approximation, system identification, chaotic time series forecasting, etc. To use a RBF network, a training algorithm is absolutely necessary for determining the network parameters. The existing training algorithms, such as orthogonal least squares (OLS) algorithm, clustering and gradient descent algorithm, have their own shortcomings respectively. In this paper, we propose a training algorithm based on a novel population-based evolutionary technique, quantum-behaved particle swarm optimization (QPSO), to train RBF neural network. The proposed QPSO-trained RBF network was tested on non-linear system identification problem and chaotic time series forecasting problem, and the results show that it can identify the system and forecast the chaotic time series more quickly and precisely than that trained by the particle swarm algorithm.

[1]  Wenbo Xu,et al.  Particle swarm optimization with particles having quantum behavior , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[2]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[3]  J. Nazuno.  Haykin, Simon. Neural networks: A comprehensive foundation, Prentice Hall, Inc. Segunda Edición, 1999 , 2000 .

[4]  Julius T. Tou,et al.  Pattern Recognition Principles , 1974 .

[5]  David S. Broomhead,et al.  Multivariable Functional Interpolation and Adaptive Networks , 1988, Complex Syst..

[6]  Zhen Wang,et al.  Time series forecasting with RBF neural network , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[7]  M. Clerc,et al.  The swarm and the queen: towards a deterministic and adaptive particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[8]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[9]  Peter J. Angeline,et al.  Evolutionary Optimization Versus Particle Swarm Optimization: Philosophy and Performance Differences , 1998, Evolutionary Programming.

[10]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[11]  Jing Liu,et al.  Using quantum-behaved particle swarm optimization algorithm to solve non-linear programming problems , 2007, Int. J. Comput. Math..

[12]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory, Third Edition , 1989, Springer Series in Information Sciences.

[13]  J. S. Vesterstrom,et al.  Division of labor in particle swarm optimisation , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[14]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[15]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[16]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[17]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[18]  Wenbo Xu,et al.  Adaptive parameter control for quantum-behaved particle swarm optimization on individual level , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[19]  Chia-Feng Juang,et al.  A hybrid of genetic algorithm and particle swarm optimization for recurrent network design , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  L. Coelho.  A quantum particle swarm optimizer with chaotic mutation operator , 2008 .

[21]  Yanchun Liang,et al.  A novel quantum swarm evolutionary algorithm and its applications , 2007, Neurocomputing.

[22]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[23]  Jun Sun,et al.  A global search strategy of quantum-behaved particle swarm optimization , 2004, IEEE Conference on Cybernetics and Intelligent Systems, 2004..

[24]  A. Engelbrecht,et al.  A new locally convergent particle swarm optimiser , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[25]  Robert M. Sanner,et al.  Gaussian Networks for Direct Adaptive Control , 1991, 1991 American Control Conference.

[26]  J. Kennedy.  Stereotyping: improving particle swarm performance with cluster analysis , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[27]  Erkki Oja,et al.  Rival penalized competitive learning for clustering analysis, RBF net, and curve detection , 1993, IEEE Trans. Neural Networks.

[28]  Martin Casdagli,et al.  Nonlinear prediction of chaotic time series , 1989 .

[29]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .