Two-stage RBF network construction based on particle swarm optimization

The conventional radial basis function (RBF) network optimization methods, such as orthogonal least squares or the two-stage selection, can produce a sparse network with satisfactory generalization capability. However, the RBF width, as a nonlinear parameter in the network, is not easy to determine. In the aforementioned methods, the width is always pre-determined, either by trial-and-error, or generated randomly. Furthermore, all hidden nodes share the same RBF width. This will inevitably reduce the network performance, and more RBF centres may then be needed to meet a desired modelling specification. In this paper we investigate a new two-stage construction algorithm for RBF networks. It utilizes the particle swarm optimization method to search for the optimal RBF centres and their associated widths. Although the new method needs more computation than conventional approaches, it can greatly reduce the model size and improve model generalization performance. The effectiveness of the proposed technique is confirmed by two numerical simulation examples.

[1]  Bing Lam Luk,et al.  Non-linear system identification using particle swarm optimisation tuned radial basis function models , 2009, Int. J. Bio Inspired Comput..

[2]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[3]  George W. Irwin,et al.  A two-stage algorithm for automatic construction of RBF neural models , 2010, Melecon 2010 - 2010 15th IEEE Mediterranean Electrotechnical Conference.

[4]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[5]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[6]  O. Nelles.  Nonlinear System Identification , 2001 .

[7]  D. Devaraj,et al.  Fault detection in centrifugal pumping systems using neural networks , 2008, Int. J. Model. Identif. Control..

[8]  Paul Sharkey,et al.  Automatic nonlinear predictive model-construction algorithm using forward regression and the PRESS statistic , 2003 .

[9]  Sheng Chen,et al.  Sparse kernel density construction using orthogonal forward regression with leave-one-out test score and local regularization , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  L. Glass,et al.  Oscillation and chaos in physiological control systems. , 1977, Science.

[11]  Stephen L. Chiu,et al.  Fuzzy Model Identification Based on Cluster Estimation , 1994, J. Intell. Fuzzy Syst..

[12]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[13]  George W. Irwin,et al.  Fast Forward RBF Network Construction Based on Particle Swarm Optimization , 2010, LSMS/ICSEE.

[14]  George W. Irwin,et al.  A hybrid linear/nonlinear training algorithm for feedforward neural networks , 1998, IEEE Trans. Neural Networks.

[15]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[16]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[17]  Christian Blum,et al.  Swarm Intelligence: Introduction and Applications , 2008, Swarm Intelligence.

[18]  Sheng Chen,et al.  Particle Swarm Optimization Aided Orthogonal Forward Regression for Unified Data Modeling , 2010, IEEE Transactions on Evolutionary Computation.

[19]  George W. Irwin,et al.  A fast nonlinear model identification method , 2005, IEEE Transactions on Automatic Control.

[20]  Sheng Chen,et al.  Local regularization assisted orthogonal least squares regression , 2006, Neurocomputing.

[21]  S. Billings,et al.  Algorithms for minimal model structure detection in nonlinear dynamic system identification , 1997 .

[22]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[23]  Kang Li,et al.  A two-stage algorithm for identification of nonlinear dynamic systems , 2006, Autom..

[24]  Sheng Chen,et al.  Model selection approaches for non-linear system identification: a review , 2008, Int. J. Syst. Sci..

[25]  Yung C. Shin,et al.  Radial basis function neural network for approximation and estimation of nonlinear stochastic dynamic systems , 1994, IEEE Trans. Neural Networks.

[26]  Kang Li,et al.  Two-Stage Mixed Discrete–Continuous Identification of Radial Basis Function (RBF) Neural Models for Nonlinear Systems , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[27]  Kevin Warwick,et al.  Mean-tracking clustering algorithm for radial basis function centre selection , 1997 .

[28]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[29]  Rosalind W. Picard,et al.  On the efficiency of the orthogonal least squares training method for radial basis function networks , 1996, IEEE Trans. Neural Networks.

[30]  Stephen A. Billings,et al.  International Journal of Control , 2004 .

[31]  De-Shuang Huang,et al.  A Hybrid Forward Algorithm for RBF Neural Network Construction , 2006, IEEE Transactions on Neural Networks.

[32]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[33]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.