A tunable radial basis function model for nonlinear system identification using particle swarm optimisation

A tunable radial basis function (RBF) network model is proposed for nonlinear system identification using particle swarm optimisation (PSO). At each stage of orthogonal forward regression (OFR) model construction, PSO optimises one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is computationally more efficient.

[1]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[2]  Héctor Pomares,et al.  Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation , 2003, IEEE Trans. Neural Networks.

[3]  Sheng Chen,et al.  Non-linear systems identification using radial basis functions , 1990 .

[4]  George W. Irwin,et al.  A hybrid linear/nonlinear training algorithm for feedforward neural networks , 1998, IEEE Trans. Neural Networks.

[5]  Leandro dos Santos Coelho,et al.  Multi-step ahead nonlinear identification of Lorenz’s chaotic system using radial basis neural network with learning by clustering and particle swarm optimization , 2008 .

[6]  Hsuan-Ming Feng,et al.  Self-generation RBFNs using evolutional PSO learning , 2006, Neurocomputing.

[7]  Sheng Chen,et al.  Recursive hybrid algorithm for non-linear system identification using radial basis function networks , 1992 .

[8]  Gary G. Yen,et al.  PSO-Based Multiobjective Optimization With Dynamic Population Size and Adaptive Local Archives , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[9]  Sheng Chen,et al.  Sparse kernel regression modeling using combined locally regularized orthogonal least squares and D-optimality experimental design , 2003, IEEE Trans. Autom. Control..

[10]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[11]  Sheng Chen,et al.  Parallel recursive prediction error algorithm for training layered neural networks , 1990 .

[12]  Sheng Chen,et al.  Robust maximum likelihood training of heteroscedastic probabilistic neural networks , 1998, Neural Networks.

[13]  S.K. Halgamuge,et al.  Particle Swarm Optimisers for Cluster formation in Wireless Sensor Networks , 2005, 2005 International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[14]  Sun-Yuan Kung,et al.  Estimation of elliptical basis function parameters by the EM algorithm with application to speaker verification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[15]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Lin Yang,et al.  Particle-Swarm-Optimization-Based Multiuser Detector for CDMA Communications , 2007, IEEE Transactions on Vehicular Technology.

[18]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[19]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[20]  Sheng Chen,et al.  Representations of non-linear systems: the NARMAX model , 1989 .

[21]  Tsung-Ying Sun,et al.  Adequate determination of a band of wavelet threshold for noise cancellation using particle swarm optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[22]  Sheng Chen,et al.  Practical identification of NARMAX models using radial basis functions , 1990 .

[23]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[24]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[25]  R. H. Myers.  Classical and modern regression with applications , 1986 .

[26]  S..  Nonlinear time series modelling and prediction using Gaussian RBF networks with enhanced clustering and RLS learning , 2004 .

[27]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[28]  Sheng Chen,et al.  Sparse modeling using orthogonal forward regression with PRESS statistic and regularization , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[29]  S. Billings,et al.  A prediction-error and stepwise-regression estimation algorithm for non-linear systems , 1986 .

[30]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.