Non-linear system identification using particle swarm optimisation tuned radial basis function models

A novel particle swarm optimisation (PSO) tuned radial basis function (RBF) network model is proposed for identification of non-linear systems. At each stage of orthogonal forward regression (OFR) model construction process, PSO is adopted to tune one RBF unit's centre vector and diagonal covariance matrix by minimising the leave-one-out (LOO) mean square error (MSE). This PSO aided OFR automatically determines how many tunable RBF nodes are sufficient for modelling. Compared with the-state-of-the-art local regularisation assisted orthogonal least squares algorithm based on the LOO MSE criterion for constructing fixed-node RBF network models, the PSO tuned RBF model construction produces more parsimonious RBF models with better generalisation performance and is often more efficient in model construction. The effectiveness of the proposed PSO aided OFR algorithm for constructing tunable node RBF models is demonstrated using three real data sets.

[1]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[2]  R. H. Myers.  Classical and modern regression with applications , 1986 .

[3]  Michael Tiefelsdorf.  The regression model , 2000 .

[4]  Bing Lam Luk,et al.  Adaptive simulated annealing for optimization in signal processing applications , 1999, Signal Process..

[5]  Sheng Chen,et al.  Recursive hybrid algorithm for non-linear system identification using radial basis function networks , 1992 .

[6]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[7]  S..  Nonlinear time series modelling and prediction using Gaussian RBF networks with enhanced clustering and RLS learning , 2004 .

[8]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[9]  Sheng Chen,et al.  Sparse modeling using orthogonal forward regression with PRESS statistic and regularization , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  S. Billings,et al.  A prediction-error and stepwise-regression estimation algorithm for non-linear systems , 1986 .

[11]  Lin Yang,et al.  Particle-Swarm-Optimization-Based Multiuser Detector for CDMA Communications , 2007, IEEE Transactions on Vehicular Technology.

[12]  Sheng Chen,et al.  Robust maximum likelihood training of heteroscedastic probabilistic neural networks , 1998, Neural Networks.

[13]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[14]  Héctor Pomares,et al.  Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation , 2003, IEEE Trans. Neural Networks.

[15]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[16]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[17]  Sheng Chen,et al.  Experiments with repeating weighted boosting search for optimization signal processing applications , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[18]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[19]  Lester Ingber,et al.  Simulated annealing: Practice versus theory , 1993 .

[20]  Sheng Chen,et al.  Sparse kernel regression modeling using combined locally regularized orthogonal least squares and D-optimality experimental design , 2003, IEEE Trans. Autom. Control..

[21]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[22]  Sheng Chen,et al.  Parallel recursive prediction error algorithm for training layered neural networks , 1990 .

[23]  S.K. Halgamuge,et al.  Particle Swarm Optimisers for Cluster formation in Wireless Sensor Networks , 2005, 2005 International Conference on Intelligent Sensors, Sensor Networks and Information Processing.

[24]  Meryl E. Wastney,et al.  21 – USING THE MODEL , 1999 .

[25]  Sun-Yuan Kung,et al.  Estimation of elliptical basis function parameters by the EM algorithm with application to speaker verification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[26]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[27]  Sheng Chen,et al.  Representations of non-linear systems: the NARMAX model , 1989 .

[28]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[29]  Tsung-Ying Sun,et al.  Adequate determination of a band of wavelet threshold for noise cancellation using particle swarm optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[30]  Sheng Chen,et al.  Practical identification of NARMAX models using radial basis functions , 1990 .

[31]  Wenbo Xu,et al.  Training RBF Neural Network Via Quantum-Behaved Particle Swarm Optimization , 2006, ICONIP.

[32]  Steven Y. Liang,et al.  Workpiece dynamic analysis and prediction during chatter of turning process , 2008 .

[33]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[34]  Sheng Chen,et al.  Non-linear systems identification using radial basis functions , 1990 .

[35]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[36]  Sam Kwong,et al.  Genetic Algorithms : Concepts and Designs , 1998 .

[37]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[38]  S. A. Billings,et al.  The identification of linear and non-linear models of a turbocharged automotive diesel engine , 1989 .

[39]  R. Brereton,et al.  Support vector machines for classification and regression. , 2010, The Analyst.

[40]  X. Wanga,et al.  Sparse support vector regression based on orthogonal forward selection for the generalised kernel model , 2005 .

[41]  Sheng Chen,et al.  Parsimonious least squares support vector regression using orthogonal forward selection with the generalised kernel model , 2006, Int. J. Model. Identif. Control..

[42]  Vladimir Cherkassky,et al.  The Nature Of Statistical Learning Theory , 1997, IEEE Trans. Neural Networks.

[43]  George W. Irwin,et al.  A hybrid linear/nonlinear training algorithm for feedforward neural networks , 1998, IEEE Trans. Neural Networks.

[44]  K. S. Tang,et al.  Genetic Algorithms: Concepts and Designs with Disk , 1999 .

[45]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[46]  Hsuan-Ming Feng,et al.  Self-generation RBFNs using evolutional PSO learning , 2006, Neurocomputing.