Identification of nonlinear systems with outliers using wavelet neural networks based on annealing dynamical learning algorithm

This paper presents an annealing dynamical learning algorithm (ADLA) to train wavelet neural networks (WNNs) for identifying nonlinear systems with outliers. In ADLA-WNNs, wavelet-based support vector regression (WSVR) is adopted to determine the initial translation and dilation of a wavelet kernel and the weights of WNNs due to the similarity between WSVR and WNNs. After initialization, ADLA with nonlinear time-varying learning rates is applied to train the WNNs. In the ADLA, the determination of the learning rates would be a key work for the trade-off between stability and speed of convergence. A computationally efficient optimization method, particle swarm optimization (PSO), is adopted to find the optimal learning rates to overcome the stagnation in the training procedure of WNNs. Due to the advantages of WSVR and ADLA (WSVR-ADLA), the WSVR-based ADLA-WNNs (WSVR-ADLA-WNNs) can robust against outliers and achieve the promising efficiency of system identifications. Three examples are simulated to confirm the performance of the proposed algorithm. From the simulated results, the feasibility and superiority of the proposed WSVR-ADLA-WNNs for identifying nonlinear systems with artificial outliers are verified.

[1]  Abdulhamit Subasi,et al.  Wavelet neural network classification of EEG signals by using AR model with MLE preprocessing , 2005, Neural Networks.

[2]  Chia-Feng Juang,et al.  A recurrent self-organizing neural fuzzy inference network , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[3]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[4]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[5]  Qinghua Zhang,et al.  Using wavelet network in nonparametric estimation , 1997, IEEE Trans. Neural Networks.

[6]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[7]  Okyay Kaynak,et al.  Fuzzy Wavelet Neural Networks for Identification and Control of Dynamic Plants—A Novel Structure and a Comparative Study , 2008, IEEE Transactions on Industrial Electronics.

[8]  Rong-Jong Wai,et al.  Hybrid control using recurrent fuzzy neural network for linear induction motor servo drive , 2001, IEEE Trans. Fuzzy Syst..

[9]  Jian-Da Wu,et al.  Faulted gear identification of a rotating machinery based on wavelet transform and artificial neural network , 2009, Expert Syst. Appl..

[10]  Shun-Feng Su,et al.  Robust support vector regression networks for function approximation with outliers , 2002, IEEE Trans. Neural Networks.

[11]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[12]  Cheng-Jian Lin.  Wavelet Neural Networks with a Hybrid Learning Approach , 2006, J. Inf. Sci. Eng..

[13]  Seyed Farid Ghaderi,et al.  An integrated artificial neural network and fuzzy clustering algorithm for performance assessment of decision making units , 2007, Appl. Math. Comput..

[14]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[15]  Jin Bae Park,et al.  Indirect adaptive control of nonlinear dynamic systems using self recurrent wavelet neural networks via adaptive learning rates , 2007, Inf. Sci..

[16]  Ying Chen,et al.  Identifying chaotic systems via a Wiener-type cascade model , 1997 .

[17]  Xuli Han,et al.  The multidimensional function approximation based on constructive wavelet RBF neural network , 2011, Appl. Soft Comput..

[18]  Chin-Wang Tao,et al.  Hybrid SVMR-GPR for modeling of chaotic time series systems with noise and outliers , 2010, Neurocomputing.

[19]  Wen Yu,et al.  Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms , 2004, Inf. Sci..

[20]  Yee Ming Chen,et al.  Dynamic parameter optimization of evolutionary computation for on-line prediction of time series with changing dynamics , 2007, Appl. Soft Comput..

[21]  Cheng-Jian Lin,et al.  A compensation-based recurrent fuzzy neural network for dynamic system identification , 2006, Eur. J. Oper. Res..

[22]  Bart Kosko,et al.  Neural networks and fuzzy systems: a dynamical systems approach to machine intelligence , 1991 .

[23]  Theodore B. Trafalis,et al.  Robust classification and regression using support vector machines , 2006, Eur. J. Oper. Res..

[24]  Stephen A. Billings,et al.  A new class of wavelet networks for nonlinear system identification , 2005, IEEE Transactions on Neural Networks.

[25]  Viviana Fernandez,et al.  Wavelet- and SVM-based forecasts: An analysis of the U.S. metal and materials manufacturing industry , 2007 .

[26]  Chein-I Chang,et al.  Robust radial basis function neural networks , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[27]  Ganesh K. Venayagamoorthy,et al.  Particle swarm optimization with quantum infusion for system identification , 2010, Eng. Appl. Artif. Intell..

[28]  Jun Zhang,et al.  Wavelet neural networks for function learning , 1995, IEEE Trans. Signal Process..

[29]  Vicky Rouss,et al.  Multi-input and multi-output neural model of the mechanical nonlinear behaviour of a PEM fuel cell system , 2008 .

[30]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[31]  Ana Maria Frattini Fileti,et al.  Identification and predictive control of a FCC unit using a MIMO neural model , 2005 .

[32]  George G. Lendaris.  Adaptive dynamic programming approach to experience-based systems identification and control , 2009, Neural Networks.

[33]  Tsung-Ying Sun,et al.  Effective Learning Rate Adjustment of Blind Source Separation Based on an Improved Particle Swarm Optimizer , 2008, IEEE Transactions on Evolutionary Computation.

[34]  Shi-jin Ren,et al.  Robust Wavelet Support Vector Machine for Regression Estimation , 2006 .

[35]  Shun-Feng Su,et al.  The annealing robust backpropagation (ARBP) learning algorithm , 2000, IEEE Trans. Neural Networks Learn. Syst..

[36]  Alexander J. Smola,et al.  Support Vector Method for Function Approximation, Regression Estimation and Signal Processing , 1996, NIPS.

[37]  M. Moraud.  Wavelet Networks , 2018, Foundations of Wavelet Networks and Applications.

[38]  Zhaoshen Li,et al.  Differential diagnosis of pancreatic cancer from normal tissue with digital imaging processing and pattern recognition based on a support vector machine of EUS images. , 2010, Gastrointestinal endoscopy.

[39]  Cameron Johnson,et al.  Comparison of a spiking neural network and an MLP for robust identification of generator dynamics in a multimachine power system , 2009, Neural Networks.

[40]  José Luis Rojo-Álvarez,et al.  Support Vector Machines for Nonlinear Kernel ARMA System Identification , 2006, IEEE Transactions on Neural Networks.

[41]  Kadir Liano,et al.  Robust error measure for supervised neural network learning with outliers , 1996, IEEE Trans. Neural Networks.

[42]  Mohamed Benrejeb,et al.  A new approach for multimodel identification of complex systems based on both neural and fuzzy clustering algorithms , 2010, Eng. Appl. Artif. Intell..

[43]  Nong Zhang,et al.  Application of evolving Takagi-Sugeno fuzzy model to nonlinear system identification , 2008, Appl. Soft Comput..

[44]  Yusuf Oysal,et al.  Fuzzy Wavelet Neural Network Models for Prediction and Identification of Dynamical Systems , 2010, IEEE Transactions on Neural Networks.

[45]  Chia-Ju Wu,et al.  Identification of MIMO systems using radial basis function networks with hybrid learning algorithm , 2009, Appl. Math. Comput..

[46]  Stephen A. Billings,et al.  An adaptive wavelet neural network for spatio-temporal system identification , 2010, Neural Networks.

[47]  Chia-Feng Juang,et al.  A TSK-type recurrent fuzzy network for dynamic systems processing by neural network and genetic algorithms , 2002, IEEE Trans. Fuzzy Syst..

[48]  Kumpati S. Narendra,et al.  Identification and control , 1998 .

[49]  Chia-Ju Wu,et al.  Annealing Robust Radial Basis Function Networks with Support Vector Regression for Nonlinear Inverse System Identification with Outliers , 2010 .

[50]  Xiangyang Wang,et al.  A wavelet-based image denoising using least squares support vector machine , 2010, Eng. Appl. Artif. Intell..

[51]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[52]  Douglas M. Hawkins.  Identification of Outliers , 1980, Monographs on Applied Probability and Statistics.

[53]  S. Ozer,et al.  Identification of bilinear systems using differential evolution algorithm , 2011 .

[54]  Charles K. Chui,et al.  An Introduction to Wavelets , 1992 .

[55]  Shian-Tang Tzeng,et al.  Design of fuzzy wavelet neural networks using the GA approach for function approximation and system identification , 2010, Fuzzy Sets Syst..

[56]  Shuyuan Yang,et al.  Applied Soft Computing Approximation of Functions with Spatial Inhomogeneity Based on " True " Ortho-ridgelet Neural Network , 2022 .

[57]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[59]  Leila Ait Gougam,et al.  A systematic investigation of a neural network for function approximation , 2008, Neural Networks.

[60]  Jin-Tsong Jeng,et al.  Annealing robust radial basis function networks for function approximation with outliers , 2004, Neurocomputing.

[61]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[62]  Heng-Da Cheng,et al.  Effective image retrieval using dominant color descriptor and fuzzy support vector machine , 2009, Pattern Recognit..

[63]  Li Zhang,et al.  Wavelet support vector machine , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[64]  Daniel W. C. Ho,et al.  A basis selection algorithm for wavelet neural networks , 2002, Neurocomputing.

[65]  Song Guangjun,et al.  The Research of Dynamic Change Learning Rate Strategy in BP Neural Network and Application in Network Intrusion Detection , 2008, 2008 3rd International Conference on Innovative Computing Information and Control.

[66]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[67]  Donald E. Grierson,et al.  Comparison among five evolutionary-based optimization algorithms , 2005, Adv. Eng. Informatics.

[68]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[69]  Chih-Hong Lin,et al.  Self-constructing fuzzy neural network speed controller for permanent-magnet synchronous motor drive , 2001, IEEE Trans. Fuzzy Syst..