Adaptive annealing learning algorithm-based robust wavelet neural networks for function approximation with outliers

In this study, a robust wavelet neural network (WNN) is proposed to approximate functions with outliers. In the proposed methodology, firstly, support vector machine with wavelet kernel function (WSVM) is adopted to determine the initial translation and dilation of a wavelet kernel and the weights of WNNs. Then, an adaptive annealing learning algorithm (AALA) is adopted to accommodate the translations, the dilations, and the weights of the WNNs. In the learning procedure, the AALA is proposed to overcome the problems of initialization and the cut-off points in the robust learning algorithm. Hence, when an initial structure of the WNNs is determined by a support vector regression (SVR) approach, the WNNs with AALA (AALA-WNNs) have fast convergence speed and can robust against outliers. Two examples are simulated to verify the feasibility and efficiency of the proposed algorithm.

[1]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[2]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[3]  Shi-jin Ren,et al.  Robust Wavelet Support Vector Machine for Regression Estimation , 2006 .

[4]  Stephen A. Billings,et al.  An adaptive wavelet neural network for spatio-temporal system identification , 2010, Neural Networks.

[5]  A. V.DavidSánchez,et al.  Robustization of a learning method for RBF networks , 1995, Neurocomputing.

[6]  Chi-Hsu Wang,et al.  Function approximation using fuzzy neural networks with robust learning algorithm , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Bart Kosko,et al.  Neural networks and fuzzy systems: a dynamical systems approach to machine intelligence , 1991 .

[8]  Jian-Da Wu,et al.  Faulted gear identification of a rotating machinery based on wavelet transform and artificial neural network , 2009, Expert Syst. Appl..

[9]  Shun-Feng Su,et al.  The annealing robust backpropagation (ARBP) learning algorithm , 2000, IEEE Trans. Neural Networks Learn. Syst..

[10]  Chia-Ju Wu,et al.  Identification of MIMO systems using radial basis function networks with hybrid learning algorithm , 2009, Appl. Math. Comput..

[11]  Xiangping Zeng,et al.  Identification of Nonlinear Dynamic System Using a Novel Recurrent Wavelet Neural Network Based on the Pipelined Architecture , 2014, IEEE Transactions on Industrial Electronics.

[12]  Yuting Wang,et al.  Very Short-Term Load Forecasting: Wavelet Neural Networks With Data Pre-Filtering , 2013, IEEE Transactions on Power Systems.

[13]  Jooyoung Park,et al.  Approximation and Radial-Basis-Function Networks , 1993, Neural Computation.

[14]  M. Nasir Uddin,et al.  A Novel Wavelet-Neural-Network-Based Robust Controller for IPM Motor Drives , 2013, IEEE Transactions on Industry Applications.

[15]  P. Torkzadeh,et al.  Structural optimization with frequency constraints by genetic algorithm using wavelet radial basis function neural network , 2008 .

[16]  Cheng-Chien Kuo,et al.  Modified particle swarm optimization algorithm with simulated annealing behavior and its numerical verification , 2011, Appl. Math. Comput..

[17]  L. Ingber.  Adaptive Simulated Annealing (ASA) , 1993 .

[18]  Mahmoud H. Alrefaei,et al.  A simulated annealing technique for multi-objective simulation optimization , 2009, Appl. Math. Comput..

[19]  Li Zhang,et al.  Wavelet support vector machine , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[20]  Daniel W. C. Ho,et al.  A basis selection algorithm for wavelet neural networks , 2002, Neurocomputing.

[21]  Xuli Han,et al.  The multidimensional function approximation based on constructive wavelet RBF neural network , 2011, Appl. Soft Comput..

[22]  Douglas M. Hawkins.  Identification of Outliers , 1980, Monographs on Applied Probability and Statistics.

[23]  S. N. Singh,et al.  Low-Order Dominant Harmonic Estimation Using Adaptive Wavelet Neural Network , 2014, IEEE Transactions on Industrial Electronics.

[24]  Jin-Tsong Jeng,et al.  Annealing robust radial basis function networks for function approximation with outliers , 2004, Neurocomputing.

[25]  Stephen A. Billings,et al.  A new class of wavelet networks for nonlinear system identification , 2005, IEEE Transactions on Neural Networks.

[26]  Pao-Ta Yu,et al.  On the optimal design of fuzzy neural networks with robust learning for function approximation , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[27]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[28]  Chein-I Chang,et al.  Robust radial basis function neural networks , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[29]  Man Ieee Systems,et al.  IEEE transactions on systems, man and cybernetics. Part B, Cybernetics , 1996 .

[30]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..