Recurrent wavelets neural networks learning via dead zone Kalman filter

Compared to normal learning algorithms, for example backpropagation, Kalman filter-based algorithm has some better properties, such as faster convergence, although this algorithm is more complex and sensitive to the nature of noises. In this paper, extended Kalman filter is applied to train recurrent wavelets neural networks for nonlinear system identification. In order to improve robustness of Kalman filter algorithm dead-zone robust modification is applied to Kalman filter. Lyapunov method is used to prove that this new training approach is stable.

[1]  Sharad Singhal,et al.  Training Multilayer Perceptrons with the Extende Kalman Algorithm , 1988, NIPS.

[2]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[3]  Fahmida N. Chowdhury.  A new approach to real‐time training of dynamic neural networks , 2003 .

[4]  Kiyoshi Nishiyama,et al.  H∞-learning of layered neural networks , 2001, IEEE Trans. Neural Networks.

[5]  Angelo Alessandri,et al.  On the convergence of EKF-based parameters optimization for neural networks , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[6]  YuWen.  Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms , 2004 .

[7]  Konrad Reif,et al.  The extended Kalman filter as an exponential observer for nonlinear systems , 1999, IEEE Trans. Signal Process..

[8]  Marios M. Polycarpou,et al.  High-order neural network structures for identification of dynamical systems , 1995, IEEE Trans. Neural Networks.

[9]  Wen Yu,et al.  Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms , 2004, Inf. Sci..

[10]  Graham C. Goodwin,et al.  Adaptive filtering prediction and control , 1984 .

[11]  Stefanos Kollias,et al.  An adaptive least squares algorithm for the efficient training of artificial neural networks , 1989 .

[12]  Lei Guo.  Estimating time-varying parameters by the Kalman filter based algorithm: stability and convergence , 1990 .

[13]  Andrew Chi-Sing Leung,et al.  On the Kalman filtering method in neural network training and pruning , 1999, IEEE Trans. Neural Networks.

[14]  Amir F. Atiya,et al.  An algorithmic approach to adaptive state filtering using recurrent neural networks , 2001, IEEE Trans. Neural Networks.

[15]  Hideaki Sakai,et al.  A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter , 1992, IEEE Trans. Signal Process..

[16]  Andrew Chi-Sing Leung,et al.  Dual extended Kalman filtering in recurrent neural networks , 2003, Neural Networks.

[17]  A. Haar.  Zur Theorie der orthogonalen Funktionensysteme , 1910 .

[18]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[19]  R. Unbehauen,et al.  Stochastic stability of the continuous-time extended Kalman filter , 2000 .

[20]  Ian R. Petersen,et al.  Robustness and risk-sensitive filtering , 2002, IEEE Trans. Autom. Control..

[21]  M. Athans,et al.  Robustness and computational aspects of nonlinear stochastic estimators and regulators , 1977, 1977 IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and A Special Symposium on Fuzzy Set Theory and Applications.

[22]  Mark E. Oxley,et al.  Comparative Analysis of Backpropagation and the Extended Kalman Filter for Training Multilayer Perceptrons , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Ingrid Daubechies,et al.  Ten Lectures on Wavelets , 1992 .

[24]  Giuseppe Carlo Calafiore,et al.  Robust filtering for discrete-time systems with bounded noise and parametric uncertainty , 2001, IEEE Trans. Autom. Control..