Nonlinear system identification with recurrent neural networks and dead-zone Kalman filter algorithm

Compared to normal learning algorithms, for example backpropagation, Kalman filter-based algorithm has some better properties, such as faster convergence, although this algorithm is more complex and sensitive to the nature of noises. In this paper, extended Kalman filter is applied to train state-space recurrent neural networks for nonlinear system identification. In order to improve robustness of Kalman filter algorithm dead-zone robust modification is applied to Kalman filter. Lyapunov method is used to prove that the Kalman filter training is stable.

[1]  R. Unbehauen,et al.  Stochastic stability of the continuous-time extended Kalman filter , 2000 .

[2]  Wen Yu,et al.  Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms , 2004, Inf. Sci..

[3]  Graham C. Goodwin,et al.  Adaptive filtering prediction and control , 1984 .

[4]  Andrew Chi-Sing Leung,et al.  Dual extended Kalman filtering in recurrent neural networks , 2003, Neural Networks.

[5]  Marios M. Polycarpou,et al.  High-order neural network structures for identification of dynamical systems , 1995, IEEE Trans. Neural Networks.

[6]  Kiyoshi Nishiyama,et al.  H∞-learning of layered neural networks , 2001, IEEE Trans. Neural Networks.

[7]  Mark E. Oxley,et al.  Comparative Analysis of Backpropagation and the Extended Kalman Filter for Training Multilayer Perceptrons , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Fahmida N. Chowdhury.  A new approach to real‐time training of dynamic neural networks , 2003 .

[9]  Amir F. Atiya,et al.  An algorithmic approach to adaptive state filtering using recurrent neural networks , 2001, IEEE Trans. Neural Networks.

[10]  M. Athans,et al.  Robustness and computational aspects of nonlinear stochastic estimators and regulators , 1977, 1977 IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and A Special Symposium on Fuzzy Set Theory and Applications.

[11]  Hideaki Sakai,et al.  A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter , 1992, IEEE Trans. Signal Process..

[12]  Konrad Reif,et al.  The extended Kalman filter as an exponential observer for nonlinear systems , 1999, IEEE Trans. Signal Process..

[13]  Ian R. Petersen,et al.  Robustness and risk-sensitive filtering , 2002, IEEE Trans. Autom. Control..

[14]  Giuseppe Carlo Calafiore,et al.  Robust filtering for discrete-time systems with bounded noise and parametric uncertainty , 2001, IEEE Trans. Autom. Control..

[15]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[16]  Andrew Chi-Sing Leung,et al.  On the Kalman filtering method in neural network training and pruning , 1999, IEEE Trans. Neural Networks.

[17]  YuWen.  Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms , 2004 .

[18]  Lei Guo.  Estimating time-varying parameters by the Kalman filter based algorithm: stability and convergence , 1990 .

[19]  Sharad Singhal,et al.  Training Multilayer Perceptrons with the Extende Kalman Algorithm , 1988, NIPS.

[20]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[21]  Angelo Alessandri,et al.  On the convergence of EKF-based parameters optimization for neural networks , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).