Dead-zone Kalman filter algorithm for recurrent neural networks

Compared to normal learning algorithms, for example backpropagation, Kalman filter-based algorithm has some better properties, such as faster convergence, although this algorithm is more complex and sensitive to the nature of noises. In this paper, extended Kalman filter is applied to train state-space recurrent neural networks for nonlinear system identification. In order to improve robustnees of Kalman filter algorithm dead-zone robust modification is applied to Kalman filter. Lyapunov method is used to prove that the Kalman filter training is stable.

[1]  Lei Guo.  Estimating time-varying parameters by the Kalman filter based algorithm: stability and convergence , 1990 .

[2]  Angelo Alessandri,et al.  On the convergence of EKF-based parameters optimization for neural networks , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[3]  Amir F. Atiya,et al.  An algorithmic approach to adaptive state filtering using recurrent neural networks , 2001, IEEE Trans. Neural Networks.

[4]  Sharad Singhal,et al.  Training Multilayer Perceptrons with the Extende Kalman Algorithm , 1988, NIPS.

[5]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[6]  Ian R. Petersen,et al.  Robustness and risk-sensitive filtering , 2002, IEEE Trans. Autom. Control..

[7]  A. Morales,et al.  Gasoline blending system modelling via static and dynamic neural networks , 2004 .

[8]  Konrad Reif,et al.  The extended Kalman filter as an exponential observer for nonlinear systems , 1999, IEEE Trans. Signal Process..

[9]  Fahmida N. Chowdhury.  A new approach to real‐time training of dynamic neural networks , 2003 .

[10]  Marios M. Polycarpou,et al.  High-order neural network structures for identification of dynamical systems , 1995, IEEE Trans. Neural Networks.

[11]  Wen Yu,et al.  Passivity analysis for dynamic multilayer neuro identifier , 2003 .

[12]  Wen Yu,et al.  Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms , 2004, Inf. Sci..

[13]  Graham C. Goodwin,et al.  Adaptive filtering prediction and control , 1984 .

[14]  Stefanos Kollias,et al.  An adaptive least squares algorithm for the efficient training of artificial neural networks , 1989 .

[15]  Hideaki Sakai,et al.  A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter , 1992, IEEE Trans. Signal Process..

[16]  Wen Yu,et al.  Discrete-time neuro identification without robust modification , 2003 .

[17]  Giuseppe Carlo Calafiore,et al.  Robust filtering for discrete-time systems with bounded noise and parametric uncertainty , 2001, IEEE Trans. Autom. Control..

[18]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[19]  Xiaoou Li,et al.  Fuzzy identification using fuzzy neural networks with stable learning algorithms , 2004, IEEE Transactions on Fuzzy Systems.

[20]  R. Unbehauen,et al.  Stochastic stability of the continuous-time extended Kalman filter , 2000 .

[21]  Mark E. Oxley,et al.  Comparative Analysis of Backpropagation and the Extended Kalman Filter for Training Multilayer Perceptrons , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Andrew Chi-Sing Leung,et al.  On the Kalman filtering method in neural network training and pruning , 1999, IEEE Trans. Neural Networks.

[23]  Kiyoshi Nishiyama,et al.  H∞-learning of layered neural networks , 2001, IEEE Trans. Neural Networks.