Online State–Space Modeling Using Recurrent Multilayer Perceptrons with Unscented Kalman Filter

A nonlinear black-box modeling approach using a state–space recurrent multilayer perceptron (RMLP) is considered in this paper. The unscented Kalman filter (UKF), which was proposed recently and is appropriate for state–space representation, is employed to train the RMLP. The UKF offers a derivative-free computation and an easy implementation, compared to the extended Kalman filter (EKF) widely used for training neural networks. In addition, the UKF has a fast convergence rate and an excellent capability of parameter estimation which are appropriate for online learning. Through modeling experiments of nonlinear systems, the effectiveness of the RMLP trained with the UKF is demonstrated.

[1]  Tet Hin Yeap,et al.  A Derivative-Free Kalman Filter for Parameter Estimation of Recurrent Neural Networks and Its Applications to Nonlinear Channel Equalization , 2003 .

[2]  P. S. Sastry,et al.  Memory neuron networks for identification and control of dynamical systems , 1994, IEEE Trans. Neural Networks.

[3]  Hugh F. Durrant-Whyte,et al.  A new method for the nonlinear transformation of means and covariances in filters and estimators , 2000, IEEE Trans. Autom. Control..

[4]  Jochen J. Steil,et al.  Tutorial: Perspectives on Learning with RNNs , 2002 .

[5]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[6]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[7]  Rudolph van der Merwe,et al.  The Unscented Kalman Filter , 2002 .

[8]  Amir F. Atiya,et al.  New results on recurrent network training: unifying the algorithms and accelerating convergence , 2000, IEEE Trans. Neural Networks Learn. Syst..

[9]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[10]  Lennart Ljung,et al.  Nonlinear black-box modeling in system identification: a unified overview , 1995, Autom..

[11]  C. F. N. Cowan,et al.  Time‐variant equalization using a novel non‐linear adaptive structure , 1998 .

[12]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[13]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[14]  J Kurths,et al.  Estimation of parameters and unobserved components for nonlinear systems from noisy time series. , 2002, Physical review. E, Statistical, nonlinear, and soft matter physics.

[15]  Léon Personnaz,et al.  BLACK-BOX MODELING WITH STATE-SPACE NEURAL NETWORKS , 1995 .

[16]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[17]  Simon Haykin,et al.  Unscented Kalman filter-trained recurrent neural equalizer for time-varying channels , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[18]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[19]  Daesik Hong,et al.  A decision feedback recurrent neural equalizer as an infinite impulse response filter , 1997, IEEE Trans. Signal Process..