A Derivative-Free Kalman Filter for Parameter Estimation of Recurrent Neural Networks and Its Applications to Nonlinear Channel Equalization

Recurrent neural networks (RNNs) trained with gradient-based algorithms such as real-time recurrent learning or back-propagation through time have a drawback of slow convergence rate. These algorithms also need the derivative calculation through the error back-propagation process. In this paper, a derivative-free Kalman filter, so called the unscented Kalman filter (UKF), for training a fully connected RNN is presented in a state-space formulation of the system. The UKF algorithm makes the RNN have fast convergence speed and good tracking performance without the derivative computation. Through experiments of nonlinear channel equalization, the performance of the RNN with the UKF is evaluated.

[1]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[2]  Rudolph van der Merwe,et al.  The Unscented Kalman Filter , 2002 .

[3]  Sheng Chen,et al.  Adaptive Bayesian equalizer with decision feedback , 1993, IEEE Trans. Signal Process..

[4]  Daesik Hong,et al.  A decision feedback recurrent neural equalizer as an infinite impulse response filter , 1997, IEEE Trans. Signal Process..

[5]  Amir F. Atiya,et al.  New results on recurrent network training: unifying the algorithms and accelerating convergence , 2000, IEEE Trans. Neural Networks Learn. Syst..

[6]  C.F.N. Cowan,et al.  Adaptive equalization of finite nonlinear channels using multilayer perceptron , 1990 .

[7]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[8]  Bhaskar D. Rao,et al.  Fast adaptive digital equalization by recurrent neural networks , 1997, IEEE Trans. Signal Process..

[9]  C. F. N. Cowan,et al.  Time‐variant equalization using a novel non‐linear adaptive structure , 1998 .

[10]  Jochen J. Steil,et al.  Tutorial: Perspectives on Learning with RNNs , 2002 .

[11]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[12]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[13]  Simon Haykin,et al.  Adaptive Filter Theory 4th Edition , 2002 .

[14]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[15]  Kyung Sup Kwak,et al.  On Modified Complex Recurrent Neural Network Adaptive Equalizer , 2002, J. Circuits Syst. Comput..

[16]  Elias S. Manolakos,et al.  Using recurrent neural networks for adaptive communication channel equalization , 1994, IEEE Trans. Neural Networks.

[17]  Raffaele Parisi,et al.  Complex discriminative learning Bayesian neural equalizer , 2001, Signal Process..

[18]  Hugh F. Durrant-Whyte,et al.  A new method for the nonlinear transformation of means and covariances in filters and estimators , 2000, IEEE Trans. Autom. Control..