A Fast U-d Factorization-based Learning Algorithm with Applications to Nonlinear System Modeling and Identification

A fast learning algorithm for training multilayer feedforward neural networks (FNN's) by using a fading memory extended Kalman filter (FMEKF) is presented first, along with a technique using a self-adjusting time-varying forgetting factor. Then a U-D factorization-based FMEKF is proposed to further improve the learning rate and accuracy of the FNN. In comparison with the backpropagation (BP) and existing EKF-based learning algorithms, the proposed U-D factorization-based FMEKF algorithm provides much more accurate learning results, using fewer hidden nodes. It has improved convergence rate and numerical stability (robustness). In addition, it is less sensitive to start-up parameters (e.g., initial weights and covariance matrix) and the randomness in the observed data. It also has good generalization ability and needs less training time to achieve a specified learning accuracy. Simulation results in modeling and identification of nonlinear dynamic systems are given to show the effectiveness and efficiency of the proposed algorithm.

[1]  X. R. Li,et al.  Hybrid training of RBF networks with application to nonlinear systems identification , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[2]  Dragan Obradovic,et al.  On-line training of recurrent neural networks with continuous topology adaptation , 1996, IEEE Trans. Neural Networks.

[3]  Guanrong Chen,et al.  Modified extended kalman filtering for supervised learning , 1993 .

[4]  Hideaki Sakai,et al.  A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter , 1992, IEEE Trans. Signal Process..

[5]  Sharad Singhal,et al.  Training feed-forward networks with the extended Kalman algorithm , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[6]  Youmin Zhang,et al.  A fast and robust recursive prediction error learning algorithm for feedforward neural networks , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[7]  Francesco Palmieri,et al.  Optimal filtering algorithms for fast learning in feedforward neural networks , 1992, Neural Networks.

[8]  Stephen A. Billings,et al.  Properties of neural networks with applications to modelling non-linear dynamical systems , 1992 .

[9]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[10]  G. Bierman.  Factorization methods for discrete sequential estimation , 1977 .

[11]  P. Kumar,et al.  Theory and practice of recursive identification , 1985, IEEE Transactions on Automatic Control.

[12]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.