A new learning algorithm for diagonal recurrent neural network

A new hybrid learning algorithm combining the extended Kalman filter (EKF) and particle filter is presented. The new algorithm is firstly applied to train diagonal recurrent neural network (DRNN). The EKF is used to train DRNN and particle filter applies the resampling algorithm to optimize the particles, namely DRNNs, with the relative network weights. These methods make the training shorter and DRNN convergent more quickly. Simulation results of the nonlinear dynamical identification verify the validity of the new algorithm.

[1]  Kwang Y. Lee,et al.  Diagonal recurrent neural networks for dynamic systems control , 1995, IEEE Trans. Neural Networks.

[2]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[3]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[4]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[5]  Ronald J. Williams,et al.  Training recurrent networks using the extended Kalman filter , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[6]  Arnaud Doucet,et al.  Sequential Monte Carlo Methods to Train Neural Network Models , 2000, Neural Computation.

[7]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.