Estimation of artificial neural network parameters for nonlinear system identification

A unified framework for representing ANN (artificial neural network) training algorithms is developed by considering weight selection as a parameter estimation problem. Three existing ANN training strategies are reviewed within this framework, i.e., gradient-descent backpropagation, the extended Kalman algorithm, and the recursive least squares method. A strikingly different approach to error backpropagation is presented, resulting in the development of a novel method of backward signal propagation and target state generation for embedded layers. The proposed technique is suitable for implementation with a linear-Kalman based update algorithm and is applied with a time-varying method of covariance modification for the elimination of transients associated with initial conditions. Results from a nonlinear identification experiment demonstrate an increased rate of convergence in comparison with backpropagation. The new algorithm displayed similar rates of parameter convergence and a decreased computational overhead compared to the extended Kalman algorithm.<<ETX>>

[1]  Paul J. Werbos,et al.  Neural networks for control and system identification , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.

[2]  Francesco Palmieri,et al.  A new algorithm for training multilayer perceptrons , 1989, Conference Proceedings., IEEE International Conference on Systems, Man and Cybernetics.

[3]  Karl Johan Åström,et al.  BOOK REVIEW SYSTEM IDENTIFICATION , 1994, Econometric Theory.

[4]  Edward K. Blum,et al.  Approximation theory and feedforward networks , 1991, Neural Networks.

[5]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[6]  Sharad Singhal,et al.  Training feed-forward networks with the extended Kalman algorithm , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[7]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[8]  T. R. Fortescue,et al.  Implementation of self-tuning regulators with variable forgetting factors , 1981, Autom..

[9]  Sheng Chen,et al.  Parallel recursive prediction error algorithm for training layered neural networks , 1990 .

[10]  K S Narendra,et al.  IDENTIFICATION AND CONTROL OF DYNAMIC SYSTEMS USING NEURAL NETWORKS , 1990 .

[11]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[12]  Rahmat A. Shoureshi,et al.  Neural networks for system identification , 1990 .

[13]  J.J. Helferty,et al.  Experiments in adaptive control using artificial neural networks , 1990, 1990 IEEE International Conference on Systems Engineering.

[14]  Kumpati S. Narendra,et al.  Adaptive identification and control of dynamical systems using neural networks , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.

[15]  Lennart Ljung,et al.  Adaptation and tracking in system identification - A survey , 1990, Autom..

[16]  Nazif Tepedelenlioglu,et al.  A fast new algorithm for training feedforward neural networks , 1992, IEEE Trans. Signal Process..

[17]  Fu-Chuang Chen,et al.  Adaptive control of nonlinear systems using neural networks , 1992 .