Comparison of four gradient-learning algorithms for neural network Wiener models

A gradient-based approach to training neural network Wiener models is presented. Calculation of the gradient or approximate gradient for the series-parallel and parallel Wiener models by the backpropagation, the sensitivity method (SM) and the backpropagation through time (BPTT) is considered in a unified framework. Four different recursive learning algorithms are derived, analysed and compared. For the truncated BPTT, it is shown that the determination of the number of unfolding time steps can be made on the basis of an impulse response function of sensitivity models. Analysis of the computational complexity of these algorithms shows that, contrary to the other recurrent neural network models, computation of the gradient in parallel Wiener models with the sensitivity method or backpropagation through time requires only a little more computational burden than the backpropagation. A simulated data example and a real data example of a laboratory two-tank system are also included to make comparison of different methods and their effectiveness and practical feasibility are shown.

[1]  W. R. Cluett,et al.  A new approach to the identification of pH processes based on the Wiener model , 1995 .

[2]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[3]  S. Billings,et al.  Theory of separable processes with applications to the identification of nonlinear systems , 1978 .

[4]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[5]  R. Pearson,et al.  Gray-box identification of block-oriented nonlinear models , 2000 .

[6]  Panos J. Antsaklis,et al.  Neural networks for control systems , 1990, IEEE Trans. Neural Networks.

[7]  Stephen A. Billings,et al.  Identification of systems containing linear dynamic and static nonlinear elements , 1982, Autom..

[8]  W. R. Cluett,et al.  Identification of Wiener-type nonlinear systems in a noisy environment , 1997 .

[9]  H. Al. Duwaish,et al.  Use of Multilayer Feedforward Neural Networks in Identification and Control of Wiener Model , 1996 .

[10]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[11]  Kumpati S. Narendra,et al.  Gradient methods for the optimization of dynamical systems containing neural networks , 1991, IEEE Trans. Neural Networks.

[12]  W. Greblicki.  Nonparametric identification of Wiener systems by orthogonal series , 1994, IEEE Trans. Autom. Control..

[13]  Wlodzimierz Greblicki,et al.  Nonparametric approach to Wiener system identification , 1997 .

[14]  Man-Wai Mak,et al.  On the improvement of the real time recurrent learning algorithm for recurrent neural networks , 1999, Neurocomputing.

[15]  Torbjörn Wigren,et al.  Recursive prediction error identification using the nonlinear wiener model , 1993, Autom..

[16]  Peter J. Gawthrop,et al.  Neural networks for control systems - A survey , 1992, Autom..

[17]  Heinz Unbehauen,et al.  Structure identification of nonlinear dynamic systems - A survey on input/output approaches , 1990, Autom..

[18]  Aarne Halme,et al.  Modeling of chromatographic separation process with Wiener-MLP representation , 2001 .