An extended ADALINE neural network trained by Levenberg-Marquardt method for system identification of linear systems

This paper presents a sliding-window version of online identification method for linear time varying systems based on the ADaptive LINear Element - ADALINE (Widrow and Lehr, 1990) neural network trained with Levenberg-Marquardt method which offers faster tracking of system parameter change. It is well known ADALINE is slow in convergence which is not appropriate for online application and identification of time varying system. To speed up convergence of learning and thus increase the capability of tracking time varying system parameters, our previous work added a momentum term to the weight adjustment. While the momentum does speed up convergence, it also shows overshooting or oscillating and also tracks noise closely. The Levenberg-Marquardt method is explored in this paper. Simulation results show that the proposed method provides indeed fast yet smoother convergence and better tracking of time varying parameters.

[1]  Bernard Widrow,et al.  30 years of adaptive neural networks: perceptron, Madaline, and backpropagation , 1990, Proc. IEEE.

[2]  Wenle Zhang,et al.  System identification based on an improved generalized ADALINE neural network , 2011, 2011 Chinese Control and Decision Conference (CCDC).

[3]  Gonzalo Joya,et al.  Gray box identification with hopfield neural networks , 2004 .

[4]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[5]  S Z Qin,et al.  Comparison of four neural net learning methods for dynamic system identification , 1992, IEEE Trans. Neural Networks.

[6]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[7]  Harpreet Singh,et al.  Single layer neural networks for linear system identification using gradient descent technique , 1993, IEEE Trans. Neural Networks.

[8]  Ferenc Szeifert,et al.  System Identification Using Delaunay Tessellation of Self-Organizing Maps , 2003 .

[9]  Kishan G. Mehrotra,et al.  Elements of artificial neural networks , 1996 .

[10]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[11]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[12]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[13]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[14]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[15]  A. Gretton,et al.  Support vector regression for black-box system identification , 2001, Proceedings of the 11th IEEE Signal Processing Workshop on Statistical Signal Processing (Cat. No.01TH8563).

[16]  Rahmat A. Shoureshi,et al.  Neural networks for system identification , 1990 .