Nonlinear dynamic system identification using recurrent neural network with multi-segment piecewise-linear connection weight

This paper introduces a new concept of the connection weight to the standard recurrent neural networks—Elman and Jordan networks. The architecture of the modified networks is the same as that of the original recurrent neural networks. However, unlike the original recurrent neural networks whose connection weight is a single real number, in the modified networks the weight of each connection is multi-valued, depending on the value of the input data involved. The backpropagation learning algorithm is also modified to suit the proposed concept. The modified networks have been benchmarked against the feedforward neural network and the original recurrent neural networks. The experimental results on twelve benchmark problems show that the modified networks are clearly superior to the other three methods.

[1]  Wen Yu,et al.  Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms , 2004, Inf. Sci..

[2]  K. Mehrotra,et al.  Nonlinear system identification using recurrent networks , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[3]  Wei-Min Qi,et al.  Dynamic properties of Elman and modified Elman neural network , 2002, Proceedings. International Conference on Machine Learning and Cybernetics.

[4]  Duc Truong Pham,et al.  Training Elman and Jordan networks for system identification using genetic algorithms , 1999, Artif. Intell. Eng..

[5]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[6]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[7]  SATYANARAYAN S. RAO,et al.  A COMPOSITE NEURAL ARCHITECTURE AND ALGORITHM FOR NONLINEAR SYSTEM IDENTIFICATION , .

[8]  Dervis Karaboga,et al.  Training recurrent neural networks for dynamic system identification using parallel tabu search algorithm , 1997, Proceedings of 12th IEEE International Symposium on Intelligent Control.

[9]  Von-Wun Soo,et al.  A comparative study of recurrent neural network architectures on learning temporal sequences , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[10]  Qiao-ling Ji,et al.  The Property of PID Elman Neural Network and Its Application in Identification of Hydraulic Unit , 2007, 2007 IEEE International Conference on Control and Automation.

[11]  Xin Yao,et al.  Empirical analysis of evolutionary algorithms with immigrants schemes for dynamic optimization , 2009, Memetic Comput..

[12]  Seppo J. Ovaska,et al.  A modified Elman neural network model with application to dynamical systems identification , 1996, 1996 IEEE International Conference on Systems, Man and Cybernetics. Information Intelligence and Systems (Cat. No.96CH35929).