Nonlinear system identification with a feedforward neural network and an optimal bounded ellipsoid algorithm

Compared to normal learning algorithms, for example backpropagation, the optimal bounded ellipsoid (OBE) algorithm has some better properties, such as faster convergence, since it has a similar structure as the Kalman filter algorithm. Optimal bounded ellipsoid algorithm has some better properties than the Kalman filter training, one is that the noise is not required to be Guassian. In this paper optimal bounded ellipsoid algorithm is applied train the weights of a feedforward neural network for nonlinear system identification. Both hidden layers and output layers can be updated. In order to improve robustness of the optimal of the optimal bounded ellipsoid algorithm, dead-zone is applied to this algorithm. From a dynamic systems point of view, such training can be useful for all neural network applications requiring real-time updating of the weights. Two examples where provided which illustrate the effectiveness of the suggested algorithm based on simulations.

[1]  Sharad Singhal,et al.  Training Multilayer Perceptrons with the Extende Kalman Algorithm , 1988, NIPS.

[2]  Wei Wei,et al.  Application of an automatic fuzzy logic based pattern recognition method for DNA microarray reader , 2006 .

[3]  Federico Thomas,et al.  An ellipsoidal calculus based on propagation and fusion , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[4]  Lee A. Feldkamp,et al.  Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks , 1994, IEEE Trans. Neural Networks.

[5]  Amir F. Atiya,et al.  An algorithmic approach to adaptive state filtering using recurrent neural networks , 2001, IEEE Trans. Neural Networks.

[6]  B. T. Polyak,et al.  Limiting Behaviour of Bounding Ellipsoids for State Estimation , 2001 .

[7]  Fahmida N. Chowdhury.  A new approach to real‐time training of dynamic neural networks , 2003 .

[8]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[9]  Stephen P. Boyd,et al.  Robust minimum variance beamforming , 2005, IEEE Transactions on Signal Processing.

[10]  José Ramón Hilera González,et al.  Redes neuronales artificiales: fundamentos, modelos y aplicaciones , 1995 .

[11]  Roman Neruda,et al.  Parameter genetic learning of perceptron networks , 2006 .

[12]  Michael J. Todd,et al.  The Ellipsoid Method: A Survey , 1980 .

[13]  I. Siller-Alcalá,et al.  Design , construction , and control of a novel tower crane , 2022 .

[14]  Mark E. Oxley,et al.  Comparative Analysis of Backpropagation and the Extended Kalman Filter for Training Multilayer Perceptrons , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  John R. Deller,et al.  Multiweight optimization in optimal bounding ellipsoid algorithms , 2006, IEEE Transactions on Signal Processing.

[16]  S. Kapoor,et al.  Tracking of Time-Varying Parameters using Optimal Bounding Ellipsoid Algorithms , 1996 .

[17]  Y. F. Huang,et al.  On the value of information in system identification - Bounded noise case , 1982, Autom..

[18]  Venkat Venkatasubramanian,et al.  Representing bounded fault classes using neural networks with ellipsoidal activation functions , 1993 .

[19]  Wen Yu,et al.  Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms , 2004, Inf. Sci..

[20]  Graham C. Goodwin,et al.  Adaptive filtering prediction and control , 1984 .

[21]  J. J. Rubio Avila,et al.  Design of a low cost electronic control system for an articulated robot arm , 2007 .

[22]  Erik Weyer,et al.  Non-asymptotic confidence ellipsoids for the least squares estimate , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[23]  Wen Yu,et al.  Nonlinear system identification with recurrent neural networks and dead-zone Kalman filter algorithm , 2007, Neurocomputing.

[24]  S. Gazor,et al.  A new NLMS algorithm for slow noise magnitude variation , 2002, IEEE Signal Processing Letters.

[25]  Fred C. Schweppe,et al.  Uncertain dynamic systems , 1973 .

[26]  Li-Xin Wang,et al.  A Course In Fuzzy Systems and Control , 1996 .

[27]  Marios M. Polycarpou,et al.  High-order neural network structures for identification of dynamical systems , 1995, IEEE Trans. Neural Networks.

[28]  Luis A. Aguirre,et al.  Using Steady-State Prior Knowledge to Constrain Parameter Estimates in Nonlinear System Identification , 2002 .

[29]  Wen Yu,et al.  On-Line Clustering for Nonlinear System Identification Using Fuzzy Neural Networks , 2005, The 14th IEEE International Conference on Fuzzy Systems, 2005. FUZZ '05..

[30]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[31]  Bart Kosko,et al.  Fuzzy function approximation with ellipsoidal rules , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[32]  Ahmad Bagheri,et al.  Adaptive neural network control of an underwater remotely operated vehicle (ROV) , 2006 .