An efficient parameterization of dynamic neural networks for nonlinear system identification

Dynamic neural networks (DNNs), which are also known as recurrent neural networks, are often used for nonlinear system identification. The main contribution of this letter is the introduction of an efficient parameterization of a class of DNNs. Having to adjust less parameters simplifies the training problem and leads to more parsimonious models. The parameterization is based on approximation theory dealing with the ability of a class of DNNs to approximate finite trajectories of nonautonomous systems. The use of the proposed parameterization is illustrated through a numerical example, using data from a nonlinear model of a magnetic levitation system.

[1]  Tommy W. S. Chow,et al.  Modeling of continuous time dynamical systems with input by recurrent neural networks , 2000 .

[2]  Erol Gelenbe,et al.  Measurement and performance of a cognitive packet network , 2001, Comput. Networks.

[3]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[4]  Roger Fletcher,et al.  Practical methods of optimization; (2nd ed.) , 1987 .

[5]  Ju-Yeop Choi,et al.  A constructive approach for nonlinear system identification using multilayer perceptrons , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[6]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[7]  Miguel Pinzolas,et al.  Improvement of the neighborhood based Levenberg-Marquardt algorithm by local adaptation of the learning coefficient , 2005, IEEE Transactions on Neural Networks.

[8]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[9]  Stephen A. Billings,et al.  International Journal of Control , 2004 .

[10]  Zhi-Hong Mao,et al.  Function approximation with spiked random networks , 1999, IEEE Trans. Neural Networks.

[11]  Ravi N. Banavar,et al.  A recurrent network for dynamic system identification , 1997, Int. J. Syst. Sci..

[12]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[13]  M. Sami Fadali,et al.  Nonlinear system identification using a Gabor/Hopfield network , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[14]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[15]  J. J. Hopfield,et al.  “Neural” computation of decisions in optimization problems , 1985, Biological Cybernetics.

[16]  Stephen A. Billings,et al.  Global analysis and model validation in nonlinear system identification , 1994, Nonlinear Dynamics.

[17]  O. Nelles.  Nonlinear System Identification , 2001 .

[18]  Erol Gelenbe,et al.  Random Neural Networks with Negative and Positive Signals and Product Form Solution , 1989, Neural Computation.

[19]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[20]  Victor M. Becerra,et al.  Dynamic Neural Networks , 2003 .

[21]  Michael I. Jordan,et al.  The Handbook of Brain Theory and Neural Networks , 2002 .

[22]  Pascal Koiran.  Dynamics of Discrete Time, Continuous State Hopfield Networks , 1994, Neural Computation.

[23]  Alberto Isidori,et al.  Nonlinear control systems: an introduction (2nd ed.) , 1989 .

[24]  Erol Gelenbe,et al.  Learning in the multiple class random neural network , 2002, IEEE Trans. Neural Networks.

[25]  Masahiro Kimura,et al.  Learning dynamical systems by recurrent neural networks from orbits , 1998, Neural Networks.

[26]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[27]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[28]  Joos Vandewalle,et al.  Efficient identification of RBF neural net models for nonlinear discrete-time multivariable dynamical systems , 1995, Neurocomputing.

[29]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[30]  Alexander S. Poznyak,et al.  Differential Neural Networks for Robust Nonlinear Control , 2004, IEEE Transactions on Neural Networks.

[31]  Stephen A. Billings,et al.  Identi cation of nonlinear systems-A survey , 1980 .

[32]  J. Hopfield,et al.  Computing with neural circuits: a model. , 1986, Science.

[33]  A. Isidori.  Nonlinear Control Systems , 1985 .

[34]  Taskin Koçak,et al.  Design and implementation of a random neural network routing engine , 2003, IEEE Trans. Neural Networks.

[35]  Victor M. Becerra,et al.  SYSTEM IDENTIFICATION USING DYNAMIC NEURAL NETWORKS: TRAINING AND INITIALIZATION ASPECTS , 2002 .

[36]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.