Training trajectories by continuous recurrent multilayer networks

This paper addresses the problem of training trajectories by means of continuous recurrent neural networks whose feedforward parts are multilayer perceptrons. Such networks can approximate a general nonlinear dynamic system with arbitrary accuracy. The learning process is transformed into an optimal control framework where the weights are the controls to be determined. A training algorithm based upon a variational formulation of Pontryagin's maximum principle is proposed for such networks. Computer examples demonstrating the efficiency of the given approach are also presented.

[1]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[2]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[3]  Seong-Whan Lee,et al.  A new recurrent neural network architecture for pattern recognition , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[4]  Athanasios Kehagias,et al.  Time-Series Segmentation Using Predictive Modular Neural Networks , 1997, Neural Computation.

[5]  Kwang Y. Lee,et al.  An optimal tracking neuro-controller for nonlinear dynamic systems , 1996, IEEE Trans. Neural Networks.

[6]  Vassilios Petridis,et al.  Predictive Modular Neural Networks: Applications to Time Series , 1998 .

[7]  E Konecny,et al.  Middle latency auditory evoked responses and electroencephalographic derived variables do not predict movement to noxious stimulation during 1 minimum alveolar anesthetic concentration isoflurane/nitrous oxide anesthesia. , 1999, Anesthesia and analgesia.

[8]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[9]  Barak A. Pearlmutter.  Learning State Space Trajectories in Recurrent Neural Networks , 1989, Neural Computation.

[10]  Herbert Witte,et al.  Structure optimization of neural networks with the A*-algorithm , 1997, IEEE Trans. Neural Networks.

[11]  Fionn Murtagh,et al.  Combining Neural Network Forecasts on Wavelet-transformed Time Series , 1997, Connect. Sci..

[12]  Herbert Witte,et al.  Learning continuous trajectories in recurrent neural networks with time-dependent weights , 1999, IEEE Trans. Neural Networks.

[13]  Nader Sadegh,et al.  A perceptron network for functional identification and control of nonlinear systems , 1993, IEEE Trans. Neural Networks.

[14]  G N Kenny,et al.  Middle latency auditory evoked potentials during repeated transitions from consciousness to unconsciousness , 1996, Anaesthesia.

[15]  N. Toomarian.  Adjoint operators and non adiabatic algorithms in neural networks , 1991 .

[16]  Athanasios Kehagias,et al.  Predictive Modular Neural Networks for Time Series Classification , 1997, Neural Networks.

[17]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[18]  J. Barhen,et al.  Adjoint-operators and non-adiabatic learning algorithms in neural networks , 1991 .

[19]  Emanuel Marom,et al.  Efficient Training of Recurrent Neural Network with Time Delays , 1997, Neural Networks.

[20]  Garrison W. Cottrell,et al.  Phase-Space Learning , 1994, NIPS.

[21]  Seong-Whan Lee,et al.  A new recurrent neural-network architecture for visual pattern recognition , 1997, IEEE Trans. Neural Networks.

[22]  Barak A. Pearlmutter.  Gradient calculations for dynamic recurrent neural networks: a survey , 1995, IEEE Trans. Neural Networks.

[23]  Amir F. Atiya,et al.  Application of the recurrent multilayer perceptron in modeling complex process dynamics , 1994, IEEE Trans. Neural Networks.

[24]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[25]  Gaetan Libert,et al.  Dynamic recurrent neural networks: a dynamical analysis , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[26]  C Thornton,et al.  Evoked potentials in anaesthesia. , 1991, European journal of anaesthesiology.

[27]  D Lehmann,et al.  Instantaneous frequency maps, dipole models and potential distributions of pattern reversal-evoked potential fields for correct recognition of stimulated hemiretinae. , 1996, Electroencephalography and clinical neurophysiology.

[28]  Jacques Ludik,et al.  A Multilayer Real-Time Recurrent Learning Algorithm for Improved Convergence , 1997, ICANN.

[29]  Marios M. Polycarpou,et al.  High-order neural network structures for identification of dynamical systems , 1995, IEEE Trans. Neural Networks.

[30]  R. P. Fedorenko.  Approximate solution of some optimal control problems , 1964 .

[31]  Miroslaw Galicki,et al.  The Planning of Robotic Optimal Motions in the Presence of Obstacles , 1998, Int. J. Robotics Res..

[32]  Ewald Konecny,et al.  INTEROBSERVER-VARIABILITY FOR EVALUATION OF MIDDLE LATENCY AUDITORY EVOKED POTENTIALS DURING ANESTHESIA , 1998 .

[33]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .