Optimal control of terminal processes using neural networks

Feedforward neural networks are capable of approximating continuous multivariate functions and, as such, can implement nonlinear state-feedback controllers. Training methods such as backpropagation-through-time (BPTT), however, do not deal with terminal control problems in which the specified cost function includes the elapsed trajectory-time. In this paper, an extension to BPTT is proposed which addresses this limitation. The controller design is reformulated as a constrained optimization problem defined over the entire field of extremals and in which the set of trajectory times is incorporated into the cost function. Necessary first-order stationary conditions are derived which correspond to standard BPTT with the addition of certain transversality conditions. The new gradient algorithm based on these conditions, called time-optimal backpropagation through time, is tested on two benchmark minimum-time control problems.

[1]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[2]  B. Widrow,et al.  The truck backer-upper: an example of self-learning in neural networks , 1989, International 1989 Joint Conference on Neural Networks.

[3]  K S Narendra,et al.  IDENTIFICATION AND CONTROL OF DYNAMIC SYSTEMS USING NEURAL NETWORKS , 1990 .

[4]  B.D.O. Anderson,et al.  Singular optimal control problems , 1975, Proceedings of the IEEE.

[5]  A. Guez,et al.  Solution to the inverse kinematics problem in robotics by neural networks , 1988, IEEE 1988 International Conference on Neural Networks.

[6]  W. Thomas Miller,et al.  Sensor-based control of robotic manipulators using a general learning algorithm , 1987, IEEE J. Robotics Autom..

[7]  Helge J. Ritter,et al.  Three-dimensional neural net for learning visuomotor coordination of a robot arm , 1990, IEEE Trans. Neural Networks.

[8]  Sharad Singhal,et al.  Training Multilayer Perceptrons with the Extende Kalman Algorithm , 1988, NIPS.

[9]  G. Josin,et al.  Robot control using neural networks , 1988, IEEE 1988 International Conference on Neural Networks.

[10]  J. B. Rosen.  The gradient projection method for nonlinear programming: Part II , 1961 .

[11]  Arthur E. Bryson,et al.  Applied Optimal Control , 1969 .

[12]  N.V. Bhat,et al.  Modeling chemical process systems via neural computation , 1990, IEEE Control Systems Magazine.

[13]  Dean Pomerleau,et al.  Efficient Training of Artificial Neural Networks for Autonomous Navigation , 1991, Neural Computation.

[14]  Michael Kuperstein,et al.  Neural controller for adaptive movements with unforeseen payloads , 1990, IEEE Trans. Neural Networks.

[15]  R. Bellman.  Dynamic programming. , 1957, Science.

[16]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[17]  W. E. Staib,et al.  The intelligent arc furnace controller: a neural network electrode position optimization system for the electric arc furnace , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[18]  Kumpati S. Narendra,et al.  Gradient methods for the optimization of dynamical systems containing neural networks , 1991, IEEE Trans. Neural Networks.

[19]  Sebastian Thrun,et al.  Exploration and model building in mobile robot domains , 1993, IEEE International Conference on Neural Networks.

[20]  M. Kuperstein,et al.  Implementation of an adaptive neural controller for sensory-motor coordination , 1989, International 1989 Joint Conference on Neural Networks.

[21]  Bernard Widrow,et al.  Adaptive inverse control , 1987, Proceedings of 8th IEEE International Symposium on Intelligent Control.

[22]  J. B. Rosen.  The Gradient Projection Method for Nonlinear Programming. Part I. Linear Constraints , 1960 .

[23]  Yaman Arkun,et al.  Neural Network Modeling and an Extended DMC Algorithm to Control Nonlinear Systems , 1990, 1990 American Control Conference.

[24]  M. Kawato,et al.  Hierarchical neural network model for voluntary movement with application to robotics , 1988, IEEE Control Systems Magazine.

[25]  Lee A. Feldkamp,et al.  Decoupled extended Kalman filter training of feedforward layered networks , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[26]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.