Reinforcement learning for the adaptive control of nonlinear systems

The adaptive control of nonlinear systems is a nontrivial problem. Examples of this class of problems are found widely in many areas of control applications. While techniques for the adaptive control of linear systems have been well-established in the literature, there are few corresponding techniques for nonlinear systems. In this work an attempt is made to present a method for the adaptive control of nonlinear systems based on a feedfoward neural network. The proposed approach incorporates a neuro-controller used within a reinforcement learning framework, which reduces the problem to one of learning a stochastic approximation of an unknown average error surface. Emphasis is placed on the fact that the neuro-controller dose not need any input/output information about the controlled system. The proposed method promises to be an efficient tool for adaptive control for both static and dynamic nonlinear systems. Several examples are included to illustrate the proposed scheme. >

[1]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[2]  Lennart Ljung,et al.  Analysis of recursive stochastic algorithms , 1977 .

[3]  Lino Guzzella,et al.  Time-optimal motions of robots in assembly tasks , 1986 .

[4]  Bernard Widrow,et al.  Adaptive Signal Processing , 1985 .

[5]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[6]  Albert Y. Zomaya,et al.  Reinforcement learning using back-propagation as a building block , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[7]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[9]  Stephen A. Billings,et al.  A comparison of the backpropagation and recursive prediction error algorithms for training neural networks , 1991 .

[10]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[11]  Vijaykumar Gullapalli,et al.  A stochastic reinforcement learning algorithm for learning real-valued functions , 1990, Neural Networks.

[12]  Albert Y. Zomaya.  Modelling and Simulation of Robot Manipulators - A Parallel Processing Approach , 1993, World Scientific Series in Robotics and Intelligent Systems.

[13]  A G Barto,et al.  Learning by statistical cooperation of self-interested neuron-like computing elements. , 1985, Human neurobiology.

[14]  Jorge Angeles,et al.  Controller estimation for the adaptive control of robotic manipulators , 1989, IEEE Trans. Robotics Autom..