Optimal Control of Unknown Continuous-Time Nonaffine Nonlinear Systems

In this chapter, we consider optimal control problems of continuous-time nonaffine nonlinear systems with completely unknown dynamics via adaptive dynamic programming (ADP) methods. First, we develop an ADP-based identifier–actor–critic architecture to obtain the approximate optimal control for continuous-time unknown nonaffine nonlinear systems. The identifier is constructed by a dynamic neural network, which transforms nonaffine nonlinear systems into a kind of affine nonlinear systems. After that, the actor–critic dual networks are employed to derive the optimal control for the newly formulated affine nonlinear systems. Second, we present an ADP-based observer–critic architecture to obtain the approximate optimal output regulation for unknown nonaffine nonlinear systems. The present observer is composed of a three-layer feedforward neural network, which aims to obtain the knowledge of system states. Meanwhile, a single critic neural network is employed for estimating the performance of the systems as well as for constructing the optimal control signal.

[1]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[2]  W. Rudin.  Principles of mathematical analysis , 1964 .

[3]  Wen Yu.  Recent Advances in Intelligent Control Systems , 2009 .

[4]  A. Michel,et al.  Stability of Dynamical Systems: On the Role of Monotonic and Non-Monotonic Lyapunov Functions , 2015 .

[5]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[6]  C. D. Meyer,et al.  Generalized inverses of linear transformations , 1979 .

[7]  Frank L. Lewis,et al.  Multimodel neural networks identification and failure detection of nonlinear systems , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[8]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[9]  Frank L. Lewis,et al.  Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2010, Autom..

[10]  Frank L. Lewis,et al.  Neural Network Control Of Robot Manipulators And Non-Linear Systems , 1998 .

[11]  Xiong Yang,et al.  Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints , 2014, Int. J. Control.

[12]  Xiaoou Li,et al.  Some new results on system identification with dynamic neural networks , 2001, IEEE Trans. Neural Networks.

[13]  Eric Walter,et al.  Identification of Parametric Models: from Experimental Data , 1997 .

[14]  Petros A. Ioannou,et al.  Robust Adaptive Control , 2012 .

[15]  Heidar Ali Talebi,et al.  A stable neural network-based observer with application to flexible-joint manipulators , 2006, IEEE Transactions on Neural Networks.

[16]  Kurt Hornik,et al.  Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks , 1990, Neural Networks.

[17]  J. Theocharis,et al.  Neural network observer for induction motor control , 1994, IEEE Control Systems.

[18]  F. Lewis,et al.  Online solution of nonquadratic two‐player zero‐sum games arising in the H ∞  control of constrained input systems , 2014 .

[19]  M. S. Ahmed,et al.  Dynamic observers-a neural net approach , 2000, J. Intell. Fuzzy Syst..

[20]  T. Apostol.  Mathematical Analysis , 1957 .

[21]  Frank L. Lewis,et al.  Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach , 2005, Autom..

[22]  Naira Hovakimyan,et al.  Neural Network Adaptive Control for a Class of Nonlinear Uncertain Dynamical Systems With Asymptotic Stability Guarantees , 2008, IEEE Transactions on Neural Networks.

[23]  Graham C. Goodwin,et al.  Dynamic System Identification: Experiment Design and Data Analysis , 2012 .

[24]  Frank L. Lewis,et al.  Optimal Control , 1986 .

[25]  Derong Liu,et al.  Adaptive Dynamic Programming for Control: Algorithms and Stability , 2012 .

[26]  Derong Liu,et al.  Discrete-time online learning control for a class of unknown nonaffine nonlinear systems using reinforcement learning , 2014, Neural Networks.

[27]  Frank L. Lewis,et al.  Optimal Control: Lewis/Optimal Control 3e , 2012 .

[28]  Derong Liu,et al.  Neural-network-observer-based optimal control for unknown nonlinear systems using adaptive dynamic programming , 2013, Int. J. Control.

[29]  Frank L. Lewis,et al.  A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems , 2013, Autom..

[30]  Khanh Pham,et al.  Modeling MR-dampers: a nonlinear blackbox approach , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[31]  Yoh-Han Pao,et al.  Stochastic choice of basis functions in adaptive function approximation and the functional-link net , 1995, IEEE Trans. Neural Networks.

[32]  Zhong-Ping Jiang,et al.  Adaptive dynamic programming and optimal control of nonlinear nonaffine systems , 2014, Autom..