Adaptive dynamic programming for infinite horizon optimal robust guaranteed cost control of a class of uncertain nonlinear systems

In this paper, an infinite horizon optimal robust guaranteed cost control scheme of a class of continuous-time uncertain nonlinear systems is established based on adaptive dynamic programming. The main idea lies in that the optimal robust guaranteed cost control problem can be transformed into an optimal control problem. Actually, the optimal cost function of the nominal system is nothing but the optimal guaranteed cost of the original uncertain system. A critic neural network is constructed to help solving the modified Hamilton-Jacobi-Bellman equation corresponding to the nominal system. Then, an additional stabilizing term is introduced to reinforce the updating process of the weight vector and reduce the requirement of an initial stabilizing control. An example is provided to illustrate the effectiveness of the present control approach.

[1]  Jennie Si,et al.  Online learning control by association and reinforcement , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[2]  Yu Jiang,et al.  Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[4]  Derong Liu,et al.  Data-based Self-learning Optimal Control: Research Progress and Prospects , 2013 .

[5]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[7]  Derong Liu,et al.  Adaptive optimal control for a class of continuous-time affine nonlinear systems with unknown internal dynamics , 2012, Neural Computing and Applications.

[8]  Frank L. Lewis,et al.  2009 Special Issue: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems , 2009 .

[9]  Haibo He,et al.  Heuristic dynamic programming with internal goal representation , 2013, Soft Comput..

[10]  Sarangapani Jagannathan,et al.  Neural Network-Based Optimal Adaptive Output Feedback Control of a Helicopter UAV , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Derong Liu,et al.  Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Ali Heydaria,et al.  Global optimality of approximate dynamic programming and its use in non-convex function minimization , 2016 .

[13]  W. Haddad,et al.  Robust nonlinear feedback control for uncertain linear systems with nonquadratic performance criteria , 1998 .

[14]  Derong Liu,et al.  Policy Iteration Algorithm for Online Design of Robust Control for a Class of Continuous-Time Nonlinear Systems , 2014, IEEE Transactions on Automation Science and Engineering.

[15]  Qinglai Wei,et al.  Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming , 2012, Autom..

[16]  Hao Xu,et al.  Stochastic optimal control of unknown linear networked control system in the presence of random delays and packet losses , 2012, Autom..

[17]  Frank L. Lewis,et al.  Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach , 2005, Autom..

[18]  Huaguang Zhang,et al.  Adaptive Dynamic Programming: An Introduction , 2009, IEEE Computational Intelligence Magazine.

[19]  Derong Liu,et al.  Neural-Network-Based Optimal Control for a Class of Unknown Discrete-Time Nonlinear Systems Using Globalized Dual Heuristic Programming , 2012, IEEE Transactions on Automation Science and Engineering.

[20]  Jean-Jacques E. Slotine,et al.  Neural Network Control of Unknown Nonlinear Systems , 1989, 1989 American Control Conference.

[21]  F. Lewis,et al.  Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers , 2012, IEEE Control Systems.

[22]  Jiaqi Liang,et al.  Wide-Area Measurement Based Dynamic Stochastic Optimal Power Flow Control for Smart Grids With High Variability and Uncertainty , 2012, IEEE Transactions on Smart Grid.

[23]  T. K. C. Peng,et al.  Adaptive Guaranteed Cost of Control of Systems with Uncertain Parameters , 1970 .

[24]  Indra Narayan Kar,et al.  Bounded robust control of nonlinear systems using neural network–based HJB solution , 2011, IEEE Transactions on Automation Science and Engineering.

[25]  Jinyu Wen,et al.  Adaptive Learning in Tracking Control Based on the Dual Critic Network Design , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Donald A. Sofge,et al.  Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches , 1992 .

[27]  Sarangapani Jagannathan,et al.  Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[28]  W. Marsden.  I and J , 2012 .

[29]  Derong Liu,et al.  Neuro-optimal control for a class of unknown nonlinear dynamic systems using SN-DHP technique , 2013, Neurocomputing.

[30]  Li Yu,et al.  An LMI approach to guaranteed cost control of linear uncertain time-delay systems , 1999, Autom..

[31]  Frank L. Lewis,et al.  A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems , 2013, Autom..

[32]  Frank L. Lewis,et al.  Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Huai-Ning Wu,et al.  Neural Network Based Online Simultaneous Policy Update Algorithm for Solving the HJI Equation in Nonlinear $H_{\infty}$ Control , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Ali Heydari,et al.  Finite-Horizon Control-Constrained Nonlinear Optimal Control Using Single Network Adaptive Critics , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Qing-Long Han,et al.  Optimal guaranteed cost control of linear uncertain systems with input constraints , 2004, Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No.04EX788).

[36]  Huaguang Zhang,et al.  Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP , 2013, IEEE Transactions on Cybernetics.

[37]  Dimitri P. Bertsekas,et al.  Missile defense and interceptor allocation by neuro-dynamic programming , 2000, IEEE Trans. Syst. Man Cybern. Part A.