Adaptive tracking control for a class of continuous-time uncertain nonlinear systems using the approximate solution of HJB equation

Abstract In this paper, an adaptive tracking control scheme is designed for a class of continuous-time uncertain nonlinear systems based on the approximate solution of the Hamilton–Jacobi–Bellman (HJB) equation. Considering matched uncertainties, the tracking control of the continuous-time uncertain nonlinear system can be transformed to the optimal tracking control of the associated nominal system. By building the nominal error system and modifying its cost function, the solution of the relevant HJB equation can be contributed to the adaptive tracking control of the continuous-time uncertain nonlinear system. In view of the complexity on solving the HJB equation, its approximate solution is pursued by the policy iteration algorithm under the adaptive dynamic programming (ADP) framework, where a critic neural network is constructed to approximate the optimal cost function, and an action network is used to directly calculate the approximate optimal control law, which constitutes the tracking control law for the original uncertain system together with the steady control law. The weight convergence of the critic network and the stability of the closed-loop system are provided as the theoretical guarantee based on the Lyapunov theory. Two simulation examples are studied to verify the theoretical results and the effectiveness of the proposed tracking control scheme.

[1]  Lei Yang,et al.  Direct Heuristic Dynamic Programming for Nonlinear Tracking Control With Filtered Tracking Error , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  Frank L. Lewis,et al.  Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2010, Autom..

[3]  Jennie Si,et al.  Online learning control by association and reinforcement. , 2001, IEEE transactions on neural networks.

[4]  Rolf Findeisen,et al.  Nonlinear Model Predictive Control for Constrained Output Path Following , 2015, IEEE Transactions on Automatic Control.

[5]  Sarangapani Jagannathan,et al.  Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamics , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[6]  Haibo He,et al.  A three-network architecture for on-line learning and optimization based on adaptive dynamic programming , 2012, Neurocomputing.

[7]  Haibo He,et al.  Data-Driven Tracking Control With Adaptive Dynamic Programming for a Class of Continuous-Time Nonlinear Systems , 2017, IEEE Transactions on Cybernetics.

[8]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[9]  Chaoxu Mu,et al.  Neural-network-based adaptive guaranteed cost control of nonlinear dynamical systems with matched uncertainties , 2017, Neurocomputing.

[10]  Saleh Mobayen,et al.  An adaptive chattering-free PID sliding mode control based on dynamic sliding manifolds for a class of uncertain nonlinear systems , 2015 .

[11]  Shaocheng Tong,et al.  Adaptive NN Controller Design for a Class of Nonlinear MIMO Discrete-Time Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Lyle Noakes,et al.  Continuous-Time Adaptive Critics , 2007, IEEE Transactions on Neural Networks.

[13]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[14]  Shuzhi Sam Ge,et al.  Adaptive control of a class of discrete-time MIMO nonlinear systems with uncertain couplings , 2010, Int. J. Control.

[15]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[16]  Shaocheng Tong,et al.  Adaptive NN Tracking Control of Uncertain Nonlinear Discrete-Time Systems With Nonaffine Dead-Zone Input , 2015, IEEE Transactions on Cybernetics.

[17]  Paul J. Werbos,et al.  Consistency of HDP applied to a simple reinforcement learning problem , 1990, Neural Networks.

[18]  R. Bellman.  Dynamic programming. , 1957, Science.

[19]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[20]  Shaocheng Tong,et al.  Fuzzy Approximation-Based Adaptive Backstepping Optimal Control for a Class of Nonlinear Discrete-Time Systems With Dead-Zone , 2016, IEEE Transactions on Fuzzy Systems.

[21]  Avimanyu Sahoo,et al.  Approximate Optimal Control of Affine Nonlinear Continuous-Time Systems Using Event-Sampled Neurodynamic Programming , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Derong Liu,et al.  Data-Based Adaptive Critic Designs for Nonlinear Robust Optimal Control With Uncertain Dynamics , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[23]  Derong Liu,et al.  Action-dependent adaptive critic designs , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[24]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[25]  F.L. Lewis,et al.  Reinforcement learning and adaptive dynamic programming for feedback control , 2009, IEEE Circuits and Systems Magazine.

[26]  Frank L. Lewis,et al.  Neural Networks in Feedback Control Systems , 2015 .

[27]  Long Cheng,et al.  Tracking Control of a Closed-Chain Five-Bar Robot With Two Degrees of Freedom by Integration of an Approximation-Based Approach and Mechanical Design , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Jorge Davila,et al.  Exact Tracking Using Backstepping Control Design and High-Order Sliding Modes , 2013, IEEE Transactions on Automatic Control.

[29]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[30]  Kwang Y. Lee,et al.  An optimal tracking neuro-controller for nonlinear dynamic systems , 1996, IEEE Trans. Neural Networks.

[31]  Frank L. Lewis,et al.  Optimal Control , 1986 .

[32]  Frank L. Lewis,et al.  Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning , 2014, Autom..

[33]  Huaguang Zhang,et al.  A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[34]  Derong Liu,et al.  Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach , 2012, Neurocomputing.

[35]  Sarangapani Jagannathan,et al.  Approximate optimal distributed control of uncertain nonlinear interconnected systems with event-sampled feedback , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[36]  Stephen P. Banks,et al.  Nonlinear optimal tracking control with application to super-tankers for autopilot design , 2004, Autom..

[37]  Warren E. Dixon,et al.  Model-based reinforcement learning for infinite-horizon approximate optimal tracking , 2014, 53rd IEEE Conference on Decision and Control.