Adaptive Tracking Control of Nonlinear Systems Subject to Matched Uncertainties

In this chapter, an adaptive tracking control scheme is designed for a class of continuous-time uncertain nonlinear systems based on the approximate solution of the HJB equation. Considering matched uncertainties, the tracking control of the continuous-time uncertain nonlinear system can be transformed to the optimal tracking control of the associated nominal system. By building the nominal error system and modifying its cost function, the solution of the relevant HJB equation can be contributed to the adaptive tracking control of the continuous-time uncertain nonlinear system. In view of the complexity on solving the HJB equation, its approximate solution is pursued by the policy iteration algorithm under the ADP framework, where a critic neural network is constructed to approximate the optimal cost function. Therein, an action network is used to directly calculate the approximate optimal control law, which constitutes the tracking control law for the original uncertain system together with the steady control law. The weight convergence of the critic network and the stability of the closed-loop system are provided as the theoretical guarantee based on the Lyapunov theory. Two simulation examples are studied to verify the theoretical results and the effectiveness of the proposed tracking control scheme.

[1]  Kwang Y. Lee,et al.  An optimal tracking neuro-controller for nonlinear dynamic systems , 1996, IEEE Trans. Neural Networks.

[2]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[3]  Frank L. Lewis,et al.  Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning , 2014, Autom..

[4]  Jorge Davila,et al.  Exact Tracking Using Backstepping Control Design and High-Order Sliding Modes , 2013, IEEE Transactions on Automatic Control.

[5]  Huaguang Zhang,et al.  A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Derong Liu,et al.  Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach , 2012, Neurocomputing.

[7]  Haibo He,et al.  Data-Driven Tracking Control With Adaptive Dynamic Programming for a Class of Continuous-Time Nonlinear Systems , 2017, IEEE Transactions on Cybernetics.

[8]  Sarangapani Jagannathan,et al.  Approximate optimal distributed control of uncertain nonlinear interconnected systems with event-sampled feedback , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[9]  Stephen P. Banks,et al.  Nonlinear optimal tracking control with application to super-tankers for autopilot design , 2004, Autom..

[10]  Haibo He,et al.  A three-network architecture for on-line learning and optimization based on adaptive dynamic programming , 2012, Neurocomputing.

[11]  Rolf Findeisen,et al.  Nonlinear Model Predictive Control for Constrained Output Path Following , 2015, IEEE Transactions on Automatic Control.

[12]  Sarangapani Jagannathan,et al.  Optimal tracking control of affine nonlinear discrete-time systems with unknown internal dynamics , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[13]  F.L. Lewis,et al.  Reinforcement learning and adaptive dynamic programming for feedback control , 2009, IEEE Circuits and Systems Magazine.

[14]  Avimanyu Sahoo,et al.  Approximate Optimal Control of Affine Nonlinear Continuous-Time Systems Using Event-Sampled Neurodynamic Programming , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Warren E. Dixon,et al.  Model-based reinforcement learning for infinite-horizon approximate optimal tracking , 2014, 53rd IEEE Conference on Decision and Control.

[16]  Shaocheng Tong,et al.  Adaptive NN Controller Design for a Class of Nonlinear MIMO Discrete-Time Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Lyle Noakes,et al.  Continuous-Time Adaptive Critics , 2007, IEEE Transactions on Neural Networks.

[18]  Shuzhi Sam Ge,et al.  Adaptive control of a class of discrete-time MIMO nonlinear systems with uncertain couplings , 2010, Int. J. Control.

[19]  Paul J. Werbos,et al.  Consistency of HDP applied to a simple reinforcement learning problem , 1990, Neural Networks.

[20]  R. Bellman.  Dynamic programming. , 1957, Science.

[21]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[22]  Shaocheng Tong,et al.  Fuzzy Approximation-Based Adaptive Backstepping Optimal Control for a Class of Nonlinear Discrete-Time Systems With Dead-Zone , 2016, IEEE Transactions on Fuzzy Systems.

[23]  Chaoxu Mu,et al.  Neural-network-based adaptive guaranteed cost control of nonlinear dynamical systems with matched uncertainties , 2017, Neurocomputing.

[24]  Lei Yang,et al.  Direct Heuristic Dynamic Programming for Nonlinear Tracking Control With Filtered Tracking Error , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Saleh Mobayen,et al.  An adaptive chattering-free PID sliding mode control based on dynamic sliding manifolds for a class of uncertain nonlinear systems , 2015 .

[26]  Jennie Si,et al.  Online learning control by association and reinforcement. , 2001, IEEE transactions on neural networks.

[27]  Frank L. Lewis,et al.  Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2010, Autom..

[28]  Shaocheng Tong,et al.  Adaptive NN Tracking Control of Uncertain Nonlinear Discrete-Time Systems With Nonaffine Dead-Zone Input , 2015, IEEE Transactions on Cybernetics.

[29]  Changyin Sun,et al.  Adaptive tracking control for a class of continuous-time uncertain nonlinear systems using the approximate solution of HJB equation , 2017, Neurocomputing.

[30]  Derong Liu,et al.  Data-Based Adaptive Critic Designs for Nonlinear Robust Optimal Control With Uncertain Dynamics , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[31]  Derong Liu,et al.  Action-dependent adaptive critic designs , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[32]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[33]  Long Cheng,et al.  Tracking Control of a Closed-Chain Five-Bar Robot With Two Degrees of Freedom by Integration of an Approximation-Based Approach and Mechanical Design , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).