Event-triggered single-network ADP method for constrained optimal tracking control of continuous-time non-linear systems

Abstract This paper investigates the optimal tracking control problem (OTCP) for continuous-time non-linear systems with input constraints. A novel event-triggered single-network adaptive dynamic programming (ADP) method is proposed to obtain the solution of constrained OTCP. By constructing an augmented system and introducing a novel discounted non-quadratic cost function, an event-triggered constrained tracking Hamilton–Jacobi–Bellman equation is formulated. Then, only a critic neural network (NN) is employed to learn the optimal value function and further obtain the optimal tracking controller, which enables the architecture of ADP implementation to be simpler. And a novel NN weights updating law is constructed, by which the restriction of initial admissible control is removed. Based on the Lyapunov theory, the convergence of critic NN weights and the stability of closed-loop system are demonstrated. The derived optimal tracking controller is updated only at the event-triggered instants decided by the designed event-triggered condition. Therefore, the communication burden can be reduced effectively. Finally, two simulation examples are given to verify the effectiveness of proposed method.

[1]  Xinghuo Yu,et al.  Quantized feedback sliding-mode control: An event-triggered approach , 2018, Autom..

[2]  Haibo He,et al.  Event-Triggered Optimal Control for Partially Unknown Constrained-Input Systems via Adaptive Dynamic Programming , 2017, IEEE Transactions on Industrial Electronics.

[3]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[4]  Tianyou Chai,et al.  Online Solution of Two-Player Zero-Sum Games for Continuous-Time Nonlinear Systems With Completely Unknown Dynamics , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Frank L. Lewis,et al.  Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Heng Wang,et al.  Decentralized adaptive fuzzy tracking control for a class of uncertain large-scale systems with actuator nonlinearities , 2018, Appl. Math. Comput..

[7]  Warren E. Dixon,et al.  Approximate optimal trajectory tracking for continuous-time nonlinear systems , 2013, Autom..

[8]  Jinde Cao,et al.  Generalized State Estimation for Markovian Coupled Networks Under Round-Robin Protocol and Redundant Channels , 2019, IEEE Transactions on Cybernetics.

[9]  Huaguang Zhang,et al.  Leader-Based Optimal Coordination Control for the Consensus Problem of Multiagent Differential Games via Fuzzy Adaptive Dynamic Programming , 2015, IEEE Transactions on Fuzzy Systems.

[10]  Haibo He,et al.  Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints , 2017, IEEE Trans. Neural Networks Learn. Syst..

[11]  Sarangapani Jagannathan,et al.  Event-Triggered Distributed Control of Nonlinear Interconnected Systems Using Online Reinforcement Learning With Exploration , 2018, IEEE Transactions on Cybernetics.

[12]  Frank L. Lewis,et al.  Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games , 2015, Inf. Sci..

[13]  Shuzhi Sam Ge,et al.  Adaptive tracking control of uncertain MIMO nonlinear systems with input constraints , 2011, Autom..

[14]  Derong Liu,et al.  An Approximate Optimal Control Approach for Robust Stabilization of a Class of Discrete-Time Nonlinear Systems With Uncertainties , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[15]  Frank L. Lewis,et al.  Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning , 2014, Autom..

[16]  Qinglai Wei,et al.  Data-Driven Zero-Sum Neuro-Optimal Control for a Class of Continuous-Time Unknown Nonlinear Systems With Disturbance Using ADP , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Derong Liu,et al.  Neural robust stabilization via event-triggering mechanism and adaptive learning technique , 2018, Neural Networks.

[18]  Chao Shen,et al.  Trajectory Tracking Control of an Autonomous Underwater Vehicle Using Lyapunov-Based Model Predictive Control , 2018, IEEE Transactions on Industrial Electronics.

[19]  Avimanyu Sahoo,et al.  Approximate Optimal Control of Affine Nonlinear Continuous-Time Systems Using Event-Sampled Neurodynamic Programming , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Minrui Fei,et al.  A two-layer networked learning control system using actor-critic neural network , 2008, Appl. Math. Comput..

[21]  K. Vamvoudakis Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems , 2014, IEEE/CAA Journal of Automatica Sinica.

[22]  Shengyuan Xu,et al.  Slow State Variables Feedback Stabilization for Semi-Markov Jump Systems With Singular Perturbations , 2018, IEEE Transactions on Automatic Control.

[23]  Frank L. Lewis,et al.  Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning , 2014, IEEE Transactions on Automatic Control.

[24]  Jie Ma,et al.  The derivation of iterative convergence calculation for a nonlinear MIMO approximate dynamic programming approach , 2013, Appl. Math. Comput..

[25]  Ding Wang,et al.  Adaptive-Critic-Based Robust Trajectory Tracking of Uncertain Dynamics and Its Application to a Spring–Mass–Damper System , 2018, IEEE Transactions on Industrial Electronics.

[26]  Huaguang Zhang,et al.  Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP , 2013, IEEE Transactions on Cybernetics.

[27]  Derong Liu,et al.  Guaranteed cost neural tracking control for a class of uncertain nonlinear systems using adaptive dynamic programming , 2016, Neurocomputing.

[28]  Derong Liu,et al.  Adaptive Dynamic Programming for Discrete-Time Zero-Sum Games , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Tingwen Huang,et al.  Reinforcement learning solution for HJB equation arising in constrained optimal control problem , 2015, Neural Networks.

[30]  Shuzhi Sam Ge,et al.  Adaptive Neural Network Tracking Control of MIMO Nonlinear Systems With Unknown Dead Zones and Control Directions , 2009, IEEE Transactions on Neural Networks.

[31]  Dongbin Zhao,et al.  Using reinforcement learning techniques to solve continuous-time non-linear optimal tracking problem without system dynamics , 2016 .

[32]  Frank L. Lewis,et al.  Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2010, Autom..

[33]  Gang Sun,et al.  A DSC approach to adaptive neural network tracking control for pure-feedback nonlinear systems , 2013, Appl. Math. Comput..

[34]  K. Vamvoudakis,et al.  Event‐triggered optimal tracking control of nonlinear systems , 2017 .

[35]  Xiong Yang,et al.  Adaptive Critic Designs for Event-Triggered Robust Control of Nonlinear Systems With Unknown Dynamics , 2019, IEEE Transactions on Cybernetics.

[36]  Derong Liu,et al.  Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints , 2015, IEEE Transactions on Cybernetics.