H∞ Tracking Control of Discrete-Time System With Delays via Data-Based Adaptive Dynamic Programming

In this article, the <inline-formula> <tex-math notation="LaTeX">$H_{\infty }$ </tex-math></inline-formula> tracking control problem for a class of discrete time-delay systems is studied using data-based adaptive dynamic programming (ADP) algorithm. First, the controlled system and the reference system are combined to form an augmented discrete time-delay system. Second, we transform it into a discrete time-delay system represented by system measured data, which allows the system state to be completely replaced by the data generated during system operation. Third, a novel data-based Bellman equation is derived according to the Bellman optimality principle. Then, an ADP-based <inline-formula> <tex-math notation="LaTeX">$H_{\infty }$ </tex-math></inline-formula> tracking control method is designed by means of the measured data. The simulation example demonstrates the effectiveness of the data-based ADP control method proposed in this article.

[1]  Zongli Lin,et al.  Output feedback Q-learning for discrete-time linear zero-sum games with application to the H-infinity control , 2018, Autom..

[2]  Derong Liu,et al.  An Indirect Data-Driven Method for Trajectory Tracking Control of a Class of Nonlinear Discrete-Time Systems , 2017, IEEE Transactions on Industrial Electronics.

[3]  Frank L. Lewis,et al.  Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning , 2014, IEEE Transactions on Automatic Control.

[4]  Frank L. Lewis,et al.  Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control , 2007, Autom..

[5]  Chaomin Luo,et al.  Discrete-Time Nonzero-Sum Games for Multiplayer Using Policy-Iteration-Based Adaptive Dynamic Programming Algorithms , 2017, IEEE Transactions on Cybernetics.

[6]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[7]  Frank L. Lewis,et al.  Optimal Tracking Control of Unknown Discrete-Time Linear Systems Using Input-Output Measured Data , 2015, IEEE Transactions on Cybernetics.

[8]  Huaguang Zhang,et al.  Global Asymptotic Stability of Recurrent Neural Networks With Multiple Time-Varying Delays , 2008, IEEE Transactions on Neural Networks.

[9]  Yang Xiong,et al.  Adaptive Dynamic Programming with Applications in Optimal Control , 2017 .

[10]  Qinglai Wei,et al.  Adaptive Dynamic Programming-Based Optimal Control Scheme for Energy Storage Systems With Solar Renewable Energy , 2017, IEEE Transactions on Industrial Electronics.

[11]  Derong Liu,et al.  Neural-network-based adaptive optimal tracking control scheme for discrete-time nonlinear systems with approximation errors , 2013, Neurocomputing.

[12]  Qichao Zhang,et al.  Data-driven adaptive dynamic programming for continuous-time fully cooperative games with partially constrained inputs , 2017, Neurocomputing.

[13]  Bruno Iannazzo,et al.  Numerical Solution of Algebraic Riccati Equations , 2012, Fundamentals of algorithms.

[14]  Huaguang Zhang,et al.  Networked Synchronization Control of Coupled Dynamic Networks With Time-Varying Delay , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Bakht Zada,et al.  On uniform exponential stability of linear switching system , 2018, Mathematical Methods in the Applied Sciences.

[16]  Frank L. Lewis,et al.  Robust Optimal Control for Disturbed Nonlinear Zero-Sum Differential Games Based on Single NN and Least Squares , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[17]  Xiong Yang,et al.  Adaptive Critic Designs for Event-Triggered Robust Control of Nonlinear Systems With Unknown Dynamics , 2019, IEEE Transactions on Cybernetics.

[18]  Yang Li,et al.  Adaptive Neural Network Control of AUVs With Control Input Nonlinearities Using Reinforcement Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[19]  Haibo He,et al.  Novel iterative neural dynamic programming for data-based approximate optimal control design , 2017, Autom..

[20]  Frank L. Lewis,et al.  Discrete-Time Deterministic $Q$ -Learning: A Novel Convergence Analysis , 2017, IEEE Transactions on Cybernetics.

[21]  Changyin Sun,et al.  ADP-Based Robust Tracking Control for a Class of Nonlinear Systems With Unmatched Uncertainties , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[22]  Huaguang Zhang,et al.  Data-driven optimal tracking control for discrete-time systems with delays using adaptive dynamic programming , 2018, J. Frankl. Inst..

[23]  Frank L. Lewis,et al.  Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  Derong Liu,et al.  Neural-network-based optimal tracking control scheme for a class of unknown discrete-time nonlinear systems using iterative ADP algorithm , 2014, Neurocomputing.

[25]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[26]  Yang Liu,et al.  Data-Based Adaptive Dynamic Programming for a Class of Discrete-Time Systems With Multiple Delays , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[27]  Derong Liu,et al.  Output Tracking Control Based on Adaptive Dynamic Programming With Multistep Policy Evaluation , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[28]  Haibo He,et al.  Adaptive Critic Learning and Experience Replay for Decentralized Event-Triggered Control of Nonlinear Interconnected Systems , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[29]  Huaguang Zhang,et al.  Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming , 2014, Int. J. Control.

[30]  Radac,et al.  Data-Driven Model-Free Tracking Reinforcement Learning Control with VRFT-based Adaptive Actor-Critic , 2019, Applied Sciences.

[31]  Huaguang Zhang,et al.  Nearly data-based optimal control for linear discrete model-free systems with delays via reinforcement learning , 2016, Int. J. Syst. Sci..

[32]  Yang Liu,et al.  ADP based optimal tracking control for a class of linear discrete-time system with multiple delays , 2016, Journal of the Franklin Institute.

[33]  J. Willems.  Least squares stationary optimal control and the algebraic Riccati equation , 1971 .

[34]  Seung-Hoon Lee,et al.  Uncertainty and Disturbance Estimator-Based Tracking Control for Fuzzy Systems , 2018, 2018 18th International Conference on Control, Automation and Systems (ICCAS).

[35]  Frank L. Lewis,et al.  H∞ control of linear discrete-time systems: Off-policy reinforcement learning , 2017, Autom..

[36]  Huaguang Zhang,et al.  Decentralized adaptive tracking control scheme for nonlinear large-scale interconnected systems via adaptive dynamic programming , 2017, Neurocomputing.

[37]  Derong Liu,et al.  Adaptive Dynamic Programming for Optimal Tracking Control of Unknown Nonlinear Systems With Application to Coal Gasification , 2014, IEEE Transactions on Automation Science and Engineering.

[38]  Frank L. Lewis,et al.  Adaptive Critic Designs for Discrete-Time Zero-Sum Games With Application to $H_{\infty}$ Control , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[39]  Doreen Eichel,et al.  Adaptive Dynamic Programming For Control Algorithms And Stability , 2016 .

[40]  Yu Liu,et al.  Optimal constrained self-learning battery sequential management in microgrid via adaptive dynamic programming , 2017, IEEE/CAA Journal of Automatica Sinica.

[41]  Huaguang Zhang,et al.  Data-Driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems With Unknown Dynamics Using Reinforcement Learning Method , 2017, IEEE Transactions on Industrial Electronics.

[42]  Frank L. Lewis,et al.  Actor–Critic-Based Optimal Tracking for Partially Unknown Nonlinear Discrete-Time Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Jinde Cao,et al.  Uniform exponential stability of periodic discrete switched linear system , 2017, J. Frankl. Inst..

[44]  Kun Zhang,et al.  Robust Optimal Control Scheme for Unknown Constrained-Input Nonlinear Systems via a Plug-n-Play Event-Sampled Critic-Only Algorithm , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[45]  Yuzhu Huang,et al.  Bounded robust control design for uncertain nonlinear systems using single-network adaptive dynamic programming , 2017, Neurocomputing.

[46]  Jinyu Wen,et al.  Adaptive Learning in Tracking Control Based on the Dual Critic Network Design , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[47]  Muhammad Arif,et al.  Criteria for the exponential stability of linear evolution difference equations , 2018, IMA J. Math. Control. Inf..

[48]  Yang Liu,et al.  Model‐free optimal tracking control for discrete‐time system with delays using reinforcement Q ‐learning , 2018, Electronics Letters.

[49]  Frank L. Lewis,et al.  Mixed Iterative Adaptive Dynamic Programming for Optimal Battery Energy Control in Smart Residential Microgrids , 2017, IEEE Transactions on Industrial Electronics.

[50]  Huaguang Zhang,et al.  Optimal Guaranteed Cost Sliding Mode Control for Constrained-Input Nonlinear Systems With Matched and Unmatched Disturbances , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[51]  Bin Wang,et al.  Dual Heuristic dynamic Programming for nonlinear discrete-time uncertain systems with state delay , 2014, Neurocomputing.

[52]  Huaguang Zhang,et al.  Model-free optimal control design for a class of linear discrete-time systems with multiple delays using adaptive dynamic programming , 2014, Neurocomputing.