Event-trigger-based robust control for nonlinear constrained-input systems using reinforcement learning method

Abstract In this paper, an online integral reinforcement learning strategy is proposed to deal with robust constrained control problems using event-triggered mechanism for nonlinear Continuous-Time (C-T) systems with external disturbances. The novel design of constrained control law is addressed together with the adaptive event-triggered condition by guaranteeing the optimal performance and system stability. An adaptive online actor-critic Neural Network (NN) reinforcement learning scheme is developed to approximate the optimal solution of the complicated Hamilton–Jacobi–Isaacs equation. Meanwhile, the convergence of NN weight errors and the event-triggered closed-loop system stability are demonstrated to be uniform ultimate bounded by Lyapunov analysis under the proposed triggering condition. Moreover, event-triggered H∞ tracking control with input constrains and limited network communication is also presented by establishing an augmented system. Finally, simulation results are provided to show the algorithm validity.

[1]  T. Basar,et al.  H/sup /spl infin//-optimal tracking control techniques for nonlinear underactuated systems , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[2]  Frank L. Lewis,et al.  $ {H}_{ {\infty }}$ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Kun Zhang,et al.  Iterative adaptive dynamic programming methods with neural network implementation for multi-player zero-sum games , 2018, Neurocomputing.

[4]  Derong Liu,et al.  Online Synchronous Approximate Optimal Learning Algorithm for Multi-Player Non-Zero-Sum Games With Unknown Dynamics , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[5]  Shaocheng Tong,et al.  Neural Networks-Based Adaptive Finite-Time Fault-Tolerant Control for a Class of Strict-Feedback Switched Nonlinear Systems , 2019, IEEE Transactions on Cybernetics.

[6]  Huaguang Zhang,et al.  Neural network-based online H∞ control for discrete-time affine nonlinear system using adaptive dynamic programming , 2016, Neurocomputing.

[7]  Haibo He,et al.  Event-Triggered Optimal Control for Partially Unknown Constrained-Input Systems via Adaptive Dynamic Programming , 2017, IEEE Transactions on Industrial Electronics.

[8]  Yan-Jun Liu,et al.  Adaptive neural network-based control for a class of nonlinear pure-feedback systems with time-varying full state constraints , 2018, IEEE/CAA Journal of Automatica Sinica.

[9]  Linda Bushnell,et al.  Stability analysis of networked control systems , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[10]  Bin Jiang,et al.  Online Adaptive Policy Learning Algorithm for $H_{\infty }$ State Feedback Control of Unknown Affine Nonlinear Discrete-Time Systems , 2014, IEEE Transactions on Cybernetics.

[11]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[12]  Huaguang Zhang,et al.  An Overview of Research on Adaptive Dynamic Programming , 2013, Acta Automatica Sinica.

[13]  Haibo He,et al.  Event-triggered adaptive dynamic programming for continuous-time nonlinear system using measured input-output data , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[14]  Yang Liu,et al.  Neural network-based H∞ sliding mode control for nonlinear systems with actuator faults and unmatched disturbances , 2018, Neurocomputing.

[15]  Derong Liu,et al.  Data-based robust adaptive control for a class of unknown nonlinear constrained-input systems via integral reinforcement learning , 2016, Inf. Sci..

[16]  Dong Yue,et al.  Event-triggered real-time scheduling stabilization of discrete-time Takagi-Sugeno fuzzy systems via a new weighted matrix approach , 2018, Inf. Sci..

[17]  Shaocheng Tong,et al.  Observer-based fuzzy adaptive optimal stabilization control for completely unknown nonlinear interconnected systems , 2018, Neurocomputing.

[18]  Shaocheng Tong,et al.  Data-based adaptive neural network optimal output feedback control for nonlinear systems with actuator saturation , 2017, Neurocomputing.

[19]  Dong-Juan Li,et al.  Adaptive Neural Tracking Control for an Uncertain State Constrained Robotic Manipulator With Unknown Time-Varying Delays , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[20]  Haibo He,et al.  Event-Driven Adaptive Robust Control of Nonlinear Systems With Uncertainties Through NDP Strategy , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[21]  Jing Wu,et al.  An event-trigger two-stage architecture for voltage control in distribution systems , 2018 .

[22]  Wei Liu,et al.  Cooperative global robust output regulation for a class of nonlinear multi-agent systems by distributed event-triggered control , 2017, Autom..

[23]  K. Vamvoudakis Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems , 2014, IEEE/CAA Journal of Automatica Sinica.

[24]  K. Vamvoudakis,et al.  Event‐triggered optimal tracking control of nonlinear systems , 2017 .

[25]  Shaocheng Tong,et al.  Adaptive control-based Barrier Lyapunov Functions for a class of stochastic nonlinear systems with full state constraints , 2018, Autom..

[26]  Gang Feng,et al.  Observer-Based Output Feedback Event-Triggered Control for Consensus of Multi-Agent Systems , 2014, IEEE Transactions on Industrial Electronics.

[27]  Frank L. Lewis,et al.  Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[28]  Haibo He,et al.  Gr-GDHP: A New Architecture for Globalized Dual Heuristic Dynamic Programming , 2017, IEEE Transactions on Cybernetics.

[29]  Qichao Zhang,et al.  Event-Triggered $H_\infty $ Control for Continuous-Time Nonlinear System via Concurrent Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[30]  Frank L. Lewis,et al.  Optimal tracking control of nonlinear partially-unknown constrained-input systems using integral reinforcement learning , 2014, Autom..

[31]  Frank L. Lewis,et al.  Actor–Critic-Based Optimal Tracking for Partially Unknown Nonlinear Discrete-Time Systems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Eugênio B. Castelan,et al.  Event-triggered tracking control of unicycle mobile robots , 2015, Autom..

[33]  Tingwen Huang,et al.  Off-Policy Reinforcement Learning for $ H_\infty $ Control Design , 2013, IEEE Transactions on Cybernetics.

[34]  Zongli Lin,et al.  Output feedback Q-learning for discrete-time linear zero-sum games with application to the H-infinity control , 2018, Autom..

[35]  Dong Yue,et al.  Relaxed Control Design of Discrete-Time Takagi–Sugeno Fuzzy Systems: An Event-Triggered Real-Time Scheduling Approach , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[36]  Yun Ho Choi,et al.  Robust event-driven tracking control with preassigned performance for uncertain input-quantized nonlinear pure-feedback systems , 2018, J. Frankl. Inst..

[37]  Qichao Zhang,et al.  Event-Triggered H ∞ Control for Continuous-Time Nonlinear System , 2015, ISNN.

[38]  Dongbin Zhao,et al.  Event-Based Robust Control for Uncertain Nonlinear Systems Using Adaptive Dynamic Programming. , 2018, IEEE transactions on neural networks and learning systems.

[39]  Derong Liu,et al.  Neural Network H ∞ Tracking Control of Nonlinear Systems Using GHJI Method , 2013, ISNN.

[40]  Huaguang Zhang,et al.  Data-Driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems With Unknown Dynamics Using Reinforcement Learning Method , 2017, IEEE Transactions on Industrial Electronics.

[41]  Kyriakos G. Vamvoudakis,et al.  Model-free event-triggered control algorithm for continuous-time linear systems with optimal performance , 2018, Autom..

[42]  Pavankumar Tallapragada,et al.  On Event Triggered Tracking for Nonlinear Systems , 2013, IEEE Transactions on Automatic Control.

[43]  F. Lewis,et al.  Online solution of nonquadratic two‐player zero‐sum games arising in the H ∞  control of constrained input systems , 2014 .

[44]  Haibo He,et al.  Event-Triggered Adaptive Dynamic Programming for Continuous-Time Systems With Control Constraints , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Dong Liu,et al.  Event-based model-free adaptive control for discrete-time non-linear processes , 2017 .