Robust optimal control for finite-horizon zero-sum differential games via a plug-n-play event-triggered scheme

Abstract This paper proposes a robust optimal control strategy for finite-horizon two-player Zero-Sum(ZS) differential games with partially unknown dynamics by incorporating an event-triggered scheme and the critic-only adaptive dynamic programming(ADP) method. Firstly, an online identifier is designed to reconstruct unknown system dynamics based on the data-driven technique. The identifier is running in the solving process rather than as a priori part of the solution, which simplifies the system structure and decreases the computational cost. To deal with the finite-horizon constraints, a time-varying value function and a additional term are considered to such that the terminal constraint error is minimised. A critic neural network(CNN) is used to solve the event-triggered Hamilton–Jacob—Isaacs (HJI) equation under a plug-n-play structure, which reduces the redundant information transmission as well as receives all measurement information immediately. According to the Lyapunov theory, the uniformly ultimately bounded (UUB) for the event-triggered closed-loop system and the CNN weight error are demonstrated, in the meantime the asymptotic stability of the identifier weight error is proved. Finally, the application in the missile-target interception system validates the feasibility and efficacy of the proposed method.

[1]  Yuzhu Huang,et al.  Neuro-observer based online finite-horizon optimal control for uncertain non-linear continuous-time systems , 2017 .

[2]  S. Tong,et al.  Observer-based direct adaptive fuzzy control of uncertain nonlinear systems and its applications , 2009 .

[3]  Chaoyang Dong,et al.  Spacecraft output feedback attitude control based on extended state observer and adaptive dynamic programming , 2019, J. Frankl. Inst..

[4]  Dongbin Zhao,et al.  Iterative Adaptive Dynamic Programming for Solving Unknown Nonlinear Zero-Sum Game Based on Online Data , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Shiping Wen,et al.  Event-triggered distributed control for synchronization of multiple memristive neural networks under cyber-physical attacks , 2020, Inf. Sci..

[6]  Huaguang Zhang,et al.  An Analysis of IRL-Based Optimal Tracking Control of Unknown Nonlinear Systems with Constrained Input , 2019, Neural Processing Letters.

[7]  Kun Zhang,et al.  Event-Triggered Adaptive Dynamic Programming for Non-Zero-Sum Games of Unknown Nonlinear Systems via Generalized Fuzzy Hyperbolic Models , 2019, IEEE Transactions on Fuzzy Systems.

[8]  Shaocheng Tong,et al.  Adaptive Neural Network Control for Active Suspension Systems With Time-Varying Vertical Displacement and Speed Constraints , 2019, IEEE Transactions on Industrial Electronics.

[9]  Youmin Zhang,et al.  Adaptive Discrete-Time Flight Control Using Disturbance Observer and Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Pingjian Zhang,et al.  Some Results On Two-Person Zero-Sum Linear Quadratic Differential Games , 2005, SIAM J. Control. Optim..

[11]  Frank L. Lewis,et al.  Optimal Control , 1986 .

[12]  Kun Zhang,et al.  Robust Optimal Control Scheme for Unknown Constrained-Input Nonlinear Systems via a Plug-n-Play Event-Sampled Critic-Only Algorithm , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  Chunsheng Liu,et al.  Disturbance observer-based robust missile autopilot design with full-state constraints via adaptive dynamic programming , 2018, J. Frankl. Inst..

[14]  T. Basar,et al.  H∞-0ptimal Control and Related Minimax Design Problems: A Dynamic Game Approach , 1996, IEEE Trans. Autom. Control..

[15]  Derong Liu,et al.  An iterative ϵ-optimal control scheme for a class of discrete-time nonlinear systems with unfixed initial state , 2012, Neural Networks.

[16]  Derong Liu,et al.  Event-Triggered Adaptive Dynamic Programming for Zero-Sum Game of Partially Unknown Continuous-Time Nonlinear Systems , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[17]  Qinglai Wei,et al.  Data-Driven Zero-Sum Neuro-Optimal Control for a Class of Continuous-Time Unknown Nonlinear Systems With Disturbance Using ADP , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Shaocheng Tong,et al.  Fuzzy-Based Multierror Constraint Control for Switched Nonlinear Systems and Its Applications , 2019, IEEE Transactions on Fuzzy Systems.

[19]  Ho-Lim Choi,et al.  Robust control for nonlinear systems with uncertain time-varying parameters coupled with non-triangular terms , 2020, Int. J. Syst. Sci..

[20]  Frank L. Lewis,et al.  Online solution of nonlinear two‐player zero‐sum games using synchronous policy iteration , 2012 .

[21]  Ke Wang,et al.  Single-network ADP for near optimal control of continuous-time zero-sum games without using initial stabilising control laws , 2018 .

[22]  Qing-Long Han,et al.  A Threshold-Parameter-Dependent Approach to Designing Distributed Event-Triggered $H_{\infty}$ Consensus Filters Over Sensor Networks , 2019, IEEE Transactions on Cybernetics.

[23]  D. Liu,et al.  Adaptive Dynamic Programming for Finite-Horizon Optimal Control of Discrete-Time Nonlinear Systems With $\varepsilon$-Error Bound , 2011, IEEE Transactions on Neural Networks.

[24]  Yongguang Yu,et al.  Pinning synchronization of fractional and impulsive complex networks via event-triggered strategy , 2020, Commun. Nonlinear Sci. Numer. Simul..

[25]  Chunsheng Liu,et al.  Finite-horizon differential games for missile–target interception system using adaptive dynamic programming with input constraints , 2018, Int. J. Syst. Sci..

[26]  Xiaohong Cui,et al.  Online finite-horizon optimal learning algorithm for nonzero-sum games with partially unknown dynamics and constrained inputs , 2016, Neurocomputing.

[27]  Guoqiang Li,et al.  Energy management strategy for parallel hybrid electric vehicles based on approximate dynamic programming and velocity forecast , 2019, J. Frankl. Inst..

[28]  Tal Shima,et al.  Cooperative Differential Games Strategies for Active Aircraft Protection from a Homing Missile , 2010 .

[29]  Qichao Zhang,et al.  Event-Based Robust Control for Uncertain Nonlinear Systems Using Adaptive Dynamic Programming , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Chaoyang Dong,et al.  Morphing aircraft control based on switched nonlinear systems and adaptive dynamic programming , 2019, Aerospace Science and Technology.

[31]  Qing Ye,et al.  Robust differential game guidance laws design for uncertain interceptor-target engagement via adaptive dynamic programming , 2017, Int. J. Control.

[32]  Hao Xu,et al.  Finite-Horizon Near Optimal Design of Nonlinear Two-Player Zero-Sum Game in Presence of Completely Unknown Dynamics , 2015 .

[33]  Xiong Yang,et al.  Adaptive Critic Designs for Event-Triggered Robust Control of Nonlinear Systems With Unknown Dynamics , 2019, IEEE Transactions on Cybernetics.

[34]  Huaguang Zhang,et al.  Leader-Based Optimal Coordination Control for the Consensus Problem of Multiagent Differential Games via Fuzzy Adaptive Dynamic Programming , 2015, IEEE Transactions on Fuzzy Systems.

[35]  T. Shima,et al.  Linear Quadratic Differential Games Guidance Law for Dual Controlled Missiles , 2007, IEEE Transactions on Aerospace and Electronic Systems.

[36]  Kun Zhang,et al.  Data-driven adaptive dynamic programming schemes for non-zero-sum games of unknown discrete-time nonlinear systems , 2018, Neurocomputing.

[37]  Frank L. Lewis,et al.  Neural Network Control Of Robot Manipulators And Non-Linear Systems , 1998 .

[38]  Debasish Ghose,et al.  Nonlinear Differential Games-Based Impact-Angle-Constrained Guidance Law , 2015 .

[39]  Frank L. Lewis,et al.  Robust Optimal Control for Disturbed Nonlinear Zero-Sum Differential Games Based on Single NN and Least Squares , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[40]  Shan Liang,et al.  Data-Based Online Optimal Temperature Tracking Control in Continuous Microwave Heating System by Adaptive Dynamic Programming , 2019, Neural Processing Letters.

[41]  Frank L. Lewis,et al.  Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics , 2014, Autom..

[42]  Jing Na,et al.  Online optimal solutions for multi-player nonzero-sum game with completely unknown dynamics , 2017, Neurocomputing.

[43]  Kun Zhang,et al.  Iterative adaptive dynamic programming methods with neural network implementation for multi-player zero-sum games , 2018, Neurocomputing.

[44]  Josef Shinar,et al.  Robust trajectory tracking: differential game/cheap control approach , 2014, Int. J. Syst. Sci..

[45]  Chunsheng Liu,et al.  Adaptive periodic event-triggered control for missile-target interception system with finite-horizon convergence , 2020, Trans. Inst. Meas. Control.