Adaptive learning: robust stabilization of two-player games with unmodeled dynamics

Consider the inherent existence of unmodeled dynamics when identifying system models, it is worth investigating control to guarantee the robust stability of the systems. This paper focuses on robust control for two-player time-invariant difference game with uncertain unmodeled dynamics by using adaptive learning way. To this end, the optimization control problem for two-player linear difference games with bounded unmodeled dynamics is formulated first. Then, dynamic programming combined with game theory and adaptive critic learning are employed for the purpose of finding the stabilizing control polices, such that an adaptive learning method is developed for stabilizing the closed-loop two-player dynamics with unmodeled dynamics. The robust stabilization of the uncertain two-player game systems is rigorously proved. Simulations are given to show the effectiveness of the proposed method.

[1]  Frank L. Lewis,et al.  Game Theory-Based Control System Algorithms with Real-Time Reinforcement Learning: How to Solve Multiplayer Games Online , 2017, IEEE Control Systems.

[2]  Stephen B. Wicker,et al.  Game theory in communications: motivation, explanation, and application to power control , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[3]  Huaguang Zhang,et al.  Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP , 2013, IEEE Transactions on Cybernetics.

[4]  Derong Liu,et al.  Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints , 2015, IEEE Transactions on Cybernetics.

[5]  Frank L. Lewis,et al.  Off-Policy Q-Learning: Set-Point Design for Optimizing Dual-Rate Rougher Flotation Operational Processes , 2018, IEEE Transactions on Industrial Electronics.

[6]  Zhengtao Ding,et al.  Fixed-Time Formation Control of Multirobot Systems: Design and Experiments , 2019, IEEE Transactions on Industrial Electronics.

[7]  Qinglai Wei,et al.  Data-Driven Zero-Sum Neuro-Optimal Control for a Class of Continuous-Time Unknown Nonlinear Systems With Disturbance Using ADP , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Haibo He,et al.  Robust controller design of continuous-time nonlinear system using neural network , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[9]  Frank L. Lewis,et al.  Adaptive Critic Designs for Discrete-Time Zero-Sum Games With Application to $H_{\infty}$ Control , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Frank L. Lewis,et al.  Off-Policy Reinforcement Learning for Synchronization in Multiagent Graphical Games , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Frank L. Lewis,et al.  $ {H}_{ {\infty }}$ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Zhengtao Ding,et al.  Predictor-Based Extended-State-Observer Design for Consensus of MASs With Delays and Disturbances , 2019, IEEE Transactions on Cybernetics.

[13]  Walid Saad,et al.  Game-Theoretic Methods for the Smart Grid: An Overview of Microgrid Systems, Demand-Side Management, and Smart Grid Communications , 2012, IEEE Signal Processing Magazine.

[14]  Frank L. Lewis,et al.  Off-Policy Interleaved $Q$ -Learning: Optimal Control for Affine Nonlinear Discrete-Time Systems , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Frank L. Lewis,et al.  Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances , 2016, IEEE Transactions on Cybernetics.

[16]  Yun Zhang,et al.  Neural Network Learning and Robust Stabilization of Nonlinear Systems With Dynamic Uncertainties , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Lacra Pavel,et al.  A noncooperative game approach to OSNR optimization in optical networks , 2006, IEEE Transactions on Automatic Control.

[18]  Marcus Johnson,et al.  Approximate $N$ -Player Nonzero-Sum Game Solution for an Uncertain Continuous Nonlinear System , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Derong Liu,et al.  Policy Iteration Algorithm for Online Design of Robust Control for a Class of Continuous-Time Nonlinear Systems , 2014, IEEE Transactions on Automation Science and Engineering.

[20]  Derong Liu,et al.  Adaptive Dynamic Programming for Discrete-Time Zero-Sum Games , 2018, IEEE Transactions on Neural Networks and Learning Systems.