Optimal distributed learning for disturbance rejection in networked non‐linear games under unknown dynamics

In this study, an online distributed optimal adaptive algorithm is introduced for continuous-time non-linear differential graphical games under unknown systems subject to external disturbances. The proposed algorithm learns online the approximate solution to the coupled Hamilton–Jacobi–Isaacs equations. Each of the players in the game uses an actor-critic network structure and an intelligent identifier to find the unknown parameters of the systems. The authors use recorded past observations concurrently with current data to speed up convergence by exploring the state space. The closed-loop stability and convergence of the policies to Nash equilibrium are ensured by using Lyapunov stability theory. Finally, a simulation example shows the efficiency of the proposed algorithm.

[1]  Kyriakos G. Vamvoudakis,et al.  Non-zero sum Nash Q-learning for unknown deterministic continuous-time linear systems , 2015, Autom..

[2]  Luy Nguyen Tan.  Distributed optimal integrated tracking control for separate kinematic and dynamic uncertain non-holonomic mobile mechanical multi-agent systems , 2017 .

[3]  Frank L. Lewis,et al.  Multi-agent differential graphical games , 2011, Proceedings of the 30th Chinese Control Conference.

[4]  Mohammad-Bagher Naghibi-Sistani,et al.  A novel distributed optimal adaptive control algorithm for nonlinear multi-agent differential graphical games , 2018, IEEE/CAA Journal of Automatica Sinica.

[5]  Frank L. Lewis,et al.  Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach , 2005, Autom..

[6]  Zhong-Ping Jiang,et al.  Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems , 2016, IEEE Transactions on Automatic Control.

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Karl Henrik Johansson,et al.  Distributed Event-Triggered Control for Multi-Agent Systems , 2012, IEEE Transactions on Automatic Control.

[9]  M. Spong.  Modeling and Control of Elastic Joint Robots , 1987 .

[10]  Guo-Ping Liu,et al.  Data‐driven consensus for non‐linear networked multi‐agent systems with switching topology and time‐varying delays , 2018, IET Control Theory & Applications.

[11]  Frank L. Lewis,et al.  Online solution of nonlinear two‐player zero‐sum games using synchronous policy iteration , 2012 .

[12]  Warren E. Dixon,et al.  Model-Based Reinforcement Learning in Differential Graphical Games , 2017, IEEE Transactions on Control of Network Systems.

[13]  Mengyin Fu,et al.  Consensus of Multi-Agent Systems With General Linear and Lipschitz Nonlinear Dynamics Using Distributed Adaptive Protocols , 2011, IEEE Transactions on Automatic Control.

[14]  F. Lewis,et al.  Online solution of nonquadratic two‐player zero‐sum games arising in the H ∞  control of constrained input systems , 2014 .

[15]  Sandip Roy,et al.  Constructing consensus controllers for networks with identical general linear agents , 2011 .

[16]  Ziyang Meng,et al.  Leaderless and Leader-Following Consensus With Communication and Input Delays Under a Directed Network Topology , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[17]  Kyriakos G. Vamvoudakis,et al.  Distributed learning algorithm for non-linear differential graphical games , 2017 .

[18]  Tamer Başar,et al.  H1-Optimal Control and Related Minimax Design Problems , 1995 .

[19]  Frank L. Lewis,et al.  Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Mohammad Bagher Menhaj,et al.  Adaptive output stationary average consensus for heterogeneous unknown linear multi-agent systems , 2018 .

[21]  Frank L. Lewis,et al.  Multi-agent zero-sum differential graphical games for disturbance rejection in distributed control , 2016, Autom..