Bipartite state synchronization of heterogeneous system with active leader on signed digraph under adversarial inputs

Abstract In this paper, the bipartite state synchronization problem of heterogeneous multi-agent systems (MASs) with active leader under adversarial signals over signed digraph is studied. The non-zero control of active leader is bounded, and we can update this control to protect followers from possible danger. Firstly, the bipartite state synchronization problem over signed digraph is converted to a regular tracking synchronization problem over nonnegative digraph after state transformations for heterogeneous MASs with active leader under adversarial signals. Then, novel distributed observers are designed to estimate the state of leader, as well as prevent disrupted data of agent from propagating the communication networks. Observer-based H∞ optimal controllers are also designed for each follower to mitigate effects on disrupted agents that agents under attack, and the inhomogeneous algebraic Riccati equations (AREs) are obtained. Meanwhile, a data-based off-policy reinforcement learning (RL) algorithm is used to solve the AREs without requiring the dynamics of agents. At last, the effectiveness of the algorithm is demonstrated by a simulation example.

[1]  Claudio Altafini,et al.  Consensus Problems on Networks With Antagonistic Interactions , 2013, IEEE Transactions on Automatic Control.

[2]  Wei Ren,et al.  Distributed consensus of linear multi-agent systems with adaptive dynamic protocols , 2011, Autom..

[3]  Long Wang,et al.  Controllability of multi-agent systems with directed and weighted signed networks , 2018, Syst. Control. Lett..

[4]  Guanghui Wen,et al.  Designing Fully Distributed Consensus Protocols for Linear Multi-Agent Systems With Directed Graphs , 2013, IEEE Transactions on Automatic Control.

[5]  Huaguang Zhang,et al.  Delay-dependent distributed event-triggered tracking control for multi-agent systems with input time delay , 2019, Neurocomputing.

[6]  Yixin Yin,et al.  Optimal Containment Control of Unknown Heterogeneous Systems With Active Leaders , 2019, IEEE Transactions on Control Systems Technology.

[7]  Hamidreza Modares,et al.  Resilient Autonomous Control of Distributed Multiagent Systems in Contested Environments , 2017, IEEE Transactions on Cybernetics.

[8]  Magnus Egerstedt,et al.  Distributed containment control with multiple stationary or dynamic leaders in fixed and switching directed networks , 2012, Autom..

[9]  Huaguang Zhang,et al.  An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games , 2011, Autom..

[10]  Lin Huang,et al.  Consensus of Multiagent Systems and Synchronization of Complex Networks: A Unified Viewpoint , 2016, IEEE Transactions on Circuits and Systems I: Regular Papers.

[11]  Qing-Long Han,et al.  Consensus control of stochastic multi-agent systems: a survey , 2017, Science China Information Sciences.

[12]  Yixin Yin,et al.  Containment control of heterogeneous systems with active leaders of bounded unknown control using reinforcement learning , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[13]  Khashayar Khorasani,et al.  $H_{\infty}$ Consensus Achievement of Multi-Agent Systems With Directed and Switching Topology Networks , 2014, IEEE Transactions on Automatic Control.

[14]  Rathinasamy Sakthivel,et al.  Finite-time leaderless consensus of uncertain multi-agent systems against time-varying actuator faults , 2019, Neurocomputing.

[15]  Hongwei Zhang,et al.  Bipartite consensus of multi‐agent systems over signed graphs: State feedback and output feedback control approaches , 2017 .

[16]  Frank L. Lewis,et al.  Bipartite tracking of homogeneous and heterogeneous linear multi-agent systems , 2019, Int. J. Control.

[17]  F. Lewis,et al.  Online solution of nonquadratic two‐player zero‐sum games arising in the H ∞  control of constrained input systems , 2014 .

[18]  Girish Chowdhary,et al.  Concurrent Learning for Parameter Estimation Using Dynamic State-Derivative Estimators , 2015, IEEE Transactions on Automatic Control.

[19]  Qing-Long Han,et al.  Dissipative control for nonlinear Markovian jump systems with actuator failures and mixed time-delays , 2018, Autom..

[20]  Wei Liu,et al.  Finite-time consensus control of heterogeneous nonlinear MASs with uncertainties bounded by positive functions , 2019, Neurocomputing.

[21]  Wei Xing Zheng,et al.  Robust $H_{\infty }$ Group Consensus for Interacting Clusters of Integrator Agents , 2017, IEEE Transactions on Automatic Control.

[22]  Frank L. Lewis,et al.  Off-Policy Actor-Critic Structure for Optimal Control of Unknown Systems With Disturbances , 2016, IEEE Transactions on Cybernetics.

[23]  Claudio Altafini,et al.  Predictable Dynamics of Opinion Forming for Networks With Antagonistic Interactions , 2015, IEEE Transactions on Automatic Control.

[24]  Frank L. Lewis,et al.  Bipartite output synchronization of heterogeneous multiagent systems on signed digraphs , 2018, International Journal of Robust and Nonlinear Control.

[25]  Frank L. Lewis,et al.  Off-Policy Integral Reinforcement Learning Method to Solve Nonlinear Continuous-Time Multiplayer Nonzero-Sum Games , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[27]  Frank L. Lewis,et al.  Multi-agent discrete-time graphical games and reinforcement learning solutions , 2014, Autom..

[28]  Frank L. Lewis,et al.  Optimal model-free output synchronization of heterogeneous systems using off-policy reinforcement learning , 2016, Autom..

[29]  Yixin Yin,et al.  Leader–Follower Output Synchronization of Linear Heterogeneous Systems With Active Leader Using Reinforcement Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Xiao Fan Wang,et al.  Flocking of Multi-Agents With a Virtual Leader , 2009, IEEE Trans. Autom. Control..

[31]  Frank L. Lewis,et al.  Distributed L2-gain output-feedback control of homogeneous and heterogeneous systems , 2016, at - Automatisierungstechnik.

[32]  Derong Liu,et al.  Integral Reinforcement Learning for Linear Continuous-Time Zero-Sum Games With Completely Unknown Dynamics , 2014, IEEE Transactions on Automation Science and Engineering.

[33]  Yongguang Yu,et al.  Neuro-adaptive leaderless consensus of fractional-order multi-agent systems , 2019, Neurocomputing.

[34]  Zhiguang Feng,et al.  Non-fragile consensus control for singular multi-agent systems with Lipschitz nonlinear dynamics , 2019, Neurocomputing.

[35]  Hongjing Liang,et al.  Adaptive Distributed Observer Approach for Cooperative Containment Control of Nonidentical Networks , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[36]  Hak-Keung Lam,et al.  Distributed Event-Based Set-Membership Filtering for a Class of Nonlinear Systems With Sensor Saturations Over Sensor Networks , 2017, IEEE Transactions on Cybernetics.

[37]  Frank L. Lewis,et al.  $ {H}_{ {\infty }}$ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[38]  Qinglai Wei,et al.  Distributed control of leader-follower systems under adversarial inputs using reinforcement learning , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[39]  Hamidreza Modares,et al.  An internal model principle for the attacker in distributed control systems , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[40]  Jinyu Wen,et al.  Adaptive Learning in Tracking Control Based on the Dual Critic Network Design , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[41]  Hak-Keung Lam,et al.  Mean-Square $H_\infty $ Consensus Control for a Class of Nonlinear Time-Varying Stochastic Multiagent Systems: The Finite-Horizon Case , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[42]  Tingwen Huang,et al.  Off-Policy Reinforcement Learning for $ H_\infty $ Control Design , 2013, IEEE Transactions on Cybernetics.