Reinforcement Learning Control for Consensus of the Leader-Follower Multi-Agent Systems

This paper considers the optimal consensus of multi-agent systems using reinforcement learning control. The system is nonlinear and the number of agents can be large. The control objective is to design the controllers for each agent such that all the agents will be consensus to the leader agent. We use the Actor-Critic Network and the Deterministic Policy Gradient method to realize the controller. The policy iteration algorithm is discussed and many simulations are provided to validate the result.

[1]  Huaguang Zhang,et al.  Leader-Based Optimal Coordination Control for the Consensus Problem of Multiagent Differential Games via Fuzzy Adaptive Dynamic Programming , 2015, IEEE Transactions on Fuzzy Systems.

[2]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[3]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[4]  Frank L. Lewis,et al.  Tracking Control for Linear Discrete-Time Networked Control Systems With Unknown Dynamics and Dropout , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Frank L. Lewis,et al.  Off-Policy Reinforcement Learning for Synchronization in Multiagent Graphical Games , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Zongli Lin,et al.  Consensus of a class of discrete-time nonlinear multi-agent systems in the presence of communication delays. , 2017, ISA transactions.

[7]  Huaguang Zhang,et al.  Data-Driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems With Unknown Dynamics Using Reinforcement Learning Method , 2017, IEEE Transactions on Industrial Electronics.

[8]  Huaguang Zhang,et al.  Optimal Output Regulation for Heterogeneous Multiagent Systems via Adaptive Dynamic Programming , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Dimitri P. Bertsekas,et al.  Value and Policy Iterations in Optimal Control and Adaptive Dynamic Programming , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Xiang Li,et al.  Pinning a complex dynamical network to its equilibrium , 2004, IEEE Trans. Circuits Syst. I Regul. Pap..

[11]  Kyriakos G. Vamvoudakis,et al.  Multi-agent differential graphical games , 2011, Proceedings of the 30th Chinese Control Conference.

[12]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[13]  Ali Heydari,et al.  An adaptive critic-based scheme for consensus control of nonlinear multi-agent systems , 2014, Int. J. Control.

[14]  Haibo He,et al.  Novel iterative neural dynamic programming for data-based approximate optimal control design , 2017, Autom..

[15]  Tingwen Huang,et al.  Reinforcement learning solution for HJB equation arising in constrained optimal control problem , 2015, Neural Networks.

[16]  Frank L. Lewis,et al.  Multi-agent discrete-time graphical games and reinforcement learning solutions , 2014, Autom..

[17]  Ali Heydari,et al.  Theoretical and Numerical Analysis of Approximate Dynamic Programming with Approximation Errors , 2014, ArXiv.