Event-triggered Multi-agent Optimal Regulation Using Adaptive Dynamic Programming

This paper develops an event-triggered multi-agent control method based on adaptive dynamic programming (ADP) techniques. Different from the traditional ADP-based multi-agent control with fixed sampling period, our method designs an adaptive controller only based on the efficiently reduced samples. The sampling instants are decided by an adaptive triggering condition to guarantee the stability of the event-triggered learning process. The theoretical analysis of the proposed method is also provided in this paper. It is proved that the designed event-triggered ADP controller can make all the agents synchronize to the leader’s dynamics with reduced sampled data, and also reach Nash equilibrium at the same time. Therefore, the proposed method can save the computational resources in the learning process. Finally, the simulation results verify the theoretical analysis and also demonstrate the performance of the developed method.

[1]  Michael D. Lemmon,et al.  Event-Triggered Feedback in Control, Estimation, and Optimization , 2010 .

[2]  Zhen Ni,et al.  Data-Driven Reinforcement Learning Design for Multi-agent Systems with Unknown Disturbances , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[3]  Haibo He,et al.  Event-triggered reinforcement learning approach for unknown nonlinear continuous-time system , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[4]  Derong Liu,et al.  Multibattery Optimal Coordination Control for Home Energy Management Systems via Distributed Iterative Adaptive Dynamic Programming , 2015, IEEE Transactions on Industrial Electronics.

[5]  Frank L. Lewis,et al.  Approximate dynamic programming solutions of multi-agent graphical games using actor-critic network structures , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[6]  K. Vamvoudakis Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems , 2014, IEEE/CAA Journal of Automatica Sinica.

[7]  Seif Haridi,et al.  Distributed Algorithms , 1992, Lecture Notes in Computer Science.

[8]  Vicsek,et al.  Novel type of phase transition in a system of self-driven particles. , 1995, Physical review letters.

[9]  Paulo Tabuada,et al.  Event-Triggered Real-Time Scheduling of Stabilizing Control Tasks , 2007, IEEE Transactions on Automatic Control.

[10]  Huaguang Zhang,et al.  Data-Driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems With Unknown Dynamics Using Reinforcement Learning Method , 2017, IEEE Transactions on Industrial Electronics.

[11]  Avimanyu Sahoo,et al.  Neural Network-Based Event-Triggered State Feedback Control of Nonlinear Continuous-Time Systems , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Huijun Gao,et al.  Data-Driven Control and Learning Systems , 2017, IEEE Trans. Ind. Electron..

[13]  Haibo He,et al.  Event-Triggered Optimal Control for Partially Unknown Constrained-Input Systems via Adaptive Dynamic Programming , 2017, IEEE Transactions on Industrial Electronics.

[14]  Huaguang Zhang,et al.  Leader-Based Optimal Coordination Control for the Consensus Problem of Multiagent Differential Games via Fuzzy Adaptive Dynamic Programming , 2015, IEEE Transactions on Fuzzy Systems.

[15]  Wenwu Yu,et al.  An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination , 2012, IEEE Transactions on Industrial Informatics.

[16]  Frank L. Lewis,et al.  Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games , 2015, Inf. Sci..

[17]  Haibo He,et al.  Adaptive Dynamic Programming for Robust Regulation and Its Application to Power Systems , 2018, IEEE Transactions on Industrial Electronics.

[18]  Doreen Eichel,et al.  Adaptive Dynamic Programming For Control Algorithms And Stability , 2016 .

[19]  Haibo He,et al.  Event-triggered adaptive dynamic programming for continuous-time nonlinear system using measured input-output data , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[20]  Avimanyu Sahoo,et al.  Near Optimal Event-Triggered Control of Nonlinear Discrete-Time Systems Using Neurodynamic Programming , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[21]  W. P. M. H. Heemels,et al.  Periodic Event-Triggered Control for Linear Systems , 2013, IEEE Trans. Autom. Control..

[22]  Gang Feng,et al.  Event-driven observer-based output feedback control for linear systems , 2014, Autom..

[23]  Ziyang Wang,et al.  A Tracking Control Method based on Event-Triggered Adaptive Dynamic Programming , 2019, 2019 Chinese Control Conference (CCC).

[24]  Richard M. Murray,et al.  Recent Research in Cooperative Control of Multivehicle Systems , 2007 .

[25]  Tao Feng,et al.  Distributed Optimal Consensus Control for Nonlinear Multiagent System With Unknown Dynamic , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Guanrong Chen,et al.  Pinning control and synchronization on complex dynamical networks , 2014, International Journal of Control, Automation and Systems.

[27]  Pavankumar Tallapragada,et al.  On Event Triggered Tracking for Nonlinear Systems , 2013, IEEE Transactions on Automatic Control.

[28]  Paul J. Werbos,et al.  Foreword: ADP - The Key Direction for Future Research in Intelligent Control and Understanding Brain Intelligence , 2008, IEEE Trans. Syst. Man Cybern. Part B.

[29]  Huijun Gao,et al.  Distributed Synchronization in Networks of Agent Systems With Nonlinearities and Random Switchings , 2013, IEEE Transactions on Cybernetics.

[30]  Paulo Tabuada,et al.  Periodic event-triggered control for nonlinear systems , 2013, 52nd IEEE Conference on Decision and Control.

[31]  Randal W. Beard,et al.  Consensus seeking in multiagent systems under dynamically changing interaction topologies , 2005, IEEE Transactions on Automatic Control.

[32]  Luc Moreau,et al.  Stability of multiagent systems with time-dependent communication links , 2005, IEEE Transactions on Automatic Control.

[33]  Huaguang Zhang,et al.  LQR-Based Optimal Distributed Cooperative Design for Linear Discrete-Time Multiagent Systems , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Qichao Zhang,et al.  Event-Based Robust Control for Uncertain Nonlinear Systems Using Adaptive Dynamic Programming , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[36]  Huijun Gao,et al.  On Discrete-Time Convergence for General Linear Multi-Agent Systems Under Dynamic Topology , 2014, IEEE Transactions on Automatic Control.

[37]  Wei Ren,et al.  Information consensus in multivehicle cooperative control , 2007, IEEE Control Systems.

[38]  Guoqiang Hu,et al.  Distributed Secure Coordinated Control for Multiagent Systems Under Strategic Attacks , 2017, IEEE Transactions on Cybernetics.

[39]  Huaguang Zhang,et al.  Optimal Output Regulation for Heterogeneous Multiagent Systems via Adaptive Dynamic Programming , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[40]  M. Degroot Reaching a Consensus , 1974 .

[41]  Haibo He,et al.  An Event-Triggered ADP Control Approach for Continuous-Time System With Unknown Internal States , 2017, IEEE Transactions on Cybernetics.

[42]  Frank L. Lewis,et al.  Multi-agent discrete-time graphical games and reinforcement learning solutions , 2014, Autom..

[43]  Wei Ren,et al.  Distributed Velocity-Constrained Consensus of Discrete-Time Multi-Agent Systems With Nonconvex Constraints, Switching Topologies, and Delays , 2017, IEEE Transactions on Automatic Control.

[44]  Richard M. Murray,et al.  Information flow and cooperative control of vehicle formations , 2004, IEEE Transactions on Automatic Control.

[45]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[46]  Wei Zhao,et al.  Distributed optimal coordination control for nonlinear multi-agent systems using event-triggered adaptive dynamic programming method. , 2019, ISA transactions.