Online optimal consensus control of unknown linear multi-agent systems via time-based adaptive dynamic programming

Abstract This paper considers the online optimal consensus control problem for unknown linear discrete-time (DT) multi-agent systems (MASs). Based on time-based adaptive dynamic programming (ADP) method, the control policies are designed by utilizing the current and recorded data of unknown MASs. The critic-actor NN frameworks are employed to approximate the performance indexes and optimal control policies, respectively. The NN weights are updated once at the sampling instant to produce real-time online control. Furthermore, the control policies are proved to effectively drive the MASs to achieve consistency and satisfy the Nash equilibrium. Finally, a numerical example is implemented to shown the feasibility of the control scheme.

[1]  Shouming Zhong,et al.  Design of adaptive backstepping dynamic surface control method with RBF neural network for uncertain nonlinear system , 2019, Neurocomputing.

[2]  Derong Liu,et al.  Generalized Policy Iteration Adaptive Dynamic Programming for Discrete-Time Nonlinear Systems , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[3]  Wei Ren.  On Consensus Algorithms for Double-Integrator Dynamics , 2008, IEEE Trans. Autom. Control..

[4]  Qichao Zhang,et al.  Data-driven adaptive dynamic programming for continuous-time fully cooperative games with partially constrained inputs , 2017, Neurocomputing.

[5]  C. L. Philip Chen,et al.  Adaptive NN event-triggered control for path following of underactuated vessels with finite-time convergence , 2020, Neurocomputing.

[6]  Zhou Quan,et al.  Scaled consensus for asynchronous high‐order discrete‐time multiagent systems , 2019, International Journal of Robust and Nonlinear Control.

[7]  Tieshan Li,et al.  Finite-Time Formation Control of Under-Actuated Ships Using Nonlinear Sliding Mode Control , 2018, IEEE Transactions on Cybernetics.

[8]  Sarangapani Jagannathan,et al.  Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Tieshan Li,et al.  Bounded Neural Network Control for Target Tracking of Underactuated Autonomous Surface Vehicles in the Presence of Uncertain Target Dynamics , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Shaocheng Tong,et al.  Fuzzy Adaptive Output Feedback Optimal Control Design for Strict-Feedback Nonlinear Systems , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[11]  James E. Steck,et al.  Adaptive Feedback Control by Constrained Approximate Dynamic Programming , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Tao Dong,et al.  Distributed optimal consensus algorithms in multi-agent systems , 2019, Neurocomputing.

[13]  Frank L. Lewis,et al.  Off-Policy Integral Reinforcement Learning Method to Solve Nonlinear Continuous-Time Multiplayer Nonzero-Sum Games , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Frank L. Lewis,et al.  Discrete-Time Local Value Iteration Adaptive Dynamic Programming: Convergence Analysis , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[15]  Qinglai Wei,et al.  Discrete-Time Impulsive Adaptive Dynamic Programming , 2020, IEEE Transactions on Cybernetics.

[16]  Hao Xu,et al.  Neural Network-Based Finite Horizon Stochastic Optimal Control Design for Nonlinear Networked Control Systems , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Huaguang Zhang,et al.  Neural-network-based learning algorithms for cooperative games of discrete-time multi-player systems with control constraints via adaptive dynamic programming , 2019, Neurocomputing.

[18]  Zhonghua Miao,et al.  Cooperative adaptive consensus tracking for multiple nonholonomic mobile robots , 2019, Int. J. Syst. Sci..

[19]  Huaguang Zhang,et al.  LQR-Based Optimal Distributed Cooperative Design for Linear Discrete-Time Multiagent Systems , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[20]  Frank L. Lewis,et al.  Optimal distributed synchronization control for continuous-time heterogeneous multi-agent differential graphical games , 2015, Inf. Sci..

[21]  Tieshan Li,et al.  Adaptive leader-following formation control with collision avoidance for a class of second-order nonlinear multi-agent systems , 2019, Neurocomputing.

[22]  Huaguang Zhang,et al.  Online optimal control of unknown discrete-time nonlinear systems by using time-based adaptive dynamic programming , 2015, Neurocomputing.

[23]  Huaguang Zhang,et al.  Data-Driven Optimal Consensus Control for Discrete-Time Multi-Agent Systems With Unknown Dynamics Using Reinforcement Learning Method , 2017, IEEE Transactions on Industrial Electronics.

[24]  Qing-Long Han,et al.  $\mathcal{H}_{\infty}$ Containment Control of Multiagent Systems Under Event-Triggered Communication Scheduling: The Finite-Horizon Case , 2020, IEEE Transactions on Cybernetics.

[25]  Huaguang Zhang,et al.  Adaptive Dynamic Programming for a Class of Complex-Valued Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Yanjun Liu,et al.  Adaptive fuzzy optimal control using direct heuristic dynamic programming for chaotic discrete-time system , 2016 .

[27]  Haibo He,et al.  Fuzzy-Based Goal Representation Adaptive Dynamic Programming , 2016, IEEE Transactions on Fuzzy Systems.

[28]  Wei Wang,et al.  Model-free optimal containment control of multi-agent systems based on actor-critic framework , 2018, Neurocomputing.

[29]  Zhang Ren,et al.  Theory and Experiment on Formation-Containment Control of Multiple Multirotor Unmanned Aerial Vehicle Systems , 2019, IEEE Transactions on Automation Science and Engineering.

[30]  Tingwen Huang,et al.  Off-Policy Reinforcement Learning for $ H_\infty $ Control Design , 2013, IEEE Transactions on Cybernetics.

[31]  Frank L. Lewis,et al.  Multi-agent differential graphical games , 2011, Proceedings of the 30th Chinese Control Conference.

[32]  Wei Chen,et al.  Distributed Resilient Filtering for Power Systems Subject to Denial-of-Service Attacks , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[33]  Frank L. Lewis,et al.  Synchronization of discrete-time multi-agent systems on graphs using Riccati design , 2012, Autom..

[34]  Jiangping Hu,et al.  Data-driven optimal tracking control of discrete-time multi-agent systems with two-stage policy iteration algorithm , 2019, Inf. Sci..

[35]  Guanrong Chen,et al.  Adaptive second-order consensus of networked mobile agents with nonlinear dynamics , 2011, Autom..