LQR-Based Optimal Distributed Cooperative Design for Linear Discrete-Time Multiagent Systems

In this paper, a novel linear quadratic regulator (LQR)-based optimal distributed cooperative design method is developed for synchronization control of general linear discrete-time multiagent systems on a fixed, directed graph. Sufficient conditions are derived for synchronization, which restrict the graph eigenvalues into a bounded circular region in the complex plane. The synchronizing speed issue is also considered, and it turns out that the synchronizing region reduces as the synchronizing speed becomes faster. To obtain more desirable synchronizing capacity, the weighting matrices are selected by sufficiently utilizing the guaranteed gain margin of the optimal regulators. Based on the developed LQR-based cooperative design framework, an approximate dynamic programming technique is successfully introduced to overcome the (partially or completely) model-free cooperative design for linear multiagent systems. Finally, two numerical examples are given to illustrate the effectiveness of the proposed design methods.

[1]  Frank L. Lewis,et al.  Lyapunov, Adaptive, and Optimal Design Techniques for Cooperative Systems on Directed Communication Graphs , 2012, IEEE Transactions on Industrial Electronics.

[2]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[3]  W. Ren.  Distributed attitude consensus among multiple networked spacecraft , 2006, 2006 American Control Conference.

[4]  Derong Liu,et al.  Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Frank L. Lewis,et al.  Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Frank L. Lewis,et al.  Cooperative Optimal Control for Multi-Agent Systems on Directed Graph Topologies , 2014, IEEE Transactions on Automatic Control.

[7]  L. C. Baird,et al.  Reinforcement learning in continuous time: advantage updating , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[8]  Huaguang Zhang,et al.  Distributed Cooperative Optimal Control for Multiagent Systems on Directed Graphs: An Inverse Optimal Approach , 2015, IEEE Transactions on Cybernetics.

[9]  Jun Zhao,et al.  Cooperative Adaptive Fuzzy Tracking Control for Networked Unknown Nonlinear Multiagent Systems With Time-Varying Actuator Faults , 2014, IEEE Transactions on Fuzzy Systems.

[10]  Lihua Xie,et al.  Network Topology and Communication Data Rate for Consensusability of Discrete-Time Multi-Agent Systems , 2011, IEEE Transactions on Automatic Control.

[11]  Jiangping Hu,et al.  Tracking control for multi-agent consensus with an active leader and variable topology , 2006, Autom..

[12]  Randal W. Beard,et al.  A decentralized scheme for spacecraft formation flying via the virtual structure approach , 2003, Proceedings of the 2003 American Control Conference, 2003..

[13]  Derong Liu,et al.  Adaptive Dynamic Programming for Optimal Tracking Control of Unknown Nonlinear Systems With Application to Coal Gasification , 2014, IEEE Transactions on Automation Science and Engineering.

[14]  Huaguang Zhang,et al.  Neural-Network-Based Constrained Optimal Control Scheme for Discrete-Time Switched Nonlinear System Using Dual Heuristic Programming , 2014, IEEE Transactions on Automation Science and Engineering.

[15]  Frank L. Lewis,et al.  Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control , 2007, Autom..

[16]  Frank L. Lewis,et al.  Integral reinforcement learning and experience replay for adaptive optimal control of partially-unknown constrained-input continuous-time systems , 2014, Autom..

[17]  Doreen Eichel,et al.  Adaptive Dynamic Programming For Control Algorithms And Stability , 2016 .

[18]  Bin Jiang,et al.  Online Adaptive Policy Learning Algorithm for $H_{\infty }$ State Feedback Control of Unknown Affine Nonlinear Discrete-Time Systems , 2014, IEEE Transactions on Cybernetics.

[19]  Frank L. Lewis,et al.  Adaptive cooperative tracking control of higher-order nonlinear systems with unknown dynamics , 2012, Autom..

[20]  Daizhan Cheng,et al.  Consensus of multi-agent linear dynamic systems† , 2008 .

[21]  Lin Huang,et al.  Consensus of Multiagent Systems and Synchronization of Complex Networks: A Unified Viewpoint , 2016, IEEE Transactions on Circuits and Systems I: Regular Papers.

[22]  Frank L. Lewis,et al.  Multi-agent discrete-time graphical games and reinforcement learning solutions , 2014, Autom..

[23]  Derong Liu,et al.  Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints , 2015, IEEE Transactions on Cybernetics.

[24]  Wassim M. Haddad,et al.  Stability margins of discrete-time nonlinear nonquadratic optimal regulators , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[25]  Guanrong Chen,et al.  Consensus of Discrete-Time Linear Multi-Agent Systems with Observer-Type Protocols , 2011, ArXiv.

[26]  Frank L. Lewis,et al.  Multi-agent differential graphical games , 2011, Proceedings of the 30th Chinese Control Conference.

[27]  Richard M. Murray,et al.  Information flow and cooperative control of vehicle formations , 2004, IEEE Transactions on Automatic Control.

[28]  Stephen P. Boyd,et al.  Fast linear iterations for distributed averaging , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[29]  Frank L. Lewis,et al.  Neuro-adaptive cooperative tracking control of unknown higher-order affine nonlinear systems , 2014, Autom..

[30]  Frank L. Lewis,et al.  Synchronization of discrete-time multi-agent systems on graphs using Riccati design , 2012, Autom..

[31]  Derong Liu,et al.  Neural-Network-Based Online HJB Solution for Optimal Robust Guaranteed Cost Control of Continuous-Time Uncertain Nonlinear Systems , 2014, IEEE Transactions on Cybernetics.

[32]  Lihua Xie,et al.  Consensusability of discrete-time multi-agent systems over directed graphs , 2011, Proceedings of the 30th Chinese Control Conference.

[33]  Huaguang Zhang,et al.  Leader-Based Optimal Coordination Control for the Consensus Problem of Multiagent Differential Games via Fuzzy Adaptive Dynamic Programming , 2015, IEEE Transactions on Fuzzy Systems.

[34]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[35]  Reza Olfati-Saber,et al.  Flocking for multi-agent dynamic systems: algorithms and theory , 2006, IEEE Transactions on Automatic Control.

[36]  Derong Liu,et al.  Finite-Approximation-Error-Based Discrete-Time Iterative Adaptive Dynamic Programming , 2014, IEEE Transactions on Cybernetics.

[37]  Li Ren,et al.  A Multiagent Q-Learning-Based Optimal Allocation Approach for Urban Water Resource Management System , 2014, IEEE Transactions on Automation Science and Engineering.

[38]  Huaguang Zhang,et al.  Near-Optimal Control for Nonzero-Sum Differential Games of Continuous-Time Nonlinear Systems Using Single-Network ADP , 2013, IEEE Transactions on Cybernetics.

[39]  J. G. Bender,et al.  An overview of systems studies of automated highway systems , 1991 .

[40]  U. Shaked.  Guaranteed stability margins for the discrete-time linear quadratic optimal regulator , 1986 .

[41]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[42]  Derong Liu,et al.  Error Bounds of Adaptive Dynamic Programming Algorithms for Solving Undiscounted Optimal Control Problems , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[43]  Timothy W. McLain,et al.  Coordinated target assignment and intercept for unmanned air vehicles , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[44]  Wenwu Yu,et al.  An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination , 2012, IEEE Transactions on Industrial Informatics.