Leader-to-Formation Stability of Multiagent Systems: An Adaptive Optimal Control Approach

This note proposes a novel data-driven solution to the cooperative adaptive optimal control problem of leader-follower multiagent systems under switching network topology. The dynamics of all the followers are unknown, and the leader is modeled by a perturbed exosystem. Through the combination of adaptive dynamic programming and internal model principle, an approximate optimal controller is iteratively learned online using real-time input-state data. Rigorous stability analysis shows that the system in closed-loop with the developed control policy is leader-to-formation stable, with guaranteed robustness to unmeasurable leader disturbance. Numerical results illustrate the effectiveness of the proposed data-driven algorithm.

[1]  F. Lewis,et al.  Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers , 2012, IEEE Control Systems.

[2]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[3]  Wei Liu,et al.  Cooperative global robust output regulation for second-order nonlinear multi-agent systems with jointly connected switching networks , 2016, 2016 American Control Conference (ACC).

[4]  Zhong-Ping Jiang,et al.  Robust Adaptive Dynamic Programming , 2017 .

[5]  Lu Liu,et al.  Adaptive Cooperative Output Regulation for a Class of Nonlinear Multi-Agent Systems , 2015, IEEE Transactions on Automatic Control.

[6]  Vijay Kumar,et al.  Leader-to-formation stability , 2004, IEEE Transactions on Robotics and Automation.

[7]  Zhong-Ping Jiang,et al.  Learning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear Systems , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Lihua Xie,et al.  Distributed Tracking Control for Linear Multiagent Systems With a Leader of Bounded Unknown Input , 2013, IEEE Transactions on Automatic Control.

[9]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[10]  W. Wonham,et al.  Probing signals for model reference identification , 1977 .

[11]  P. R. Kumar,et al.  Optimal Adaptive Control of Linear-Quadratic-Gaussian Systems , 1983 .

[12]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[13]  Frank L. Lewis,et al.  Linear Quadratic Tracking Control of Partially-Unknown Continuous-Time Systems Using Reinforcement Learning , 2014, IEEE Transactions on Automatic Control.

[14]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[15]  Frank L. Lewis,et al.  Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[16]  Zhong-Ping Jiang,et al.  Small-gain theorem for ISS systems and applications , 1994, Math. Control. Signals Syst..

[17]  Haibo He,et al.  Data-Driven Tracking Control With Adaptive Dynamic Programming for a Class of Continuous-Time Nonlinear Systems , 2017, IEEE Transactions on Cybernetics.

[18]  Zhong-Ping Jiang,et al.  Robust adaptive dynamic programming for linear and nonlinear systems: An overview , 2013, Eur. J. Control.

[19]  D. Kleinman.  On an iterative technique for Riccati equation computations , 1968 .

[20]  Zhong-Ping Jiang,et al.  Sampled-data-based adaptive optimal output-feedback control of a 2-degree-of-freedom helicopter , 2016 .

[21]  Derong Liu,et al.  An Approximate Optimal Control Approach for Robust Stabilization of a Class of Discrete-Time Nonlinear Systems With Uncertainties , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[22]  Luigi Fortuna,et al.  Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control , 2009 .

[23]  F. Lewis,et al.  Online adaptive algorithm for optimal control with integral reinforcement learning , 2014 .

[24]  Jie Huang,et al.  Cooperative Output Regulation of Linear Multi-Agent Systems , 2012, IEEE Transactions on Automatic Control.

[25]  Zhengtao Ding,et al.  Consensus Output Regulation of a Class of Heterogeneous Nonlinear Systems , 2013, IEEE Transactions on Automatic Control.

[26]  Eduardo Sontag.  Smooth stabilization implies coprime factorization , 1989, IEEE Transactions on Automatic Control.

[27]  B. Francis.  The linear multivariable regulator problem , 1976, 1976 IEEE Conference on Decision and Control including the 15th Symposium on Adaptive Processes.

[28]  Wei Xing Zheng,et al.  Adaptive tracking control of leader-follower systems with unknown dynamics and partial measurements , 2014, Autom..

[29]  Domenico Prattichizzo,et al.  Discussion of paper by , 2003 .

[30]  Dimos V. Dimarogonas,et al.  Leader-follower cooperative attitude control of multiple rigid bodies , 2008, ACC.

[31]  Guang-Hong Yang,et al.  Adaptive Actor–Critic Design-Based Integral Sliding-Mode Control for Partially Unknown Nonlinear Systems With Input Disturbances , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Zhong-Ping Jiang,et al.  Adaptive Dynamic Programming and Adaptive Optimal Output Regulation of Linear Systems , 2016, IEEE Transactions on Automatic Control.

[33]  Riccardo Marino,et al.  Output regulation for linear systems via adaptive internal model , 2003, IEEE Trans. Autom. Control..

[34]  Zhong-Ping Jiang,et al.  Nonlinear and Adaptive Suboptimal Control of Connected Vehicles: A Global Adaptive Dynamic Programming Approach , 2017, J. Intell. Robotic Syst..

[35]  Zhong-Ping Jiang,et al.  Output-feedback adaptive optimal control of interconnected systems based on robust adaptive dynamic programming , 2016, Autom..

[36]  Frank L. Lewis,et al.  Multi-agent discrete-time graphical games and reinforcement learning solutions , 2014, Autom..

[37]  Eduardo Sontag.  Input to State Stability: Basic Concepts and Results , 2008 .

[38]  Zhong-Ping Jiang,et al.  A Distributed Control Approach to A Robust Output Regulation Problem for Multi-Agent Linear Systems , 2010, IEEE Transactions on Automatic Control.

[39]  Jie Huang,et al.  Nonlinear Output Regulation: Theory and Applications , 2004 .

[40]  Jie Huang,et al.  Cooperative Output Regulation With Application to Multi-Agent Consensus Under Switching Network , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[41]  Xudong Ye,et al.  Cooperative Output Regulation of Heterogeneous Multi-Agent Systems: An $H_{\infty}$ Criterion , 2014, IEEE Transactions on Automatic Control.

[42]  Frank L. Lewis,et al.  Data-Driven Flotation Industrial Process Operational Optimal Control Based on Reinforcement Learning , 2018, IEEE Transactions on Industrial Informatics.