Cooperative Path Integral Control for Stochastic Multi-Agent Systems

A distributed stochastic optimal control solution is presented for cooperative multi-agent systems. The network of agents is partitioned into multiple factorial subsystems, each of which consists of a central agent and neighboring agents. Local control actions that rely only on agents' local observations are designed to optimize the joint cost functions of subsystems. When solving for the local control actions, the joint optimality equation for each subsystem is cast as a linear partial differential equation and solved using the Feynman-Kac formula. The solution and the optimal control action are then formulated as path integrals and approximated by a Monte-Carlo method. Numerical verification is provided through a simulation example consisting of a team of cooperative UAVs.

[1]  Ross P. Anderson,et al.  Stochastic optimal enhancement of distributed formation control using Kalman smoothers , 2014, Robotica.

[2]  Hilbert J. Kappen,et al.  Graphical Model Inference in Optimal Control of Stochastic Multi-Agent Systems , 2008, J. Artif. Intell. Res..

[3]  Wenwu Yu,et al.  An Overview of Recent Progress in the Study of Distributed Multi-Agent Coordination , 2012, IEEE Transactions on Industrial Informatics.

[4]  Long Wang,et al.  Recent Advances in Consensus of Multi-Agent Systems: A Brief Survey , 2017, IEEE Transactions on Industrial Electronics.

[5]  Evangelos A. Theodorou,et al.  Sample Efficient Path Integral Control under Uncertainty , 2015, NIPS.

[6]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[7]  Evangelos A. Theodorou,et al.  Iterative path integral stochastic optimal control: Theory and applications to motor control , 2011 .

[8]  Naira Hovakimyan,et al.  Safe Coordinated Maneuvering of Teams of Multirotor Unmanned Aerial Vehicles: A Cooperative Control Framework for Multivehicle, Time-Critical Missions , 2016, IEEE Control Systems.

[9]  John N. Tsitsiklis,et al.  Efficiency loss in a network resource allocation game: the case of elastic supply , 2004, IEEE Transactions on Automatic Control.

[10]  Neng Wan,et al.  Hierarchical path generation for distributed mission planning of UAVs , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[11]  J. L. Gall,et al.  Brownian Motion, Martingales, and Stochastic Calculus , 2016 .

[12]  Frank L. Lewis,et al.  Cooperative Control of Multi-Agent Systems: Optimal and Adaptive Design Approaches , 2013 .

[13]  Hyo-Sung Ahn,et al.  A survey of multi-agent formation control , 2015, Autom..

[14]  Evangelos A. Theodorou,et al.  Model Predictive Path Integral Control: From Theory to Parallel Computation , 2017 .

[15]  Francesco Bullo,et al.  Breaking the Hierarchy: Distributed Control and Economic Optimality in Microgrids , 2014, IEEE Transactions on Control of Network Systems.

[16]  Olivier Sigaud,et al.  Path Integral Policy Improvement with Covariance Matrix Adaptation , 2012, ICML.

[17]  Vicenç Gómez,et al.  Policy Search for Path Integral Control , 2014, ECML/PKDD.

[18]  Emanuel Todorov,et al.  Compositionality of optimal control laws , 2009, NIPS.

[19]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[20]  W. Fleming.  Exit probabilities and optimal stochastic control , 1977 .

[21]  Emanuel Todorov,et al.  Linearly-solvable Markov decision problems , 2006, NIPS.

[22]  Petros G. Voulgaris,et al.  Social optimization problems with decentralized and selfish optimal strategies , 2017, 2017 IEEE 56th Annual Conference on Decision and Control (CDC).

[23]  Akihiko Yokoyama,et al.  Autonomous Distributed V2G (Vehicle-to-Grid) Satisfying Scheduled Charging , 2012, IEEE Transactions on Smart Grid.

[24]  James M. Rehg,et al.  Information-Theoretic Model Predictive Control: Theory and Applications to Autonomous Driving , 2017, IEEE Transactions on Robotics.