Nash, Social and Centralized Solutions to Consensus Problems via Mean Field Control Theory

The purpose of this paper is to synthesize initial mean consensus behavior of a set of agents from the fundamental optimization principles of i) stochastic dynamic games, and ii) optimal control. In the stochastic dynamic game model each agent seeks to minimize its individual quadratic discounted cost function involving the mean of the states of all other agents. In this formulation we derive the limiting infinite population mean field equation system and explicitly compute its unique solution. The resulting mean field (MF) control strategies drive each agent to track the overall population's initial state distribution mean, and by applying these decentralized strategies, any finite population system reaches mean consensus asymptotically as time goes to infinity. Furthermore, these control laws possess an ε<i>N</i>-Nash equilibrium property where ε<i>N</i> goes to zero as the population size N goes to infinity. Finally, the analysis is extended to the case of random mean field couplings. In the social cooperative formulation the basic objective is to minimize a social cost as the sum of the individual cost functions containing mean field coupling. In this formulation we show that for any individual agent the decentralized mean field social (MF Social) control strategy is the same as the mean field Nash (MF Nash) equilibrium strategy. Hence MF-Nash Controls <i>U</i><sub>Nash</sub><i>N</i>=MF - Social Controls <i>U</i><sub>Soc</sub><i>N</i>. On the other hand, the solution to the centralized LQR optimal control formulation yields the Standard Consensus (SC) algorithm whenever the graph representing the corresponding topology of the network is Completely Connected (CC). Hence Cent. LQR Controls <i>U</i><sub>Cent</sub><i>N</i>=SC-CC Controls <i>U</i><sub>SC</sub><i>N</i>. Moreover, a system with centralized control laws reaches consensus on the initial state distribution mean as time and population size N go to infinity. Hence, asymptotically in time MF-Nash Controls <i>U</i><sub>Nash</sub><i>N</i>=MF-Social Controls <i>U</i><sub>Soc</sub><i>N</i> = Cent. LQR Controls <i>U</i><sub>Cent</sub><sup>∞</sup> = SC-CC Controls <i>U</i><sub>SC</sub><sup>∞</sup>. Finally, the analysis is extended to the long time average (LTA) cost functions case.

[1]  Alain Haurie,et al.  On Existence of Overtaking Optimal Trajectories Over an Infinite Time Horizon , 1976, Math. Oper. Res..

[2]  John N. Tsitsiklis,et al.  Distributed Asynchronous Deterministic and Stochastic Gradient Optimization Algorithms , 1984, 1984 American Control Conference.

[3]  P. Caines,et al.  Individual and mass behaviour in large population stochastic wireless power control problems: centralized and Nash equilibrium solutions , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[4]  Jie Lin,et al.  Coordination of groups of mobile autonomous agents using nearest neighbor rules , 2003, IEEE Trans. Autom. Control..

[5]  Richard M. Murray,et al.  Consensus problems in networks of agents with switching topology and time-delays , 2004, IEEE Transactions on Automatic Control.

[6]  Benjamin Van Roy,et al.  Oblivious Equilibrium: A Mean Field Approximation for Large-Scale Dynamic Games , 2005, NIPS.

[7]  E.M. Atkins,et al.  A survey of consensus problems in multi-agent coordination , 2005, Proceedings of the 2005, American Control Conference, 2005..

[8]  Randal W. Beard,et al.  Consensus seeking in multiagent systems under dynamically changing interaction topologies , 2005, IEEE Transactions on Automatic Control.

[9]  Laura Giarré,et al.  Non-linear protocols for optimal distributed consensus in networks of dynamic agents , 2006, Syst. Control. Lett..

[10]  P. Lions,et al.  Jeux à champ moyen. I – Le cas stationnaire , 2006 .

[11]  Peter E. Caines,et al.  Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle , 2006, Commun. Inf. Syst..

[12]  P. Lions,et al.  Jeux à champ moyen. II – Horizon fini et contrôle optimal , 2006 .

[13]  Jan Lorenz,et al.  Continuous Opinion Dynamics under Bounded Confidence: A Survey , 2007, 0707.1762.

[14]  P. Lions,et al.  Mean field games , 2007 .

[15]  Minyi Huang,et al.  Large-Population Cost-Coupled LQG Problems With Nonuniform Agents: Individual-Mass Behavior and Decentralized $\varepsilon$-Nash Equilibria , 2007, IEEE Transactions on Automatic Control.

[16]  Elham Semsar-Kazerooni,et al.  Optimal Control and Game Theoretic Approaches to Cooperative Control of a Team of Multi-Vehicle Unmanned Systems , 2007, 2007 IEEE International Conference on Networking, Sensing and Control.

[17]  Maurizio Porfiri,et al.  Consensus Seeking Over Random Weighted Directed Graphs , 2007, IEEE Transactions on Automatic Control.

[18]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[19]  Peter E. Caines,et al.  An Invariance Principle in Large Population Stochastic Dynamic Games , 2007, J. Syst. Sci. Complex..

[20]  Felipe Cucker,et al.  Emergent Behavior in Flocks , 2007, IEEE Transactions on Automatic Control.

[21]  Sandro Zampieri,et al.  On rendezvous control with randomly switching communication graphs , 2007, Networks Heterog. Media.

[22]  Tao Li,et al.  Asymptotically Optimal Decentralized Control for Large Population Stochastic Multiagent Systems , 2008, IEEE Transactions on Automatic Control.

[23]  Randal W. Beard,et al.  Distributed Consensus in Multi-vehicle Cooperative Control - Theory and Applications , 2007, Communications and Control Engineering.

[24]  Jonathan H. Manton,et al.  Coordination and Consensus of Networked Agents with Noisy Measurements: Stochastic Algorithms and Asymptotic Behavior , 2009, SIAM J. Control. Optim..

[25]  P. Caines,et al.  Social optima in mean field LQG control: Centralized and decentralized strategies , 2009 .

[26]  Z. Qu,et al.  Cooperative Control of Dynamical Systems: Applications to Autonomous Vehicles , 2009 .

[27]  Peter E. Caines,et al.  The NCE (Mean Field) Principle With Locality Dependent Cost Interactions , 2010, IEEE Transactions on Automatic Control.

[28]  P. Caines,et al.  A Solution to the Consensus Problem via Stochastic Mean Field Control , 2010 .

[29]  Yongcan Cao,et al.  Optimal Linear-Consensus Algorithms: An LQR Perspective , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[30]  Tao Li,et al.  Consensus Conditions of Multi-Agent Systems With Time-Varying Topologies and Stochastic Communication Noises , 2010, IEEE Transactions on Automatic Control.

[31]  Christian Dogbé,et al.  Modeling crowd dynamics by the mean-field limit approach , 2010, Math. Comput. Model..

[32]  Luca Schenato,et al.  A Survey on Distributed Estimation and Control Applications Using Linear Consensus Algorithms , 2010 .

[33]  Jonathan H. Manton,et al.  Stochastic Consensus Seeking With Noisy and Directed Inter-Agent Communication: Fixed and Randomly Varying Topologies , 2010, IEEE Transactions on Automatic Control.

[34]  S. Meyn,et al.  Synchronization of coupled oscillators is a game , 2010, ACC 2010.

[35]  Peter E. Caines,et al.  Mean Field LQG Control in Leader-Follower Stochastic Multi-Agent Systems: Likelihood Ratio Based Adaptation , 2012, IEEE Transactions on Automatic Control.