Distributed Seeking of Nash Equilibria With Applications to Mobile Sensor Networks

We consider the problem of distributed convergence to a Nash equilibrium in a noncooperative game where the players generate their actions based only on online measurements of their individual cost functions, corrupted with additive measurement noise. Exact analytical forms and/or parameters of the cost functions, as well as the current actions of the players may be unknown. Additionally, the players' actions are subject to linear dynamic constraints. We propose an algorithm based on discrete-time stochastic extremum seeking using sinusoidal perturbations and prove its almost sure convergence to a Nash equilibrium. We show how the proposed algorithm can be applied to solving coordination problems in mobile sensor networks, where motion dynamics of the players can be modeled as: 1) single integrators (velocity-actuated vehicles), 2) double integrators (force-actuated vehicles), and 3) unicycles (a kinematic model with nonholonomic constraints). Examples are given in which the cost functions are selected such that the problems of connectivity control, formation control, rendezvous and coverage control are solved in an adaptive and distributed way. The methodology is illustrated through simulations.

[1]  Tamer Basar,et al.  Analysis of Recursive Stochastic Algorithms , 2001 .

[2]  Milos S. Stankovic,et al.  Nash equilibrium seeking in multi-vehicle systems : A Lie bracket approximation-based approach , 2012 .

[3]  L. Shapley,et al.  Potential Games , 1994 .

[4]  Han-Fu Chen Stochastic approximation and its applications , 2002 .

[5]  C. Tomlin,et al.  Decentralized optimization, with application to multiple aircraft coordination , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[6]  George J. Pappas,et al.  Flocking while preserving network connectivity , 2007, 2007 46th IEEE Conference on Decision and Control.

[7]  Christos G. Cassandras,et al.  Sensor Networks and Cooperative Control , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[8]  Jeff S. Shamma,et al.  Dynamic fictitious play, dynamic gradient play, and distributed convergence to Nash equilibria , 2005, IEEE Transactions on Automatic Control.

[9]  Jason R. Marden,et al.  Designing Games for Distributed Optimization , 2013, IEEE J. Sel. Top. Signal Process..

[10]  J. Nash,et al.  NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[11]  RoughgardenTim Intrinsic robustness of the price of anarchy , 2012 .

[12]  F. Bullo,et al.  On collective motion in sensor networks: sample problems and distributed algorithms , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[13]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[14]  Jason R. Marden,et al.  Cooperative Control and Potential Games , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Srdjan S. Stankovic,et al.  Decentralized Parameter Estimation by Consensus Based Stochastic Approximation , 2007, IEEE Transactions on Automatic Control.

[16]  Jason R. Marden,et al.  Payoff-Based Dynamics for Multiplayer Weakly Acyclic Games , 2009, SIAM J. Control. Optim..

[17]  Tim Roughgarden,et al.  Guest Editorial Non-Cooperative Behavior in Networking , 2007, IEEE Journal on Selected Areas in Communications.

[18]  Kartik B. Ariyur,et al.  Real-Time Optimization by Extremum-Seeking Control , 2003 .

[19]  Milos S. Stankovic,et al.  Extremum seeking under stochastic noise and applications to mobile sensors , 2010, Autom..

[20]  G. Dimitoglou Collectives and the Design of Complex Systems by George Dimitoglou , 2009 .

[21]  Miroslav Krstic,et al.  Real-Time Optimization by Extremum-Seeking Control: Ariyur/Extremum Seeking , 2004 .

[22]  Sonia Martínez,et al.  Robust rendezvous for mobile autonomous agents via proximity graphs in arbitrary dimensions , 2006, IEEE Transactions on Automatic Control.

[23]  L. Shapley,et al.  REGULAR ARTICLEPotential Games , 1996 .

[24]  Miroslav Krstic,et al.  Nash equilibrium seeking with finitely- and infinitely-many players , 2010 .

[25]  Magnus Egerstedt,et al.  Graph Theoretic Methods in Multiagent Networks , 2010, Princeton Series in Applied Mathematics.

[26]  Siddhartha S. Srinivasa,et al.  Decentralized estimation and control of graph connectivity in mobile sensor networks , 2008, ACC.

[27]  Milos S. Stankovic,et al.  Distributed seeking of Nash equilibria in mobile sensor networks , 2010, 49th IEEE Conference on Decision and Control (CDC).

[28]  Miroslav Krstic,et al.  Extremum seeking for moderately unstable systems and for autonomous vehicle target tracking without position measurements , 2006, 2006 American Control Conference.

[29]  Pradeep Dubey,et al.  Inefficiency of Nash Equilibria , 1986, Math. Oper. Res..

[30]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[31]  H. Peyton Young,et al.  Learning by trial and error , 2009, Games Econ. Behav..

[32]  Claire J. Tomlin,et al.  Mobile Sensor Network Control Using Mutual Information Methods and Particle Filters , 2010, IEEE Transactions on Automatic Control.

[33]  Milos S. Stankovic,et al.  Stochastic extremum seeking with applications to mobile sensor networks , 2009, 2009 American Control Conference.

[34]  J. Goodman Note on Existence and Uniqueness of Equilibrium Points for Concave N-Person Games , 1965 .

[35]  Reza Olfati-Saber,et al.  Consensus and Cooperation in Networked Multi-Agent Systems , 2007, Proceedings of the IEEE.

[36]  Miroslav Krstic,et al.  Nash Equilibrium Seeking for Dynamic Systems with Non-Quadratic Payoffs , 2011 .

[37]  K. Johansson,et al.  Distributed positioning of autonomous mobile sensors with application to coverage control , 2011, Proceedings of the 2011 American Control Conference.

[38]  Ying Tan,et al.  On non-local stability properties of extremum seeking control , 2006, Autom..

[39]  Sonia Martínez,et al.  Coverage control for mobile sensing networks , 2002, IEEE Transactions on Robotics and Automation.

[40]  Miroslav Krstic,et al.  Source seeking with non-holonomic unicycle without position measurement and with tuning of forward velocity , 2007, Syst. Control. Lett..

[41]  Jonathan H. Manton,et al.  Stochastic Consensus Seeking With Noisy and Directed Inter-Agent Communication: Fixed and Randomly Varying Topologies , 2010, IEEE Transactions on Automatic Control.

[42]  Siddhartha S. Srinivasa,et al.  Decentralized estimation and control of graph connectivity in mobile sensor networks , 2008, 2008 American Control Conference.

[43]  Jin Soo Lee,et al.  Extremum seeking control for discrete-time systems , 2002, IEEE Trans. Autom. Control..

[44]  Jorge Cortés,et al.  Distributed Motion Constraints for Algebraic Connectivity of Robotic Networks , 2008, 2008 47th IEEE Conference on Decision and Control.

[45]  Karl Henrik Johansson,et al.  Bounded control of network connectivity in multi-agent systems , 2010 .

[46]  Sonia Martínez,et al.  Distributed coverage games for mobile visual sensors (I): Reaching the set of Nash equilibria , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[47]  Milos S. Stankovic,et al.  Discrete time extremum seeking by autonomous vehicles in a stochastic environment , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[48]  Jason R. Marden,et al.  Payoff based dynamics for multi-player weakly acyclic games , 2007, 2007 46th IEEE Conference on Decision and Control.

[49]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[50]  Tamer Basar,et al.  Distributed algorithms for the computation of noncooperative equilibria , 1987, Autom..