Quick Polytope Approximation of All Correlated Equilibria in Stochastic Games

Stochastic or Markov games serve as reasonable models for a variety of domains from biology to computer security, and are appealing due to their versatility. In this paper we address the problem of finding the complete set of correlated equilibria for general-sum stochastic games with perfect information. We present QPACE – an algorithm orders of magnitude more efficient than previous approaches while maintaining a guarantee of convergence and bounded error. Finally, we validate our claims and demonstrate the limits of our algorithm with extensive empirical tests.

[1]  Dilip Abreu.  On the Theory of Infinitely Repeated Games with Discounting , 1988 .

[2]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[3]  M. Cronshaw,et al.  Algorithms for Finding Repeated Game Equilibria , 1997 .

[4]  Michael L. Littman,et al.  Friend-or-Foe Q-learning in General-Sum Games , 2001, ICML.

[5]  R. Amir.  STOCHASTIC GAMES IN ECONOMICS AND RELATED FIELDS: AN OVERVIEW , 2001 .

[6]  K. Judd,et al.  Computing Supergame Equilibria , 2003 .

[7]  A. Arkin,et al.  Motifs, modules and games in bacteria. , 2003, Current opinion in microbiology.

[8]  Keith B. Hall,et al.  Correlated Q-Learning , 2003, ICML.

[9]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[10]  Michael L. Littman,et al.  Cyclic Equilibria in Markov Games , 2005, NIPS.

[11]  Geoffrey J. Gordon,et al.  Multi-Robot Negotiation: Approximating the Set of Subgame Perfect Equilibria in General-Sum Stochastic Games , 2006, NIPS.

[12]  Geoffrey J. Gordon.  Agendas for multi-agent learning , 2007, Artif. Intell..

[13]  Tansu Alpcan,et al.  Stochastic games for security in networks with interdependent nodes , 2009, 2009 International Conference on Game Theory for Networks.

[14]  Charles L. Isbell,et al.  Solving Stochastic Games , 2009, NIPS.

[15]  Johannes Horner,et al.  Recursive Methods in Discounted Stochastic Games: An Algorithm for δ → 1 and a Folk Theorem , 2010 .

[16]  Brahim Chaib-draa,et al.  An Approximate Subgame-Perfect Equilibrium Computation Technique for Repeated Games , 2010, AAAI.