Markov Games of Incomplete Information for Multi-Agent Reinforcement Learning

Partially observable stochastic games (POSGs) are an attractive model for many multi-agent domains, but are computationally extremely difficult to solve. We present a new model, Markov games of incomplete information (MGII) which imposes a mild restriction on POSGs while overcoming their primary computational bottleneck. Finally we show how to convert a MGII into a continuous but bounded fully observable stochastic game. MGIIs represents the most general tractable model for multi-agent reinforcement learning to date.

[1]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[2]  Tuomas Sandholm,et al.  The State of Solving Large Incomplete-Information Games, and Application to Poker , 2010, AI Mag..

[3]  Charles L. Isbell,et al.  Solving Stochastic Games , 2009, NIPS.

[4]  Prashant Doshi,et al.  Monte Carlo Sampling Methods for Approximating Interactive POMDPs , 2009, J. Artif. Intell. Res..

[5]  Shlomo Zilberstein,et al.  Memory-Bounded Dynamic Programming for DEC-POMDPs , 2007, IJCAI.

[6]  F. Forges Correlated Equilibrium in Games with Incomplete Information Revisited , 2006 .

[7]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[8]  G. Gordon,et al.  Finding Correlated Equilibria in General Sum Stochastic Games , 2007 .

[9]  Leslie Pack Kaelbling,et al.  Multi-Agent Filtering with Infinitely Nested Beliefs , 2008, NIPS.

[10]  Leslie Pack Kaelbling,et al.  Acting Optimally in Partially Observable Stochastic Domains , 1994, AAAI.

[11]  Indrajit Ray,et al.  A difficulty in implementing correlated equilibrium distributions , 2010, Games Econ. Behav..

[12]  Jeff G. Schneider,et al.  Approximate solutions for partially observable stochastic games with common payoffs , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[13]  Tuomas Sandholm,et al.  Solving two-person zero-sum repeated games of incomplete information , 2008, AAMAS.

[14]  Charles Lee Isbell,et al.  Quick Polytope Approximation of All Correlated Equilibria in Stochastic Games , 2011, AAAI.