Abstraction for Solving Large Incomplete-Information Games

Most real-world games and many recreational games are games of incomplete information. Over the last dozen years, abstraction has emerged as a key enabler for solving large incomplete-information games. First, the game is abstracted to generate a smaller, abstract game that is strategically similar to the original game. Second, an approximate equilibrium is computed in the abstract game. Third, the strategy from the abstract game is mapped back to the original game. In this paper, I will review key developments in the field. I present reasons for abstracting games, and point out the issue of abstraction pathology. I then review the practical algorithms for information abstraction and action abstraction. I then cover recent theoretical breakthroughs that beget bounds on the quality of the strategy from the abstract game, when measured in the original game. I then discuss how to reverse map the opponent's action into the abstraction if the opponent makes a move that is not in the abstraction. Finally, I discuss other topics of current and future research.

[1]  Philip Wolfe,et al.  Contributions to the theory of games , 1953 .

[2]  Michael L. Littman,et al.  Abstraction Methods for Game Theoretic Poker , 2000, Computers and Games.

[3]  Jonathan Schaeffer,et al.  Approximating Game-Theoretic Optimal Strategies for Full-scale Poker , 2003, IJCAI.

[4]  Michael P. Wellman,et al.  Approximate Strategic Reasoning through Hierarchical Reduction of Large Symmetric Games , 2005, AAAI.

[5]  Tuomas Sandholm,et al.  A Competitive Texas Hold'em Poker Player via Automated Abstraction and Real-Time Equilibrium Computation , 2006, AAAI.

[6]  Tuomas Sandholm,et al.  Finding equilibria in large sequential games of imperfect information , 2006, EC '06.

[7]  Michael P. Wellman.  Methods for Empirical Game-Theoretic Analysis , 2006, AAAI.

[8]  Bill Chen,et al.  The Mathematics of Poker , 2006 .

[9]  Michael P. Wellman,et al.  Methods for empirical game-theoretic analysis (extended abstract) , 2006 .

[10]  Tuomas Sandholm,et al.  Better automated abstraction techniques for imperfect information games, with application to Texas Hold'em poker , 2007, AAMAS '07.

[11]  Tuomas Sandholm,et al.  Lossless abstraction of imperfect information games , 2007, JACM.

[12]  Troels Bjerre Lund,et al.  Potential-Aware Automated Abstraction of Sequential Games, and Holistic Equilibrium Analysis of Texas Hold'em Poker , 2007, AAAI.

[13]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[14]  Troels Bjerre Lund,et al.  A heads-up no-limit Texas Hold'em poker player: discretized betting models and automatically generated equilibrium-finding programs , 2008, AAMAS.

[15]  Michael H. Bowling,et al.  Probabilistic State Translation in Extensive Games with Large Action Sets , 2009, IJCAI.

[16]  Christopher Archibald,et al.  Modeling billiards games , 2009, AAMAS.

[17]  Kevin Waugh,et al.  A Practical Use of Imperfect Recall , 2009, SARA.

[18]  Kevin Waugh,et al.  Abstraction pathologies in extensive games , 2009, AAMAS.

[19]  Tuomas Sandholm,et al.  The State of Solving Large Incomplete-Information Games, and Application to Poker , 2010, AI Mag..

[20]  Duane Szafron,et al.  Automated Action Abstraction of Imperfect Information Extensive-Form Games , 2011, AAAI.

[21]  Kevin Waugh,et al.  Accelerating Best Response Calculation in Large Extensive Games , 2011, IJCAI.

[22]  Nicola Basilico,et al.  Automated Abstractions for Patrolling Security Games , 2011, AAAI.

[23]  Michael H. Bowling,et al.  No-Regret Learning in Extensive-Form Games with Imperfect Recall , 2012, ICML.

[24]  Kevin Waugh,et al.  Strategy purification and thresholding: effective non-equilibrium approaches for playing large games , 2012, AAMAS.

[25]  Tuomas Sandholm,et al.  Lossy stochastic game abstraction with bounds , 2012, EC '12.

[26]  Duane Szafron,et al.  Using Sliding Windows to Generate Action Abstractions in Extensive-Form Games , 2012, AAAI.

[27]  T. Sandholm,et al.  Tartanian 5 : A Heads-Up No-Limit Texas Hold ’ em Poker-Playing Program ∗ , 2012 .

[28]  Ian D. Watson,et al.  Case-based strategies in computer poker , 2012, AI Commun..

[29]  Michael Johanson,et al.  Measuring the Size of Large No-Limit Poker Games , 2013, ArXiv.

[30]  Giuseppe Persiano,et al.  Special Issue on Algorithmic Game Theory , 2013, Theory of Computing Systems.

[31]  Michael H. Bowling,et al.  Evaluating state-space abstractions in extensive-form games , 2013, AAMAS.

[32]  Tuomas Sandholm,et al.  Action Translation in Extensive-Form Games with Large Action Spaces: Axioms, Paradoxes, and the Pseudo-Harmonic Mapping , 2013, IJCAI.

[33]  Eric Griffin Jackson,et al.  Slumbot NL: Solving Large Games with Counterfactual Regret Minimization Using Sampling and Distributed Processing , 2013, AAAI 2013.

[34]  Tuomas Sandholm,et al.  Potential-Aware Imperfect-Recall Abstraction with Earth Mover's Distance in Imperfect-Information Games , 2014, AAAI.

[35]  Tuomas Sandholm,et al.  Extensive-form game abstraction with bounds , 2014, EC.

[36]  Tuomas Sandholm,et al.  Extensive-Form Game Imperfect-Recall Abstractions With Bounds , 2014, ArXiv.

[37]  Tuomas Sandholm,et al.  Regret Transfer and Parameter Optimization , 2014, AAAI.

[38]  Tuomas Sandholm,et al.  Tartanian7: A Champion Two-Player No-Limit Texas Hold'em Poker-Playing Program , 2015, AAAI.

[39]  Tuomas Sandholm,et al.  Hierarchical Abstraction, Distributed Equilibrium Computation, and Post-Processing, with Application to a Champion No-Limit Texas Hold'em Agent , 2015, AAAI Workshop: Computer Poker and Imperfect Information.