Finding Optimal Abstract Strategies in Extensive-Form Games

Extensive-form games are a powerful model for representing interactions between agents. Nash equilibrium strategies are a common solution concept for extensive-form games and, in two-player zero-sum games, there are efficient algorithms for calculating such strategies. In large games, this computation may require too much memory and time to be tractable. A standard approach in such cases is to apply a lossy state-space abstraction technique to produce a smaller abstract game that can be tractably solved, while hoping that the resulting abstract game equilibrium is close to an equilibrium strategy in the unabstracted game. Recent work has shown that this assumption is unreliable, and an arbitrary Nash equilibrium in the abstract game is unlikely to be even near the least suboptimal strategy that can be represented in that space. In this work, we present for the first time an algorithm which efficiently finds optimal abstract strategies -- strategies with minimal exploitability in the unabstracted game. We use this technique to find the least exploitable strategy ever reported for two-player limit Texas hold'em.

[1]  Nicola Gatti,et al.  Extending the alternating-offers protocol in the presence of competition: models and theoretical analysis , 2009, Annals of Mathematics and Artificial Intelligence.

[2]  Troels Bjerre Lund,et al.  Potential-Aware Automated Abstraction of Sequential Games, and Holistic Equilibrium Analysis of Texas Hold'em Poker , 2007, AAAI.

[3]  Kevin Waugh,et al.  A Practical Use of Imperfect Recall , 2009, SARA.

[4]  Giuseppe Persiano,et al.  Special Issue on Algorithmic Game Theory , 2013, Theory of Computing Systems.

[5]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[6]  Kevin Waugh,et al.  Accelerating Best Response Calculation in Large Extensive Games , 2011, IJCAI.

[7]  Ariel D. Procaccia,et al.  Extensive-Form Argumentation Games , 2005, EUMAS.

[8]  Michael H. Bowling,et al.  Computing Robust Counter-Strategies , 2007, NIPS.

[9]  Peter McCracken,et al.  Safe Strategies for Agent Modelling in Games , 2004, AAAI Technical Report.

[10]  Michael H. Bowling,et al.  Efficient Nash equilibrium approximation through Monte Carlo counterfactual regret minimization , 2012, AAMAS.

[11]  Tuomas Sandholm,et al.  The State of Solving Large Incomplete-Information Games, and Application to Poker , 2010, AI Mag..

[12]  Kevin Waugh,et al.  Monte Carlo Sampling for Regret Minimization in Extensive Games , 2009, NIPS.

[13]  Jonathan Schaeffer,et al.  Approximating Game-Theoretic Optimal Strategies for Full-scale Poker , 2003, IJCAI.

[14]  Javier Peña,et al.  Smoothing Techniques for Computing Nash Equilibria of Sequential Games , 2010, Math. Oper. Res..

[15]  Tuomas Sandholm,et al.  Finding equilibria in large sequential games of imperfect information , 2006, EC '06.

[16]  Duane Szafron,et al.  Using counterfactual regret minimization to create competitive multiplayer poker agents , 2010, AAMAS 2010.

[17]  Kevin Waugh,et al.  Abstraction pathologies in extensive games , 2009, AAMAS.

[18]  Alessandro Lazaric,et al.  Reinforcement learning in extensive form games with incomplete information: the bargaining case study , 2007, AAMAS '07.

[19]  Victor R. Lesser,et al.  Self-interested database managers playing the view maintenance game , 2008, AAMAS.

[20]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[21]  Michael L. Littman,et al.  The 2006 AAAI Computer Poker Competition , 2006 .