Heads-up limit hold’em poker is solved

I'll see your program and raise you mine One of the fundamental differences between playing chess and two-handed poker is that the chessboard and the pieces on it are visible throughout the entire game, but an opponent's cards in poker are private. This informational deficit increases the complexity and the uncertainty in calculating the best course of action—to raise, to fold, or to call. Bowling et al. now report that they have developed a computer program that can do just that for the heads-up variant of poker known as Limit Texas Hold 'em (see the Perspective by Sandholm). Science, this issue p. 145; see also p. 122 A computer goes to Las Vegas. [Also see Perspective by Sandholm] Poker is a family of games that exhibit imperfect information, where players do not have full knowledge of past events. Whereas many perfect-information games have been solved (e.g., Connect Four and checkers), no nontrivial imperfect-information game played competitively by humans has previously been solved. Here, we announce that heads-up limit Texas hold’em is now essentially weakly solved. Furthermore, this computation formally proves the common wisdom that the dealer in the game holds a substantial advantage. This result was enabled by a new algorithm, CFR+, which is capable of solving extensive-form games orders of magnitude larger than previously possible.

[1]  Jean-Pierre Bourguignon,et al.  Mathematische Annalen , 1893 .

[2]  G. H. BRYAN.  Proceedings of the Fifth International Congress of Mathematicians , 1914, Nature.

[3]  J. Neumann.  Zur Theorie der Gesellschaftsspiele , 1928 .

[4]  J. Neumann,et al.  Theory of Games and Economic Behavior. , 1945 .

[5]  J. Thomson,et al.  Philosophical Magazine , 1945, Nature.

[6]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[7]  Claude E. Shannon,et al.  Programming a computer for playing chess , 1950 .

[8]  Philip Wolfe,et al.  Contributions to the theory of games , 1953 .

[9]  Claude E. Shannon,et al.  XXII. Programming a Computer for Playing Chess 1 , 1950 .

[10]  C. Babbage.  Passages from the Life of a Philosopher , 1968 .

[11]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[12]  Robert M Thrall,et al.  Mathematics of Operations Research. , 1978 .

[13]  R. J. Joenk,et al.  IBM journal of research and development: information for authors , 1978 .

[14]  Minsky,et al.  From the New York Times Magazine , 1979 .

[15]  R. Fox.  The ascent of man , 1983, Nature.

[16]  Nancy A. Lynch,et al.  Proceedings of the fifteenth annual ACM symposium on Theory of computing , 1983, STOC 1983.

[17]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, Comb..

[18]  Proceedings of the 16th Annual ACM Symposium on Theory of Computing, April 30 - May 2, 1984, Washington, DC, USA , 1984, Symposium on the Theory of Computing.

[19]  L. Victor Allis,et al.  A Knowledge-Based Approach of Connect-Four , 1988, J. Int. Comput. Games Assoc..

[20]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[21]  Philip Mirowski.  What Were von Neumann and Morgenstern Trying to Accomplish , 1992 .

[22]  D. Koller,et al.  The complexity of two-person zero-sum games in extensive form , 1992 .

[23]  L. V. Allis,et al.  Searching for solutions in games and artificial intelligence , 1994 .

[24]  Jonathan Schaeffer,et al.  CHINOOK: The World Man-Machine Checkers Champion , 1996, AI Mag..

[25]  D. Koller,et al.  Efficient Computation of Equilibria for Extensive Two-Person Games , 1996 .

[26]  Michael J. Todd,et al.  Mathematical programming , 2004, Handbook of Discrete and Computational Geometry, 2nd Ed..

[27]  Avi Pfeffer,et al.  Representations and Solutions for Game-Theoretic Problems , 1997, Artif. Intell..

[28]  Michael L. Littman,et al.  Abstraction Methods for Game Theoretic Poker , 2000, Computers and Games.

[29]  Ian Frank,et al.  Revised Papers from the Second International Conference on Computers and Games , 2000 .

[30]  Jonathan Schaeffer,et al.  The challenge of poker , 2002, Artif. Intell..

[31]  Murray Campbell,et al.  Deep Blue , 2002, Artif. Intell..

[32]  Michael Buro,et al.  Solving the Oshi-Zumo Game , 2003, ACG.

[33]  Jonathan Schaeffer,et al.  Approximating Game-Theoretic Optimal Strategies for Full-scale Poker , 2003, IJCAI.

[34]  News Item , 2004, Acta Neuropathologica.

[35]  Michael H. Bowling,et al.  Bayes' Bluff: Opponent Modelling in Poker , 2005, UAI 2005.

[36]  Michael Craig.  The Professor, the Banker, and the Suicide King: Inside the Richest Poker Game of All Time , 2005 .

[37]  Yurii Nesterov,et al.  Excessive Gap Technique in Nonsmooth Convex Minimization , 2005, SIAM J. Optim..

[38]  Dan Suciu,et al.  Journal of the ACM , 2006 .

[39]  Javier Peña,et al.  Gradient-Based Algorithms for Finding Nash Equilibria in Extensive Form Games , 2007, WINE.

[40]  Tuomas Sandholm,et al.  Lossless abstraction of imperfect information games , 2007, JACM.

[41]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[42]  Zoubin Ghahramani,et al.  Proceedings of the 24th international conference on Machine learning , 2007, ICML 2007.

[43]  Jonathan Schaeffer,et al.  Checkers Is Solved , 2007, Science.

[44]  Andrew McCallum,et al.  Proceedings, Twenty-fifth International Conference on Machine Learning , 2008 .

[45]  Javier Peña,et al.  First-Order Algorithm with O(ln(1/e)) Convergence for e-Equilibrium in Two-Person Zero-Sum Games , 2008, AAAI.

[46]  Duane Szafron,et al.  Strategy evaluation in extensive games with importance sampling , 2008, ICML '08.

[47]  Christos H. Papadimitriou,et al.  Proceedings of the 4th International Workshop on Internet and Network Economics , 2008 .

[48]  Kevin Waugh,et al.  Strategy Grafting in Extensive Games , 2009, NIPS.

[49]  Javier Peña,et al.  Smoothing Techniques for Computing Nash Equilibria of Sequential Games , 2010, Math. Oper. Res..

[50]  Tuomas Sandholm,et al.  The State of Solving Large Incomplete-Information Games, and Application to Poker , 2010, AI Mag..

[51]  Kokolo Ikeda,et al.  Advances in Computer Games , 2011, Lecture Notes in Computer Science.

[52]  Ian D. Watson,et al.  Computer poker: A review , 2011, Artif. Intell..

[53]  Oriol Carbonell-Nicolau.  Games and Economic Behavior , 2011 .

[54]  Kevin Waugh,et al.  Accelerating Best Response Calculation in Large Extensive Games , 2011, IJCAI.

[55]  Daniele Magazzeni,et al.  Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI-11). , 2011, International Joint Conference on Artificial Intelligence.

[56]  Toby Walsh,et al.  Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three , 2011, International Joint Conference on Artificial Intelligence.

[57]  Tommi S. Jaakkola,et al.  Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence (2005) , 2012, ArXiv.

[58]  Javier Peña,et al.  First-order algorithm with $${\mathcal{O}({\rm ln}(1{/}\epsilon))}$$ convergence for $${\epsilon}$$-equilibrium in two-person zero-sum games , 2012, Math. Program..

[59]  Nicholas I. M. Gould,et al.  SIAM Journal on Optimization , 2012 .

[60]  Michael H. Bowling,et al.  Finding Optimal Abstract Strategies in Extensive-Form Games , 2012, AAAI.

[61]  David A. Ferrucci,et al.  Introduction to "This is Watson" , 2012, IBM J. Res. Dev..

[62]  Michael H. Bowling,et al.  Efficient Nash equilibrium approximation through Monte Carlo counterfactual regret minimization , 2012, AAMAS.

[63]  Michael H. Bowling,et al.  Tractable Objectives for Robust Policy Optimization , 2012, NIPS.

[64]  S. Barry Cooper,et al.  Digital Computers Applied to Games , 2013 .

[65]  Joseph Y. Halpern,et al.  Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence , 2014, AAAI 2014.

[66]  Richard G. Gibson.  Regret Minimization in Games and the Development of Champion Multiplayer Computer Poker-Playing Agents , 2014 .

[67]  Sam Ganzfried.  AAAI-15 Workshop on Computer Poker and Imperfect Information , 2015 .

[68]  Kevin Waugh,et al.  A Unified View of Large-Scale Zero-Sum Equilibrium Computation , 2014, AAAI Workshop: Computer Poker and Imperfect Information.

[69]  Michael H. Bowling,et al.  Solving Heads-Up Limit Texas Hold'em , 2015, IJCAI.

[70]  Michael Bradley Johanson.  Robust Strategies and Counter-Strategies: From Superhuman to Optimal Play , 2016 .

[71]  Juliane Hahn,et al.  Security And Game Theory Algorithms Deployed Systems Lessons Learned , 2016 .

[72]  Michael H. Bowling,et al.  Heads-up limit hold'em poker is solved , 2017, Commun. ACM.