Libratus: The Superhuman AI for No-Limit Poker

No-limit Texas Hold'em is the most popular variant of poker in the world. Heads-up no-limit Texas Hold'em is the main benchmark challenge for AI in imperfect-information games. We present Libratus, the first--and so far only--AI to defeat top human professionals in that game. Libratus's architecture features three main modules, each of which has new algorithms: pre-computing a solution to an abstraction of the game which provides a high-level blueprint for the strategy of the AI, a new nested subgame-solving algorithm which repeatedly calculates a more detailed strategy as play progresses, and a self-improving module which augments the pre-computed blueprint over time.

[1]  Tuomas Sandholm,et al.  Reduced Space and Faster Convergence in Imperfect-Information Games via Pruning , 2017, ICML.

[2]  Milan Hladík,et al.  Refining Subgames in Large Imperfect Information Games , 2016, AAAI.

[3]  Tuomas Sandholm,et al.  Safe and Nested Subgame Solving for Imperfect-Information Games , 2017, NIPS.

[4]  E. Jackson.  A Time and Space Efficient Algorithm for Approximately Solving Large Imperfect Information Games , 2014 .

[5]  Neil Burch,et al.  Heads-up limit hold’em poker is solved , 2015, Science.

[6]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[7]  Nicholas I. M. Gould,et al.  SIAM Journal on Optimization , 2012 .

[8]  Jonathan Schaeffer,et al.  Checkers Is Solved , 2007, Science.

[9]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[10]  Kevin Waugh,et al.  Monte Carlo Sampling for Regret Minimization in Extensive Games , 2009, NIPS.

[11]  Tuomas Sandholm,et al.  Endgame Solving in Large Imperfect-Information Games , 2015, AAAI Workshop: Computer Poker and Imperfect Information.

[12]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[13]  Michael H. Bowling,et al.  Solving Imperfect Information Games Using Decomposition , 2013, AAAI.

[14]  Michael H. Bowling,et al.  Solving Heads-Up Limit Texas Hold'em , 2015, IJCAI.

[15]  Kevin Waugh,et al.  Accelerating Best Response Calculation in Large Extensive Games , 2011, IJCAI.

[16]  Tuomas Sandholm,et al.  Strategy-Based Warm Starting for Regret Minimization in Games , 2016, AAAI.

[17]  Michael H. Bowling,et al.  Asymmetric abstractions for adversarial settings , 2014, AAMAS.

[18]  L. Christophorou.  Science , 2018, Emerging Dynamics: Science, Energy, Society and Values.

[19]  Chandler Jake,et al.  Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) , 2016 .

[20]  Yurii Nesterov,et al.  Excessive Gap Technique in Nonsmooth Convex Minimization , 2005, SIAM J. Optim..

[21]  Tuomas Sandholm,et al.  Hierarchical Abstraction, Distributed Equilibrium Computation, and Post-Processing, with Application to a Champion No-Limit Texas Hold'em Agent , 2015, AAAI Workshop: Computer Poker and Imperfect Information.

[22]  Michael H. Bowling,et al.  Regret Minimization in Games with Incomplete Information , 2007, NIPS.

[23]  Tuomas Sandholm,et al.  Regret Transfer and Parameter Optimization , 2014, AAAI.

[24]  Tuomas Sandholm,et al.  Baby Tartanian8: Winning Agent from the 2016 Annual Computer Poker Competition , 2016, IJCAI.

[25]  Tuomas Sandholm,et al.  Regret-Based Pruning in Extensive-Form Games , 2015, NIPS.

[26]  Kevin Waugh,et al.  Theoretical and Practical Advances on Smoothing for Extensive-Form Games , 2017, EC.