Practical exact algorithm for trembling-hand equilibrium refinements in games

Nash equilibrium strategies have the known weakness that they do not prescribe rational play in situations that are reached with zero probability according to the strategies themselves, for example, if players have made mistakes. Trembling-hand refinements---such as extensive-form perfect equilibria and quasi-perfect equilibria---remedy this problem in sound ways. Despite their appeal, they have not received attention in practice since no known algorithm for computing them scales beyond toy instances. In this paper, we design an exact polynomial-time algorithm for finding trembling-hand equilibria in zero-sum extensive-form games. It is several orders of magnitude faster than the best prior ones, numerically stable, and quickly solves game instances with tens of thousands of nodes in the game tree. It enables, for the first time, the use of trembling-hand refinements in practice.

[1]  Peter Bro Miltersen,et al.  Computing a quasi-perfect equilibrium of a two-player game , 2010 .

[2]  Nicola Gatti,et al.  Extensive-Form Perfect Equilibrium Computation in Two-Player Games , 2017, AAAI.

[3]  Tuomas Sandholm,et al.  Smoothing Method for Approximate Extensive-Form Perfect Equilibrium , 2017, IJCAI.

[4]  S. Ross.  GOOFSPIEL -- THE GAME OF PURE STRATEGY , 1971 .

[5]  Noam Brown,et al.  Superhuman AI for heads-up no-limit poker: Libratus beats top professionals , 2018, Science.

[6]  Tuomas Sandholm,et al.  Endgame Solving in Large Imperfect-Information Games , 2015, AAAI Workshop: Computer Poker and Imperfect Information.

[7]  John N. Tsitsiklis,et al.  Introduction to linear optimization , 1997, Athena scientific optimization and computation series.

[8]  E. Vandamme.  Stability and perfection of nash equilibria , 1987 .

[9]  R. Myerson.  Refinements of the Nash equilibrium concept , 1978 .

[10]  R. Selten.  Reexamination of the perfectness concept for equilibrium points in extensive games , 1975, Classics in Game Theory.

[11]  John Hillas.  ON THE RELATION BETWEEN PERFECT EQUILIBRIA IN EXTENSIVE FORM GAMES AND PROPER EQUILIBRIA IN NORMAL FORM GAMES , 1996 .

[12]  D. Koller,et al.  Efficient Computation of Equilibria for Extensive Two-Person Games , 1996 .

[13]  D. Griffel.  Linear programming 2: Theory and extensions , by G. B. Dantzig and M. N. Thapa. Pp. 408. £50.00. 2003 ISBN 0 387 00834 9 (Springer). , 2004, The Mathematical Gazette.

[14]  Yoav Shoham,et al.  Multiagent Systems - Algorithmic, Game-Theoretic, and Logical Foundations , 2009 .

[15]  B. Stengel,et al.  Efficient Computation of Behavior Strategies , 1996 .

[16]  J. Nash.  Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[17]  E. Damme.  Refinements of the Nash Equilibrium Concept , 1983 .

[18]  Branislav Bosanský,et al.  Practical Performance of Refinements of Nash Equilibria in Extensive-Form Zero-Sum Games , 2014, ECAI.

[19]  Tuomas Sandholm,et al.  Safe and Nested Subgame Solving for Imperfect-Information Games , 2017, NIPS.

[20]  Peter Bro Miltersen,et al.  Computing sequential equilibria for two-player games , 2006, SODA '06.

[21]  Tuomas Sandholm,et al.  Reduced Space and Faster Convergence in Imperfect-Information Games via Pruning , 2017, ICML.

[22]  E. Kohlberg,et al.  Foundations of Strategic Equilibrium , 1996 .