On the Rate of Convergence of Fictitious Play

Fictitious play is a simple learning algorithm for strategic games that proceeds in rounds. In each round, the players play a best response to a mixed strategy that is given by the empirical frequencies of actions played in previous rounds. There is a close relationship between fictitious play and the Nash equilibria of a game: if the empirical frequencies of fictitious play converge to a strategy profile, this strategy profile is a Nash equilibrium. While fictitious play does not converge in general, it is known to do so for certain restricted classes of games, such as constantsum games, non-degenerate 2×n games, and potential games. We study the rate of convergence of fictitious play and show that, in all the classes of games mentioned above, fictitious play may require an exponential number of rounds (in the size of the representation of the game) before some equilibrium action is eventually played. In particular, we show the above statement for symmetric constant-sum win-lose-tie games.

[1]  L. Shapley.  SOME TOPICS IN TWO-PERSON GAMES , 1963 .

[2]  J. Neumann.  Zur Theorie der Gesellschaftsspiele , 1928 .

[3]  Ulrich Berger,et al.  Brown's original fictitious play , 2007, J. Econ. Theory.

[4]  J. Robinson.  AN ITERATIVE METHOD OF SOLVING A GAME , 1951, Classics in Game Theory.

[5]  Yoav Shoham,et al.  New Criteria and a New Algorithm for Learning in Multi-Agent Systems , 2004, NIPS.

[6]  L. Shapley,et al.  Fictitious Play Property for Games with Identical Interests , 1996 .

[7]  D. Monderer,et al.  A 2£ 2 Game without the Fictitious Play Property ⁄ , 1996 .

[8]  Peter R. Wurman,et al.  Structural leverage and fictitious play in sequential auctions , 2002, AAAI/IAAI.

[9]  Ulrich Berger,et al.  Fictitious play in 2×n games , 2005, J. Econ. Theory.

[10]  Howard Raiffa,et al.  Games and Decisions: Introduction and Critical Survey. , 1958 .

[11]  Tuomas Sandholm,et al.  Computing an approximate jam/fold equilibrium for 3-player no-limit Texas Hold'em tournaments , 2008, AAMAS.

[12]  Steven Gjerstad.  The rate of convergence of continuous fictitious play , 1996 .

[13]  Philip Wolfe,et al.  Contributions to the theory of games , 1953 .

[14]  John Nachbar.  “Evolutionary” selection dynamics in games: Convergence and limit properties , 1990 .

[15]  L. Shapley,et al.  REGULAR ARTICLEPotential Games , 1996 .

[16]  S. Hahn.  The convergence of fictitious play in 3×3 games with strategic complementarities , 1999 .

[17]  J. Neumann.  A numerical method to determine optimum strategy , 1954 .

[18]  V. Conitzer,et al.  Approximation guarantees for fictitious play , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[19]  Samuel Karlin,et al.  Mathematical Methods and Theory in Games, Programming, and Economics , 1961 .

[20]  H. N. Shapiro.  Note on a Computation Method in the Theory of Games , 1958 .

[21]  Ulrich Berger.  Learning in games with strategic complementarities revisited , 2008, J. Econ. Theory.

[22]  Ulrich Berger.  The convergence of fictitious play in games with strategic complementarities: A Comment , 2009 .

[23]  D. Fudenberg,et al.  Consistency and Cautious Fictitious Play , 1995 .

[24]  Vincent Conitzer,et al.  AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents , 2003, Machine Learning.

[25]  R. Bellman.  ON A NEW ITERATIVE ALGORITHM FOR FINDING THE SOLUTIONS OF GAMES AND LINEAR PROGRAMMING PROBLEMS , 1953 .

[26]  Felix A. Fischer,et al.  On the Hardness and Existence of Quasi-Strict Equilibria , 2008, SAGT.