On the Rate of Convergence of Fictitious Play

Fictitious play is a simple learning algorithm for strategic games that proceeds in rounds. In each round, the players play a best response to a mixed strategy that is given by the empirical frequencies of actions played in previous rounds. There is a close relationship between fictitious play and the Nash equilibria of a game: if the empirical frequencies of fictitious play converge to a strategy profile, this strategy profile is a Nash equilibrium. While fictitious play does not converge in general, it is known to do so for certain restricted classes of games, such as constant-sum games, non-degenerate 2×n games, and potential games. We study the rate of convergence of fictitious play and show that, in all the classes of games mentioned above, fictitious play may require an exponential number of rounds (in the size of the representation of the game) before some equilibrium action is eventually played. In particular, we show the above statement for symmetric constant-sum win-lose-tie games.

[1]  K. Miyasawa.  ON THE CONVERGENCE OF THE LEARNING PROCESS IN A 2 X 2 NON-ZERO-SUM TWO-PERSON GAME , 1961 .

[2]  Ulrich Berger.  Learning in games with strategic complementarities revisited , 2008, J. Econ. Theory.

[3]  H. N. Shapiro.  Note on a Computation Method in the Theory of Games , 1958 .

[4]  Troels Bjerre Lund,et al.  On the approximation performance of fictitious play in finite games , 2011, Int. J. Game Theory.

[5]  H. Raiffa,et al.  Games and Decisions: Introduction and Critical Survey. , 1958 .

[6]  A. W. Tucker,et al.  Advances in game theory , 1964 .

[7]  J. Neumann.  Zur Theorie der Gesellschaftsspiele , 1928 .

[8]  Ulf-Peter Schroeder,et al.  Proceedings of the 1st International Symposium on Algorithmic Game Theory , 2008 .

[9]  Tuomas Sandholm,et al.  Computing an approximate jam/fold equilibrium for 3-player no-limit Texas Hold'em tournaments , 2008, AAMAS.

[10]  David Gale,et al.  Review: R. Duncan Luce and Howard Raiffa, Games and decisions: Introduction and critical survey , 1958 .

[11]  Ulrich Berger,et al.  Brown's original fictitious play , 2007, J. Econ. Theory.

[12]  J. Robinson.  AN ITERATIVE METHOD OF SOLVING A GAME , 1951, Classics in Game Theory.

[13]  L. Shapley,et al.  Potential Games , 1994 .

[14]  Peter R. Wurman,et al.  Structural leverage and fictitious play in sequential auctions , 2002, AAAI/IAAI.

[15]  Ulrich Berger.  The convergence of fictitious play in games with strategic complementarities: A Comment , 2009 .

[16]  D. Fudenberg,et al.  Consistency and Cautious Fictitious Play , 1995 .

[17]  Marc Nerlove,et al.  Mathematical Methods and Theory in Games, Programming, and Economics, Vol. I: Matrix Games, Programming, and Mathematical Economics. Vol. II: The Theory of Infinite Games. Samuel Karlin , 1960 .

[18]  Yoav Shoham,et al.  New Criteria and a New Algorithm for Learning in Multi-Agent Systems , 2004, NIPS.

[19]  Ulrich Berger,et al.  Fictitious play in 2×n games , 2005, J. Econ. Theory.

[20]  Vincent Conitzer,et al.  AWESOME: A general multiagent learning algorithm that converges in self-play and learns a best response against stationary opponents , 2003, Machine Learning.

[21]  L. Shapley,et al.  Fictitious Play Property for Games with Identical Interests , 1996 .

[22]  L. Shapley.  SOME TOPICS IN TWO-PERSON GAMES , 1963 .

[23]  L. Shapley,et al.  REGULAR ARTICLEPotential Games , 1996 .

[24]  D. Monderer,et al.  A 2£ 2 Game without the Fictitious Play Property ⁄ , 1996 .

[25]  J. Neumann.  A numerical method to determine optimum strategy , 1954 .

[26]  V. Conitzer,et al.  Approximation guarantees for fictitious play , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[27]  Samuel Karlin,et al.  Mathematical Methods and Theory in Games, Programming, and Economics , 1961 .

[28]  R. Bellman.  ON A NEW ITERATIVE ALGORITHM FOR FINDING THE SOLUTIONS OF GAMES AND LINEAR PROGRAMMING PROBLEMS , 1953 .

[29]  Felix A. Fischer,et al.  On the Hardness and Existence of Quasi-Strict Equilibria , 2008, SAGT.

[30]  S. Hahn.  The convergence of fictitious play in 3×3 games with strategic complementarities , 1999 .

[31]  Philip Wolfe,et al.  Contributions to the theory of games , 1953 .

[32]  Steven Gjerstad.  The rate of convergence of continuous fictitious play★ , 1995 .

[33]  John Nachbar.  “Evolutionary” selection dynamics in games: Convergence and limit properties , 1990 .

[34]  O. H. Brownlee,et al.  ACTIVITY ANALYSIS OF PRODUCTION AND ALLOCATION , 1952 .