Stochastic Approximations with Constant Step Size and Differential Inclusions

We consider stochastic approximation processes with constant step size whose associated deterministic system is an upper semicontinuous differential inclusion. We prove that over any finite time span, the sample paths of the stochastic process are closely approximated by a solution of the differential inclusion with high probability. We then analyze infinite horizon behavior, showing that if the process is Markov, its stationary measures must become concentrated on the Birkhoff center of the deterministic system. Our results extend those of Benaim for settings in which the deterministic system is Lipschitz continuous and build on the work of Benaim, Hofbauer, and Sorin for the case of decreasing step sizes. We apply our results to models of population dynamics in games, obtaining new conclusions about the medium and long run behavior of myopic optimizing agents.

[1]  P. Walters.  Introduction to Ergodic Theory , 1977 .

[2]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[3]  Bruno Gaujal,et al.  Mean field limit of non-smooth systems and differential inclusions , 2010, PERV.

[4]  J. Hofbauer.  From Nash and Brown to Maynard Smith: Equilibria, Dynamics and ESS , 2001 .

[5]  Anna Nagurney,et al.  Projected Dynamical Systems in the Formulation, Stability Analysis, and Computation of Fixed-Demand Traffic Network Equilibria , 1997, Transp. Sci..

[6]  G. Brown.  SOME NOTES ON COMPUTATION OF GAMES SOLUTIONS , 1949 .

[7]  J. Hofbauer,et al.  BEST RESPONSE DYNAMICS FOR CONTINUOUS ZERO{SUM GAMES , 2005 .

[8]  Ziv Gorodeisky.  Deterministic approximation of best-response dynamics for the Matching Pennies game , 2009, Games Econ. Behav..

[9]  David M. Kreps,et al.  Learning Mixed Equilibria , 1993 .

[10]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[11]  William H. Sandholm,et al.  Sampling Best Response Dynamics and Deterministic Equilibrium Selection , 2014 .

[12]  Gregory Roth,et al.  Stochastic Approximations of Set-Valued Dynamical Systems: Convergence with Positive Probability to an Attractor , 2009, Math. Oper. Res..

[13]  William H. Sandholm,et al.  Population Games and Deterministic Evolutionary Dynamics , 2015 .

[14]  Ergodic Properties of weak Asymptotic Pseudotrajectories for Set-valued Dynamical Systems , 2011, 1101.2154.

[15]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[16]  I. Gilboa,et al.  Social Stability and Equilibrium , 1991 .

[17]  T. Kurtz.  Solutions of ordinary differential equations as limits of pure jump markov processes , 1970, Journal of Applied Probability.

[18]  William H. Sandholm,et al.  Large population potential games , 2009, J. Econ. Theory.

[19]  Michel Bena,et al.  Asymptotic Pseudotrajectories and Chain Recurrent Flows, with Applications , 1996 .

[20]  E. C. Zeeman,et al.  Population dynamics from game theory , 1980 .

[21]  Josef Hofbauer,et al.  Stochastic Approximations and Differential Inclusions , 2005, SIAM J. Control. Optim..

[22]  M. Benaïm,et al.  Deterministic Approximation of Stochastic Evolution in Games , 2003 .

[23]  Josef Hofbauer,et al.  Evolution in games with randomly disturbed payoffs , 2007, J. Econ. Theory.

[24]  D. Stroock,et al.  Probability Theory: An Analytic View. , 1995 .

[25]  Ziv Gorodeisky.  Stochastic Approximation of Discontinuous Dynamics , 2008 .

[26]  R. Mañé,et al.  Ergodic Theory and Differentiable Dynamics , 1986 .

[27]  Tamer Basar,et al.  Analysis of Recursive Stochastic Algorithms , 2001 .

[28]  M. Hirsch,et al.  Mixed Equilibria and Dynamical Systems Arising from Fictitious Play in Perturbed Games , 1999 .

[29]  L. Shapley,et al.  Potential Games , 1994 .

[30]  M. Benaïm.  Dynamics of stochastic approximation algorithms , 1999 .

[31]  Josef Hofbauer,et al.  Stable games and their dynamics , 2009, J. Econ. Theory.

[32]  Stephen Morris,et al.  P-dominance and belief potential , 2010 .

[33]  M. Métivier,et al.  Théorèmes de convergence presque sure pour une classe d'algorithmes stochastiques à pas décroissant , 1987 .

[34]  Josef Hofbauer,et al.  Refined Best-Response Correspondence and Dynamics , 2009 .

[35]  William H. Sandholm,et al.  ON THE GLOBAL CONVERGENCE OF STOCHASTIC FICTITIOUS PLAY , 2002 .

[36]  Olivier Tercieux,et al.  p-Best response set , 2006, J. Econ. Theory.

[37]  William H. Sandholm,et al.  Evolution and equilibrium under inexact information , 2003, Games Econ. Behav..

[38]  William H. Sandholm,et al.  Population Games And Evolutionary Dynamics , 2010, Economic learning and social evolution.

[39]  G. Smirnov.  Introduction to the Theory of Differential Inclusions , 2002 .

[40]  J. M. Smith,et al.  The Logic of Animal Conflict , 1973, Nature.

[41]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[42]  P. Taylor,et al.  Evolutionarily Stable Strategies and Game Dynamics , 1978 .

[43]  William H. Sandholm,et al.  Almost global convergence to p-dominant equilibrium , 2001, Int. J. Game Theory.

[44]  M. Mirzakhani,et al.  Introduction to Ergodic theory , 2010 .

[45]  William H. Sandholm,et al.  The projection dynamic and the geometry of population games , 2008, Games Econ. Behav..

[46]  M. Benaïm.  Recursive algorithms, urn processes and chaining number of chain recurrent sets , 1998, Ergodic Theory and Dynamical Systems.

[47]  William H. Sandholm,et al.  Potential Games with Continuous Player Sets , 2001, J. Econ. Theory.