A Class of Self-Interacting Processes with Applications to Games and Reinforced Random Walks

This paper studies a class of non-Markovian and nonhomogeneous stochastic processes on a finite state space. Relying on a recent paper by Benaim, Hofbauer, and Sorin [SIAM J. Control Optim., 44 (2005), pp. 328-348] it is shown that, under certain assumptions, the asymptotic behavior of occupation measures can be described in terms of a certain set-valued deterministic dynamical system. This provides a unified approach to simulated annealing type processes and permits the study of new models of vertex reinforced random walks and new models of learning in games such as Markovian fictitious play.

[1]  M. Benaïm.  Vertex-reinforced random walks and a conjecture of Pemantle , 1997 .

[2]  L. Saloff-Coste,et al.  Lectures on finite Markov chains , 1997 .

[3]  R. Pemantle.  Vertex-reinforced random walk , 1992, math/0404041.

[4]  L. Shapley,et al.  Fictitious Play Property for Games with Identical Interests , 1996 .

[5]  M. Benaïm.  Dynamics of stochastic approximation algorithms , 1999 .

[6]  D. Stroock,et al.  Simulated annealing via Sobolev inequalities , 1988 .

[7]  S. Volkov.  Vertex-reinforced random walk on arbitrary graphs , 1999, math/9907196.

[8]  M. Benaim,et al.  VERTEX-REINFORCED RANDOM WALKS AND A CONJECTURE OF PEMANTLE , 2002 .

[9]  A survey of random processes with reinforcement , 2007, math/0610076.

[10]  M. Benaïm,et al.  Self-interacting diffusions. III. Symmetric interactions , 2005 .

[11]  Bruce E. Hajek,et al.  Cooling Schedules for Optimal Annealing , 1988, Math. Oper. Res..

[12]  Stanislav Volkov,et al.  Vertex-reinforced random walk on Z has finite range , 1999 .

[13]  David M. Kreps,et al.  Learning Mixed Equilibria , 1993 .

[14]  M. Hirsch,et al.  Asymptotic pseudotrajectories and chain recurrent flows, with applications , 1996 .

[15]  William H. Sandholm,et al.  ON THE GLOBAL CONVERGENCE OF STOCHASTIC FICTITIOUS PLAY , 2002 .

[16]  Michel Ledoux,et al.  Self-interacting diffusions , 2002 .

[17]  Josef Hofbauer,et al.  Stochastic Approximations and Differential Inclusions, Part II: Applications , 2006, Math. Oper. Res..

[18]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[19]  Michel Benaïm,et al.  SELF-INTERACTING DIFFUSIONS II: CONVERGENCE IN LAW , 2003 .

[20]  Olivier Raimond.  Self-interacting diffusions: a simulated annealing version , 2009 .

[21]  Tamer Basar,et al.  Analysis of Recursive Stochastic Algorithms , 2001 .

[22]  M. Hirsch,et al.  Mixed Equilibria and Dynamical Systems Arising from Fictitious Play in Perturbed Games , 1999 .

[23]  Laurent Miclo.  Recuit simulé sans potentiel sur un ensemble fini , 1992 .

[24]  Josef Hofbauer,et al.  Stochastic Approximations and Differential Inclusions , 2005, SIAM J. Control. Optim..

[25]  M. Métivier,et al.  Théorèmes de convergence presque sure pour une classe d'algorithmes stochastiques à pas décroissant , 1987 .