Stochastic Approximations of Set-Valued Dynamical Systems: Convergence with Positive Probability to an Attractor

A successful method to describe the asymptotic behavior of a discrete time stochastic process governed by some recursive formula is to relate it to the limit sets of a well-chosen mean differential equation. Under an attainability condition, Benaim proved that convergence to a given attractor of the flow induced by this dynamical system occurs with positive probability for a class of Robbins Monro algorithms. Benaim, Hofbauer, and Sorin generalised this approach for stochastic approximation algorithms whose average behavior is related to a differential inclusion instead. We pursue the analogy by extending to this setting the result of convergence with positive probability to an attractor.

[1]  R. Bowen.  ω-Limit sets for Axiom A diffeomorphisms , 1975 .

[2]  J. Mertens,et al.  ON THE STRATEGIC STABILITY OF EQUILIBRIA , 1986 .

[3]  Josef Hofbauer,et al.  Stochastic Approximations and Differential Inclusions, Part II: Applications , 2006, Math. Oper. Res..

[4]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[5]  Josef Hofbauer,et al.  Stochastic Approximations and Differential Inclusions , 2005, SIAM J. Control. Optim..

[6]  M. Benaïm.  A Dynamical System Approach to Stochastic Approximations , 1996 .

[7]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[8]  David M. Kreps,et al.  Learning Mixed Equilibria , 1993 .

[9]  M. Benaïm.  Dynamics of stochastic approximation algorithms , 1999 .

[10]  J. Aubin,et al.  Differential inclusions set-valued maps and viability theory , 1984 .

[11]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[12]  D. Fudenberg,et al.  The Theory of Learning in Games , 1998 .

[13]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[14]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[15]  Tamer Basar,et al.  Analysis of Recursive Stochastic Algorithms , 2001 .

[16]  J. Szpunar,et al.  Part II applications , 2003 .

[17]  R. Aumann.  INTEGRALS OF SET-VALUED FUNCTIONS , 1965 .

[18]  C. Conley.  Isolated Invariant Sets and the Morse Index , 1978 .

[19]  Michel Benaim,et al.  A class of non homogeneous self interacting random processes with applications to learning in games and vertex-reinforced random walks , 2008, 0806.0806.

[20]  M. Hirsch,et al.  Asymptotic pseudotrajectories and chain recurrent flows, with applications , 1996 .

[21]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[22]  F. Smithies.  Linear Operators , 2019, Nature.