Stochastic Approximations via Large Deviations: Asymptotic Properties

Asymptotic properties of Robbins–Munro and Kiefer–Wolfowitz type stochastic approximation algorithms are obtained via the theory of large deviations. The conditions are weak and can even yield w.p.l. convergence results. The probability of escape of the iterates from a neighborhood of a stable point of the algorithm is estimated and shown to be considerably smaller than suggested by the classical “asymptotic normality of local normalized errors” method of getting the asymptotic properties. The escape probabilities are a natural quantity of interest. In many applications, they are more useful than the “local normalized mean square errors.” Other large deviations estimates are also obtained. Typically, if ${{a_n = 1} / {n^\rho }}$, $\rho \leqq 1$, then the probability of escape from a neighborhood of a stable point in some (normalized) time interval $[n,m]:\sum_n^m {a_i \sim T} $ is $\exp - n^\rho V_\rho $, where $V_\rho $ does not depend on $\rho $ for $\rho < 1$ and is the solution to an optimal control p...