Stochastic approximation and large deviations: upper bounds and w.p.1 convergence

With probability one convergence results are obtained for stochastic recursive approximation algorithms under very general conditions. The gain sequence $\{ a_n \} $ can go to zero very slowly and state-dependent noise, discontinuous dynamical equations, and the projected or constrained algorithm are all treated. The basic technique is the theory of large deviations. Prior results obtained via this theory are extended in many directions. Let $\dot x = \bar b(x)$ denote the “mean” equation for the algorithm, let $\delta > 0$ be given, and let $G(\theta )$ be a neighborhood of a stable point $\theta $ of that ordinary differential equation. Then, asymptotic upper bounds to $a_N \log P\{ X_n \notin G(\theta ),n \geqq N\mid | {X_N - \theta } | \leqq \delta \} $, are obtained. These are often more informative than the usual classical rate of convergence results (that use a “local linearization”) and, furthermore, are obtained for the constrained and nonsmooth cases, for which there are no “rate of convergence”...

[1]  M. T. Wasan.  Stochastic Approximation , 1969 .

[2]  S. Varadhan,et al.  Asymptotic evaluation of certain Markov process expectations for large time , 1975 .

[3]  Lennart Ljung,et al.  Analysis of recursive stochastic algorithms , 1977 .

[4]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[5]  M. Freidlin,et al.  THE AVERAGING PRINCIPLE AND THEOREMS ON LARGE DEVIATIONS , 1978 .

[6]  A. Ioffe,et al.  Theory of extremal problems , 1979 .

[7]  H. Kushner,et al.  Asymptotic Properties of Stochastic Approximations with Constant Coefficients. , 1981 .

[8]  Harold J. Kushner.  AN AVERAGING METHOD FOR STOCHASTIC APPROXIMATIONS WITH DISCONTINUOUS DYNAMICS, CONSTRAINTS, AND STATE DEPENDENT NOISE , 1983 .

[9]  H. Kushner.  Asymptotic behavior of stochastic approximation and large deviations , 1984, The 22nd IEEE Conference on Decision and Control.

[10]  H. Kushner.  Robustness and Approximation of Escape Times and Large Deviations Estimates for Systems with Small Noise Effects , 1984 .

[11]  S. Varadhan.  Large Deviations and Applications , 1984 .

[12]  M. Freidlin,et al.  Random Perturbations of Dynamical Systems , 1984 .

[13]  H. Kushner,et al.  An Invariant Measure Approach to the Convergence of Stochastic Approximations with State Dependent Noise. , 1984 .

[14]  M. Metivier,et al.  Applications of a Kushner and Clark lemma to general classes of stochastic algorithms , 1984, IEEE Trans. Inf. Theory.

[15]  P. Dupuis,et al.  Stochastic Approximations via Large Deviations: Asymptotic Properties , 1985 .

[16]  Constrained stochastic approximation via the theory of large deviations , 1986 .

[17]  Yasumasa Saisho,et al.  Stochastic differential equations for multi-dimensional domain with reflecting boundary , 1987 .

[18]  H. Kushner.  Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: Global minimization via Monte Carlo , 1987 .

[19]  P. Dupuis.  Large Deviations Analysis of Some Recursive Algorithms with State Dependent Noise , 1988 .