Théorèmes de convergence presque sure pour une classe d'algorithmes stochastiques à pas décroissant

SummaryThe paper studies the pathwise asymptotic behaviour of stochastic algorithms of the following general form $$\theta _{n + 1} = \theta _n + \gamma _{n + 1} f(\theta _n ,Y_{n + 1} ),$$ the hypotheses allowing discontinuities on the adaptation termf. The process (Yn)n≧0 is a Markov chain “controlled by (θn)”. For each θ fixed the Markov chain (Ynθ)n≧0 is essentially of positive recurrent type.A first theorem generalizes to this situation a Ljung's theorem (L. Ljung, IEEE Trans. AC 22, 2, p. 551–575 (1977)).An almost sure convergence theorem is proved under the existence of a global Lyapunov function for the associated deterministic differential equation. $$\frac{{d\bar \theta (t)}}{{d(t)}} = h(\bar \theta (t))$$ whereh(θ) = ∫f(θ, y) Γθ(d, y) and Γθ is the invariant probability of the Markov chain (Ynθ)n≧0.