On the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates

We analyse the convergence of stochastic algorithms with Markovian noise when the ergodicity of the Markov chain governing the noise rapidly decreases as the control parameter tends to infinity. In such a case, there may be a positive probability of divergence of the algorithm in the classic Robbins-Monro form. We provide sufficient condition which ensure convergence. Moreover, we analyse the asymptotic behaviour of these algorithms and state a diffusion approximation theorem

[1]  P. L. Dobruschin.  The Description of a Random Field by Means of Conditional Probabilities and Conditions of Its Regularity , 1968 .

[2]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[3]  R. Dobrushin.  Prescribing a System of Random Variables by Conditional Distributions , 1970 .

[4]  J. Besag.  Spatial Interaction and the Statistical Analysis of Lattice Systems , 1974 .

[5]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  M. Métivier,et al.  Théorèmes de convergence presque sure pour une classe d'algorithmes stochastiques à pas décroissant , 1987 .

[7]  L. Younes.  Estimation and annealing for Gibbsian fields , 1988 .

[8]  Catherine Bouton.  Approximation gaussienne d'algorithmes stochastiques à dynamique markovienne , 1988 .

[9]  R. M. Dudley,et al.  Real Analysis and Probability , 1989 .

[10]  L. Younes.  Parametric Inference for imperfectly observed Gibbsian fields , 1989 .

[11]  D. Geman.  Random fields and inverse problems in imaging , 1990 .

[12]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[13]  P. Dupuis,et al.  On sampling controlled stochastic approximation , 1991 .

[14]  D. Stroock,et al.  The logarithmic sobolev inequality for discrete spin systems on a lattice , 1992 .

[15]  C. Geyer,et al.  Constrained Monte Carlo Maximum Likelihood for Dependent Data , 1992 .

[16]  Salvatore Ingrassia,et al.  Geometric Approaches to the Estimation of the Spectral Gap of Reversible Markov Chains , 1993, Combinatorics, Probability and Computing.

[17]  S. Ingrassia.  ON THE RATE OF CONVERGENCE OF THE METROPOLIS ALGORITHM AND GIBBS SAMPLER BY GEOMETRIC BOUNDS , 1994 .