Convergence of stochastic approximation algorithms under irregular conditions

We consider a class of stochastic approximation (SA) algorithms for solving a system of estimating equations. The standard condition for the convergence of the SA algorithms is that the estimating functions are locally Lipschitz continuous. Here, we show that this condition can be relaxed to the extent that the estimating functions are bounded and continuous almost everywhere. As a consequence, the use of the SA algorithm can be extended to some problems with irregular estimating functions. Our theoretical results are illustrated by solving an estimation problem for exponential power mixture models.

[1]  K. Abromeit.  Music Received , 2023, Notes.

[2]  F. Downton.  Stochastic Approximation , 1969, Nature.

[3]  H. Kushner.  Stochastic approximation with discontinuous dynamics and state dependent noise: w.p. 1 and weak convergence☆ , 1981 .

[4]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[5]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[6]  S. Kotz,et al.  Symmetric Multivariate and Related Distributions , 1989 .

[7]  W. Newey,et al.  16 Efficient estimation of models with conditional moment restrictions , 1993 .

[8]  V. Tadić.  Convergence of stochastic approximation under general noise and stability conditions , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[9]  Han-Fu Chen.  Stochastic approximation and its applications , 2002 .

[10]  T. Lai.  Stochastic approximation: invited paper , 2003 .

[11]  B. Efron.  Large-Scale Simultaneous Hypothesis Testing , 2004 .

[12]  Michael C. Fu,et al.  Convergence of simultaneous perturbation stochastic approximation for nondifferentiable optimization , 2003, IEEE Trans. Autom. Control..

[13]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[14]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[15]  L. Tian,et al.  Statistical inference based on non-smooth estimating functions , 2004 .

[16]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[17]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[18]  F. Liang,et al.  Estimating the false discovery rate using the stochastic approximation algorithm , 2008 .

[19]  Asymptotic inference for a nonstationary double AR(1) model , 2008 .

[20]  M. Newton.  Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis , 2008 .

[21]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..