New combinatorial direction stochastic approximation algorithms

The stochastic approximation problem is to find some roots or minimizers of a nonlinear function whose expression is unknown and whose evaluations are contaminated with noise. In order to accelerate the classical RM algorithm, this paper proposes a new three-term combinatorial direction stochastic approximation algorithm and its general framework which employ a weighted combination of the current noisy gradient and several previous noisy gradients as the iterative direction. Both the almost sure convergence and the asymptotic rate of convergence of the new algorithms are established. Numerical experiments show that the new algorithm outperforms the RM algorithm and another existing combined direction algorithm.

[1]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[2]  J. Blum.  Multidimensional Stochastic Approximation Methods , 1954 .

[3]  V. Fabian.  On Asymptotic Normality in Stochastic Approximation , 1968 .

[4]  Yuhong Dai.  Nonlinear Conjugate Gradient Methods , 2011 .

[5]  Dai,et al.  A Stochastic Approximation Frame Algorithm with Adaptive Directions , 2008 .

[6]  Zi Xu,et al.  A combined direction stochastic approximation algorithm , 2010, Optim. Lett..

[7]  F. Downton.  Stochastic Approximation , 1969, Nature.

[8]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[9]  Michael C. Fu,et al.  Optimization via simulation: A review , 1994, Ann. Oper. Res..

[10]  James C. Spall,et al.  Adaptive stochastic approximation by the simultaneous perturbation method , 2000, IEEE Trans. Autom. Control..

[11]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[12]  MICHAEL C. FU,et al.  Techniques for optimization via simulation: an experimental study on an (s,S) inventory system , 1997 .

[13]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[14]  Mikhail Borisovich Nevelʹson,et al.  Stochastic Approximation and Recursive Estimation , 1976 .

[15]  Nicol N. Schraudolph,et al.  Combining Conjugate Direction Methods with Stochastic Approximation of Gradients , 2003, AISTATS.

[16]  Bernard Delyon,et al.  Accelerated Stochastic Approximation , 1993, SIAM J. Optim..

[17]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[18]  John N. Tsitsiklis,et al.  Gradient Convergence in Gradient methods with Errors , 1999, SIAM J. Optim..

[19]  Y. H. Dai,et al.  Convergence of Three-term Conjugate Gradient Methods ∗ , 2003 .