Stochastic Approximation Methods and Their Finite-Time Convergence Properties

This chapter surveys some recent advances in the design and analysis of two classes of stochastic approximation methods: stochastic first- and zeroth-order methods for stochastic optimization. We focus on the finite-time convergence properties (i.e., iteration complexity) of these algorithms by providing bounds on the number of iterations required to achieve a certain accuracy. We point out that many of these complexity bounds are theoretically optimal for solving different classes of stochastic optimization problems.

[1]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[2]  K. Chung.  On a Stochastic Approximation Method , 1954 .

[3]  J. Sacks.  Asymptotic Distribution of Stochastic Approximation Procedures , 1958 .

[4]  V. Strassen.  The Existence of Probability Measures with Given Marginals , 1965 .

[5]  A. A. Gaivoronskii.  Nonstationary stochastic programming problems , 1978 .

[6]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[7]  Y. Ermoliev.  Stochastic quasigradient methods and their application to system optimization , 1983 .

[8]  A. Ruszczynski,et al.  A method of aggregate stochastic subgradients with on-line stepsize rules for convex stochastic programming problems , 1986 .

[9]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[10]  George Ch. Pflug,et al.  Optimization of Stochastic Models , 1996 .

[11]  Alexander Shapiro,et al.  The Sample Average Approximation Method for Stochastic Discrete Optimization , 2002, SIAM J. Optim..

[12]  A. Shapiro.  Monte Carlo Sampling Methods , 2003 .

[13]  Alexander V. Nazin,et al.  Recursive Aggregation of Estimators by the Mirror Descent Algorithm with Averaging , 2005, Probl. Inf. Transm..

[14]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[15]  James C. Spall,et al.  Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C. , 2007 .

[16]  A. Juditsky,et al.  Learning by mirror averaging , 2005, math/0511468.

[17]  Katya Scheinberg,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[18]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[19]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[20]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[21]  Guanghui Lan,et al.  An optimal method for stochastic composite optimization , 2011, Mathematical Programming.

[22]  Saeed Ghadimi,et al.  Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework , 2012, SIAM J. Optim..

[23]  Nicholas I. M. Gould,et al.  On the Oracle Complexity of First-Order and Derivative-Free Algorithms for Smooth Nonconvex Minimization , 2012, SIAM J. Optim..

[24]  Alexander Shapiro,et al.  Validation analysis of mirror descent stochastic approximation method , 2012, Math. Program..

[25]  Saeed Ghadimi,et al.  Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization, II: Shrinking Procedures and Optimal Algorithms , 2013, SIAM J. Optim..

[26]  Luís Nunes Vicente,et al.  Worst case complexity of direct search , 2013, EURO J. Comput. Optim..

[27]  Saeed Ghadimi,et al.  Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming , 2013, SIAM J. Optim..

[28]  L. N. Vicente,et al.  Smoothing and worst-case complexity for direct-search methods in nonsmooth optimization , 2013 .

[29]  Yurii Nesterov,et al.  Random Gradient-Free Minimization of Convex Functions , 2015, Foundations of Computational Mathematics.