Dynamic stochastic optimization

A framework for sequentially solving stochastic optimization problems with stochastic gradient descent is introduced. Two tracking criteria are considered, one based on being accurate with respect to the mean trajectory and the other based on being accurate in high probability (IHP). An off-line optimization problem is solved to find the constant step size and number of iterations to achieve the desired tracking accuracy. Simulations are used to confirm that this approach provides the desired tracking accuracy.

[1]  Eric Moulines,et al.  Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning , 2011, NIPS.

[2]  Martin Zinkevich,et al.  Online Convex Programming and Generalized Infinitesimal Gradient Ascent , 2003, ICML.

[3]  R. Rockafellar,et al.  Implicit Functions and Solution Mappings , 2009 .

[4]  Peter L. Bartlett,et al.  Adaptive Online Gradient Descent , 2007, NIPS.

[5]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[6]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[7]  Sham M. Kakade,et al.  Mind the Duality Gap: Logarithmic regret algorithms for online optimization , 2008, NIPS.

[8]  Rong Jin,et al.  25th Annual Conference on Learning Theory Online Optimization with Gradual Variations , 2022 .

[9]  Xuan Kong,et al.  Adaptive Signal Processing Algorithms: Stability and Performance , 1994 .

[10]  Lin Xiao,et al.  Dual Averaging Methods for Regularized Stochastic Learning and Online Optimization , 2009, J. Mach. Learn. Res..

[11]  R. Rockafellar,et al.  Implicit Functions and Solution Mappings: A View from Variational Analysis , 2009 .

[12]  H. Kushner,et al.  Analysis of adaptive step size SA algorithms for parameter tracking , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[13]  Ali H. Sayed,et al.  Adaptive Filters , 2008 .

[14]  Y. Singer,et al.  Logarithmic Regret Algorithms for Strongly Convex Repeated Games , 2007 .

[15]  I. Yamada,et al.  The Adaptive Projected Subgradient Method over the Fixed Point Set of Strongly Attracting Nonexpansive Mappings , 2006 .

[16]  B. Schölkopf,et al.  Convex Repeated Games and Fenchel Duality , 2007 .

[17]  Elad Hazan,et al.  Logarithmic regret algorithms for online convex optimization , 2006, Machine Learning.

[18]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[19]  Yoram Singer,et al.  Efficient Online and Batch Learning Using Forward Backward Splitting , 2009, J. Mach. Learn. Res..

[20]  Karthik Sridharan,et al.  Online Learning with Predictable Sequences , 2012, COLT.

[21]  I. Yamada,et al.  Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions , 2005 .

[22]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[23]  Isao Yamada,et al.  Diffusion Least-Mean Squares With Adaptive Combiners: Formulation and Performance Analysis , 2010, IEEE Transactions on Signal Processing.

[24]  Arnaud Doucet,et al.  Particle Filtering and Smoothing: Fifteen years later , 2008 .

[25]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[26]  A. Müller.  Integral Probability Metrics and Their Generating Classes of Functions , 1997, Advances in Applied Probability.

[27]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[28]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .