Stochastic First-Order Methods with Random Constraint Projection

We consider convex optimization problems with structures that are suitable for sequential treatment or online sampling. In particular, we focus on problems where the objective function is an expect...

[1]  Andrzej Cegielski,et al.  Relaxed Alternating Projection Methods , 2008, SIAM J. Optim..

[2]  Adrian S. Lewis,et al.  Alternating Projections on Manifolds , 2008, Math. Oper. Res..

[3]  Frank Deutsch,et al.  The rate of convergence for the cyclic projections algorithm III: Regularity of convex sets , 2008, J. Approx. Theory.

[4]  Vivek S. Borkar,et al.  Distributed Asynchronous Incremental Subgradient Methods , 2001 .

[5]  Angelia Nedic,et al.  Random algorithms for convex minimization problems , 2011, Math. Program..

[6]  Adrian S. Lewis,et al.  Randomized Methods for Linear Constraints: Convergence Rates and Conditioning , 2008, Math. Oper. Res..

[7]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[8]  Frank Deutsch,et al.  The rate of convergence for the cyclic projections algorithm II: Norms of nonlinear operators , 2006, J. Approx. Theory.

[9]  Dimitri P. Bertsekas,et al.  Incremental constraint projection methods for variational inequalities , 2014, Math. Program..

[10]  Martin J. Wainwright,et al.  Information-Theoretic Lower Bounds on the Oracle Complexity of Stochastic Convex Optimization , 2010, IEEE Transactions on Information Theory.

[11]  Frank Deutsch,et al.  The rate of convergence for the cyclic projections algorithm I: Angles between convex sets , 2006, J. Approx. Theory.

[12]  Boris Polyak,et al.  The method of projections for finding the common point of convex sets , 1967 .

[13]  Heinz H. Bauschke,et al.  Projection algorithms and monotone operators , 1996 .

[14]  Heinz H. Bauschke,et al.  On Projection Algorithms for Solving Convex Feasibility Problems , 1996, SIAM Rev..

[15]  Dimitri P. Bertsekas,et al.  Incremental proximal methods for large scale convex optimization , 2011, Math. Program..

[16]  D. Bertsekas,et al.  Convergen e Rate of In remental Subgradient Algorithms , 2000 .

[17]  Angelia Nedic,et al.  Random projection algorithms for convex set intersection problems , 2010, 49th IEEE Conference on Decision and Control (CDC).

[18]  P. Tseng.  Successive Projection under a Quasi-Cyclic Order , 1990 .

[19]  Heinz H. Bauschke,et al.  The method of cyclic projections for closed convex sets in Hilbert space , 1997 .

[20]  Ohad Shamir,et al.  Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes , 2012, ICML.