Modified Accelerated Bundle-Level Methods and Their Application in Two-Stage Stochastic Programming

The accelerated prox-level (APL) and uniform smoothing level (USL) methods recently proposed by Lan (Math Program, 149: 1–45, 2015) can achieve uniformly optimal complexity when solving black-box convex programming (CP) and structure non-smooth CP problems. In this paper, we propose two modified accelerated bundle-level type methods, namely, the modified APL (MAPL) and modified USL (MUSL) methods. Compared with the original APL and USL methods, the MAPL and MUSL methods reduce the number of subproblems by one in each iteration, thereby improving the efficiency of the algorithms. Conclusions of optimal iteration complexity of the proposed algorithms are established. Furthermore, the modified methods are applied to the two-stage stochastic programming, and numerical experiments are implemented to illustrate the advantages of our methods in terms of efficiency and accuracy.

[1]  Antonio Frangioni,et al.  Incremental Bundle Methods using Upper Models , 2018, SIAM J. Optim..

[2]  Yurii Nesterov,et al.  New variants of bundle methods , 1995, Math. Program..

[3]  David P. Morton,et al.  Monte Carlo bounding techniques for determining solution quality in stochastic programs , 1999, Oper. Res. Lett..

[4]  Linfeng Yang,et al.  Projected mixed integer programming formulations for unit commitment problem , 2015 .

[5]  Marc Teboulle,et al.  Interior Gradient and Proximal Methods for Convex and Conic Optimization , 2006, SIAM J. Optim..

[6]  Andrzej Ruszczynski,et al.  A regularized decomposition method for minimizing a sum of polyhedral functions , 1986, Math. Program..

[7]  Y. Nesterov.  A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[8]  Franck Iutzeler,et al.  Asynchronous level bundle methods , 2019, Mathematical Programming.

[9]  Guanghui Lan,et al.  Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization , 2013, Mathematical Programming.

[10]  Stephen J. Wright,et al.  Decomposition Algorithms for Stochastic Programming on a Computational Grid , 2001, Comput. Optim. Appl..

[11]  J. Jian,et al.  Tight Relaxation Method for Unit Commitment Problem Using Reformulation and Lift-and-Project , 2015, IEEE Transactions on Power Systems.

[12]  Robert D. Doverspike,et al.  Network planning with random demand , 1994, Telecommun. Syst..

[13]  Alexander Shapiro,et al.  The empirical behavior of sampling methods for stochastic programming , 2006, Ann. Oper. Res..

[14]  Christoph Helmberg,et al.  A Parallel Bundle Framework for Asynchronous Subspace Optimization of Nonsmooth Convex Functions , 2014, SIAM J. Optim..

[15]  A. Ruszczynski,et al.  Accelerating the regularized decomposition method for two stage stochastic linear problems , 1997 .

[16]  Victor M. Zavala,et al.  An Asynchronous Bundle-Trust-Region Method for Dual Decomposition of Stochastic Mixed-Integer Programming , 2019, SIAM J. Optim..

[17]  Peter Richtárik,et al.  Approximate Level Method for Nonsmooth Convex Minimization , 2011, Journal of Optimization Theory and Applications.

[18]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[19]  Arkadi Nemirovski,et al.  Non-euclidean restricted memory level method for large-scale convex optimization , 2005, Math. Program..

[20]  Krzysztof C. Kiwiel,et al.  An aggregate subgradient method for nonsmooth convex minimization , 1983, Math. Program..

[21]  Renato D. C. Monteiro,et al.  Primal-dual first-order methods with $${\mathcal {O}(1/\epsilon)}$$ iteration-complexity for cone programming , 2011, Math. Program..

[22]  Guanghui Lan,et al.  An optimal method for stochastic composite optimization , 2011, Mathematical Programming.

[23]  Yunmei Chen,et al.  Fast bundle-level methods for unconstrained and ball-constrained convex optimization , 2014, Comput. Optim. Appl..

[24]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[25]  L. Qi,et al.  Newton's method for quadratic stochastic programs with recourse , 1995 .

[26]  Yan Xu,et al.  A novel projected two-binary-variable formulation for unit commitmentin power systems , 2017 .

[27]  Krzysztof C. Kiwiel,et al.  Proximity control in bundle methods for convex nondifferentiable minimization , 1990, Math. Program..

[28]  Guoyin Li,et al.  A proximal-projection partial bundle method for convex constrained minimax problems , 2019, Journal of Industrial & Management Optimization.

[29]  Arthur F. Veinott,et al.  The Supporting Hyperplane Method for Unimodal Programming , 1967, Oper. Res..