A Generalized Alternating Linearization Bundle Method for Structured Convex Optimization with Inexact First-Order Oracles

In this paper, we consider a class of structured optimization problems whose objective function is the summation of two convex functions: f and h, which are not necessarily differentiable. We focus particularly on the case where the function f is general and its exact first-order information (function value and subgradient) may be difficult to obtain, while the function h is relatively simple. We propose a generalized alternating linearization bundle method for solving this class of problems, which can handle inexact first-order information of on-demand accuracy. The inexact information can be very general, which covers various oracles, such as inexact, partially inexact and asymptotically exact oracles, and so forth. At each iteration, the algorithm solves two interrelated subproblems: one aims to find the proximal point of the polyhedron model of f plus the linearization of h; the other aims to find the proximal point of the linearization of f plus h. We establish global convergence of the algorithm under different types of inexactness. Finally, some preliminary numerical results on a set of two-stage stochastic linear programming problems show that our method is very encouraging.

[1]  P. Wolfe.  Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974 .

[2]  K. Kiwiel.  Approximations in proximal bundle methods and decomposition of convex programs , 1995 .

[3]  Jonathan Eckstein.  Some Saddle-function splitting methods for convex programming , 1994 .

[4]  Guanghui Lan,et al.  Bundle-level type methods uniformly optimal for smooth and nonsmooth convex optimization , 2013, Mathematical Programming.

[5]  Krzysztof C. Kiwiel,et al.  An alternating linearization bundle method for convex optimization and nonlinear multicommodity flow problems , 2011, Math. Program..

[6]  Marko Mäkelä,et al.  Survey of Bundle Methods for Nonsmooth Optimization , 2002, Optim. Methods Softw..

[7]  Michael Hintermüller,et al.  A Proximal Bundle Method Based on Approximate Subgradients , 2001, Comput. Optim. Appl..

[8]  C. Lemaréchal.  An extension of davidon methods to non differentiable problems , 1975 .

[9]  S. M. Robinson,et al.  A quadratically-convergent algorithm for general nonlinear programming problems , 1972, Math. Program..

[10]  Yan Xu,et al.  A novel projected two-binary-variable formulation for unit commitmentin power systems , 2017 .

[11]  Guoyin Li,et al.  A proximal-projection partial bundle method for convex constrained minimax problems , 2019, Journal of Industrial & Management Optimization.

[12]  Jorge Nocedal,et al.  Optimization Methods for Large-Scale Machine Learning , 2016, SIAM Rev..

[13]  Shiqian Ma,et al.  Fast alternating linearization methods for minimizing the sum of two convex functions , 2009, Math. Program..

[14]  P. D. Tao,et al.  Partial regularization of the sum of two maximal monotone operators , 1993 .

[15]  Golbon Zakeri,et al.  Inexact Cuts in Benders Decomposition , 1999, SIAM J. Optim..

[16]  Zhao Yang Dong,et al.  A Distributed Dual Consensus ADMM Based on Partition for DC-DOPF With Carbon Emission Trading , 2020, IEEE Transactions on Industrial Informatics.

[17]  Jérôme Malick,et al.  Uncontrolled inexact information within bundle methods , 2017, EURO J. Comput. Optim..

[18]  Andrzej Ruszczynski,et al.  Proximal Decomposition Via Alternating Linearization , 1999, SIAM J. Optim..

[19]  J. Jian,et al.  Tight Relaxation Method for Unit Commitment Problem Using Reformulation and Lift-and-Project , 2015, IEEE Transactions on Power Systems.

[20]  Robert D. Doverspike,et al.  Network planning with random demand , 1994, Telecommun. Syst..

[21]  Myungjoo Kang,et al.  Efficient Nonsmooth Nonconvex Optimization for Image Restoration and Segmentation , 2015, J. Sci. Comput..

[22]  Bingsheng He,et al.  Linearized Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming , 2011 .

[23]  Miantao Chao,et al.  A Linearized Alternating Direction Method of Multipliers with Substitution Procedure , 2015, Asia Pac. J. Oper. Res..

[24]  Krzysztof C. Kiwiel,et al.  A Proximal Bundle Method with Approximate Subgradient Linearizations , 2006, SIAM J. Optim..

[25]  Claudia A. Sagastizábal,et al.  Level bundle methods for oracles with on-demand accuracy , 2014, Optim. Methods Softw..

[26]  Masao Fukushima,et al.  Application of the alternating direction method of multipliers to separable convex programming problems , 1992, Comput. Optim. Appl..

[27]  Susana Scheimberg,et al.  Inexact Bundle Methods for Two-Stage Stochastic Programming , 2011, SIAM J. Optim..

[28]  Bingsheng He,et al.  Convergence Rate Analysis for the Alternating Direction Method of Multipliers with a Substitution Procedure for Separable Convex Programming , 2017, Math. Oper. Res..

[29]  Zhao Yang Dong,et al.  Multiple Perspective-Cuts Outer Approximation Method for Risk-Averse Operational Planning of Regional Energy Service Providers , 2017, IEEE Transactions on Industrial Informatics.

[30]  David P. Morton,et al.  Monte Carlo bounding techniques for determining solution quality in stochastic programs , 1999, Oper. Res. Lett..

[31]  Krzysztof C. Kiwiel.  A Proximal-Projection Bundle Method for Lagrangian Relaxation, Including Semidefinite Programming , 2006, SIAM J. Optim..

[32]  P. Tseng.  Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[33]  Claude Lemaréchal,et al.  Convex proximal bundle methods in depth: a unified analysis for inexact oracles , 2014, Math. Program..

[34]  K. Kiwiel.  An algorithm for nonsmooth convex minimization with errors , 1985 .