A Subgradient Method Based on Gradient Sampling for Solving Convex Optimization Problems

Based on the gradient sampling technique, we present a subgradient algorithm to solve the nondifferentiable convex optimization problem with an extended real-valued objective function. A feature of our algorithm is the approximation of subgradient at a point via random sampling of (relative) gradients at nearby points, and then taking convex combinations of these (relative) gradients. We prove that our algorithm converges to an optimal solution with probability 1. Numerical results demonstrate that our algorithm performs favorably compared with existing subgradient algorithms on applications considered.

[1]  Michael Patriksson,et al.  On the convergence of conditional epsilon-subgradient methods for convex programs and convex-concave saddle-point problems , 2003, Eur. J. Oper. Res..

[2]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[3]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[4]  Krzysztof C. Kiwiel,et al.  An aggregate subgradient method for nonsmooth convex minimization , 1983, Math. Program..

[5]  Adrian S. Lewis,et al.  A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization , 2005, SIAM J. Optim..

[6]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[7]  Torbjörn Larsson,et al.  The Efficiency of Ballstep Subgradient Level Methods for Convex Optimization , 1999, Math. Oper. Res..

[8]  Yu. M. Ermol’ev.  Methods of solution of nonlinear extremal problems , 1966 .

[9]  Jean-Louis Goffin,et al.  Convergence of a simple subgradient level method , 1999, Math. Program..

[10]  Andrzej Ruszczynski,et al.  A merit function approach to the subgradient method with averaging , 2008, Optim. Methods Softw..

[11]  Krzysztof C. Kiwiel,et al.  Convergence of Approximate and Incremental Subgradient Methods for Convex Optimization , 2003, SIAM J. Optim..

[12]  Xiaoqi Yang,et al.  Inexact subgradient methods for quasi-convex optimization problems , 2015, Eur. J. Oper. Res..

[13]  C. Yalçin Kaya,et al.  On a Modified Subgradient Algorithm for Dual Problems via Sharp Augmented Lagrangian* , 2006, J. Glob. Optim..

[14]  Marc Teboulle,et al.  Mirror descent and nonlinear projected subgradient methods for convex optimization , 2003, Oper. Res. Lett..

[15]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[16]  Adrian S. Lewis,et al.  Approximating Subdifferentials by Random Sampling of Gradients , 2002, Math. Oper. Res..

[17]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[18]  Franziska Wulf,et al.  Minimization Methods For Non Differentiable Functions , 2016 .

[19]  Torbjörn Larsson,et al.  Lagrangian Relaxation via Ballstep Subgradient Methods , 2007, Math. Oper. Res..

[20]  M. Patriksson,et al.  Ergodic convergence in subgradient optimization , 1998 .

[21]  K. Kiwiel.  The Efficiency of Subgradient Projection Methods for Convex Optimization , 1996 .

[22]  Leon Hirsch,et al.  Fundamentals Of Convex Analysis , 2016 .

[23]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[24]  Sehun Kim,et al.  Convergence of a generalized subgradient method for nondifferentiable convex optimization , 1991, Math. Program..

[25]  A. Banerjee.  Convex Analysis and Optimization , 2006 .

[26]  James E. Smith,et al.  Information Relaxations, Duality, and Convex Stochastic Dynamic Programs , 2014, Oper. Res..

[27]  K. Kiwiel.  The efficiency of subgradient projection methods for convex optimization, part I: general level methods , 1996 .

[28]  Asuman E. Ozdaglar,et al.  Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods , 2008, SIAM J. Optim..

[29]  A. Lewis,et al.  Two numerical methods for optimizing matrix stability , 2002 .

[30]  Rafail N. Gasimov,et al.  Augmented Lagrangian Duality and Nondifferentiable Optimization Methods in Nonconvex Programming , 2002, J. Glob. Optim..