Aggregate subgradient method for nonsmooth DC optimization

The aggregate subgradient method is developed for solving unconstrained nonsmooth difference of convex (DC) optimization problems. The proposed method shares some similarities with both the subgradient and the bundle methods. Aggregate subgradients are defined as a convex combination of subgradients computed at null steps between two serious steps. At each iteration search directions are found using only two subgradients: the aggregate subgradient and a subgradient computed at the current null step. It is proved that the proposed method converges to a critical point of the DC optimization problem and also that the number of null steps between two serious steps is finite. The new method is tested using some academic test problems and compared with several other nonsmooth DC optimization solvers.

[1]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[2]  Adil M. Bagirov,et al.  An Approximate ADMM for Solving Linearly Constrained Nonsmooth Optimization Problems with Two Blocks of Variables , 2019, Nonsmooth Optimization and Its Applications.

[3]  Laurence A. Wolsey,et al.  Two “well-known” properties of subgradient optimization , 2009, Math. Program..

[4]  H. Tuy.  Convex analysis and global optimization , 1998 .

[5]  Franz Kappel,et al.  An Implementation of Shor's r-Algorithm , 2000, Comput. Optim. Appl..

[6]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[7]  Adil M. Bagirov,et al.  Introduction to Nonsmooth Optimization , 2014 .

[8]  Adil M. Bagirov,et al.  Codifferential method for minimizing nonsmooth DC functions , 2011, J. Glob. Optim..

[9]  Defeng Sun,et al.  Quasi-Newton Bundle-Type Methods for Nondifferentiable Convex Optimization , 1998, SIAM J. Optim..

[10]  Le Thi Hoai An,et al.  The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems , 2005, Ann. Oper. Res..

[11]  Kaisa Miettinen,et al.  Globally convergent limited memory bundle method for large-scale nonsmooth optimization , 2007, Math. Program..

[12]  Adil M. Bagirov,et al.  Introduction to Nonsmooth Optimization: Theory, Practice and Software , 2014 .

[13]  Alexander S. Strekalovsky.  Global Optimality Conditions for Nonconvex Optimization , 1998, J. Glob. Optim..

[14]  Naum Zuselevich Shor,et al.  Minimization Methods for Non-Differentiable Functions , 1985, Springer Series in Computational Mathematics.

[15]  Kaisa Miettinen,et al.  New limited memory bundle method for large-scale nonsmooth optimization , 2004, Optim. Methods Softw..

[16]  Antonio Fuduli,et al.  Minimizing Nonconvex Nonsmooth Functions via Cutting Planes and Proximity Control , 2003, SIAM J. Optim..

[17]  Adil M. Bagirov,et al.  Minimizing nonsmooth DC functions via successive DC piecewise-affine approximations , 2018, J. Glob. Optim..

[18]  Jan Vlcek,et al.  A bundle-Newton method for nonsmooth unconstrained minimization , 1998, Math. Program..

[19]  Hoai An Le Thi,et al.  Solving a Class of Linearly Constrained Indefinite Quadratic Problems by D.C. Algorithms , 1997 .

[20]  Adil M. Bagirov,et al.  Subgradient Method for Nonconvex Nonsmooth Optimization , 2013, J. Optim. Theory Appl..

[21]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[22]  Le Thi Hoai An,et al.  Solving a Class of Linearly Constrained Indefinite Quadratic Problems by D.C. Algorithms , 1997, J. Glob. Optim..

[23]  K. Kiwiel.  Methods of Descent for Nondifferentiable Optimization , 1985 .

[24]  Antoine Soubeyran,et al.  Global convergence of a proximal linearized algorithm for difference of convex functions , 2015, Optimization Letters.

[25]  P. Neittaanmäki,et al.  Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control , 1992 .

[26]  Adil M. Bagirov,et al.  A quasisecant method for minimizing nonsmooth functions , 2010, Optim. Methods Softw..

[27]  Adil M. Bagirov,et al.  A proximal bundle method for nonsmooth DC optimization utilizing nonconvex cutting planes , 2017, J. Glob. Optim..

[28]  Adil M. Bagirov,et al.  Double Bundle Method for finding Clarke Stationary Points in Nonsmooth DC Programming , 2018, SIAM J. Optim..

[29]  Warren Hare,et al.  A Redistributed Proximal Bundle Method for Nonconvex Optimization , 2010, SIAM J. Optim..

[30]  A. Bagirov,et al.  Discrete Gradient Method: Derivative-Free Method for Nonsmooth Optimization , 2008 .

[31]  Antonio Fuduli,et al.  A DC piecewise affine model and a bundling technique in nonconvex nonsmooth minimization , 2004, Optim. Methods Softw..

[32]  C. Beltran,et al.  An Effective Line Search for the Subgradient Method , 2005 .

[33]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[34]  Adil M. Bagirov,et al.  Nonsmooth DC programming approach to the minimum sum-of-squares clustering problems , 2016, Pattern Recognit..