A Redistributed Proximal Bundle Method for Nonconvex Optimization

Proximal bundle methods have been shown to be highly successful optimization methods for unconstrained convex problems with discontinuous first derivatives. This naturally leads to the question of whether proximal variants of bundle methods can be extended to a nonconvex setting. This work proposes an approach based on generating cutting-planes models, not of the objective function as most bundle methods do but of a local convexification of the objective function. The corresponding convexification parameter is calculated “on the fly” in such a way that the algorithm can inform the user as to what proximal parameters are sufficiently large that the objective function is likely to have well-defined proximal points. This novel approach, shown to be sound from both the objective function and subdifferential modelling perspectives, opens the way to create workable nonconvex algorithms based on nonconvex $\mathcal{VU}$ theory. Both theoretical convergence analysis and some encouraging preliminary numerical experience are provided.

[1]  Krzysztof C. Kiwiel,et al.  A Linearization Algorithm for Nonsmooth Minimization , 1985, Math. Oper. Res..

[2]  C. Lemaréchal An extension of davidon methods to non differentiable problems , 1975 .

[3]  Robert Mifflin,et al.  Primal-Dual Gradient Structured Functions: Second-Order Results; Links to Epi-Derivatives and Partly Smooth Functions , 2003, SIAM J. Optim..

[4]  C. Lemaréchal,et al.  ON A BUNDLE ALGORITHM FOR NONSMOOTH OPTIMIZATION , 1981 .

[5]  A. M. Gupal,et al.  A method for the minimization of almost-differentiable functions , 1977 .

[6]  W. Hare,et al.  Prox-Regularity and Stability of the Proximal Mapping , 2007 .

[7]  Danny C. Sorensen,et al.  Nondifferential and Variational Techniques in Optimization , 1982 .

[8]  Claude Lemaréchal,et al.  Lagrangian Relaxation , 2000, Computational Combinatorial Optimization.

[9]  Kaisa Miettinen,et al.  Globally convergent limited memory bundle method for large-scale nonsmooth optimization , 2007, Math. Program..

[10]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[11]  Claude Lemaréchal,et al.  Convergence of some algorithms for convex minimization , 1993, Math. Program..

[12]  R. Mifflin Semismooth and Semiconvex Functions in Constrained Optimization , 1977 .

[13]  Antonio Fuduli,et al.  A DC piecewise affine model and a bundling technique in nonconvex nonsmooth minimization , 2004, Optim. Methods Softw..

[14]  Aude Rondepierre,et al.  A Proximity Control Algorithm to Minimize Nonsmooth and Nonconvex Functions , 2008 .

[15]  C. Ferrier Computation of the distance to semi-algebraic sets , 2000 .

[16]  A. Lewis,et al.  Identifying active constraints via partial smoothness and prox-regularity , 2003 .

[17]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[18]  Robert Mifflin,et al.  𝒱𝒰-smoothness and proximal point results for some nonconvex functions , 2004, Optim. Methods Softw..

[19]  P. Neittaanmäki,et al.  Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control , 1992 .

[20]  Krzysztof C. Kiwiel,et al.  Restricted Step and Levenberg-Marquardt Techniques in Proximal Bundle Methods for Nonconvex Nondifferentiable Optimization , 1996, SIAM J. Optim..

[21]  Warren Hare,et al.  Computing proximal points of nonconvex functions , 2008, Math. Program..

[22]  K. Kiwiel A Method for Solving Certain Quadratic Programming Problems Arising in Nonsmooth Optimization , 1986 .

[23]  Antonio Fuduli,et al.  Minimizing Nonconvex Nonsmooth Functions via Cutting Planes and Proximity Control , 2003, SIAM J. Optim..

[24]  Antonio Frangioni,et al.  Solving semidefinite quadratic problems within nonsmooth optimization algorithms , 1996, Comput. Oper. Res..

[25]  Jan Vlcek,et al.  A bundle-Newton method for nonsmooth unconstrained minimization , 1998, Math. Program..

[26]  Christophe Ferrier Bornes duales de problèmes d'optimisation polynomiaux , 1997 .

[27]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization , 2001 .

[28]  Adil M. Bagirov,et al.  A new nonsmooth optimization algorithm for minimum sum-of-squares clustering problems , 2006, Eur. J. Oper. Res..

[29]  R. Mifflin A modification and an extension of Lemarechal’s algorithm for nonsmooth minimization , 1982 .

[30]  J. Hiriart-Urruty Generalized Differentiability / Duality and Optimization for Problems Dealing with Differences of Convex Functions , 1985 .

[31]  P. Neittaanmäki,et al.  Nonsmooth Optimization Theory , 1992 .

[32]  Krzysztof C. Kiwiel A Method of Centers with Approximate Subgradient Linearizations for Nonsmooth Convex Optimization , 2008, SIAM J. Optim..

[33]  R. Rockafellar,et al.  Prox-regular functions in variational analysis , 1996 .

[34]  A. Bagirov,et al.  Discrete Gradient Method: Derivative-Free Method for Nonsmooth Optimization , 2008 .

[35]  Adrian S. Lewis,et al.  A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization , 2005, SIAM J. Optim..