Gradient set splitting in nonconvex nonsmooth numerical optimization

We present a numerical bundle-type method for local minimization of a real function of several variables, which is supposed to be locally Lipschitz. We provide a short survey of some optimization algorithms from the literature, which are able to deal with both nonsmoothness and nonconvexity of the objective function. We focus on possible extensions of classical bundle-type methods, originally conceived to deal with convex nonsmooth optimization. They are all based on a convex cutting plane model which has the property of both minorizing everywhere and interpolating at certain points the objective function. Such properties may be lost whenever nonconvexity is present and the case may be described in terms of possible negative values of certain linearization errors. We describe some alternative ways the problem is dealt with in the literature. Here, on the basis of a classification of the limit points of gradient sequences, we define two distinct cutting plane approximations. We derive an algorithm which uses both such models. In particular, only the convex model is primarily adopted to find a tentative displacement from the current stability centre, while the concave one enters into the play only when the convex model has failed in providing a sufficient decrease step. Termination of the method is proved and the results of some numerical experiments are reported.

[1]  Jochem Zowe,et al.  A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results , 1992, SIAM J. Optim..

[2]  A. Ioffe Proximal Analysis and Approximate Subdifferentials , 1990 .

[3]  A. Bagirov Continuous Subdifferential Approximations and Their Applications , 2003 .

[4]  Warren Hare,et al.  A Redistributed Proximal Bundle Method for Nonconvex Optimization , 2010, SIAM J. Optim..

[5]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[6]  M. Pullan CONVEX ANALYSIS AND MINIMIZATION ALGORITHMS Volumes I and II (Comprehensive Studies in Mathematics 305, 306) , 1995 .

[7]  Enrico Gorgone,et al.  Piecewise linear approximations in nonconvex nonsmooth optimization , 2009, Numerische Mathematik.

[8]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization , 2001 .

[9]  Philip Wolfe,et al.  Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974, Math. Program..

[10]  R. Mifflin A modification and an extension of Lemarechal’s algorithm for nonsmooth minimization , 1982 .

[11]  A. Bagirov,et al.  Discrete Gradient Method: Derivative-Free Method for Nonsmooth Optimization , 2008 .

[12]  Antonio Fuduli,et al.  A DC piecewise affine model and a bundling technique in nonconvex nonsmooth minimization , 2004, Optim. Methods Softw..

[13]  Defeng Sun,et al.  Quasi-Newton Bundle-Type Methods for Nondifferentiable Convex Optimization , 1998, SIAM J. Optim..

[14]  Antonio Fuduli,et al.  Minimizing Nonconvex Nonsmooth Functions via Cutting Planes and Proximity Control , 2003, SIAM J. Optim..

[15]  Giovanna Miglionico,et al.  An Incremental Method for Solving Convex Finite Min-Max Problems , 2006, Math. Oper. Res..

[16]  Krzysztof C. Kiwiel,et al.  An aggregate subgradient method for nonsmooth convex minimization , 1983, Math. Program..

[17]  Robert Mifflin,et al.  An Algorithm for Constrained Optimization with Semismooth Functions , 1977, Math. Oper. Res..

[18]  A. Ioffe Nonsmooth analysis: differential calculus of nondifferentiable mappings , 1981 .

[19]  P. Wolfe Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974 .

[20]  Jan Vlcek,et al.  A bundle-Newton method for nonsmooth unconstrained minimization , 1998, Math. Program..

[21]  J. Frédéric Bonnans,et al.  A family of variable metric proximal methods , 1995, Math. Program..

[22]  J. Penot Calcul sous-differentiel et optimisation , 1978 .

[23]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[24]  V. F. Demʹi︠a︡nov,et al.  Constructive nonsmooth analysis , 1995 .

[25]  Adrian S. Lewis,et al.  A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization , 2005, SIAM J. Optim..

[26]  M. Overton NONSMOOTH OPTIMIZATION VIA BFGS , 2008 .

[27]  K. G. Murty,et al.  Nonsmooth optimization , 2007 .

[28]  B. N. Pshenichnyi Necessary Conditions for an Extremum , 1971 .

[29]  Donald Goldfarb,et al.  A numerically stable dual method for solving strictly convex quadratic programs , 1983, Math. Program..

[30]  K. Kiwiel Methods of Descent for Nondifferentiable Optimization , 1985 .