Comparing different nonsmooth minimization methods and software

Most nonsmooth optimization (NSO) methods can be divided into two main groups: subgradient methods and bundle methods. In this paper, we test and compare different methods from both groups as well as some methods which may be considered as hybrids of these two and/or some others. All the solvers tested are so-called general black box methods which, at least in theory, can be applied to solve almost all NSO problems. The test set includes a large number of unconstrained nonsmooth convex and nonconvex problems of different size. In particular, it includes piecewise linear and quadratic problems. The aim of this work is not to foreground some methods over the others but to get some insight on which method to select for certain types of problems.

[1]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[2]  A. Bihain.  Optimization of upper semidifferentiable functions , 1984 .

[3]  Ladislav Luksan,et al.  Dual method for solving a special problem of quadratic programming as a subproblem at linearly constrained nonlinear minimax approximation , 1984, Kybernetika.

[4]  K. Kiwiel.  Methods of Descent for Nondifferentiable Optimization , 1985 .

[5]  J. Moreau,et al.  Topics in Nonsmooth Mechanics , 1988 .

[6]  C. Lemaréchal.  Chapter VII Nondifferentiable optimization , 1989 .

[7]  Krzysztof C. Kiwiel,et al.  Proximity control in bundle methods for convex nondifferentiable minimization , 1990, Math. Program..

[8]  S. Uryas'ev.  New variable-metric algorithms for nondifferentiable optimization problems , 1991 .

[9]  P. Neittaanmäki,et al.  Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control , 1992 .

[10]  Jochem Zowe,et al.  A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results , 1992, SIAM J. Optim..

[11]  M. F. Monaco,et al.  Variants to the cutting plane approach for convex nondifferentiable optimization , 1992 .

[12]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[13]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[14]  J. Haslinger,et al.  Finite Element Approximation for Optimal Shape, Material and Topology Design , 1996 .

[15]  Yu. S. Ledyaev,et al.  Nonsmooth analysis and control theory , 1998 .

[16]  Michal Kočvara,et al.  Nonsmooth approach to optimization problems with equilibrium constraints : theory, applications, and numerical results , 1998 .

[17]  Jan Vlcek,et al.  A bundle-Newton method for nonsmooth unconstrained minimization , 1998, Math. Program..

[18]  Stephen M. Robinson,et al.  Linear convergence of epsilon-subgradient descent methods for a class of convex functions , 1999, Math. Program..

[19]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Convex Nonsmooth Unconstrained Minimization1 , 1999 .

[20]  M. M. MÄKELÄ,et al.  Comparing Nonsmooth Nonconvex Bundle Methods in Solving Hemivariational Inequalities , 1999, J. Glob. Optim..

[21]  Paul S. Bradley,et al.  Mathematical Programming for Data Mining: Formulations and Challenges , 1999, INFORMS J. Comput..

[22]  Franz Kappel,et al.  An Implementation of Shor's r-Algorithm , 2000, Comput. Optim. Appl..

[23]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization , 2001 .

[24]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[25]  Adil M. Bagirov,et al.  A method of truncated codifferential with application to some problems of cluster analysis , 2002, J. Glob. Optim..

[26]  Marko Mäkelä,et al.  Survey of Bundle Methods for Nonsmooth Optimization , 2002, Optim. Methods Softw..

[27]  Marc Teboulle,et al.  Mirror descent and nonlinear projected subgradient methods for convex optimization , 2003, Oper. Res. Lett..

[28]  A. Bagirov.  Continuous Subdifferential Approximations and Their Applications , 2003 .

[29]  Tommi Kärkkäinen,et al.  Robust Formulations for Training Multilayer Perceptrons , 2004, Neural Computation.

[30]  Antonio Fuduli,et al.  A DC piecewise affine model and a bundling technique in nonconvex nonsmooth minimization , 2004, Optim. Methods Softw..

[31]  Kaisa Miettinen,et al.  New limited memory bundle method for large-scale nonsmooth optimization , 2004, Optim. Methods Softw..

[32]  Antonio Fuduli,et al.  Minimizing Nonconvex Nonsmooth Functions via Cutting Planes and Proximity Control , 2003, SIAM J. Optim..

[33]  Marjo S. Haarala.  Large-scale nonsmooth optimization : variable metric bundle method with limited memory , 2004 .

[34]  Adrian S. Lewis,et al.  A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization , 2005, SIAM J. Optim..

[35]  Claudia A. Sagastizábal,et al.  An Infeasible Bundle Method for Nonsmooth Convex Constrained Optimization without a Penalty Function or a Filter , 2005, SIAM J. Optim..

[36]  Arkadi Nemirovski,et al.  Non-euclidean restricted memory level method for large-scale convex optimization , 2005, Math. Program..

[37]  Kaisa Miettinen,et al.  Synchronous approach in interactive multiobjective optimization , 2006, Eur. J. Oper. Res..

[38]  Sami Äyrämö,et al.  Knowledge mining using robust clustering , 2006 .

[39]  Kaisa Miettinen,et al.  Globally convergent limited memory bundle method for large-scale nonsmooth optimization , 2007, Math. Program..

[40]  M. M. Ali,et al.  Limited memory interior point bundle method for large inequality constrained nonsmooth minimization , 2008, Appl. Math. Comput..

[41]  Aude Rondepierre,et al.  A Proximity Control Algorithm to Minimize Nonsmooth and Nonconvex Functions , 2008 .

[42]  A. Bagirov,et al.  Discrete Gradient Method: Derivative-Free Method for Nonsmooth Optimization , 2008 .

[43]  Samir Elhedhli,et al.  Nondifferentiable Optimization , 2009, Encyclopedia of Optimization.

[44]  Adil M. Bagirov,et al.  Empirical and Theoretical Comparisons of Several Nonsmooth Minimization Methods and Software , 2009 .

[45]  Warren Hare,et al.  A Redistributed Proximal Bundle Method for Nonconvex Optimization , 2010, SIAM J. Optim..

[46]  Adil M. Bagirov,et al.  A quasisecant method for minimizing nonsmooth functions , 2010, Optim. Methods Softw..

[47]  Georgios E. Stavroulakis,et al.  Nonconvex Optimization in Mechanics , 2013 .