Survey of Bundle Methods for Nonsmooth Optimization

Bundle methods are at the moment the most efficient and promising methods for nonsmooth optimization. They have been successfully used in many practical applications, for example, in economics, mechanics, engineering and optimal control. The aim of this paper is to give an overview of the development and history of the bundle methods from the seventies to the present. For simplicity, we first concentrate on the convex unconstrained case with a single objective function. The methods are later extended to nonconvex, constrained and multicriteria cases.

[1]  R. Tyrrell Rockafellar,et al.  Generalized Hessian Properties of Regularized Nonsmooth Functions , 1996, SIAM J. Optim..

[2]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization , 2001 .

[3]  Krzysztof C. Kiwiel.  Approximations in Decomposition of Large-Scale Convex Programs via a Nondifferentiable Optimization Method , 1990 .

[4]  K. Kiwiel.  A constraint linearization method for nondifferentiable convex minimization , 1987 .

[5]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[6]  J. Haslinger,et al.  Finite Element Approximation for Optimal Shape, Material and Topology Design , 1996 .

[7]  Kaisa Miettinen,et al.  Comparative evaluation of some interactive reference point-based methods for multi-objective optimisation , 1999, J. Oper. Res. Soc..

[8]  R. Rockafellar.  The theory of subgradients and its applications to problems of optimization : convex and nonconvex functions , 1981 .

[9]  Per Olov Lindberg,et al.  A descent proximal level bundle method for convex nondifferentiable optimization , 1995, Oper. Res. Lett..

[10]  Kaisa Miettinen,et al.  Interactive multiobjective optimization system WWW-NIMBUS on the Internet , 2000, Comput. Oper. Res..

[11]  K. Kiwiel.  Methods of Descent for Nondifferentiable Optimization , 1985 .

[12]  P. Neittaanmäki,et al.  Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control , 1992 .

[13]  K. Miettinen,et al.  Interactive bundle-based method for nondifferentiable multiobjeective optimization: nimbus § , 1995 .

[14]  Yu. S. Ledyaev,et al.  Nonsmooth analysis and control theory , 1998 .

[15]  M. M. MÄKELÄ,et al.  Comparing Nonsmooth Nonconvex Bundle Methods in Solving Hemivariational Inequalities , 1999, J. Glob. Optim..

[16]  P. Wolfe.  Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974 .

[17]  K. Kiwiel.  An Exact Penalty Function Algorithm for Non-smooth Convex Constrained Minimization Problems , 1985 .

[18]  Robert Mifflin,et al.  An Algorithm for Constrained Optimization with Semismooth Functions , 1977, Math. Oper. Res..

[19]  J. Frédéric Bonnans,et al.  A family of variable metric proximal methods , 1995, Math. Program..

[20]  K. Kiwiel.  A descent method for nonsmooth convex multiobjective minimization , 1985 .

[21]  K. Kiwiel.  Proximal Minimization Methods with Generalized Bregman Functions , 1997 .

[22]  M. Gaudioso,et al.  Some Techniques for Finding the Search Direction in Nonsmooth Minimization Problems , 1989 .

[23]  A. A. Goldstein,et al.  Newton's method for convex programming and Tchebycheff approximation , 1959, Numerische Mathematik.

[24]  Robert Mifflin,et al.  A quasi-second-order proximal bundle algorithm , 1996, Math. Program..

[25]  Claude Lemaréchal,et al.  An approach to variable metric bundle methods , 1993, System Modelling and Optimization.

[26]  A. Bihain.  Optimization of upper semidifferentiable functions , 1984 .

[27]  Ulf Brännlund,et al.  A descent method with relaxation type step , 1993, System Modelling and Optimization.

[28]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[29]  Philip Wolfe,et al.  Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974, Math. Program..

[30]  R. Mifflin.  A modification and an extension of Lemarechal’s algorithm for nonsmooth minimization , 1982 .

[31]  C. Lemaréchal.  Chapter VII Nondifferentiable optimization , 1989 .

[32]  H. Mukai.  Algorithms for multicriterion optimization , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[33]  Krzysztof C. Kiwiel,et al.  Proximity control in bundle methods for convex nondifferentiable minimization , 1990, Math. Program..

[34]  Krzysztof C. Kiwiel,et al.  Exact penalty functions in proximal bundle methods for constrained convex nondifferentiable minimization , 1991, Math. Program..

[35]  Jean-Louis Goffin,et al.  On convergence rates of subgradient optimization methods , 1977, Math. Program..

[36]  Andrzej P. Wierzbick.  Basic properties of scalarizmg functionals for multiobjective optimization , 1977 .

[37]  K. Kiwiel.  Efficiency of Proximal Bundle Methods , 2000 .

[38]  A. Auslender.  Numerical methods for nondifferentiable convex optimization , 1987 .

[39]  Xiaojun Chen,et al.  Proximal quasi-Newton methods for nondifferentiable convex optimization , 1999, Math. Program..

[40]  Manlio Gaudioso,et al.  A bundle type approach to the unconstrained minimization of convex nonsmooth functions , 1982, Math. Program..

[41]  C. Lemaréchal,et al.  ON A BUNDLE ALGORITHM FOR NONSMOOTH OPTIMIZATION , 1981 .

[42]  S. Uryas'ev.  New variable-metric algorithms for nondifferentiable optimization problems , 1991 .

[43]  K. Kiwiel.  Approximations in proximal bundle methods and decomposition of convex programs , 1995 .

[44]  Kok Lay Teo,et al.  On constrained optimization problems with nonsmooth cost functionals , 1988 .

[45]  K. H. Kim.  The theory of subgradients and its applications to problems of optimization: Convex and nonconvex functions: R.T. Rockafeller, Berlin: Heldermann Verlag, 1981. pp. 107, DM 28.00/$12.00 , 1983 .

[46]  Krzysztof C. Kiwiel.  A method of linearizations for linearly constrained nonconvex nonsmooth minimization , 1986, Math. Program..

[47]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Convex Nonsmooth Unconstrained Minimization1 , 1999 .

[48]  Krzysztof C. Kiwiel,et al.  Restricted Step and Levenberg-Marquardt Techniques in Proximal Bundle Methods for Nonconvex Nondifferentiable Optimization , 1996, SIAM J. Optim..

[49]  R. Rockafellar.  Monotone Operators and the Proximal Point Algorithm , 1976 .

[50]  Per Olov Lindberg,et al.  Preliminary computational experience with a descent level method for convex nondifferentiable optimization , 1996 .

[51]  C. Lemaréchal.  Nonsmooth Optimization and Descent Methods , 1978 .

[52]  Jan Vlcek,et al.  A bundle-Newton method for nonsmooth unconstrained minimization , 1998, Math. Program..

[53]  Claude Lemaréchal,et al.  Variable metric bundle methods: From conceptual to implementable forms , 1997, Math. Program..

[54]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[55]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[56]  M. F. Monaco,et al.  Variants to the cutting plane approach for convex nondifferentiable optimization , 1992 .

[57]  V. N. Tarasov,et al.  A Modification of the Cutting-Plane Method with Accelerated Convergence , 1985 .

[58]  Yurii Nesterov,et al.  New variants of bundle methods , 1995, Math. Program..

[59]  Nimrod Megiddo,et al.  Progress in Mathematical Programming , 1989 .

[60]  C. Lemaréchal.  An extension of davidon methods to non differentiable problems , 1975 .

[61]  B. N. Pshenichnyi,et al.  Numerical Methods in Extremal Problems. , 1978 .

[62]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[63]  James V. Burke,et al.  On the superlinear convergence of the variable metric proximal point algorithm using Broyden and BFGS matrix secant updating , 2000, Math. Program..

[64]  Z. Wei,et al.  Convergence Analysis of Some Methods for Minimizing a Nonsmooth Convex Function , 2003 .

[65]  K. Kiwiel.  An algorithm for nonsmooth convex minimization with errors , 1985 .

[66]  Kaisa Miettinen,et al.  Optimal Control of Continuous Casting by Nondifferentiable Multiobjective Optimization , 1998, Comput. Optim. Appl..

[67]  A. Auslender,et al.  Penalty-proximal methods in convex programming , 1987 .

[68]  J. E. Kelley,et al.  The Cutting-Plane Method for Solving Convex Programs , 1960 .

[69]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[70]  Michal Kočvara,et al.  Nonsmooth approach to optimization problems with equilibrium constraints : theory, applications, and numerical results , 1998 .

[71]  Jochem Zowe,et al.  A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results , 1992, SIAM J. Optim..

[72]  Claude Lemaréchal,et al.  Practical Aspects of the Moreau-Yosida Regularization: Theoretical Preliminaries , 1997, SIAM J. Optim..

[73]  D. Mayne,et al.  On the Extension of Constrained Optimization Algorithms from Differentiable to Nondifferentiable Problems , 1983 .

[74]  Krzysztof C. Kiwiel,et al.  A tilted cutting plane proximal bundle method for convex nondifferentiable optimization , 1991, Oper. Res. Lett..

[75]  Krzysztof C. Kiwiel.  A bundle Bregman proximal method for convex nondifferentiable minimization , 1999, Math. Program..

[76]  Kaisa Miettinen,et al.  Nonlinear multiobjective optimization , 1998, International series in operations research and management science.

[77]  Krzysztof C. Kiwiel,et al.  Proximal level bundle methods for convex nondifferentiable optimization, saddle-point problems and variational inequalities , 1995, Math. Program..

[78]  J. Moreau,et al.  Topics in Nonsmooth Mechanics , 1988 .

[79]  Jean-Jacques Strodiot,et al.  ε-Optimal solutions in nondifferentiable convex programming and some related questions , 1983, Math. Program..