Globally convergent limited memory bundle method for large-scale nonsmooth optimization

Many practical optimization problems involve nonsmooth (that is, not necessarily differentiable) functions of thousands of variables. In the paper [Haarala, Miettinen, Mäkelä, Optimization Methods and Software, 19, (2004), pp. 673–692] we have described an efficient method for large-scale nonsmooth optimization. In this paper, we introduce a new variant of this method and prove its global convergence for locally Lipschitz continuous objective functions, which are not necessarily differentiable or convex. In addition, we give some encouraging results from numerical experiments.

[1]  Dumitru Motreanu,et al.  Evolutionary Variational-hemivariational Inequalities: Existence and Comparison Results , 2008 .

[2]  Jochem Zowe,et al.  A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results , 1992, SIAM J. Optim..

[3]  P. Panagiotopoulos.  Static Hemivariational Inequalities , 1993 .

[4]  G. Stavroulakis,et al.  Nonconvex Optimization in Mechanics: Algorithms, Heuristics and Engineering Applications , 1997 .

[5]  J. Moreau,et al.  Topics in Nonsmooth Mechanics , 1988 .

[6]  K. Kiwiel,et al.  Power management in a hydro-thermal system under uncertainty by Lagrangian relaxation , 2002 .

[7]  C. Lemaréchal,et al.  ON A BUNDLE ALGORITHM FOR NONSMOOTH OPTIMIZATION , 1981 .

[8]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[9]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization , 2001 .

[10]  C. Lemaréchal.  Nondifferentiable optimization , 1989 .

[11]  Michal Kočvara,et al.  Nonsmooth approach to optimization problems with equilibrium constraints : theory, applications, and numerical results , 1998 .

[12]  C Greengard,et al.  Decision Making Under Uncertainty: Energy and Power (The IMA Volumes in Mathematics and its Applications) , 2002 .

[13]  L. Chambers.  Practical methods of optimization (2nd edn) , by R. Fletcher. Pp. 436. £34.95. 2000. ISBN 0 471 49463 1 (Wiley). , 2001, The Mathematical Gazette.

[14]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[15]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[16]  K. Kiwiel.  A Method for Solving Certain Quadratic Programming Problems Arising in Nonsmooth Optimization , 1986 .

[17]  Matthias Peter Nowak,et al.  Stochastic Programming for Power Production and Trading under Uncertainty , 2003 .

[18]  W. Jäger,et al.  Mathematics – key technology for the future , 2003 .

[19]  Tommi Kärkkäinen,et al.  Comparison of formulations and solution methods for image restoration problems , 2001 .

[20]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[21]  P. D. Panagiotopoulos,et al.  Mathematical Theory of Hemivariational Inequalities and Applications , 1994 .

[22]  K. Kiwiel.  Methods of Descent for Nondifferentiable Optimization , 1985 .

[23]  Kaisa Miettinen,et al.  New limited memory bundle method for large-scale nonsmooth optimization , 2004, Optim. Methods Softw..

[24]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Convex Nonsmooth Unconstrained Minimization1 , 1999 .

[25]  Claude Lemaréchal,et al.  Bundle Methods in Stochastic Optimal Power Management: A Disaggregated Approach Using Preconditioners , 2001, Comput. Optim. Appl..

[26]  Claude Lemaréchal,et al.  Some numerical experiments with variable-storage quasi-Newton algorithms , 1989, Math. Program..

[27]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[28]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[29]  Marjo S. Haarala.  Large-scale nonsmooth optimization : variable metric bundle method with limited memory , 2004 .

[30]  P. Neittaanmäki,et al.  Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control , 1992 .

[31]  Yu. S. Ledyaev,et al.  Nonsmooth analysis and control theory , 1998 .

[32]  M. M. MÄKELÄ,et al.  Comparing Nonsmooth Nonconvex Bundle Methods in Solving Hemivariational Inequalities , 1999, J. Glob. Optim..

[33]  Marko Mäkelä,et al.  Survey of Bundle Methods for Nonsmooth Optimization , 2002, Optim. Methods Softw..

[34]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[35]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[36]  A. Bihain.  Optimization of upper semidifferentiable functions , 1984 .

[37]  R. Mifflin.  A modification and an extension of Lemarechal’s algorithm for nonsmooth minimization , 1982 .

[38]  C. Lemaréchal.  Chapter VII Nondifferentiable optimization , 1989 .