Limited memory bundle algorithm for large bound constrained nonsmooth minization problems

Typically, practical optimization problems involve nonsmooth functions of hundreds or thousands of variables. As a rule, the variables in such problems are restricted to certain meaningful intervals. In this paper, we propose an efficient adaptive limited memory bundle method for large-scale nonsmooth, possibly nonconvex, bound constrained optimization. The method combines the nonsmooth variable metric bundle method and the smooth limited memory variable metric method, while the constraint handling is based on the projected gradient method and the dual subspace minimization. The preliminary numerical experiments to be presented confirm the usability of the method.

[1]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[2]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[3]  C. Lemaréchal,et al.  ON A BUNDLE ALGORITHM FOR NONSMOOTH OPTIMIZATION , 1981 .

[4]  R. Mifflin.  A modification and an extension of Lemarechal’s algorithm for nonsmooth minimization , 1982 .

[5]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[6]  K. Kiwiel.  Methods of Descent for Nondifferentiable Optimization , 1985 .

[7]  Krzysztof C. Kiwiel.  A method of linearizations for linearly constrained nonconvex nonsmooth minimization , 1986, Math. Program..

[8]  Roger Fletcher,et al.  Practical methods of optimization; (2nd ed.) , 1987 .

[9]  K. Kiwiel.  A constraint linearization method for nondifferentiable convex minimization , 1987 .

[10]  Eliane R. Panier,et al.  An active set method for solving linearly constrained nonsmooth optimization problems , 1987, Math. Program..

[11]  P. Toint,et al.  Testing a class of methods for solving minimization problems with simple bounds on the variables , 1988 .

[12]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[13]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[14]  P. Toint,et al.  Global convergence of a class of trust region algorithms for optimization with simple bounds , 1988 .

[15]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[16]  C. Lemaréchal.  Nondifferentiable optimization , 1989 .

[17]  Claude Lemaréchal,et al.  Some numerical experiments with variable-storage quasi-Newton algorithms , 1989, Math. Program..

[18]  P. Neittaanmäki,et al.  Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control , 1992 .

[19]  Jochem Zowe,et al.  A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results , 1992, SIAM J. Optim..

[20]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[21]  P. D. Panagiotopoulos,et al.  Mathematical Theory of Hemivariational Inequalities and Applications , 1994 .

[22]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[23]  J. Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[24]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Convex Nonsmooth Unconstrained Minimization1 , 1999 .

[25]  M. M. MÄKELÄ,et al.  Comparing Nonsmooth Nonconvex Bundle Methods in Solving Hemivariational Inequalities , 1999, J. Glob. Optim..

[26]  Tommi Kärkkäinen,et al.  Comparison of formulations and solution methods for image restoration problems , 2001 .

[27]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization , 2001 .

[28]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[29]  Kaisa Miettinen,et al.  New limited memory bundle method for large-scale nonsmooth optimization , 2004, Optim. Methods Softw..

[30]  Marjo S. Haarala.  Large-scale nonsmooth optimization : variable metric bundle method with limited memory , 2004 .

[31]  Arkadi Nemirovski,et al.  Non-euclidean restricted memory level method for large-scale convex optimization , 2005, Math. Program..

[32]  Kaisa Miettinen,et al.  Globally convergent limited memory bundle method for large-scale nonsmooth optimization , 2007, Math. Program..