A New Nonsmooth Trust Region Algorithm for Locally Lipschitz Unconstrained Optimization Problems

In this paper, a new nonsmooth trust region algorithm is proposed for solving unconstrained minimization problems with locally Lipschitz objective functions. At first, by using an approximation of the steepest descent direction, a local model is presented for locally Lipschitz functions. More precisely, in the quadratic model of classical trust region methods, the gradient vector is replaced by an approximation of the steepest descent direction. We then apply one of the efficient approaches of classical trust region methods in order to solve the obtained model. Using the BFGS updating formula for the Hessian approximation of the model, we show that the proposed algorithm is convergent under some mild and standard conditions on the objective function. Finally, the presented algorithm is implemented in the MATLAB environment and applied on some nonsmooth test problems.

[1]  Johannes O. Royset,et al.  Algorithms for Finite and Semi-Infinite Min–Max–Min Problems Using Adaptive Smoothing Techniques , 2003 .

[2]  R. Fletcher.  A model algorithm for composite nondifferentiable optimization problems , 1982 .

[3]  D. Bertsekas,et al.  A DESCENT NUMERICAL METHOD FOR OPTIMIZATION PROBLEMS WITH NONDIFFERENTIABLE COST FUNCTIONALS , 1973 .

[4]  Antonio Frangioni,et al.  Generalized Bundle Methods , 2002, SIAM J. Optim..

[5]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[6]  P. Wolfe.  Note on a method of conjugate subgradients for minimizing nondifferentiable functions , 1974 .

[7]  P. Wolfe,et al.  A METHOD OF CONJUGATE SUBGRADIENTS FOR , 1975 .

[8]  Richard A. Tapia,et al.  A unified approach to global convergence of trust region methods for nonsmooth optimization , 1995, Math. Program..

[9]  Marko Mäkelä,et al.  Survey of Bundle Methods for Nonsmooth Optimization , 2002, Optim. Methods Softw..

[10]  Liqun Qi,et al.  A trust region algorithm for minimization of locally Lipschitzian functions , 1994, Math. Program..

[11]  Jochem Zowe,et al.  A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results , 1992, SIAM J. Optim..

[12]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[13]  Kaisa Miettinen,et al.  Globally convergent limited memory bundle method for large-scale nonsmooth optimization , 2007, Math. Program..

[14]  Nezam Mahdavi-Amiri,et al.  An Effective Nonsmooth Optimization Algorithm for Locally Lipschitz Functions , 2012, J. Optim. Theory Appl..

[15]  A. Bagirov,et al.  Limited memory discrete gradient bundle method for nonsmooth derivative-free optimization , 2012 .

[16]  K. Kiwiel.  Methods of Descent for Nondifferentiable Optimization , 1985 .

[17]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[18]  A. Bagirov.  Continuous Subdifferential Approximations and Their Applications , 2003 .

[19]  Jan Vlcek,et al.  A bundle-Newton method for nonsmooth unconstrained minimization , 1998, Math. Program..

[20]  Adrian S. Lewis,et al.  A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization , 2005, SIAM J. Optim..

[21]  Adrian S. Lewis,et al.  Nonsmooth optimization via quasi-Newton methods , 2012, Mathematical Programming.

[22]  R. Tapia,et al.  A Global Convergence Theory for Arbitrary Norm Trust-Region Methods for Nonlinear Equations , 1995 .

[23]  Manlio Gaudioso,et al.  A bundle type approach to the unconstrained minimization of convex nonsmooth functions , 1982, Math. Program..

[24]  J. Nocedal,et al.  The use of linear programming for the solution of sparse sets of nonlinear equations , 1987 .

[25]  A. A. Goldstein,et al.  Optimization of lipschitz continuous functions , 1977, Math. Program..

[26]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Convex Nonsmooth Unconstrained Minimization1 , 1999 .

[27]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[28]  Krzysztof C. Kiwiel,et al.  Convergence of the Gradient Sampling Algorithm for Nonsmooth Nonconvex Optimization , 2007, SIAM J. Optim..

[29]  Leon S. Lasdon,et al.  An Improved Successive Linear Programming Algorithm , 1985 .

[30]  L. Luksan,et al.  Globally Convergent Variable Metric Method for Nonconvex Nondifferentiable Unconstrained Minimization , 2001 .

[31]  Naum Zuselevich Shor,et al.  Minimization Methods for Non-Differentiable Functions , 1985, Springer Series in Computational Mathematics.