An algorithm for nonsmooth optimization by successive piecewise linearization

We present an optimization method for Lipschitz continuous, piecewise smooth (PS) objective functions based on successive piecewise linearization. Since, in many realistic cases, nondifferentiabilities are caused by the occurrence of abs(), max(), and min(), we concentrate on these nonsmooth elemental functions. The method’s idea is to locate an optimum of a PS objective function by explicitly handling the kink structure at the level of piecewise linear models. This piecewise linearization can be generated in its abs-normal-form by minor extension of standard algorithmic, or automatic, differentiation tools. In this paper it is shown that the new method when started from within a compact level set generates a sequence of iterates whose cluster points are all Clarke stationary. Numerical results including comparisons with other nonsmooth optimization methods then illustrate the capabilities of the proposed approach.

[1]  Andreas Griewank,et al.  On Lipschitz optimization based on gray-box piecewise linearization , 2016, Math. Program..

[2]  Adrian S. Lewis,et al.  A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization , 2005, SIAM J. Optim..

[3]  P. Neittaanmäki,et al.  Nonsmooth Optimization: Analysis and Algorithms with Applications to Optimal Control , 1992 .

[4]  Claudia Sagastizábal,et al.  BUNDLE METHODS IN THE XXIst CENTURY: A BIRD'S-EYE VIEW , 2014 .

[5]  Marko M. Mäkelä,et al.  Limited memory bundle method for large bound constrained nonsmooth optimization: convergence analysis , 2010, Optim. Methods Softw..

[6]  J. Goffin.  Subgradient Optimization in Nonsmooth Optimization (including the Soviet Revolution) , 2012 .

[7]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[8]  Adrian S. Lewis,et al.  Nonsmooth optimization via quasi-Newton methods , 2012, Mathematical Programming.

[9]  Claude Lemaréchal,et al.  Variable metric bundle methods: From conceptual to implementable forms , 1997, Math. Program..

[10]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[11]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[12]  Yurii Nesterov,et al.  Lexicographic differentiation of nonsmooth functions , 2005, Math. Program..

[13]  Andreas Griewank,et al.  First- and second-order optimality conditions for piecewise smooth objective functions , 2016, Optim. Methods Softw..

[14]  Mert Gürbüzbalaban,et al.  On Nesterov's Nonsmooth Chebyshev-Rosenbrock Functions , 2012 .

[15]  S. Scholtes.  Introduction to Piecewise Differentiable Equations , 2012 .

[16]  Kaisa Miettinen,et al.  New limited memory bundle method for large-scale nonsmooth optimization , 2004, Optim. Methods Softw..

[17]  Andreas Griewank,et al.  On stable piecewise linearization and generalized algorithmic differentiation , 2013, Optim. Methods Softw..

[18]  A. Griewank,et al.  Solving piecewise linear systems in abs-normal form , 2015, 1701.00753.

[19]  N. Shor.  Nondifferentiable Optimization and Polynomial Problems , 1998 .