Priority Programme 1962 Beyond the Oracle: Opportunities of Piecewise Differentiation

For more than thirty years much of the research and development in nonsmooth optimization has been predicated on the assumption that the user provides an oracle that evaluates at any given x ∈ R the objective function value φ(x) and a generalized gradient g ∈ ∂φ(x) in the sense of Clarke. We will argue here that, if there is a realistic possibility of computing a vector g that is guaranteed to be a generalized gradient, then one must know so much about the way φ : R → R is calculated that more information about the behavior of φ in a neighborhood of the evaluation point can be extracted. Moreover, the latter can be achieved with reasonable effort and in a stable manner so that the derivative information provided varies Lipschitz continuously with respect to x. In particular we describe the calculation of directionally active generalized gradients, generalized ε-gradients and the checking of first and second order optimality conditions. All this is based on the abs-linearization of a piecewise smooth objective in abs-normal form. 1 Motivation and Introduction It is well understood that the convex set ∂φ(x) of generalized gradients is highly volatile with respect to variations in x, never mind that it is by definition outer semi-continuous as a set-valued mapping ∂φ : R ⇒ R. Moreover, due to Rademacher’s theorem for Lipschitzian functions on a Euclidean space, we can expect that almost everywhere we get a singleton ∂φ(x) = {∂φ(x)} so Andreas Griewank School of Mathematical and Computational Sciences, Yachay Tech, Urcuqúı, Imbabura, Ecuador, e-mail: griewank@yachaytech.edu.ec Andrea Walther Institut für Mathematik, Universität Paderborn, Paderborn, Germany e-mail: andrea.walther@uni-paderborn.de

[1]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[2]  Yurii Nesterov,et al.  Lexicographic differentiation of nonsmooth functions , 2005, Math. Program..

[3]  S. Scholtes Introduction to Piecewise Differentiable Equations , 2012 .

[4]  Diethard Klatte,et al.  Nonsmooth Equations in Optimization: "Regularity, Calculus, Methods And Applications" , 2006 .

[5]  Paul I. Barton,et al.  Computationally relevant generalized derivatives: theory, evaluation and applications , 2018, Optim. Methods Softw..

[6]  Adil M. Bagirov,et al.  Introduction to Nonsmooth Optimization , 2014 .

[7]  A. Lewis,et al.  Nonsmooth optimization via quasi-Newton methods , 2012, Mathematical programming.

[8]  Leo Liberti,et al.  Euclidean Distance Geometry and Applications , 2012, SIAM Rev..

[9]  Robert Mifflin,et al.  A Science Fiction Story in Nonsmooth Optimization Originating at IIASA , 2012 .

[10]  Andreas Griewank,et al.  An algorithm for nonsmooth optimization by successive piecewise linearization , 2019, Math. Program..

[11]  J. Outrata On mathematical programs with complementarity constraints , 2000 .

[12]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[13]  Andreas Griewank,et al.  Piecewise linear secant approximation via algorithmic piecewise differentiation , 2017, Optim. Methods Softw..

[14]  Paul I. Barton,et al.  Evaluating an element of the Clarke generalized Jacobian of a composite piecewise differentiable function , 2013, TOMS.

[15]  Andreas Griewank,et al.  First- and second-order optimality conditions for piecewise smooth objective functions , 2016, Optim. Methods Softw..

[16]  F. Clarke Optimization And Nonsmooth Analysis , 1983 .

[17]  Adrian S. Lewis,et al.  Stabilization via Nonsmooth, Nonconvex Optimization , 2006, IEEE Transactions on Automatic Control.

[18]  Adrian S. Lewis,et al.  Active Sets, Nonsmoothness, and Sensitivity , 2002, SIAM J. Optim..

[19]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[20]  Andreas Griewank,et al.  On stable piecewise linearization and generalized algorithmic differentiation , 2013, Optim. Methods Softw..

[21]  Kamil A. Khan Branch-locking AD techniques for nonsmooth composite functions and nonsmooth implicit functions , 2018, Optim. Methods Softw..

[22]  Adil M. Bagirov,et al.  Introduction to Nonsmooth Optimization: Theory, Practice and Software , 2014 .

[23]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[24]  Marko M. Mäkelä,et al.  Limited memory bundle method for large bound constrained nonsmooth optimization: convergence analysis , 2010, Optim. Methods Softw..

[25]  Horst W. Hamacher,et al.  Classification of location models , 1998 .

[26]  D. Klatte Nonsmooth equations in optimization , 2002 .

[27]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[28]  A. Griewank,et al.  CHARACTERIZING AND TESTING SUBDIFFERENTIAL REGULARITY FOR PIECEWISE SMOOTH OBJECTIVE FUNCTIONS , 2017 .

[29]  Christian Kirches,et al.  On the relation between MPECs and optimization problems in abs-normal form , 2020, Optim. Methods Softw..

[30]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[31]  Andreas Griewank,et al.  Cubic overestimation and secant updating for unconstrained optimization of C2, 1 functions , 2014, Optim. Methods Softw..

[32]  Robert E. Mahony,et al.  Convergence of the Iterates of Descent Methods for Analytic Cost Functions , 2005, SIAM J. Optim..

[33]  Paul I. Barton,et al.  A vector forward mode of automatic differentiation for generalized derivative evaluation , 2015, Optim. Methods Softw..

[34]  Michael Hintermüller,et al.  A semi‐smooth Newton method for constrained linear‐quadratic control problems , 2003 .

[35]  Laurent Hascoët,et al.  The Tapenade automatic differentiation tool: Principles, model, and specification , 2013, TOMS.

[36]  Andreas Griewank,et al.  Finite convergence of an active signature method to local minima of piecewise linear functions , 2018, Optim. Methods Softw..

[37]  A. Griewank Automatic Directional Differentiation of Nonsmooth Composite Functions , 1995 .