On the Quality of First-Order Approximation of Functions with Hölder Continuous Gradient

We show that Hölder continuity of the gradient is not only a sufficient condition, but also a necessary condition for the existence of a global upper bound on the error of the first-order Taylor approximation. We also relate this global upper bound to the Hölder constant of the gradient. This relation is expressed as an interval, depending on the Hölder constant, in which the error of the first-order Taylor approximation is guaranteed to be. We show that, for the Lipschitz continuous case, the interval cannot be reduced. An application to the norms of quadratic forms is proposed, which allows us to derive a novel characterization of Euclidean norms.

[1]  T. Motzkin,et al.  Maxima for Graphs and a New Proof of a Theorem of Turán , 1965, Canadian Journal of Mathematics.

[2]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[3]  K. Ball.  Ellipsoids of maximal volume in convex bodies , 1990, math/9201217.

[4]  P. Absil,et al.  Erratum to: ``Global rates of convergence for nonconvex optimization on manifolds'' , 2016, IMA Journal of Numerical Analysis.

[5]  F. John.  Extremum Problems with Inequalities as Subsidiary Conditions , 2014 .

[6]  Maryam Yashtini,et al.  On the global convergence rate of the gradient descent method for functions with Hölder continuous gradients , 2016, Optim. Lett..

[7]  Yurii Nesterov,et al.  Gradient methods for minimizing composite functions , 2012, Mathematical Programming.

[8]  D. Amir.  Characterizations of Inner Product Spaces , 1986 .

[9]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[10]  Yurii Nesterov,et al.  Universal gradient methods for convex optimization problems , 2015, Math. Program..

[11]  A. Friedman.  Foundations of modern analysis , 1970 .

[12]  Joseph E. Borzellino,et al.  When is a Trigonometric Polynomial Not a Trigonometric Polynomial , 1935 .

[13]  Nicholas I. M. Gould,et al.  Worst-case evaluation complexity of regularization methods for smooth unconstrained optimization using Hölder continuous gradients , 2017, Optim. Methods Softw..