Performance estimation toolbox (PESTO): Automated worst-case analysis of first-order optimization methods

We present a Matlab toolbox that automatically computes tight worst-case performance guarantees for a broad class of first-order methods for convex optimization. The class of methods includes those performing explicit, projected, proximal, conditional and inexact (sub)gradient steps. The toolbox relies on the performance estimation (PE) framework, which recently emerged through works of Drori and Teboulle and the authors. The PE approach is a very systematic manner of obtaining non-improvable worst-case guarantees for first-order numerical optimization schemes. However, using the PE methodology requires modelling efforts from the user, along with some knowledge of semidefinite programming. The goal of this work is to ease the use of the performance estimation methodology, by providing a toolbox that implicitly does the modelling job. In short, its aim is to (i) let the user write the algorithm in a natural way, as he/she would have implemented it, and (ii) let the computer perform the modelling and worst-case analysis parts automatically.

[1]  Marc Teboulle,et al.  Performance of first-order methods for smooth convex minimization: a novel approach , 2012, Mathematical Programming.

[2]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[3]  Donghwan Kim,et al.  Optimized first-order methods for smooth convex minimization , 2014, Mathematical Programming.

[4]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[5]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[6]  Shang-Hua Teng,et al.  Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time , 2001, STOC '01.

[7]  Adrien B. Taylor,et al.  Exact Worst-Case Performance of First-Order Methods for Composite Convex Optimization , 2015, SIAM J. Optim..

[8]  Kim-Chuan Toh,et al.  SDPT3 -- A Matlab Software Package for Semidefinite Programming , 1996 .

[9]  Jeffrey A. Fessler,et al.  Generalizing the Optimized Gradient Method for Smooth Convex Minimization , 2016, SIAM J. Optim..

[10]  Benjamin Recht,et al.  Analysis and Design of Optimization Algorithms via Integral Quadratic Constraints , 2014, SIAM J. Optim..

[11]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[12]  Marc Teboulle,et al.  An optimal variant of Kelley’s cutting-plane method , 2014, Math. Program..

[13]  Yoel Drori,et al.  The exact information-based complexity of smooth convex minimization , 2016, J. Complex..

[14]  Etienne de Klerk,et al.  On the worst-case complexity of the gradient method with exact line search for smooth strongly convex functions , 2016, Optimization Letters.

[15]  Yurii Nesterov,et al.  First-order methods of smooth convex optimization with inexact oracle , 2013, Mathematical Programming.

[16]  Adrien B. Taylor,et al.  Exact Worst-Case Convergence Rates of the Proximal Gradient Method for Composite Convex Minimization , 2017, Journal of Optimization Theory and Applications.

[17]  Johan Löfberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004 .

[18]  Jeffrey A. Fessler,et al.  Another Look at the Fast Iterative Shrinkage/Thresholding Algorithm (FISTA) , 2016, SIAM J. Optim..

[19]  Adrien B. Taylor,et al.  Convex interpolation and performance estimation of first-order methods for convex optimization , 2017 .

[20]  Adrien B. Taylor,et al.  Smooth strongly convex interpolation and exact worst-case performance of first-order methods , 2015, Mathematical Programming.

[21]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .