Dynamic smoothness parameter for fast gradient methods

We present and computationally evaluate a variant of the fast gradient method by Nesterov that is capable of exploiting information, even if approximate, about the optimal value of the problem. This information is available in some applications, among which the computation of bounds for hard integer programs. We show that dynamically changing the smoothness parameter of the algorithm using this information results in a better convergence profile of the algorithm in practice.

[1]  Naum Zuselevich Shor,et al.  Minimization Methods for Non-Differentiable Functions , 1985, Springer Series in Computational Mathematics.

[2]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[3]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[4]  Antonio Frangioni,et al.  Convergence Analysis of Deflected Conditional Approximate Subgradient Methods , 2009, SIAM J. Optim..

[5]  Antonio Frangioni,et al.  On the computational efficiency of subgradient methods: a case study with Lagrangian bounds , 2017, Mathematical Programming Computation.

[6]  Mervat Chouman,et al.  Commodity Representations and Cut-Set-Based Inequalities for Multicommodity Capacitated Fixed-Charge Network Design , 2017, Transp. Sci..

[7]  Masoud Ahookhosh,et al.  Optimal subgradient algorithms with application to large-scale linear inverse problems , 2014, 1402.7291.

[8]  A. Frangioni,et al.  On the Computational Efficiency of Subgradient Methods : A Case Study in Combinatorial Optimization , 2015 .

[9]  Antonio Frangioni,et al.  Bundle methods for sum-functions with “easy” components: applications to multicommodity network design , 2013, Mathematical Programming.

[10]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[11]  R. Boţ,et al.  A variable smoothing algorithm for solving convex optimization problems , 2012, 1207.3254.

[12]  Marc Teboulle,et al.  Smoothing and First Order Methods: A Unified Framework , 2012, SIAM J. Optim..

[13]  Paul Tseng,et al.  Approximation accuracy, gradient methods, and error bound for structured convex optimization , 2010, Math. Program..

[14]  Antonio Frangioni,et al.  Generalized Bundle Methods , 2002, SIAM J. Optim..

[15]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[16]  Masoud Ahookhosh,et al.  An optimal subgradient algorithm for large-scale convex optimization in simple domains , 2015, 1501.01451.