Improved Algorithms for Convex Minimization in Relative Scale

In this paper we propose two modifications to Nesterov’s algorithms for minimizing convex functions in relative scale. The first is based on a bisection technique and leads to improved theoretical iteration complexity, and the second is a heuristic for avoiding restarting behavior. The fastest of our algorithms produces a solution within relative error O(1/k) of the optimum, with k being the iteration counter.

[1]  Michael J. Todd,et al.  On Khachiyan's algorithm for the computation of minimum-volume enclosing ellipsoids , 2007, Discret. Appl. Math..

[2]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[3]  J. Borwein,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[4]  Peter Richtárik.  Simultaneously solving seven optimization problems in relative scale , 2009 .

[5]  J. Neumann,et al.  The Theory of Games and Economic Behaviour , 1944 .

[6]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[7]  Naum Zuselevich Shor,et al.  Minimization Methods for Non-Differentiable Functions , 1985, Springer Series in Computational Mathematics.

[8]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[9]  Jean-Louis Goffin,et al.  On convergence rates of subgradient optimization methods , 1977, Math. Program..

[10]  Y. Nesterov.  A method for unconstrained convex minimization problem with the rate of convergence o(1/k^2) , 1983 .

[11]  Peng Sun,et al.  Computation of Minimum Volume Covering Ellipsoids , 2002, Oper. Res..

[12]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[13]  Yurii Nesterov.  Unconstrained Convex Minimization in Relative Scale , 2003 .

[14]  Peter Richtárik,et al.  Approximate Level Method for Nonsmooth Convex Minimization , 2011, Journal of Optimization Theory and Applications.

[15]  Adrian S. Lewis,et al.  Convex Analysis And Nonlinear Optimization , 2000 .

[16]  Yurii Nesterov,et al.  Rounding of convex sets and efficient gradient methods for linear programming problems , 2004, Optim. Methods Softw..

[17]  Robert M. Freund,et al.  On the symmetry function of a convex set , 2008, Math. Program..

[18]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[19]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[20]  Peng Sun,et al.  Linear convergence of a modified Frank–Wolfe algorithm for computing minimum-volume enclosing ellipsoids , 2008, Optim. Methods Softw..

[21]  Yurii Nesterov,et al.  Barrier subgradient method , 2011, Math. Program..

[22]  Michael J. Todd,et al.  Some algorithms for large-scale linear and convex minimization in relative scale , 2007 .

[23]  Leonid Khachiyan,et al.  Rounding of Polytopes in the Real Number Model of Computation , 1996, Math. Oper. Res..

[24]  Y. Nesterov.  Rounding of Convex Sets and Efficient Gradient Methods for Linear Programming Problems , 2004 .

[25]  Piyush Kumar,et al.  Minimum-Volume Enclosing Ellipsoids and Core Sets , 2005 .

[26]  Fabián A. Chudak,et al.  Improved Approximation Schemes for Linear Programming Relaxations of Combinatorial Optimization Problems , 2005, IPCO.

[27]  Peter Richtárik,et al.  Approximate level method , 2008 .

[28]  Yurii Nesterov,et al.  Unconstrained Convex Minimization in Relative Scale , 2003, Math. Oper. Res..