Subgradient ellipsoid method for nonsmooth convex problems

In this paper, we present a new ellipsoid-type algorithm for solving nonsmooth problems with convex structure. Examples of such problems include nonsmooth convex minimization problems, convex-concave saddle-point problems and variational inequalities with monotone operator. Our algorithm can be seen as a combination of the standard Subgradient and Ellipsoid methods. However, in contrast to the latter one, the proposed method has a reasonable convergence rate even when the dimensionality of the problem is sufficiently large. For generating accuracy certificates in our algorithm, we propose an efficient technique, which ameliorates the previously known recipes.

[1]  Guanghui Lan,et al.  First-order and Stochastic Optimization Methods for Machine Learning , 2020 .

[2]  Alexander Gasnikov,et al.  Stochastic Intermediate Gradient Method for Convex Problems with Stochastic Inexact Oracle , 2016, Journal of Optimization Theory and Applications.

[3]  Aharon Ben-Tal,et al.  Lectures on modern convex optimization , 1987 .

[4]  Claude Lemaréchal,et al.  An Algorithm for Minimizing Convex Functions , 1974, IFIP Congress.

[5]  A. Auslender,et al.  Resolution Numerique D’inegalites Variationnelles , 1974 .

[6]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[7]  Mohit Singh,et al.  A geometric alternative to Nesterov's accelerated gradient descent , 2015, ArXiv.

[8]  M. Todd,et al.  The Ellipsoid Method: A Survey , 1980 .

[9]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[10]  Uriel G. Rothblum,et al.  Accuracy Certificates for Computational Problems with Convex Structure , 2010, Math. Oper. Res..

[11]  Guanghui Lan,et al.  An optimal method for stochastic composite optimization , 2011, Mathematical Programming.

[12]  N. Z. Shor.  Cut-off method with space extension in convex programming problems , 1977, Cybernetics.

[13]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[14]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[15]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[16]  Alexander Shapiro,et al.  Stochastic Approximation approach to Stochastic Programming , 2013 .

[17]  Martin Grötschel,et al.  The ellipsoid method and its consequences in combinatorial optimization , 1981, Comb..

[18]  Yin Tat Lee,et al.  Black-box Optimization with a Politician , 2016, ICML.

[19]  D. J. Newman,et al.  Location of the Maximum on Unimodal Surfaces , 1965, JACM.

[20]  Yurii Nesterov,et al.  Lectures on Convex Optimization , 2018 .