Subgradient methods for huge-scale optimization problems

We consider a new class of huge-scale problems, the problems with sparse subgradients. The most important functions of this type are piece-wise linear. For optimization problems with uniform sparsity of corresponding linear operators, we suggest a very efficient implementation of subgradient iterations, which total cost depends logarithmically in the dimension. This technique is based on a recursive update of the results of matrix/vector products and the values of symmetric functions. It works well, for example, for matrices with few nonzero diagonals and for max-type functions. We show that the updating technique can be efficiently coupled with the simplest subgradient methods, the unconstrained minimization method by B.Polyak, and the constrained minimization scheme by N.Shor. Similar results can be obtained for a new nonsmooth random variant of a coordinate descent scheme. We present also the promising results of preliminary computational experiments.

[1]  Naum Zuselevich Shor,et al.  Minimization Methods for Non-Differentiable Functions , 1985, Springer Series in Computational Mathematics.

[2]  Paul Tseng,et al.  On the Convergence Rate of Dual Ascent Methods for Linearly Constrained Convex Minimization , 1993, Math. Oper. Res..

[3]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[4]  Bernt Øksendal,et al.  Optimal time to invest when the price processes are geometric Brownian motions , 1998, Finance Stochastics.

[5]  J. Huriot,et al.  Economics of Cities , 2000 .

[6]  Daniel Bienstock,et al.  Potential Function Methods for Approximately Solving Linear Programming Problems: Theory and Practice , 2002 .

[7]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[8]  Javier Peña,et al.  First-Order Algorithm with O(ln(1/e)) Convergence for e-Equilibrium in Two-Person Zero-Sum Games , 2008, AAAI.

[9]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[10]  Boris S. Mordukhovich.  Characterizations of linear suboptimality for mathematical programs with equilibrium constraints , 2009, Math. Program..

[11]  J. Dávila,et al.  Implementing Steady State Efficiency in Overlapping Generations Economies with Environmental Externalities , 2014 .

[12]  Paul Belleflamme,et al.  Industrial Organization: Markets and Strategies , 2010 .

[13]  S. Bellegem.  Locally stationary volatility modelling , 2011 .

[14]  Y. Smeers,et al.  A real options model for electricity capacity expansion , 2011 .

[15]  M. Fleurbaey,et al.  A Theory of Fairness and Social Welfare , 2011 .

[16]  T. Mayer,et al.  The Economics of Clusters: Lessons from the French Experience , 2011 .

[17]  H. Vandenbussche,et al.  Moving Up the Quality Ladder? Eu-China Trade Dynamics in Clothing , 2011 .

[18]  M. V. Vyve,et al.  Linear prices for non-convex electricity markets: models and algorithms , 2011 .

[19]  Y. Smeers,et al.  Sufficient and necessary conditions for perpetual multi-assets exchange options , 2011 .

[20]  O. Devolder,et al.  Stochastic first order methods in smooth convex optimization , 2011 .

[21]  P. Pestieau,et al.  Indirect Taxes for Redistribution: Should Necessity Goods Be Favored? , 2011 .

[22]  L. Bauwens,et al.  Estimating and forecasting structural breaks in financial time series , 2011 .

[23]  J. Gabszewicz,et al.  Migration, wage differentials and fiscal competition , 2011 .

[24]  P. Pestieau,et al.  Private, social and self insurance for long-term care: A political economy analysis , 2011 .

[25]  J. Dávila.  Optimal population and education , 2011 .

[26]  D. Paolini,et al.  Tax Treaties and the Allocation of Taxing Rights With Developing Countries , 2011 .

[27]  B. Miguel,et al.  Hierarchical shrinkage in time-varying parameter models , 2011 .

[28]  F. Mayneris,et al.  French Firms at the Conquest of Asian Markets: The Role of Export Spillovers , 2011 .

[29]  F. Mayneris.  A new perspective on the firm size-growth relationship: Shape of profits, investment and heterogeneous credit constraints , 2011 .

[30]  K. Munk.  Optimal taxation in the presence of a congested public good and an application to transport policy , 2011 .

[31]  E. Schokkaert,et al.  Behavioral fair social choice , 2011 .

[32]  Yurii Nesterov,et al.  Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems , 2012, SIAM J. Optim..

[33]  Javier Peña,et al.  First-order algorithm with $${\mathcal{O}({\rm ln}(1{/}\epsilon))}$$ convergence for $${\epsilon}$$-equilibrium in two-person zero-sum games , 2012, Math. Program..

[34]  Nicolas Gillis,et al.  Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization , 2011, Neural Computation.

[35]  Peter Neary,et al.  International trade with endogenous mode of competition in general equilibrium , 2012 .

[36]  P. Pestieau,et al.  Childbearing Age, Family Allowances and Social Security , 2013 .

[37]  Pierre Pestieau,et al.  Measuring poverty without the Mortality Paradox , 2012, Social Choice and Welfare.

[38]  Michael Poss,et al.  Benders Decomposition for the Hop-Constrained Survivable Network Design Problem , 2013, INFORMS J. Comput..

[39]  Pau Olivella,et al.  Multidimensional Screening in a Monopolistic Insurance Market , 2011, The Geneva Risk and Insurance Review.

[40]  Peter Richtárik,et al.  Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function , 2011, Mathematical Programming.