Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods

In this paper, we study methods for generating approximate primal solutions as a byproduct of subgradient methods applied to the Lagrangian dual of a primal convex (possibly nondifferentiable) constrained optimization problem. Our work is motivated by constrained primal problems with a favorable dual problem structure that leads to efficient implementation of dual subgradient methods, such as the recent resource allocation problems in large-scale networks. For such problems, we propose and analyze dual subgradient methods that use averaging schemes to generate approximate primal optimal solutions. These algorithms use a constant stepsize in view of its simplicity and practical significance. We provide estimates on the primal infeasibility and primal suboptimality of the generated approximate primal solutions. These estimates are given per iteration, thus providing a basis for analyzing the trade-offs between the desired level of error and the selection of the stepsize value. Our analysis relies on the Slater condition and the inherited boundedness properties of the dual problem under this condition. It also relies on the boundedness of subgradients, which is ensured by assuming the compactness of the constraint set.

[1]  Arkadi Nemirovski,et al.  Non-euclidean restricted memory level method for large-scale convex optimization , 2005, Math. Program..

[2]  Vivek S. Borkar,et al.  Distributed Asynchronous Incremental Subgradient Methods , 2001 .

[3]  N. G. Zhurbenko,et al.  Choice of fleet composition and allocation of aircraft to civil airline routes , 1976 .

[4]  K. Kiwiel,et al.  Parallel Subgradient Methods for Convex Optimization , 2001 .

[5]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[6]  Frank Kelly,et al.  Rate control for communication networks: shadow prices, proportional fairness and stability , 1998, J. Oper. Res. Soc..

[7]  Stephen P. Boyd,et al.  Processor Speed Control With Thermal Constraints , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  Samir Elhedhli,et al.  Nondifferentiable Optimization , 2009, Encyclopedia of Optimization.

[9]  Dimitri P. Bertsekas,et al.  Incremental Subgradient Methods for Nondifferentiable Optimization , 2001, SIAM J. Optim..

[10]  R. Srikant,et al.  Network Optimization and Control , 2008, Found. Trends Netw..

[11]  Francisco Barahona,et al.  The volume algorithm: producing primal solutions with a subgradient method , 2000, Math. Program..

[12]  Steven H. Low,et al.  Optimization flow control—I: basic algorithm and convergence , 1999, TNET.

[13]  V. F. Dem'yanov,et al.  Nondifferentiable Optimization , 1985 .

[14]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[15]  Hanif D. Sherali,et al.  A class of convergent primal-dual subgradient algorithms for decomposable convex programs , 1986, Math. Program..

[16]  A. Robert Calderbank,et al.  Layering as Optimization Decomposition: A Mathematical Theory of Network Architectures , 2007, Proceedings of the IEEE.

[17]  D. Bertsekas,et al.  Convergen e Rate of In remental Subgradient Algorithms , 2000 .

[18]  Derong Liu The Mathematics of Internet Congestion Control , 2005, IEEE Transactions on Automatic Control.

[19]  Andrzej Ruszczynski,et al.  A merit function approach to the subgradient method with averaging , 2008, Optim. Methods Softw..

[20]  T. Larsson,et al.  A Lagrangean Relaxation Scheme for Structured Linear Programs With Application To Multicommodity Network Flows , 1997 .

[21]  M. Patriksson,et al.  Ergodic results and bounds on the optimal value in subgradient optimization , 1995 .

[22]  Rayadurgam Srikant,et al.  The Mathematics of Internet Congestion Control , 2003 .

[23]  Hanif D. Sherali,et al.  Recovery of primal solutions when using subgradient optimization methods to solve Lagrangian duals of linear programs , 1996, Oper. Res. Lett..

[24]  Claude Lemaréchal,et al.  Convergence of some algorithms for convex minimization , 1993, Math. Program..

[25]  A. Banerjee Convex Analysis and Optimization , 2006 .

[26]  P. Luh,et al.  On the Surrogate Gradient Algorithm for Lagrangian Relaxation , 2007 .

[27]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[28]  Torbjörn Larsson,et al.  Lagrangian Relaxation via Ballstep Subgradient Methods , 2007, Math. Oper. Res..

[29]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[30]  M. Patriksson,et al.  Ergodic convergence in subgradient optimization , 1998 .

[31]  Michael Patriksson,et al.  Ergodic, primal convergence in dual subgradient schemes for convex programming , 1999, Mathematical programming.