Fast inexact decomposition algorithms for large-scale separable convex optimization

In this paper, we propose a new inexact dual decomposition algorithm for solving separable convex optimization problems. This algorithm is a combination of three techniques: dual Lagrangian decomposition, smoothing and excessive gap. The algorithm has low computational complexity since it consists in only one primal step and two dual steps at each iteration and allows one to solve the subproblem of each component inexactly and in parallel. Moreover, the algorithmic parameters are updated automatically without any tuning strategy as it happens in augmented Lagrangian approaches. We analyse the convergence of the algorithm and estimate its analytical worst-case complexity for both the primal–dual suboptimality and the primal feasibility violation, where is a given accuracy. Extensive numerical tests confirm that our method is numerically more efficient than the classical decomposition methods from the literature.

[1]  Dinh Quoc Tran,et al.  Combining Lagrangian decomposition and excessive gap smoothing technique for solving large-scale separable convex optimization problems , 2011, Comput. Optim. Appl..

[2]  Jacob Ponstein Convexity and Duality in Optimization , 1985 .

[3]  Nimrod Megiddo,et al.  Horizontal and vertical decomposition in interior point methods for linear programs , 1994 .

[4]  Ion Necoara,et al.  Rate Analysis of Inexact Dual First-Order Methods Application to Dual Decomposition , 2014, IEEE Transactions on Automatic Control.

[5]  Jonathan E. Spingarn,et al.  Applications of the method of partial inverses to convex programming: Decomposition , 1985, Math. Program..

[6]  Philippe Mahey,et al.  Accelerating convergence of a Separable Augmented Lagrangian Algorithm , 2007 .

[7]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[8]  K. Holmberg,et al.  Experiments with primal - dual decomposition and subgradient methods for the uncapacitatied facility location problem , 2001 .

[9]  N. Komodakis,et al.  MRF Energy Minimization & Beyond via Dual Decomposition , 2010 .

[10]  R. Rockafellar Convex Analysis: (pms-28) , 1970 .

[11]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[12]  Abdelouahed Hamdi,et al.  Two-level primal-dual proximal decomposition technique to solve large scale optimization problems , 2005, Appl. Math. Comput..

[13]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[14]  Nikos Komodakis,et al.  MRF Energy Minimization and Beyond via Dual Decomposition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[16]  Masao Fukushima,et al.  Application of the alternating direction method of multipliers to separable convex programming problems , 1992, Comput. Optim. Appl..

[17]  Yurii Nesterov,et al.  Correlation between Two Projected Matrices Under Isometry Constraints , 2005 .

[18]  Shih-Ping Han A parallel algorithm for a class of convex programs , 1988 .

[19]  Gongyun Zhao,et al.  A Lagrangian Dual Method with Self-Concordant Barriers for Multi-Stage Stochastic Convex Programming , 2005, Math. Program..

[20]  P. L. Combettes,et al.  A proximal decomposition method for solving convex variational inverse problems , 2008, 0807.2617.

[21]  Paul Tseng,et al.  Alternating Projection-Proximal Methods for Convex Programming and Variational Inequalities , 1997, SIAM J. Optim..

[22]  Yurii Nesterov,et al.  Excessive Gap Technique in Nonsmooth Convex Minimization , 2005, SIAM J. Optim..

[23]  Johan A. K. Suykens,et al.  Application of a Smoothing Technique to Decomposition in Convex Optimization , 2008, IEEE Transactions on Automatic Control.

[24]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[25]  J. Suykens,et al.  An Interior-Point Lagrangian Decomposition Method for Separable Convex Optimization , 2013, 1302.3136.

[26]  Sanjay Mehrotra,et al.  Decomposition Based Interior Point Methods for Two-Stage Stochastic Convex Quadratic Programs with Recourse , 2009, Oper. Res..

[27]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[28]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[29]  M. Moonen,et al.  Improved Dual Decomposition Based Optimization for DSL Dynamic Spectrum Management , 2010, IEEE Transactions on Signal Processing.

[30]  Andrzej Ruszczynski,et al.  On Convergence of an Augmented Lagrangian Decomposition Method for Sparse Convex Optimization , 1995, Math. Oper. Res..

[31]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[32]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[33]  Antonio J. Conejo,et al.  Decomposition Techniques in Mathematical Programming: Engineering and Science Applications , 2006 .

[34]  Martin J. Wainwright,et al.  Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling , 2010, IEEE Transactions on Automatic Control.

[35]  R. Tyrrell Rockafellar Monotropic Programming: A Generalization of Linear Programming and Network Programming , 1985 .

[36]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[37]  John S. Baras,et al.  An Optimal Distributed Routing Algorithm using Dual Decomposition Techniques , 2008, Commun. Inf. Syst..

[38]  Kaj Holmberg,et al.  Mean value cross decomposition for nonlinear convex problems , 2006, Optim. Methods Softw..

[39]  Stephen P. Boyd,et al.  Distributed estimation via dual decomposition , 2007, 2007 European Control Conference (ECC).

[40]  Shiqian Ma,et al.  Fast Multiple-Splitting Algorithms for Convex Optimization , 2009, SIAM J. Optim..

[41]  Stephen J. Wright,et al.  Distributed MPC Strategies With Application to Power System Automatic Generation Control , 2008, IEEE Transactions on Control Systems Technology.

[42]  S. Kontogiorgis,et al.  Alternating direction splitting for block Angular parallel optimization , 1996 .

[43]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[44]  Dinh Quoc Tran,et al.  An Inexact Perturbed Path-Following Method for Lagrangian Decomposition in Large-Scale Separable Convex Optimization , 2011, SIAM J. Optim..