On linear convergence of a distributed dual gradient algorithm for linearly constrained separable convex problems

In this paper we propose a fully distributed dual gradient algorithm for minimizing linearly constrained separable convex problems and analyze its rate of convergence. In particular, we prove that under the assumption of strong convexity and Lipschitz continuity of the gradient of the primal objective function we have a global error bound type property for the dual problem. Using this error bound property we devise a fully distributed dual gradient scheme, i.e. a gradient scheme based on a weighted step size, for which we derive global linear rate of convergence for both dual and primal suboptimality and for primal feasibility violation. Numerical simulations are also provided to confirm our theory.

[1]  J. Suykens,et al.  An Interior-Point Lagrangian Decomposition Method for Separable Convex Optimization , 2013, 1302.3136.

[2]  Ion Necoara,et al.  Iteration complexity analysis of dual first-order methods for conic convex programming , 2014, Optim. Methods Softw..

[3]  Paul Tseng,et al.  On the Convergence Rate of Dual Ascent Methods for Linearly Constrained Convex Minimization , 1993, Math. Oper. Res..

[4]  Hanif D. Sherali,et al.  A class of convergent primal-dual subgradient algorithms for decomposable convex programs , 1986, Math. Program..

[5]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[6]  Asuman E. Ozdaglar,et al.  On the O(1=k) convergence of asynchronous distributed alternating Direction Method of Multipliers , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[7]  Marc Teboulle,et al.  An $O(1/k)$ Gradient Method for Network Resource Allocation Problems , 2014, IEEE Transactions on Control of Network Systems.

[8]  Ion Necoara,et al.  Parallel and distributed optimization methods for estimation and control in networks , 2011, 1302.3103.

[9]  Jong-Shi Pang,et al.  A Posteriori Error Bounds for the Linearly-Constrained Variational Inequality Problem , 1987, Math. Oper. Res..

[10]  Marcello Farina,et al.  Distributed predictive control: A non-cooperative algorithm with neighbor-to-neighbor communication for linear systems , 2012, Autom..

[11]  Stephen J. Wright,et al.  Cooperative distributed model predictive control , 2010, Syst. Control. Lett..

[12]  Asuman E. Ozdaglar,et al.  Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods , 2008, SIAM J. Optim..

[13]  David Q. Mayne,et al.  Constrained model predictive control: Stability and optimality , 2000, Autom..

[14]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[15]  Z.-Q. Luo,et al.  Error bounds and convergence analysis of feasible descent methods: a general approach , 1993, Ann. Oper. Res..

[16]  Bart De Schutter,et al.  Accelerated gradient methods and dual decomposition in distributed model predictive control , 2013, Autom..

[17]  Lucas Barcelos de Oliveira,et al.  Distributed Optimization for Model Predictive Control of Linear-Dynamic Networks , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[18]  Chih-Jen Lin,et al.  Iteration complexity of feasible descent methods for convex optimization , 2014, J. Mach. Learn. Res..

[19]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[20]  A. Bakirtzis,et al.  A decentralized solution to the DC-OPF of interconnected power systems , 2003 .

[21]  Manfred Morari,et al.  Towards computational complexity certification for constrained MPC based on Lagrange Relaxation and the fast gradient method , 2011, IEEE Conference on Decision and Control and European Control Conference.

[22]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[23]  Ion Necoara,et al.  Computational Complexity of Inexact Gradient Augmented Lagrangian Methods: Application to Constrained MPC , 2013, SIAM J. Control. Optim..

[24]  Alberto Bemporad,et al.  An Accelerated Dual Gradient-Projection Algorithm for Embedded Linear Model Predictive Control , 2014, IEEE Transactions on Automatic Control.

[25]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[26]  A. Ozdaglar,et al.  Optimal Distributed Gradient Methods for Network Resource Allocation Problems , 2013 .

[27]  Dragos N. Clipici,et al.  Distributed coordinate descent methods for composite minimization , 2013 .

[28]  Rolf Findeisen,et al.  Fast predictive control of linear systems combining Nesterov's gradient method and the method of multipliers , 2011, IEEE Conference on Decision and Control and European Control Conference.

[29]  Asuman E. Ozdaglar,et al.  A distributed Newton method for Network Utility Maximization , 2010, 49th IEEE Conference on Decision and Control (CDC).

[30]  S. M. Robinson.  Bounds for error in the solution set of a perturbed linear program , 1973 .

[31]  Johan A. K. Suykens,et al.  Application of a Smoothing Technique to Decomposition in Convex Optimization , 2008, IEEE Transactions on Automatic Control.

[32]  Dragos N. Clipici,et al.  Parallel coordinate descent methods for composite minimization , 2013 .

[33]  Bart De Schutter,et al.  A distributed optimization-based approach for hierarchical model predictive control of large-scale systems with coupled dynamics and constraints , 2011, ArXiv.

[34]  Michael Ulbrich,et al.  A class of distributed optimization methods with event-triggered communication , 2013, Computational Optimization and Applications.

[35]  Ion Necoara,et al.  Rate Analysis of Inexact Dual First-Order Methods Application to Dual Decomposition , 2014, IEEE Transactions on Automatic Control.