Adaptive inexact fast augmented Lagrangian methods for constrained convex optimization

In this paper we study two inexact fast augmented Lagrangian algorithms for solving linearly constrained convex optimization problems. Our methods rely on a combination of the excessive-gap-like smoothing technique introduced in Nesterov (SIAM J Optim 16(1):235–249, 2005) and the general inexact oracle framework studied in Devolder (Math Program 146:37–75, 2014). We develop and analyze two augmented based algorithmic instances with constant and adaptive smoothness parameters, and derive a total computational complexity estimate in terms of projections on a simple primal feasible set for each algorithm. For the constant parameter algorithm we obtain the overall computational complexity of order $$\mathcal {O}(\frac{1}{\epsilon ^{5/4}})$$O(1ϵ5/4), while for the adaptive one we obtain $$\mathcal {O}(\frac{1}{\epsilon })$$O(1ϵ) total number of projections onto the primal feasible set in order to achieve an $$\epsilon $$ϵ-optimal solution for the original problem.

[1]  Bingsheng He,et al.  A modified augmented Lagrangian method for a class of monotone variational inequalities , 2004, Eur. J. Oper. Res..

[2]  I. Necoara,et al.  Fast inexact decomposition algorithms for large-scale separable convex optimization , 2012, 1212.4275.

[3]  Bingsheng He,et al.  On the O(1/n) Convergence Rate of the Douglas-Rachford Alternating Direction Method , 2012, SIAM J. Numer. Anal..

[4]  Dimitri P. Bertsekas,et al.  Convex Optimization Theory , 2009 .

[5]  Patrick L. Combettes,et al.  A Monotone+Skew Splitting Model for Composite Monotone Inclusions in Duality , 2010, SIAM J. Optim..

[6]  Yurii Nesterov,et al.  Primal-dual subgradient methods for convex problems , 2005, Math. Program..

[7]  Yurii Nesterov,et al.  Subgradient methods for huge-scale optimization problems , 2013, Mathematical Programming.

[8]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[9]  Arkadi Nemirovski,et al.  Lectures on modern convex optimization - analysis, algorithms, and engineering applications , 2001, MPS-SIAM series on optimization.

[10]  Renato D. C. Monteiro,et al.  Iteration-complexity of first-order augmented Lagrangian methods for convex programming , 2015, Mathematical Programming.

[11]  Ion Necoara,et al.  Rate Analysis of Inexact Dual First-Order Methods Application to Dual Decomposition , 2014, IEEE Transactions on Automatic Control.

[12]  Necdet Serhat Aybat,et al.  A First-Order Augmented Lagrangian Method for Compressed Sensing , 2010, SIAM J. Optim..

[13]  N. S. Aybat,et al.  An Augmented Lagrangian Method for Conic Convex Programming , 2013, 1302.6322.

[14]  Ion Necoara,et al.  Complexity of first-order inexact Lagrangian and penalty methods for conic convex programming , 2015, Optim. Methods Softw..

[15]  Ion Necoara,et al.  Complexity certifications of first order inexact Lagrangian and penalty methods for conic convex programming , 2015 .

[16]  Ion Necoara,et al.  Iteration complexity analysis of dual first-order methods for conic convex programming , 2014, Optim. Methods Softw..

[17]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[18]  A. Ozdaglar,et al.  Optimal Distributed Gradient Methods for Network Resource Allocation Problems , 2013 .

[19]  Bingsheng He,et al.  Linearized Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming , 2011 .

[20]  Asuman E. Ozdaglar,et al.  Approximate Primal Solutions and Rate Analysis for Dual Subgradient Methods , 2008, SIAM J. Optim..

[21]  Arkadi Nemirovski,et al.  Prox-Method with Rate of Convergence O(1/t) for Variational Inequalities with Lipschitz Continuous Monotone Operators and Smooth Convex-Concave Saddle Point Problems , 2004, SIAM J. Optim..

[22]  Yurii Nesterov,et al.  First-order methods of smooth convex optimization with inexact oracle , 2013, Mathematical Programming.

[23]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[24]  Yurii Nesterov,et al.  Excessive Gap Technique in Nonsmooth Convex Minimization , 2005, SIAM J. Optim..

[25]  Johan A. K. Suykens,et al.  Application of a Smoothing Technique to Decomposition in Convex Optimization , 2008, IEEE Transactions on Automatic Control.

[26]  Xiaoming Yuan,et al.  A Proximal Strictly Contractive Peaceman-Rachford Splitting Method for Convex Programming with Applications to Imaging , 2015, SIAM J. Imaging Sci..

[27]  Dinh Quoc Tran,et al.  Combining Lagrangian decomposition and excessive gap smoothing technique for solving large-scale separable convex optimization problems , 2011, Comput. Optim. Appl..

[28]  Ion Necoara,et al.  Computational Complexity of Inexact Gradient Augmented Lagrangian Methods: Application to Constrained MPC , 2013, SIAM J. Control. Optim..

[29]  V. Cevher,et al.  A Primal-Dual Algorithmic Framework for Constrained Convex Minimization , 2014, 1406.5403.

[30]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[31]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[32]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[33]  I. Necoara,et al.  DuQuad: an inexact (augmented) dual first order algorithm for quadratic programming , 2015, 1504.05708.