An alternating direction-based contraction method for linearly constrained separable convex programming problems

The classical alternating direction method (ADM) has been well studied in the context of linearly constrained convex programming and variational inequalities where the involved operator is formed as the sum of two individual functions without crossed variables. Recently, ADM has found many novel applications in diversified areas, such as image processing and statistics. However, it is still not clear whether ADM can be extended to the case where the operator is the sum of more than two individual functions. In this article, we extend the spirit of ADM to solve the general case of the linearly constrained separable convex programming problems whose involved operator is separable into finitely many individual functions. As a result, an alternating direction-based contraction-type method is developed. The realization of tackling this class of problems broadens the applicable scope of ADM substantially.

[1]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[2]  Xiaoming Yuan,et al.  Alternating Direction Method for Covariance Selection Models , 2011, Journal of Scientific Computing.

[3]  Xiaoming Yuan,et al.  Matrix completion via an alternating direction method , 2012 .

[4]  Bingsheng He,et al.  Solving Large-Scale Least Squares Semidefinite Programming by Alternating Direction Methods , 2011, SIAM J. Matrix Anal. Appl..

[5]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[6]  Su Zhang,et al.  A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs , 2010, Eur. J. Oper. Res..

[7]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[8]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[9]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[10]  Xiaoming Yuan,et al.  New Parallel Descent-like Method for Solving a Class of Variational Inequalities , 2010 .

[11]  Simon Setzer,et al.  Split Bregman Algorithm, Douglas-Rachford Splitting and Frame Shrinkage , 2009, SSVM.

[12]  Bingsheng He,et al.  Parallel splitting augmented Lagrangian methods for monotone structured variational inequalities , 2009, Comput. Optim. Appl..

[13]  Ernie Esser,et al.  Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman , 2009 .

[14]  Xiaoming Yuan,et al.  A descent method for structured monotone variational inequalities , 2007, Optim. Methods Softw..

[15]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[16]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[17]  Bingsheng He,et al.  Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities , 1998, Oper. Res. Lett..

[18]  Robert R. Meyer,et al.  A variable-penalty alternating directions method for convex optimization , 1998, Math. Program..

[19]  Paul Tseng,et al.  Alternating Projection-Proximal Methods for Convex Programming and Variational Inequalities , 1997, SIAM J. Optim..

[20]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[21]  Jonathan Eckstein.  Some Saddle-function splitting methods for convex programming , 1994 .

[22]  Masao Fukushima,et al.  Some Reformulations and Applications of the Alternating Direction Method of Multipliers , 1994 .

[23]  W. Hager,et al.  Large Scale Optimization : State of the Art , 1993 .

[24]  Masao Fukushima,et al.  Application of the alternating direction method of multipliers to separable convex programming problems , 1992, Comput. Optim. Appl..

[25]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[26]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[27]  R. Glowinski,et al.  Numerical Methods for Nonlinear Variational Problems , 1985 .

[28]  D. Gabay.  Applications of the method of multipliers to variational inequalities , 1983 .

[29]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[30]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[31]  M. Powell.  A method for nonlinear constraints in minimization problems , 1969 .

[32]  H. H. Rachford,et al.  On the numerical solution of heat conduction problems in two and three space variables , 1956 .