An ADM-based splitting method for separable convex programming

We consider the convex minimization problem with linear constraints and a block-separable objective function which is represented as the sum of three functions without coupled variables. To solve this model, it is empirically effective to extend straightforwardly the alternating direction method of multipliers (ADM for short). But, the convergence of this straightforward extension of ADM is still not proved theoretically. Based on ADM’s straightforward extension, this paper presents a new splitting method for the model under consideration, which is empirically competitive to the straightforward extension of ADM and meanwhile the global convergence can be proved under standard assumptions. At each iteration, the new method corrects the output of the straightforward extension of ADM by some slight correction computation to generate a new iterate. Thus, the implementation of the new method is almost as easy as that of ADM’s straightforward extension. We show the numerical efficiency of the new method by some applications in the areas of image processing and statistics.

[1]  Robert R. Meyer,et al.  A variable-penalty alternating directions method for convex optimization , 1998, Math. Program..

[2]  Richard A. Haddad,et al.  Adaptive median filters: new algorithms and results , 1995, IEEE Trans. Image Process..

[3]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[4]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[5]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[6]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[7]  Wotao Yin,et al.  Alternating direction augmented Lagrangian methods for semidefinite programming , 2010, Math. Program. Comput..

[8]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[9]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[10]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[11]  Bingsheng He,et al.  Solving Large-Scale Least Squares Semidefinite Programming by Alternating Direction Methods , 2011, SIAM J. Matrix Anal. Appl..

[12]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[13]  Xiaoming Yuan,et al.  Alternating Direction Method for Covariance Selection Models , 2011, Journal of Scientific Computing.

[14]  Xiaoming Yuan,et al.  Matrix completion via an alternating direction method , 2012 .

[15]  Michael K. Ng,et al.  Fast Image Restoration Methods for Impulse and Gaussian Noises Removal , 2009, IEEE Signal Processing Letters.

[16]  Victor Vianu,et al.  Invited articles section foreword , 2010, JACM.

[17]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[18]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[19]  Bingsheng He,et al.  Linearized Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming , 2011 .

[20]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[21]  Xiaoming Yuan,et al.  A splitting method for separable convex programming , 2015 .

[22]  Jian-Feng Cai,et al.  Two-phase approach for deblurring images corrupted by impulse plus gaussian noise , 2008 .

[23]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[24]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[25]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[26]  Donald Goldfarb,et al.  A Line Search Multigrid Method for Large-Scale Nonlinear Optimization , 2009, SIAM J. Optim..

[27]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[28]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra, and Filtering (Fundamentals of Algorithms 3) (Fundamentals of Algorithms) , 2006 .

[29]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[30]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[31]  Michael K. Ng,et al.  Inexact Alternating Direction Methods for Image Recovery , 2011, SIAM J. Sci. Comput..

[32]  Masao Fukushima,et al.  Application of the alternating direction method of multipliers to separable convex programming problems , 1992, Comput. Optim. Appl..

[33]  John Wright,et al.  RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Bingsheng He,et al.  Parallel splitting augmented Lagrangian methods for monotone structured variational inequalities , 2009, Comput. Optim. Appl..

[35]  Su Zhang,et al.  A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs , 2010, Eur. J. Oper. Res..

[36]  Xiaoming Yuan,et al.  Solving Large-Scale Least Squares Semidefinite Programming by Alternating Direction Methods , 2011 .

[37]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra and Filtering , 2006, J. Electronic Imaging.