An augmented Lagrangian based parallel splitting method for separable convex minimization with applications to image processing

This paper considers the convex minimization problem with linear constraints and a separable objective function which is the sum of many individual functions without coupled variables. An algorithm is developed by splitting the augmented Lagrangian function in a parallel way. The new algorithm differs substantially from existing splitting methods in alternating style which require solving the decomposed subproblems sequentially, while it remains the main superiority of existing splitting methods in that the resulting subproblems could be simple enough to have closed-form solutions for such an application whose functions in the objective are simple. We show applicability and encouraging efficiency of the new algorithm by some applications in image processing.

[1]  Richard A. Haddad,et al.  Adaptive median filters: new algorithms and results , 1995, IEEE Trans. Image Process..

[2]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[3]  Shiqian Ma,et al.  An alternating direction method for total variation denoising , 2011, Optim. Methods Softw..

[4]  Wotao Yin,et al.  Analysis and Generalizations of the Linearized Bregman Method , 2010, SIAM J. Imaging Sci..

[5]  Mila Nikolova,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004, Journal of Mathematical Imaging and Vision.

[6]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[7]  Xiaoming Yuan,et al.  An alternating direction-based contraction method for linearly constrained separable convex programming problems , 2013 .

[8]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[9]  Shiqian Ma,et al.  Fast Multiple-Splitting Algorithms for Convex Optimization , 2009, SIAM J. Optim..

[10]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[11]  Stephen Gould,et al.  Projected Subgradient Methods for Learning Sparse Gaussians , 2008, UAI.

[12]  Shiqian Ma,et al.  Fast alternating linearization methods for minimizing the sum of two convex functions , 2009, Math. Program..

[13]  Bingsheng He,et al.  Parallel splitting augmented Lagrangian methods for monotone structured variational inequalities , 2009, Comput. Optim. Appl..

[14]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[15]  Xiaoming Yuan,et al.  A Note on the Alternating Direction Method of Multipliers , 2012, J. Optim. Theory Appl..

[16]  William K. Pratt,et al.  Digital Image Processing: PIKS Inside , 2001 .

[17]  Nirmal K. Bose,et al.  High‐resolution image reconstruction with multisensors , 1998, Int. J. Imaging Syst. Technol..

[18]  Jian-Feng Cai,et al.  Two-phase approach for deblurring images corrupted by impulse plus gaussian noise , 2008 .

[19]  Patrick L. Combettes,et al.  A Parallel Splitting Method for Coupled Monotone Inclusions , 2009, SIAM J. Control. Optim..

[20]  John Wright,et al.  RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Stanley Osher,et al.  A Unified Primal-Dual Algorithm Framework Based on Bregman Iteration , 2010, J. Sci. Comput..

[22]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[23]  Bin Dong,et al.  Fast Linearized Bregman Iteration for Compressive Sensing and Sparse Denoising , 2011, ArXiv.

[24]  Jian-Feng Cai,et al.  Linearized Bregman iterations for compressed sensing , 2009, Math. Comput..

[25]  Shiqian Ma,et al.  Accelerated Linearized Bregman Method , 2011, J. Sci. Comput..

[26]  James G. Nagy,et al.  Structured linear algebra problems in adaptive optics imaging , 2011, Adv. Comput. Math..

[27]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[28]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[29]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[30]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[31]  Bingsheng He,et al.  Linearized Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming , 2011 .

[32]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[33]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[34]  Gabriele Steidl,et al.  Deblurring Poissonian images by split Bregman techniques , 2010, J. Vis. Commun. Image Represent..

[35]  Rachid Deriche,et al.  Regularizing Flows for Constrained Matrix-Valued Images , 2004 .

[36]  Michael K. Ng,et al.  Fast Image Restoration Methods for Impulse and Gaussian Noises Removal , 2009, IEEE Signal Processing Letters.

[37]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[38]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[39]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[40]  Xavier Bresson,et al.  Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction , 2010, SIAM J. Imaging Sci..

[41]  Gilles Aubert,et al.  Efficient Schemes for Total Variation Minimization Under Constraints in Image Processing , 2009, SIAM J. Sci. Comput..

[42]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[43]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[44]  Benar Fux Svaiter,et al.  General Projective Splitting Methods for Sums of Maximal Monotone Operators , 2009, SIAM J. Control. Optim..

[45]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[46]  Jian-Feng Cai,et al.  Linearized Bregman Iterations for Frame-Based Image Deblurring , 2009, SIAM J. Imaging Sci..

[47]  Tony F. Chan,et al.  Aspects of Total Variation Regularized L[sup 1] Function Approximation , 2005, SIAM J. Appl. Math..

[48]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[49]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[50]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[51]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[52]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[53]  Xiaoming Yuan,et al.  New Parallel Descent-like Method for Solving a Class of Variational Inequalities , 2010 .

[54]  Bingsheng He,et al.  Proximal-Point Algorithm Using a Linear Proximal Term , 2009 .

[55]  J. Spingarn.  Partial inverse of a monotone operator , 1983 .