Symmetric Gauss–Seidel Technique-Based Alternating Direction Methods of Multipliers for Transform Invariant Low-Rank Textures Problem

Transform invariant low-rank textures, referred to as TILT, can accurately and robustly extract textural or geometric information in a 3D from user-specified windows in 2D in spite of significant corruptions and warping. It was discovered that the task can be characterized, both theoretically and numerically, by solving a sequence of matrix nuclear-norm and $$\ell _1$$ℓ1-norm involved convex minimization problems. For solving this problem, the direct extension of alternating direction method of multipliers (ADMM) in an usual Gauss–Seidel manner often performs numerically well in practice but there is no theoretical guarantee on its convergence. In this paper, we resolve this dilemma by using the novel symmetric Gauss–Seidel (sGS)-based ADMM developed by Li et al. (Math Program 155:333–373, 2016). The sGS-ADMM is guaranteed to converge, and we shall demonstrate in this paper that it is also practically efficient than the directly extended ADMM. When the sGS technique is applied to this particular problem, we show that only one variable needs to be re-updated, and this updating hardly imposes any excessive computational cost. The sGS decomposition theorem of Li et al. (Math. Program., 2018. https://doi.org/10.1007/s10107-018-1247-7) establishes the equivalent between sGS-ADMM and the classical ADMM with an additional semi-proximal term, so the convergence result is followed directly. Extensive experiments illustrate that the sGS-ADMM and its generalized variant have superior numerical efficiency over the directly extended ADMM.

[1]  Xiang Ren,et al.  Linearized Alternating Direction Method with Adaptive Penalty and Warm Starts for Fast Solving Transform Invariant Low-Rank Textures , 2012, International Journal of Computer Vision.

[2]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[3]  John Wright,et al.  RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Kim-Chuan Toh,et al.  A Schur complement based semi-proximal ADMM for convex quadratic conic programming and extensions , 2014, Mathematical Programming.

[5]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[6]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[7]  R. Rockafellar Convex Analysis: (pms-28) , 1970 .

[8]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[9]  Jean-Michel Morel,et al.  ASIFT: A New Framework for Fully Affine Invariant Image Comparison , 2009, SIAM J. Imaging Sci..

[10]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[11]  Kim-Chuan Toh,et al.  A Convergent 3-Block SemiProximal Alternating Direction Method of Multipliers for Conic Programming with 4-Type Constraints , 2014, SIAM J. Optim..

[12]  Xiaoming Yuan,et al.  Alternating algorithms for total variation image reconstruction from random projections , 2012 .

[13]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[14]  Yi Ma,et al.  TILT: Transform Invariant Low-Rank Textures , 2010, ACCV.

[15]  Victor Vianu,et al.  Invited articles section foreword , 2010, JACM.

[16]  Liang Chen,et al.  A generalized alternating direction method of multipliers with semi-proximal terms for convex composite conic programming , 2015, Mathematical Programming Computation.

[17]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[18]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[19]  Kim-Chuan Toh,et al.  A block symmetric Gauss–Seidel decomposition theorem for convex composite quadratic programming and its applications , 2017, Mathematical Programming.

[20]  Kim-Chuan Toh,et al.  An efficient inexact symmetric Gauss–Seidel based majorized ADMM for high-dimensional convex composite conic programming , 2015, Mathematical Programming.

[21]  Yi Ma,et al.  Unwrapping low-rank textures on generalized cylindrical surfaces , 2011, 2011 International Conference on Computer Vision.

[22]  Yun-Hai Xiao,et al.  An Inexact Alternating Directions Algorithm for Constrained Total Variation Regularized Compressive Sensing Problems , 2011, Journal of Mathematical Imaging and Vision.

[23]  Defeng Sun,et al.  A rank-corrected procedure for matrix completion with fixed basis coefficients , 2012, Math. Program..

[24]  Kim-Chuan Toh,et al.  QSDPNAL: a two-phase augmented Lagrangian method for convex quadratic semidefinite programming , 2015, Mathematical Programming Computation.

[25]  Kim-Chuan Toh,et al.  Fast Algorithms for Large-Scale Generalized Distance Weighted Discrimination , 2016, 1604.05473.

[26]  Bingsheng He,et al.  The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent , 2014, Mathematical Programming.

[27]  Jonathan Eckstein Some Saddle-function splitting methods for convex programming , 1994 .

[28]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .