Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations

Many problems can be characterized by the task of recovering the low-rank and sparse components of a given matrix. Recently, it was discovered that this nondeterministic polynomial-time hard (NP-hard) task can be well accomplished, both theoretically and numerically, via heuristically solving a convex relaxation problem where the widely acknowledged nuclear norm and $l_1$ norm are utilized to induce low-rank and sparsity. This paper studies the recovery task in the general settings that only a fraction of entries of the matrix can be observed and the observation is corrupted by both impulsive and Gaussian noise. We show that the resulting model falls into the applicable scope of the classical augmented Lagrangian method. Moreover, the separable structure of the new model enables us to solve the involved subproblems more efficiently by splitting the augmented Lagrangian function. Hence, some splitting numerical algorithms are developed for solving the new recovery model. Some preliminary numerical experiments verify that these augmented-Lagrangian-based splitting algorithms are easily implementable and surprisingly efficient for tackling the new recovery model.

[1]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[2]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[3]  Arvind Ganesh,et al.  Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix , 2009 .

[4]  Junfeng Yang,et al.  ALTERNATING DIRECTION ALGORITHMS FOR TOTAL VARIATION DECONVOLUTION IN IMAGE RECONSTRUCTION , 2009 .

[5]  Leslie G. Valiant,et al.  Graph-Theoretic Arguments in Low-Level Complexity , 1977, MFCS.

[6]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[7]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[8]  R. Glowinski,et al.  Numerical Methods for Nonlinear Variational Problems , 1985 .

[9]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..

[10]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[11]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[12]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[13]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[14]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[15]  Michael I. Jordan Graphical Models , 2003 .

[16]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[17]  Heinz H. Bauschke,et al.  A Weak-to-Strong Convergence Principle for Fejé-Monotone Methods in Hilbert Spaces , 2001, Math. Oper. Res..

[18]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[19]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .

[20]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[21]  W. Hager,et al.  Large Scale Optimization : State of the Art , 1993 .

[22]  D. Gabay Applications of the method of multipliers to variational inequalities , 1983 .

[23]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, 2009 IEEE International Symposium on Information Theory.

[24]  Stephen P. Boyd,et al.  Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices , 2003, Proceedings of the 2003 American Control Conference, 2003..

[25]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[26]  Bingsheng He,et al.  A new inexact alternating directions method for monotone variational inequalities , 2002, Math. Program..

[27]  Paul Tseng,et al.  Alternating Projection-Proximal Methods for Convex Programming and Variational Inequalities , 1997, SIAM J. Optim..

[28]  Bingsheng He,et al.  Parallel splitting augmented Lagrangian methods for monotone structured variational inequalities , 2009, Comput. Optim. Appl..

[29]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[30]  Jonathan Eckstein Some Saddle-function splitting methods for convex programming , 1994 .

[31]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[32]  Junfeng Yang,et al.  A Fast Algorithm for Edge-Preserving Variational Multichannel Image Restoration , 2009, SIAM J. Imaging Sci..

[33]  Su Zhang,et al.  A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs , 2010, Eur. J. Oper. Res..

[34]  Masao Fukushima,et al.  Application of the alternating direction method of multipliers to separable convex programming problems , 1992, Comput. Optim. Appl..

[35]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[36]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[37]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[38]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[39]  Bingsheng He,et al.  Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities , 1998, Oper. Res. Lett..

[40]  Thomas Kailath,et al.  Phase-shifting masks for microlithography: automated design and mask requirements , 1994 .

[41]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[42]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[43]  R. Larsen Lanczos Bidiagonalization With Partial Reorthogonalization , 1998 .

[44]  N. Higham COMPUTING A NEAREST SYMMETRIC POSITIVE SEMIDEFINITE MATRIX , 1988 .

[45]  Xiaoming Yuan,et al.  Alternating Direction Methods for Sparse Covariance Selection * , 2009 .

[46]  Robert R. Meyer,et al.  A variable-penalty alternating directions method for convex optimization , 1998, Math. Program..

[47]  Xiaoming Yuan,et al.  A descent method for structured monotone variational inequalities , 2007, Optim. Methods Softw..

[48]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[49]  Gregory Piatetsky-Shapiro,et al.  High-Dimensional Data Analysis: The Curses and Blessings of Dimensionality , 2000 .

[50]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .