An alternating minimization method for robust principal component analysis

ABSTRACT This paper focuses on the solution of robust principal component analysis (RPCA) problems that arise in fields such as information theory, statistics, and engineering. We adopt a model that minimizes the sum of the observation error and sparsity measurement subject to the rank constraint. To solve this problem, we propose a two-step alternating minimization method. In the first step, a symmetric low-rank product minimization, which is essentially a partial singular value decomposition, is efficiently solved with moderate accuracy. The second step then derives a closed-form solution. The proposed approach is almost parameter-free, and global convergence to a strict local minimizer is guaranteed under very loose conditions. We compare the proposed approach with some existing solvers, and numerical experiments demonstrate the outstanding performance of our approach in solving synthetic and real RPCA test problems. In particular, we illustrate the significant potential of the proposed approach to solve large-size problems with moderate accuracy.

[1]  Lars Elden,et al.  Matrix methods in data mining and pattern recognition , 2007, Fundamentals of algorithms.

[2]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[3]  A. Willsky,et al.  Sparse and low-rank matrix decompositions , 2009 .

[4]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[5]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[6]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation , 2010 .

[7]  Katya Scheinberg,et al.  Block Coordinate Descent Methods for Semidefinite Programming , 2012 .

[8]  Xiaodong Li,et al.  Dense error correction for low-rank matrices via Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[9]  Bingsheng He,et al.  The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent , 2014, Mathematical Programming.

[10]  Arvind Ganesh,et al.  Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix , 2009 .

[11]  John Wright,et al.  Scalable Robust Matrix Recovery: Frank-Wolfe Meets Proximal Methods , 2014, SIAM J. Sci. Comput..

[12]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[13]  Lieven Vandenberghe,et al.  Interior-Point Method for Nuclear Norm Approximation with Application to System Identification , 2009, SIAM J. Matrix Anal. Appl..

[14]  Yin Zhang,et al.  An Efficient Gauss-Newton Algorithm for Symmetric Low-Rank Product Matrix Approximations , 2015, SIAM J. Optim..

[16]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[17]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..

[18]  Kim-Chuan Toh,et al.  A Convergent 3-Block Semi-Proximal ADMM for Convex Minimization Problems with One Strongly Convex Block , 2014, Asia Pac. J. Oper. Res..

[19]  Stephen J. Wright Coordinate descent algorithms , 2015, Mathematical Programming.

[20]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[21]  Xiaoming Yuan,et al.  A Note on the Alternating Direction Method of Multipliers , 2012, J. Optim. Theory Appl..

[22]  Yurii Nesterov,et al.  Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems , 2012, SIAM J. Optim..

[23]  Takeo Kanade,et al.  A sequential factorization method for recovering shape and motion from image streams , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Yurii Nesterov,et al.  Lectures on Convex Optimization , 2018 .

[25]  Caihua Chen,et al.  On the Convergence Analysis of the Alternating Direction Method of Multipliers with Three Blocks , 2013 .

[26]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[27]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[28]  Iteration Bounds for Finding the $\epsilon$-Stationary Points for Structured Nonconvex Optimization , 2014, 1410.4066.

[29]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[30]  Ting Kei Pong,et al.  Peaceman–Rachford splitting for a class of nonconvex optimization problems , 2015, Comput. Optim. Appl..

[31]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[32]  Jieping Ye,et al.  Orthogonal Rank-One Matrix Pursuit for Low Rank Matrix Completion , 2014, SIAM J. Sci. Comput..

[33]  P. Tseng Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[34]  Y. Zhang,et al.  Augmented Lagrangian alternating direction method for matrix separation based on low-rank factorization , 2014, Optim. Methods Softw..

[35]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..

[36]  Wotao Yin,et al.  Global Convergence of ADMM in Nonconvex Nonsmooth Optimization , 2015, Journal of Scientific Computing.

[37]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[39]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[41]  Brendt Wohlberg,et al.  Fast principal component pursuit via alternating minimization , 2013, 2013 IEEE International Conference on Image Processing.

[42]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[43]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.