Accelerated Alternating Projections for Robust Principal Component Analysis

We study robust PCA for the fully observed setting, which is about separating a low rank matrix $\boldsymbol{L}$ and a sparse matrix $\boldsymbol{S}$ from their sum $\boldsymbol{D}=\boldsymbol{L}+\boldsymbol{S}$. In this paper, a new algorithm, dubbed accelerated alternating projections, is introduced for robust PCA which significantly improves the computational efficiency of the existing alternating projections proposed in [Netrapalli, Praneeth, et al., 2014] when updating the low rank factor. The acceleration is achieved by first projecting a matrix onto some low dimensional subspace before obtaining a new estimate of the low rank matrix via truncated SVD. Exact recovery guarantee has been established which shows linear convergence of the proposed algorithm. Empirical performance evaluations establish the advantage of our algorithm over other state-of-the-art algorithms for robust PCA.

[1]  Paris Smaragdis,et al.  Singing-voice separation from monaural recordings using robust principal component analysis , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[3]  Colin N. Jones,et al.  Complexity Certification of the Fast Alternating Minimization Algorithm for Linear MPC , 2017, IEEE Transactions on Automatic Control.

[4]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[5]  Yousef Saad,et al.  Scaled Gradients on Grassmann Manifolds for Matrix Completion , 2012, NIPS.

[6]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[7]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[8]  Constantine Caramanis,et al.  Fast Algorithms for Robust PCA via Gradient Descent , 2016, NIPS.

[9]  Tony F. Chan,et al.  Guarantees of Riemannian Optimization for Low Rank Matrix Recovery , 2015, SIAM J. Matrix Anal. Appl..

[10]  Bamdev Mishra,et al.  A Riemannian geometry for low-rank matrix completion , 2012, ArXiv.

[11]  Hossein Mobahi,et al.  Holistic 3D reconstruction of urban structures from low-rank textures , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[12]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[13]  Robert W. Heath,et al.  Interference alignment via alternating minimization , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Gilad Lerman,et al.  Robust Locally Linear Analysis with Applications to Image Denoising and Blind Inpainting , 2013, SIAM J. Imaging Sci..

[15]  Bamdev Mishra,et al.  R3MC: A Riemannian three-factor algorithm for low-rank matrix completion , 2013, 53rd IEEE Conference on Decision and Control.

[16]  Prateek Jain,et al.  Learning Sparsely Used Overcomplete Dictionaries , 2014, COLT.

[17]  Yang Wang,et al.  Fast Rank-One Alternating Minimization Algorithm for Phase Retrieval , 2017, Journal of Scientific Computing.

[18]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[19]  Raghunandan H. Keshavan.  Efficient algorithms for collaborative filtering , 2012 .

[20]  Zhaoran Wang,et al.  Low-Rank and Sparse Structure Pursuit via Alternating Minimization , 2016, AISTATS.

[21]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[22]  Moritz Hardt,et al.  On the Provable Convergence of Alternating Minimization for Matrix Completion , 2013, ArXiv.

[23]  Prateek Jain,et al.  Phase Retrieval Using Alternating Minimization , 2013, IEEE Transactions on Signal Processing.

[24]  T. Chan,et al.  Guarantees of riemannian optimization for low rank matrix completion , 2016, Inverse Problems & Imaging.

[25]  Joseph A. O'Sullivan,et al.  Alternating Minimization Algorithms for Transmission Tomography , 2007, IEEE Transactions on Medical Imaging.

[26]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[27]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[28]  Martin J. Wainwright,et al.  Fast low-rank estimation by projected gradient descent: General statistical and algorithmic guarantees , 2015, ArXiv.

[29]  Qi Tian,et al.  Statistical modeling of complex backgrounds for foreground object detection , 2004, IEEE Transactions on Image Processing.

[30]  Sujay Sanghavi,et al.  Clustering Sparse Graphs , 2012, NIPS.

[31]  Bamdev Mishra,et al.  Fixed-rank matrix factorizations and Riemannian low-rank optimization , 2012, Comput. Stat..

[32]  T. Chan,et al.  Convergence of the alternating minimization algorithm for blind deconvolution , 2000 .

[33]  J. Tanner,et al.  Low rank matrix completion by alternating steepest descent methods , 2016 .

[34]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[35]  Bart Vandereycken,et al.  Low-Rank Matrix Completion by Riemannian Optimization , 2013, SIAM J. Optim..

[36]  Moritz Hardt,et al.  Understanding Alternating Minimization for Matrix Completion , 2013, 2014 IEEE 55th Annual Symposium on Foundations of Computer Science.

[37]  Khaled Ben Letaief,et al.  Alternating Minimization Algorithms for Hybrid Precoding in Millimeter Wave MIMO Systems , 2016, IEEE Journal of Selected Topics in Signal Processing.

[38]  Didier Maquin,et al.  Fault Detection and Isolation with Robust Principal Component Analysis , 2008, 2008 16th Mediterranean Conference on Control and Automation.

[39]  Teng Zhang,et al.  Phase Retrieval Using Alternating Minimization in a Batch Setting , 2017, 2018 Information Theory and Applications Workshop (ITA).