Robust PCA With Partial Subspace Knowledge

In recent work, robust Principal Components Analysis (PCA) has been posed as a problem of recovering a low-rank matrix L and a sparse matrix S from their sum, M: = L + S and a provably exact convex optimization solution called PCP has been proposed. This work studies the following problem. Suppose that we have partial knowledge about the column space of the low rank matrix L. Can we use this information to improve the PCP solution, i.e., allow recovery under weaker assumptions? We propose here a simple but useful modification of the PCP idea, called modified-PCP, that allows us to use this knowledge. We derive its correctness result which shows that, when the available subspace knowledge is accurate, modified-PCP indeed requires significantly weaker incoherence assumptions than PCP. Extensive simulations are also used to illustrate this. Comparisons with PCP and other existing work are shown for a stylized real application as well. Finally, we explain how this problem naturally occurs in many applications involving time series data, i.e., in what is called the online or recursive robust PCA problem. A corollary for this case is also given.

[1]  Namrata Vaswani,et al.  Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise , 2012, IEEE Transactions on Information Theory.

[2]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[3]  Gou Hosoya,et al.  国際会議参加報告:2014 IEEE International Symposium on Information Theory , 2014 .

[4]  Ali Jalali,et al.  Low-Rank Matrix Recovery From Errors and Erasures , 2013, IEEE Transactions on Information Theory.

[5]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[6]  John Wright,et al.  Compressive principal component pursuit , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[7]  John Wright,et al.  Principal Component Pursuit with reduced linear measurements , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[8]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Namrata Vaswani,et al.  An Online Algorithm for Separating Sparse and Low-Dimensional Signal Sequences From Their Sum , 2013, IEEE Transactions on Signal Processing.

[10]  Martin Kleinsteuber,et al.  pROST : A Smoothed Lp-norm Robust Online Subspace Tracking Method for Realtime Background Subtraction in Video , 2013, ArXiv.

[11]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[12]  Guillermo Sapiro,et al.  Online Learning for Matrix Factorization and Sparse Coding , 2009, J. Mach. Learn. Res..

[13]  L. Lovász Combinatorial problems and exercises , 1979 .

[14]  Martin Kleinsteuber,et al.  Robust PCA and subspace tracking from incomplete observations using $$\ell _0$$ℓ0-surrogates , 2012, Comput. Stat..

[15]  John Wright,et al.  Dense Error Correction Via $\ell^1$-Minimization , 2010, IEEE Transactions on Information Theory.

[16]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[17]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[18]  J. Hiriart-Urruty,et al.  Fundamentals of Convex Analysis , 2004 .

[19]  Sham M. Kakade,et al.  Robust Matrix Decomposition With Sparse Corruptions , 2011, IEEE Transactions on Information Theory.

[20]  Linear Operators and Adjoints , 2022 .

[21]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..

[22]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[23]  Namrata Vaswani,et al.  Recursive sparse recovery in large but correlated noise , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[24]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[25]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[26]  Morteza Mardani,et al.  Dynamic Anomalography: Tracking Network Anomalies Via Sparsity and Low Rank , 2012, IEEE Journal of Selected Topics in Signal Processing.

[27]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[28]  J. Tropp,et al.  Two proposals for robust PCA using semidefinite programming , 2010, 1012.1086.

[29]  Robert D. Nowak,et al.  Online identification and tracking of subspaces from highly incomplete information , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[30]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[31]  János Komlós,et al.  The eigenvalues of random symmetric matrices , 1981, Comb..

[32]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[33]  Martin J. Wainwright,et al.  Noisy matrix decomposition via convex relaxation: Optimal rates in high dimensions , 2011, ICML.

[34]  Stephen Becker,et al.  Quantum state tomography via compressed sensing. , 2009, Physical review letters.

[35]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[36]  Adrian Lewis,et al.  The mathematics of eigenvalue optimization , 2003, Math. Program..

[37]  Shie Mannor,et al.  Online PCA for Contaminated Data , 2013, NIPS.

[38]  Dimitris Achlioptas,et al.  Fast computation of low rank matrix approximations , 2001, STOC '01.

[39]  G. Watson Characterization of the subdifferential of some matrix norms , 1992 .

[40]  Sham M. Kakade,et al.  Robust Matrix Decomposition with Outliers , 2010, ArXiv.

[41]  Namrata Vaswani,et al.  A correctness result for online robust PCA , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[43]  Wei Lu,et al.  Modified-CS: Modifying compressive sensing for problems with partially known support , 2009, 2009 IEEE International Symposium on Information Theory.

[44]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[45]  Motaz El-Saban,et al.  FRPCA: Fast Robust Principal Component Analysis for online observations , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[46]  A. Robert Calderbank,et al.  PETRELS: Parallel Subspace Estimation and Tracking by Recursive Least Squares From Partial Observations , 2012, IEEE Transactions on Signal Processing.

[47]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[48]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[49]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[51]  Xiaodong Li,et al.  Dense error correction for low-rank matrices via Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[52]  John Wright,et al.  Dense Error Correction via L1-Minimization , 2008, 0809.0199.

[53]  Namrata Vaswani,et al.  Modified-CS: Modifying compressive sensing for problems with partially known support , 2009, ISIT.

[54]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[55]  Namrata Vaswani,et al.  Real-time Robust Principal Components' Pursuit , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[56]  Gilad Lerman,et al.  A novel M-estimator for robust PCA , 2011, J. Mach. Learn. Res..

[57]  Namrata Vaswani,et al.  Robust PCA With Partial Subspace Knowledge , 2015, IEEE Trans. Signal Process..

[58]  Ronald Fagin,et al.  Proceedings of the thirty-seventh annual ACM symposium on Theory of computing , 2005, STOC 2005.

[59]  James M. Rehg,et al.  GOSUS: Grassmannian Online Subspace Updates with Structured-Sparsity , 2013, 2013 IEEE International Conference on Computer Vision.

[60]  Van H. Vu,et al.  Spectral norm of random matrices , 2005, STOC '05.

[61]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..