Provable Dynamic Robust PCA or Robust Subspace Tracking

Dynamic robust principal component analysis (PCA) refers to the dynamic (time-varying) extension of robust PCA (RPCA). It assumes that the true (uncorrupted) data lie in a low-dimensional subspace that can change with time, albeit slowly. The goal is to track this changing subspace over time in the presence of sparse outliers. We develop and study a novel algorithm, which we call simple-ReProCS, based on the recently introduced Recursive Projected Compressive Sensing (ReProCS) framework. This paper provides the first guarantee for dynamic RPCA that holds under weakened versions of standard RPCA assumptions, slow subspace change, and a lower bound assumption on most outlier magnitudes. Our result is significant because: 1) it removes the strong assumptions needed by the two previous complete guarantees for ReProCS-based algorithms; 2) it shows that it is possible to achieve significantly improved outlier tolerance, compared with all existing RPCA or dynamic RPCA solutions by exploiting the above-mentioned two simple extra assumptions; and 3) it proves that simple-ReProCS is online (after initialization), fast, and, has near-optimal memory complexity.

[1]  Prateek Jain,et al.  Nearly Optimal Robust Matrix Completion , 2016, ICML.

[2]  Bin Yang,et al.  Asymptotic convergence analysis of the projection approximation subspace tracking algorithms , 1996, Signal Process..

[3]  L. Ammann.  Robust Principal Components , 1989 .

[4]  Robert D. Nowak,et al.  Online identification and tracking of subspaces from highly incomplete information , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[5]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..

[6]  Sajid Javed,et al.  Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery , 2017, IEEE Signal Processing Magazine.

[7]  Danijel Skocaj,et al.  Weighted and robust incremental method for subspace learning , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[8]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[9]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Namrata Vaswani,et al.  An Online Algorithm for Separating Sparse and Low-Dimensional Signal Sequences From Their Sum , 2013, IEEE Transactions on Signal Processing.

[11]  Namrata Vaswani,et al.  Static and Dynamic Robust PCA and Matrix Completion: A Review , 2018, Proceedings of the IEEE.

[12]  Namrata Vaswani,et al.  Real-time Robust Principal Components' Pursuit , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[13]  Yang Shi,et al.  Streaming Robust PCA U . , 2016 .

[14]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[15]  Namrata Vaswani,et al.  Separating sparse and low-dimensional signal sequences from time-varying undersampled projections of their sums , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Namrata Vaswani,et al.  Robust PCA With Partial Subspace Knowledge , 2015, IEEE Trans. Signal Process..

[17]  Namrata Vaswani,et al.  Online (and Offline) Robust PCA: Novel Algorithms and Performance Guarantees , 2016, AISTATS.

[18]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[19]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[20]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[21]  Namrata Vaswani,et al.  Online matrix completion and online robust PCA , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[22]  Namrata Vaswani,et al.  A Fast and Memory-Efficient Algorithm for Robust PCA (MEROP) , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[23]  Namrata Vaswani,et al.  Correlated-PCA: Principal Components' Analysis when Data and Noise are Correlated , 2016, NIPS.

[24]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[25]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[26]  Namrata Vaswani,et al.  Recursive robust PCA or recursive sparse recovery in large but structured noise , 2013, ICASSP.

[27]  James M. Rehg,et al.  GOSUS: Grassmannian Online Subspace Updates with Structured-Sparsity , 2013, 2013 IEEE International Conference on Computer Vision.

[28]  Lin Xiao,et al.  A Proximal-Gradient Homotopy Method for the L1-Regularized Least-Squares Problem , 2012, ICML.

[29]  Stephen J. Wright,et al.  Local Convergence of an Algorithm for Subspace Identification from Partial Data , 2013, Found. Comput. Math..

[30]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[31]  A. Robert Calderbank,et al.  PETRELS: Parallel Subspace Estimation and Tracking by Recursive Least Squares From Partial Observations , 2012, IEEE Transactions on Signal Processing.

[32]  Jason Morphett,et al.  An integrated algorithm of incremental and robust PCA , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[33]  Constantine Caramanis,et al.  Fast Algorithms for Robust PCA via Gradient Descent , 2016, NIPS.

[34]  Namrata Vaswani,et al.  Finite sample guarantees for PCA in non-isotropic and data-dependent noise , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[35]  Sham M. Kakade,et al.  Robust Matrix Decomposition With Sparse Corruptions , 2011, IEEE Transactions on Information Theory.

[36]  Namrata Vaswani,et al.  NEARLY OPTIMAL ROBUST SUBSPACE TRACKING: A UNIFIED APPROACH , 2017, 2018 IEEE Data Science Workshop (DSW).

[37]  Namrata Vaswani,et al.  Provable Dynamic Robust PCA or Robust Subspace Tracking , 2017, 2018 IEEE International Symposium on Information Theory (ISIT).

[38]  Namrata Vaswani,et al.  Recursive sparse recovery in large but correlated noise , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[39]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.