Provable Dynamic Robust PCA or Robust Subspace Tracking

Dynamic robust PCA refers to the dynamic (time-varying) extension of the robust PCA (RPCA) problem. It assumes that the true (uncorrupted) data lies in a low-dimensional subspace that can change with time, albeit slowly. The goal is to track this changing subspace over time in the presence of sparse outliers. This work provides the first guarantee for dynamic RPCA that holds under weakened standard RPCA assumptions, slow subspace change and two mild assumptions. We analyze a simple algorithm based on the Recursive Projected Compressive Sensing (ReProCS) framework. Our result is significant because (i) it removes the strong assumptions needed by the two previous complete guarantees for ReProCS-based algorithms; (ii) it shows that it is possible to achieve significantly improved outlier tolerance than all existing provable RPCA methods by exploiting slow subspace change and a lower bound on outlier magnitudes; and (iii) it proves that the proposed algorithm is online, fast, and memory-efficient.

[1]  Praneeth Narayanamurthy,et al.  Nearly Optimal Robust Subspace Tracking and Dynamic Robust PCA , 2017, ICML 2018.

[2]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[3]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[4]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Namrata Vaswani,et al.  Online matrix completion and online robust PCA , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[6]  Namrata Vaswani,et al.  A Fast and Memory-Efficient Algorithm for Robust PCA (MEROP) , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Sajid Javed,et al.  Robust Subspace Learning: Robust PCA, Robust Subspace Tracking, and Robust Subspace Recovery , 2017, IEEE Signal Processing Magazine.

[8]  Sajid Javed,et al.  Robust PCA, Subspace Learning, and Tracking , 2017 .

[9]  Namrata Vaswani,et al.  Static and Dynamic Robust PCA and Matrix Completion: A Review , 2018, Proceedings of the IEEE.

[10]  Namrata Vaswani,et al.  Correlated-PCA: Principal Components' Analysis when Data and Noise are Correlated , 2016, NIPS.

[11]  Sham M. Kakade,et al.  Robust Matrix Decomposition With Sparse Corruptions , 2011, IEEE Transactions on Information Theory.

[12]  Namrata Vaswani,et al.  Online (and Offline) Robust PCA: Novel Algorithms and Performance Guarantees , 2016, AISTATS.

[13]  Lin Xiao,et al.  A Proximal-Gradient Homotopy Method for the L1-Regularized Least-Squares Problem , 2012, ICML.

[14]  Robert D. Nowak,et al.  Online identification and tracking of subspaces from highly incomplete information , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[15]  Simon Haykin,et al.  Adaptive Signal Processing: Next Generation Solutions , 2010 .

[16]  Prateek Jain,et al.  Nearly Optimal Robust Matrix Completion , 2016, ICML.

[17]  Bin Yang,et al.  Asymptotic convergence analysis of the projection approximation subspace tracking algorithms , 1996, Signal Process..

[18]  Namrata Vaswani,et al.  NEARLY OPTIMAL ROBUST SUBSPACE TRACKING: A UNIFIED APPROACH , 2017, 2018 IEEE Data Science Workshop (DSW).

[19]  A. Robert Calderbank,et al.  PETRELS: Parallel Subspace Estimation and Tracking by Recursive Least Squares From Partial Observations , 2012, IEEE Transactions on Signal Processing.

[20]  Roman Vershynin,et al.  Introduction to the non-asymptotic analysis of random matrices , 2010, Compressed Sensing.

[21]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[22]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[23]  Namrata Vaswani,et al.  An Online Algorithm for Separating Sparse and Low-Dimensional Signal Sequences From Their Sum , 2013, IEEE Transactions on Signal Processing.

[24]  Namrata Vaswani,et al.  Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise , 2012, IEEE Transactions on Information Theory.

[25]  Stephen J. Wright,et al.  Local Convergence of an Algorithm for Subspace Identification from Partial Data , 2013, Found. Comput. Math..

[26]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[27]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[28]  Danijel Skocaj,et al.  Weighted and robust incremental method for subspace learning , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[29]  Jason Morphett,et al.  An integrated algorithm of incremental and robust PCA , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[30]  Constantine Caramanis,et al.  Fast Algorithms for Robust PCA via Gradient Descent , 2016, NIPS.

[31]  Namrata Vaswani,et al.  Finite sample guarantees for PCA in non-isotropic and data-dependent noise , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[32]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[33]  Namrata Vaswani,et al.  Robust PCA With Partial Subspace Knowledge , 2014, IEEE Transactions on Signal Processing.

[34]  Namrata Vaswani,et al.  Provable Dynamic Robust PCA or Robust Subspace Tracking , 2017, IEEE Transactions on Information Theory.

[35]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..