A correctness result for online robust PCA

We study the problem of sequentially recovering a sparse vector x<sub>t</sub> and a vector from a low-dimensional subspace ℓ<sub>t</sub> from knowledge of their sum m<sub>t</sub> = x<sub>t</sub> + ℓ<sub>t</sub>. If the primary goal is to recover the low-dimensional subspace where the ℓ<sub>t</sub>'s lie, then the problem is one of online or recursive robust principal components analysis (PCA). To the best of our knowledge, this is the first correctness result for this problem. We prove that if a good estimate of the initial subspace is available; the ℓ<sub>t</sub>'s obey certain denseness and slow subspace change assumptions; and the support of x<sub>t</sub> changes either at every frame or at least every so often, then with high probability, the support of x<sub>t</sub> will be recovered exactly, and the error made in estimating x<sub>t</sub> and ℓ<sub>t</sub> will be small. An example where this problem occurs is in separating a sparse foreground and a slowly changing dense background from surveillance videos.

[1]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[2]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[3]  B. Nadler.  Finite sample approximation results for principal component analysis: a matrix perturbation approach , 2009, 0901.3245.

[4]  John Wright,et al.  Dense Error Correction Via $\ell^1$-Minimization , 2010, IEEE Transactions on Information Theory.

[5]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[6]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[7]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[8]  Gonzalo Mateos,et al.  Robust PCA as Bilinear Decomposition With Outlier-Sparsity Regularization , 2011, IEEE Transactions on Signal Processing.

[9]  Namrata Vaswani,et al.  Robust PCA With Partial Subspace Knowledge , 2014, IEEE Transactions on Signal Processing.

[10]  Joel A. Tropp,et al.  Sharp recovery bounds for convex deconvolution, with applications , 2012, ArXiv.

[11]  J. Tropp,et al.  Two proposals for robust PCA using semidefinite programming , 2010, 1012.1086.

[12]  Namrata Vaswani,et al.  Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise , 2012, IEEE Transactions on Information Theory.

[13]  Ali Jalali,et al.  Low-Rank Matrix Recovery From Errors and Erasures , 2013, IEEE Transactions on Information Theory.

[14]  Morteza Mardani,et al.  Dynamic Anomalography: Tracking Network Anomalies Via Sparsity and Low Rank , 2012, IEEE Journal of Selected Topics in Signal Processing.

[15]  Namrata Vaswani,et al.  An Online Algorithm for Separating Sparse and Low-Dimensional Signal Sequences From Their Sum , 2013, IEEE Transactions on Signal Processing.

[16]  Gilad Lerman,et al.  A novel M-estimator for robust PCA , 2011, J. Mach. Learn. Res..

[17]  Namrata Vaswani,et al.  Performance guarantees for ReProCS - Correlated low-rank matrix entries case , 2014, 2014 IEEE International Symposium on Information Theory.

[18]  Namrata Vaswani,et al.  Performance guarantees for undersampled recursive sparse recovery in large but structured noise , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[19]  MaYi,et al.  Dense error correction via l1-minimization , 2010 .

[20]  Joel A. Tropp,et al.  Sharp Recovery Bounds for Convex Demixing, with Applications , 2012, Found. Comput. Math..

[21]  John C. S. Lui,et al.  Online Robust Subspace Tracking from Partial Information , 2011, ArXiv.

[22]  Shie Mannor,et al.  Online PCA for Contaminated Data , 2013, NIPS.

[23]  Marco Muselli,et al.  On convergence properties of pocket algorithm , 1997, IEEE Trans. Neural Networks.

[24]  Sham M. Kakade,et al.  Robust Matrix Decomposition with Outliers , 2010, ArXiv.

[25]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[26]  Joel A. Tropp,et al.  User-Friendly Tail Bounds for Sums of Random Matrices , 2010, Found. Comput. Math..

[27]  Michael J. Black,et al.  A Framework for Robust Subspace Learning , 2003, International Journal of Computer Vision.

[28]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[29]  Sham M. Kakade,et al.  Robust Matrix Decomposition With Sparse Corruptions , 2011, IEEE Transactions on Information Theory.

[30]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[31]  Emmanuel J. Cand.  The Restricted Isometry Property and Its Implications for Compressed Sensing , 2008 .

[32]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.