Subspace Tracking from Missing and Outlier Corrupted Data

We study the related problems of subspace tracking in the presence of missing data (ST-miss) as well as robust subspace tracking with missing data (RST-miss). Here "robust" refers to robustness to sparse outliers. In recent work, we have studied the RST problem without missing data. In this work, we show that simple modifications of our solution approach for RST also provably solve ST-miss and RST-miss under weaker and similar assumptions respectively. To our knowledge, our result is the first complete guarantee for both ST-miss and RST-miss. This means we are able to show that, under assumptions on only the algorithm inputs (input data and/or initialization), the output subspace estimates are close to the true data subspaces at all times. Our guarantees hold under mild and easily interpretable assumptions and handle time-varying subspaces (unlike all previous work). We also show that our algorithm and its extensions are fast and have competitive experimental performance when compared with existing methods.

[1]  Sergios Theodoridis,et al.  Robust Subspace Tracking With Missing Entries: The Set-Theoretic Approach , 2015, IEEE Transactions on Signal Processing.

[2]  Robert D. Nowak,et al.  Online identification and tracking of subspaces from highly incomplete information , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[3]  Tengyu Ma,et al.  Matrix Completion has No Spurious Local Minimum , 2016, NIPS.

[4]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[5]  Akshay Krishnamurthy,et al.  Low-Rank Matrix and Tensor Completion via Adaptive Sampling , 2013, NIPS.

[6]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[7]  Selin Aviyente,et al.  Recursive Tensor Subspace Tracking for Dynamic Brain Network Analysis , 2017, IEEE Transactions on Signal and Information Processing over Networks.

[8]  Stephen J. Wright,et al.  Local Convergence of an Algorithm for Subspace Identification from Partial Data , 2013, Found. Comput. Math..

[9]  Ali Jalali,et al.  Low-Rank Matrix Recovery From Errors and Erasures , 2013, IEEE Transactions on Information Theory.

[10]  Namrata Vaswani,et al.  Time Invariant Error Bounds for Modified-CS-Based Sparse Signal Sequence Recovery , 2015, IEEE Trans. Inf. Theory.

[11]  Namrata Vaswani,et al.  Online (and Offline) Robust PCA: Novel Algorithms and Performance Guarantees , 2016, AISTATS.

[12]  Robert D. Nowak,et al.  High-dimensional Matched Subspace Detection when data are missing , 2010, 2010 IEEE International Symposium on Information Theory.

[13]  Yudong Chen,et al.  Coherent Matrix Completion , 2013, ICML.

[14]  Sham M. Kakade,et al.  Provable Efficient Online Matrix Completion via Non-convex Stochastic Gradient Descent , 2016, NIPS.

[15]  Namrata Vaswani,et al.  Modified-CS: Modifying compressive sensing for problems with partially known support , 2009, ISIT.

[16]  Namrata Vaswani,et al.  Real-time Robust Principal Components' Pursuit , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[17]  Namrata Vaswani,et al.  NEARLY OPTIMAL ROBUST SUBSPACE TRACKING: A UNIFIED APPROACH , 2017, 2018 IEEE Data Science Workshop (DSW).

[18]  Constantine Caramanis,et al.  Fast Algorithms for Robust PCA via Gradient Descent , 2016, NIPS.

[19]  Namrata Vaswani,et al.  Finite sample guarantees for PCA in non-isotropic and data-dependent noise , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[20]  E. Oja Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[21]  Hassan Mansour,et al.  A robust online subspace estimation and tracking algorithm , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[23]  Namrata Vaswani,et al.  PCA in Sparse Data-Dependent Noise , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[24]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[25]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Prateek Jain,et al.  Low-rank matrix completion using alternating minimization , 2012, STOC '13.

[27]  G. Golub,et al.  Tracking a few extreme singular values and vectors in signal processing , 1990, Proc. IEEE.

[28]  Dejiao Zhang,et al.  Global Convergence of a Grassmannian Gradient Descent Algorithm for Subspace Estimation , 2015, AISTATS.

[29]  Ioannis Mitliagkas Streaming PCA with Many Missing Entries , 2014 .

[30]  Namrata Vaswani,et al.  Provable Dynamic Robust PCA or Robust Subspace Tracking , 2017, 2018 IEEE International Symposium on Information Theory (ISIT).

[31]  Namrata Vaswani,et al.  Static and Dynamic Robust PCA and Matrix Completion: A Review , 2018, Proceedings of the IEEE.

[32]  Lek-Heng Lim,et al.  Schubert Varieties and Distances between Subspaces of Different Dimensions , 2014, SIAM J. Matrix Anal. Appl..

[33]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, ISIT.

[34]  A. Robert Calderbank,et al.  PETRELS: Parallel Subspace Estimation and Tracking by Recursive Least Squares From Partial Observations , 2012, IEEE Transactions on Signal Processing.

[35]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[36]  Piotr Indyk,et al.  Fast recovery from a union of subspaces , 2016, NIPS.

[37]  Prateek Jain,et al.  Fast Exact Matrix Completion with Finite Samples , 2014, COLT.

[38]  Mary Wootters,et al.  Fast matrix completion without the condition number , 2014, COLT.

[39]  Prateek Jain,et al.  Universal Matrix Completion , 2014, ICML.

[40]  Prateek Jain,et al.  Nearly Optimal Robust Matrix Completion , 2016, ICML.

[41]  Bin Yang,et al.  Asymptotic convergence analysis of the projection approximation subspace tracking algorithms , 1996, Signal Process..

[42]  Yonina C. Eldar,et al.  Subspace Estimation From Incomplete Observations: A High-Dimensional Analysis , 2018, IEEE Journal of Selected Topics in Signal Processing.

[43]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[44]  Yonina C. Eldar,et al.  Subspace Learning with Partial Information , 2014, J. Mach. Learn. Res..

[45]  Namrata Vaswani,et al.  Recursive robust PCA or recursive sparse recovery in large but structured noise , 2013, ICASSP.

[46]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[47]  Yuejie Chi,et al.  Streaming PCA and Subspace Tracking: The Missing Data Case , 2018, Proceedings of the IEEE.

[48]  Yi Zheng,et al.  No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis , 2017, ICML.

[49]  Dejiao Zhang,et al.  Enhanced online subspace estimation via adaptive sensing , 2017, 2017 51st Asilomar Conference on Signals, Systems, and Computers.

[50]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[51]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..