Online Robust Subspace Tracking from Partial Information

This paper presents GRASTA (Grassmannian Robust Adaptive Subspace Tracking Algorithm), an ecient and robust online algorithm for tracking subspaces from highly incomplete information. The algorithm uses a robust l 1 -norm cost function in order to estimate and track non-stationary subspaces when the streaming data vectors are corrupted with outliers. We apply GRASTA to the problems of robust matrix completion and real-time separation of background from foreground in video. In this second application, we show that GRASTA performs highquality separation of moving objects from background at exceptional speeds: In one popular benchmark video example [28], GRASTA achieves a rate of 57 frames per second, even when run in MATLAB on a personal laptop.

[1]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[2]  Qi Tian,et al.  Statistical modeling of complex backgrounds for foreground object detection , 2004, IEEE Transactions on Image Processing.

[3]  Joos Vandewalle,et al.  A Singular Value Decomposition Updating Algorithm for Subspace Tracking , 1992, SIAM J. Matrix Anal. Appl..

[4]  Jar-Ferr Yang,et al.  Adaptive eigensubspace algorithms for direction or frequency estimation and tracking , 1988, IEEE Trans. Acoust. Speech Signal Process..

[5]  Christian H. Bischof,et al.  On updating signal subspaces , 1992, IEEE Trans. Signal Process..

[6]  Robert D. Nowak,et al.  High-dimensional Matched Subspace Detection when data are missing , 2010, 2010 IEEE International Symposium on Information Theory.

[7]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..

[8]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[9]  Yoram Bresler,et al.  Efficient and guaranteed rank minimization by atomic decomposition , 2009, 2009 IEEE International Symposium on Information Theory.

[10]  Max A. Viergever,et al.  Adaptive Stochastic Gradient Descent Optimisation for Image Registration , 2009, International Journal of Computer Vision.

[11]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[12]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[13]  R. O. Schmidt,et al.  Multiple emitter location and signal Parameter estimation , 1986 .

[14]  Andrea Montanari,et al.  Matrix Completion from Noisy Entries , 2009, J. Mach. Learn. Res..

[15]  M. Brand,et al.  Fast low-rank modifications of the thin singular value decomposition , 2006 .

[16]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[17]  A. Edelman,et al.  On conjugate gradient-like methods for eigen-like problems , 1996 .

[18]  Jean Pierre Delmas,et al.  Performance analysis of an adaptive algorithm for tracking dominant subspaces , 1998, IEEE Trans. Signal Process..

[19]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[20]  V. Rich.  Personal communication , 1989, Nature.

[21]  Soura Dasgupta,et al.  Adaptive estimation of eigensubspace , 1995, IEEE Trans. Signal Process..

[22]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[23]  G. Giannakis,et al.  Sparsity control for robust principal component analysis , 2010, 2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers.

[24]  G. W. Stewart,et al.  An updating algorithm for subspace tracking , 1992, IEEE Trans. Signal Process..

[25]  Steven Thomas Smith,et al.  Geometric Optimization Methods for Adaptive Filtering , 2013, ArXiv.

[26]  B. Ripley,et al.  Robust Statistics , 2018, Wiley Series in Probability and Statistics.

[27]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[28]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[29]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[30]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[31]  Olgica Milenkovic,et al.  Subspace Evolution and Transfer (SET) for Low-Rank Matrix Completion , 2010, IEEE Transactions on Signal Processing.

[32]  G. Golub,et al.  Tracking a few extreme singular values and vectors in signal processing , 1990, Proc. IEEE.

[33]  Mark Crovella,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM '04.

[34]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[35]  A. Plakhov,et al.  A Stochastic Approximation Algorithm with Step-Size Adaptation , 2004 .

[36]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[37]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[38]  Eric Moulines,et al.  Subspace methods for the blind identification of multichannel FIR filters , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[39]  Robert D. Nowak,et al.  Online identification and tracking of subspaces from highly incomplete information , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[40]  M. Moonen,et al.  AN SVD UPDATING ALGORITHM FOR SUBSPACE TRACKING , 1992 .

[41]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[42]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..