Compressive Online Robust Principal Component Analysis via $n$ - $\ell_1$ Minimization

This paper considers online robust principal component analysis (RPCA) in time-varying decomposition problems such as video foreground-background separation. We propose a compressive online RPCA algorithm that decomposes recursively a sequence of data vectors (e.g., frames) into sparse and low-rank components. Different from conventional batch RPCA, which processes all the data directly, our approach considers a small set of measurements taken per data vector (frame). Moreover, our algorithm can incorporate multiple prior information from previous decomposed vectors via proposing an <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> minimization method. At each time instance, the algorithm recovers the sparse vector by solving the <inline-formula> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula>-<inline-formula> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> minimization problem—which promotes not only the sparsity of the vector but also its correlation with multiple previously recovered sparse vectors—and, subsequently, updates the low-rank component using incremental singular value decomposition. We also establish theoretical bounds on the number of measurements required to guarantee successful compressive separation under the assumptions of static or slowly changing low-rank components. We evaluate the proposed algorithm using numerical experiments and online video foreground-background separation experiments. The experimental results show that the proposed method outperforms the existing methods.

[1]  Yongmin Li,et al.  On incremental and robust subspace learning , 2004, Pattern Recognit..

[2]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[3]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[4]  James M. Rehg,et al.  GOSUS: Grassmannian Online Subspace Updates with Structured-Sparsity , 2013, 2013 IEEE International Conference on Computer Vision.

[5]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[6]  Lawrence Carin,et al.  Bayesian Robust Principal Component Analysis , 2011, IEEE Transactions on Image Processing.

[7]  Namrata Vaswani,et al.  An Online Algorithm for Separating Sparse and Low-Dimensional Signal Sequences From Their Sum , 2013, IEEE Transactions on Signal Processing.

[8]  Namrata Vaswani,et al.  Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise , 2012, IEEE Transactions on Information Theory.

[9]  Volkan Cevher,et al.  Dynamic sparse state estimation using ℓ1-ℓ1 minimization: Adaptive-rate measurement bounds, algorithms and applications , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[11]  André Kaup,et al.  Incorporating Prior Information in Compressive Online Robust Principal Component Analysis , 2017, ArXiv.

[12]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[13]  Namrata Vaswani,et al.  Real-time Robust Principal Components' Pursuit , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[14]  Gonzalo Mateos,et al.  Robust PCA as Bilinear Decomposition With Outlier-Sparsity Regularization , 2011, IEEE Transactions on Signal Processing.

[15]  Brendt Wohlberg,et al.  Incremental Principal Component Pursuit for Video Background Modeling , 2015, Journal of Mathematical Imaging and Vision.

[16]  Andrzej Cichocki,et al.  Total Variation Regularized Tensor RPCA for Background Subtraction From Compressive Measurements , 2015, IEEE Transactions on Image Processing.

[17]  Gonzalo Mateos,et al.  Modeling and Optimization for Big Data Analytics: (Statistical) learning tools for our era of data deluge , 2014, IEEE Signal Processing Magazine.

[18]  André Kaup,et al.  A Reconstruction Algorithm with Multiple Side Information for Distributed Compression of Sparse Sources , 2016, 2016 Data Compression Conference (DCC).

[19]  Yoram Bresler,et al.  ADMiRA: Atomic Decomposition for Minimum Rank Approximation , 2009, IEEE Transactions on Information Theory.

[20]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Peter Schelkens,et al.  On the side-information dependency of the temporal correlation in Wyner-Ziv video coding , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[23]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[24]  Matthew Brand,et al.  Incremental Singular Value Decomposition of Uncertain Data with Missing Values , 2002, ECCV.

[25]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[26]  André Kaup,et al.  Compressive online robust principal component analysis with multiple prior information , 2017, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[27]  Namrata Vaswani,et al.  Recursive Recovery of Sparse Signal Sequences From Compressive Measurements: A Review , 2016, IEEE Transactions on Signal Processing.

[28]  Laura Balzano,et al.  Incremental gradient on the Grassmannian for online foreground and background separation in subsampled video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Miguel R. D. Rodrigues,et al.  Compressed sensing with side information: Geometrical interpretation and performance bounds , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[30]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[31]  Volkan Cevher,et al.  Compressive Sensing for Background Subtraction , 2008, ECCV.

[32]  Lei Zhang,et al.  Robust Online Matrix Factorization for Dynamic Background Subtraction , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Brendt Wohlberg,et al.  Fast principal component pursuit via alternating minimization , 2013, 2013 IEEE International Conference on Image Processing.

[34]  John Wright,et al.  Compressive principal component pursuit , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[35]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[36]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[37]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[38]  André Kaup,et al.  Sparse signal reconstruction with multiple side information using adaptive weights for multiview sources , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[39]  오승준 [서평]「Digital Video Processing」 , 1996 .

[40]  Yonina C. Eldar,et al.  Compressed sensing for longitudinal MRI: An adaptive-weighted approach. , 2014, Medical physics.

[41]  Hassan Mansour,et al.  A robust online subspace estimation and tracking algorithm , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Thomas S. Huang,et al.  Base selection in estimating sparse foreground in video , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[43]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[44]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[45]  Aswin C. Sankaranarayanan,et al.  SpaRCS: Recovering low-rank and sparse matrices from compressive measurements , 2011, NIPS.

[46]  Namrata Vaswani,et al.  Stability (over time) of modified-CS for recursive causal sparse reconstruction , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[47]  Ling Chen,et al.  Online compressed robust PCA , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[48]  Qi Tian,et al.  Statistical modeling of complex backgrounds for foreground object detection , 2004, IEEE Transactions on Image Processing.

[49]  Volkan Cevher,et al.  Adaptive-Rate Reconstruction of Time-Varying Signals With Application in Compressive Foreground Extraction , 2016, IEEE Transactions on Signal Processing.

[50]  Deyu Meng,et al.  Robust Low-Rank Matrix Factorization Under General Mixture Noise Distributions , 2016, IEEE Transactions on Image Processing.

[51]  Wei Lu,et al.  Modified-CS: Modifying compressive sensing for problems with partially known support , 2009, 2009 IEEE International Symposium on Information Theory.

[52]  André Kaup,et al.  Compressive Online Robust Principal Component Analysis with Optical Flow for Video Foreground-Background Separation , 2017, SoICT.

[53]  Miguel R. D. Rodrigues,et al.  Compressed Sensing With Prior Information: Strategies, Geometry, and Bounds , 2017, IEEE Transactions on Information Theory.

[54]  Rama Chellappa,et al.  Adaptive-Rate Compressive Sensing Using Side Information , 2014, IEEE Transactions on Image Processing.