Online robust principal component analysis via truncated nuclear norm regularization

Robust principal component analysis (RPCA) has been widely used to deal with high dimensional noisy data in many applications. Traditional RPCA approaches consider all the samples to recover the low dimensional subspace in a batch manner, which incur expensive storage cost and fail to update the low dimensional subspace efficiently for stream data. Thus it is urgent to develop online RPCA methods. In this paper, we propose a novel online RPCA algorithm by adopting a recently proposed truncated nuclear norm as a tighter approximation of low rank constraint. Then we decompose the objective function as a summation of sample-wise cost. And we design an efficient alternating optimization algorithm in an online manner. Experimental results show that our proposed method can achieve more accurate low dimensional subspace estimation performance compared with state-of-the-art online RPCA algorithms.

[1]  John C. S. Lui,et al.  Online Robust Subspace Tracking from Partial Information , 2011, ArXiv.

[2]  Hong Cheng,et al.  Robust Principal Component Analysis with Missing Data , 2014, CIKM.

[3]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[4]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[5]  John Wright,et al.  Decomposing background topics from keywords by principal component pursuit , 2010, CIKM.

[6]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[7]  Paris Smaragdis,et al.  Singing-voice separation from monaural recordings using robust principal component analysis , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Xuelong Li,et al.  Matrix completion by Truncated Nuclear Norm Regularization , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[10]  Xuelong Li,et al.  Fast and Accurate Matrix Completion via Truncated Nuclear Norm Regularization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[12]  Yao Hu,et al.  Large scale multi-class classification with truncated nuclear norm regularization , 2015, Neurocomputing.

[13]  Dacheng Tao,et al.  GoDec: Randomized Lowrank & Sparse Matrix Decomposition in Noisy Case , 2011, ICML.

[14]  G. James Blaine,et al.  Continuous Monitoring of Physiologic Variables with a Dedicated Minicomputer , 1975, Computer.

[15]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[16]  Yao Hu,et al.  Salient Object Detection via Fast Iterative Truncated Nuclear Norm Recovery , 2013, IScIDE.

[17]  Gilad Lerman,et al.  A novel M-estimator for robust PCA , 2011, J. Mach. Learn. Res..

[18]  Ping Li,et al.  Online optimization for max-norm regularization , 2014, Machine Learning.

[19]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[20]  Jieping Ye,et al.  Robust principal component analysis via capped norms , 2013, KDD.

[21]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[22]  Gilad Lerman,et al.  Robust Stochastic Principal Component Analysis , 2014, AISTATS.

[23]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.

[24]  Xiaoming Yuan,et al.  Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations , 2011, SIAM J. Optim..

[25]  Tommi S. Jaakkola,et al.  Maximum-Margin Matrix Factorization , 2004, NIPS.

[26]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[27]  John Wright,et al.  RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images , 2012, IEEE Trans. Pattern Anal. Mach. Intell..