RAP: Scalable RPCA for Low-rank Matrix Recovery

Recovering low-rank matrices is a problem common in many applications of data mining and machine learning, such as matrix completion and image denoising. Robust Principal Component Analysis (RPCA) has emerged for handling such kinds of problems; however, the existing RPCA approaches are usually computationally expensive, due to the fact that they need to obtain the singular value decomposition (SVD) of large matrices. In this paper, we propose a novel RPCA approach that eliminates the need for SVD of large matrices. Scalable algorithms are designed for several variants of our approach, which are crucial for real world applications on large scale data. Extensive experimental results confirm the effectiveness of our approach both quantitatively and visually.

[1]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[2]  I. Jolliffe.  Principal Component Analysis , 2002 .

[3]  Jieping Ye,et al.  Efficient Sparse Group Feature Selection via Nonconvex Optimization , 2012, ICML.

[4]  N. S. Aybat,et al.  Fast First-Order Methods for Stable Principal Component Pursuit , 2011, 1105.2126.

[5]  C. Croux,et al.  Principal Component Analysis Based on Robust Estimators of the Covariance or Correlation Matrix: Influence Functions and Efficiencies , 2000 .

[6]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[7]  Ming Yang,et al.  Feature Selection Embedded Subspace Clustering , 2016, IEEE Signal Processing Letters.

[8]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[9]  Lawrence Carin,et al.  Bayesian Robust Principal Component Analysis , 2011, IEEE Transactions on Image Processing.

[10]  Takeo Kanade,et al.  Robust L/sub 1/ norm factorization in the presence of outliers and missing data by alternative convex programming , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[12]  Prateek Jain,et al.  Provable non-convex robust PCA , 2014, NIPS 2014.

[13]  Zhao Kang,et al.  Robust PCA Via Nonconvex Rank Approximation , 2015, 2015 IEEE International Conference on Data Mining.

[14]  Zhaoran Wang,et al.  OPTIMAL COMPUTATIONAL AND STATISTICAL RATES OF CONVERGENCE FOR SPARSE NONCONVEX LEARNING PROBLEMS. , 2013, Annals of statistics.

[15]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[16]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[17]  Zhao Kang,et al.  Subspace Clustering Using Log-determinant Rank Approximation , 2015, KDD.

[18]  Zhao Kang,et al.  Robust Subspace Clustering via Tighter Rank Approximation , 2015, CIKM.

[19]  Constantine Caramanis,et al.  Robust PCA via Outlier Pursuit , 2010, IEEE Transactions on Information Theory.

[20]  Cun-Hui Zhang.  Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[21]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..