Learned Robust PCA: A Scalable Deep Unfolding Approach for High-Dimensional Outlier Detection

Robust principal component analysis (RPCA) is a critical tool in modern machine learning, which detects outliers in the task of low-rank matrix reconstruction. In this paper, we propose a scalable and learnable non-convex approach for highdimensional RPCA problems, which we call Learned Robust PCA (LRPCA). LRPCA is highly efficient, and its free parameters can be effectively learned to optimize via deep unfolding. Moreover, we extend deep unfolding from finite iterations to infinite iterations via a novel feedforward-recurrent-mixed neural network model. We establish the recovery guarantee of LRPCA under mild assumptions for RPCA. Numerical experiments show that LRPCA outperforms the state-of-the-art RPCA algorithms, such as ScaledGD and AltProj, on both synthetic datasets and real-world applications.

[1]  Yong Xu,et al.  RPCA-Based Tumor Classification Using Gene Expression Data , 2015, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[2]  Rudrasis Chakraborty,et al.  Intrinsic Grassmann Averages for Online Linear, Robust and Nonlinear Subspace Learning , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Constantine Caramanis,et al.  Fast Algorithms for Robust PCA via Gradient Descent , 2016, NIPS.

[4]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[5]  Prateek Jain,et al.  Nearly Optimal Robust Matrix Completion , 2016, ICML.

[6]  Shiyu Chang,et al.  Training Stronger Baselines for Learning to Optimize , 2020, NeurIPS.

[7]  Jonas Adler,et al.  Learned Primal-Dual Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[8]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[9]  Deanna Needell,et al.  Fast Robust Tensor Principal Component Analysis via Fiber CUR Decomposition * , 2021, 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW).

[10]  Yonina C. Eldar,et al.  Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing , 2021, IEEE Signal Processing Magazine.

[11]  Richard G. Baraniuk,et al.  Learned D-AMP: Principled Neural Network based Compressive Image Recovery , 2017, NIPS.

[12]  Yuejie Chi,et al.  Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent , 2020, J. Mach. Learn. Res..

[13]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[15]  Sujay Sanghavi,et al.  Clustering Sparse Graphs , 2012, NIPS.

[16]  Keaton Hamm,et al.  Rapid Robust Principal Component Analysis: CUR Accelerated Inexact Low Rank Estimation , 2020, IEEE Signal Processing Letters.

[17]  Wotao Yin,et al.  Learning A Minimax Optimizer: A Pilot Study , 2021, ICLR.

[18]  Yonina C. Eldar,et al.  Deep Unfolded Robust PCA with Application to Clutter Suppression in Ultrasound , 2018, bioRxiv.

[19]  John Wright,et al.  RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Bernard Ghanem,et al.  ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Wen Gao,et al.  Maximal Sparsity with Deep Networks? , 2016, NIPS.

[22]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[23]  Xiaohan Chen,et al.  ALISTA: Analytic Weights Are As Good As Learned Weights in LISTA , 2018, ICLR.

[24]  Didier Maquin,et al.  Fault Detection and Isolation with Robust Principal Component Analysis , 2008, 2008 16th Mediterranean Conference on Control and Automation.

[25]  Yann LeCun,et al.  Learning Fast Approximations of Sparse Coding , 2010, ICML.

[26]  Jeremy Nixon,et al.  Understanding and correcting pathologies in the training of learned optimizers , 2018, ICML.

[27]  Sham M. Kakade,et al.  Robust Matrix Decomposition With Sparse Corruptions , 2011, IEEE Transactions on Information Theory.

[28]  Martin Vetterli,et al.  Euclidean Distance Matrices: Essential theory, algorithms, and applications , 2015, IEEE Signal Processing Magazine.

[29]  Xiaohan Chen,et al.  Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholds , 2018, NeurIPS.

[30]  J. Teichmann,et al.  Denise: Deep Learning based Robust PCA for Positive Semidefinite Matrices , 2020, ArXiv.

[31]  Søren Hauberg,et al.  Scalable Robust Principal Component Analysis Using Grassmann Averages , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[33]  Jian-Feng Cai,et al.  Generalized Low-rank plus Sparse Tensor Estimation by Fast Riemannian Optimization , 2021 .

[34]  Deanna Needell,et al.  Robust CUR Decomposition: Theory and Imaging Applications , 2021, SIAM J. Imaging Sci..

[35]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[36]  Chang-Su Kim,et al.  Primary Object Segmentation in Videos via Alternate Convex Optimization of Foreground and Background Distributions , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Misha Denil,et al.  Learned Optimizers that Scale and Generalize , 2017, ICML.

[38]  Jian-Feng Cai,et al.  Accelerated Structured Alternating Projections for Robust Spectrally Sparse Signal Recovery , 2021, IEEE Transactions on Signal Processing.

[39]  Zhaoran Wang,et al.  Low-Rank and Sparse Structure Pursuit via Alternating Minimization , 2016, AISTATS.

[40]  Marcin Andrychowicz,et al.  Learning to learn by gradient descent by gradient descent , 2016, NIPS.

[41]  Yonina C. Eldar,et al.  Deep Convolutional Robust PCA with Application to Ultrasound Imaging , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Ali Jalali,et al.  Low-Rank Matrix Recovery From Errors and Erasures , 2013, IEEE Transactions on Information Theory.

[43]  Jian Sun,et al.  Deep ADMM-Net for Compressive Sensing MRI , 2016, NIPS.

[44]  Jian-Feng Cai,et al.  Accelerated Alternating Projections for Robust Principal Component Analysis , 2017, J. Mach. Learn. Res..

[45]  Yonina C. Eldar,et al.  A Deep-Unfolded Reference-Based RPCA Network For Video Foreground-Background Separation , 2020, 2020 28th European Signal Processing Conference (EUSIPCO).