Sparse Bayesian Learning for Robust PCA

In this paper, we propose a new Bayesian model to solve the Robust PCA problem - recovering the underlying low-rank matrix and sparse matrix from their noisy compositions. We first derive and analyze a new objective function, which is proven to be equivalent to the fundamental minimizing "rank+sparsity" objective. To solve this objective, we develop a concise Sparse Bayesian Learning (SBL) method that has minimum assumptions and effectively deals with the crux of the problem. The concise modeling allows simple and effective Empirical Bayesian inference via MAP-EM. Simulation studies demonstrate the superiority of the proposed method over the existing state-of-the-art methods. The efficacy of the method is further verified through a text extraction image processing task.

[1]  Bhaskar D. Rao,et al.  Algorithms for robust linear regression by exploiting the connection to sparse signal recovery , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  David P. Wipf,et al.  Non-Convex Rank Minimization via an Empirical Bayesian Approach , 2012, UAI.

[3]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[4]  Guangming Shi,et al.  Compressive Sensing via Nonlocal Low-Rank Regularization , 2014, IEEE Transactions on Image Processing.

[5]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[6]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[7]  Aggelos K. Katsaggelos,et al.  Sparse Bayesian Methods for Low-Rank Matrix Estimation , 2011, IEEE Transactions on Signal Processing.

[8]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision , 2016, International Journal of Computer Vision.

[9]  Rama Chellappa,et al.  IEEE TRANSACTIONS ON SIGNAL PROCESSING 1 Analysis of Sparse Regularization Based Robust Regression Approaches , 2022 .

[10]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[11]  Maya R. Gupta,et al.  EM Demystified: An Expectation-Maximization Tutorial , 2010 .

[12]  Necdet Serhat Aybat,et al.  An alternating direction method with increasing penalty for stable principal component pursuit , 2013, Computational Optimization and Applications.

[13]  Lawrence Carin,et al.  Bayesian Robust Principal Component Analysis , 2011, IEEE Transactions on Image Processing.

[14]  Bhaskar D. Rao,et al.  Robust PCA via ℓ0-ℓ1 Regularization , 2019, IEEE Trans. Signal Process..

[15]  Prateek Jain,et al.  Non-convex Robust PCA , 2014, NIPS.

[16]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[17]  John Wright,et al.  Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization , 2009, NIPS.

[18]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[19]  Dit-Yan Yeung,et al.  Bayesian Robust Matrix Factorization for Image and Video Processing , 2013, 2013 IEEE International Conference on Computer Vision.

[20]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[21]  Bhaskar D. Rao,et al.  L_0-norm Minimization for Basis Selection , 2004, NIPS.

[22]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[23]  Tae-Hyun Oh,et al.  A Pseudo-Bayesian Algorithm for Robust PCA , 2016, NIPS.

[24]  Bhaskar D. Rao,et al.  Sparse Bayesian Learning for Robust PCA: Algorithms and Analyses , 2019, IEEE Transactions on Signal Processing.

[25]  Stephen P. Boyd,et al.  Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices , 2003, Proceedings of the 2003 American Control Conference, 2003..