Bayesian rank penalization

Rank minimization is a key component of many computer vision and machine learning methods, including robust principal component analysis (RPCA) and low-rank representations (LRR). However, usual methods rely on optimization to produce a point estimate without characterizing uncertainty in this estimate, and also face difficulties in tuning parameter choice. Both of these limitations are potentially overcome with Bayesian methods, but there is currently a lack of general purpose Bayesian approaches for rank penalization. We address this gap using a positive generalized double Pareto prior, illustrating the approach in RPCA and LRR. Posterior computation relies on hybrid Gibbs sampling and geodesic Monte Carlo algorithms. We assess performance in simulation examples, and benchmark data sets.

[1]  Aggelos K. Katsaggelos,et al.  Sparse Bayesian Methods for Low-Rank Matrix Estimation , 2011, IEEE Transactions on Signal Processing.

[2]  Alfred O. Hero,et al.  Semi-Blind Sparse Image Reconstruction With Application to MRFM , 2012, IEEE Transactions on Image Processing.

[3]  Yong Yu,et al.  Robust Subspace Segmentation by Low-Rank Representation , 2010, ICML.

[4]  Yi Ma,et al.  TILT: Transform Invariant Low-Rank Textures , 2010, ACCV 2010.

[5]  Yi Ma,et al.  Robust and Practical Face Recognition via Structured Sparsity , 2012, ECCV.

[6]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[8]  Jian Yu,et al.  Saliency Detection by Multitask Sparsity Pursuit , 2012, IEEE Transactions on Image Processing.

[9]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[10]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[11]  John Wright,et al.  RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images , 2012, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[13]  Lawrence Carin,et al.  Bayesian Robust Principal Component Analysis , 2011, IEEE Transactions on Image Processing.

[14]  Xian-Da Zhang,et al.  Matrix Analysis and Applications , 2017 .

[15]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[16]  Michael E. Tipping.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[17]  Qi Tian,et al.  Statistical modeling of complex backgrounds for foreground object detection , 2004, IEEE Transactions on Image Processing.

[18]  David P. Wipf,et al.  Non-Convex Rank Minimization via an Empirical Bayesian Approach , 2012, UAI.

[19]  Chris Hans.  Bayesian lasso regression , 2009 .

[20]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[21]  J. Griffin,et al.  Inference with normal-gamma prior distributions in regression problems , 2010 .

[22]  M. Girolami,et al.  Geodesic Monte Carlo on Embedded Manifolds , 2013, Scandinavian journal of statistics, theory and applications.

[23]  J. Leroy Folks,et al.  The Inverse Gaussian Distribution: Theory: Methodology, and Applications , 1988 .

[24]  Yong Yu,et al.  Robust Recovery of Subspace Structures by Low-Rank Representation , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[26]  Mário A. T. Figueiredo.  Adaptive Sparseness for Supervised Learning , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[28]  René Vidal,et al.  Subspace Clustering , 2011, IEEE Signal Processing Magazine.

[29]  Shuicheng Yan,et al.  Multi-task low-rank affinity pursuit for image segmentation , 2011, 2011 International Conference on Computer Vision.

[30]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[32]  James G. Scott,et al.  The horseshoe estimator for sparse signals , 2010 .

[33]  Shinichi Nakajima,et al.  Probabilistic Low-Rank Subspace Clustering , 2012, NIPS.

[34]  René Vidal,et al.  A closed form solution to robust subspace estimation and clustering , 2011, CVPR 2011.

[35]  D. Brie,et al.  Simulation of postive normal variables using several proposal distributions , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[36]  Michael Elad,et al.  On the Role of Sparse and Redundant Representations in Image Processing , 2010, Proceedings of the IEEE.

[37]  E. Stiefel.  Richtungsfelder und Fernparallelismus in n-dimensionalen Mannigfaltigkeiten , 1935 .

[38]  Jaeyong Lee,et al.  GENERALIZED DOUBLE PARETO SHRINKAGE. , 2011, Statistica Sinica.

[39]  Hossein Mobahi,et al.  Face recognition with contiguous occlusion using markov random fields , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.