Bayesian Nonnegative Matrix Factorization with a Truncated Spike-and-Slab Prior

Non-negative matrix factorization (NMF) is a challenging problem due to its ill-posed nature. The key for the success of NMF is to exploit appropriate prior models for those two decomposed factor matrices. Although lots of effective sparsity-inducing prior models have been developed for NMF, they are often rooted in either ℓ_p regularization with p > 0, which only provide an approximation to the ℓ_0 sparsity, ultimately resulting in a sub-optimal solution. To address this problem, we propose a novel truncated spike-and-slab prior based Bayesian NMF method. Through integrating a Bernoulli distribution with a truncated Gaussian distribution together, the proposed prior is capable of imposing the exact ℓ_0 regularization as well as the non-negativity constraint on the factor matrices. Further, the proposed prior can be extended to robust NMF problem. Experimental results in blind source separation, face images representation and image denoising demonstrate the advantage of the proposed method.

[1]  Miguel Lázaro-Gredilla,et al.  Spike and Slab Variational Inference for Multi-Task and Multiple Kernel Learning , 2011, NIPS.

[2]  Lars Kai Hansen,et al.  Approximate L0 constrained non-negative matrix and tensor factorization , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[3]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[4]  Thomas T. Liu,et al.  Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem , 2016, IEEE Transactions on Signal Processing.

[5]  Ole Winther,et al.  Bayesian Non-negative Matrix Factorization , 2009, ICA.

[6]  Lucas C. Parra,et al.  Recovery of constituent spectra using non-negative matrix factorization , 2003, SPIE Optics + Photonics.

[7]  A. V. D. Vaart,et al.  BAYESIAN LINEAR REGRESSION WITH SPARSE PRIORS , 2014, 1403.0735.

[8]  Lei Zhang,et al.  Deblurring Natural Image Using Super-Gaussian Fields , 2018, ECCV.

[9]  Xiaoqiang Lu,et al.  Substance Dependence Constrained Sparse NMF for Hyperspectral Unmixing , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Yoshua Bengio,et al.  Spike-and-Slab Sparse Coding for Unsupervised Feature Discovery , 2012, ArXiv.

[11]  Jinhui Tang,et al.  Robust Structured Nonnegative Matrix Factorization for Image Representation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[12]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[13]  Yuan Gao,et al.  Improving molecular cancer class discovery through sparse non-negative matrix factorization , 2005 .

[14]  David Zhang,et al.  FSIM: A Feature Similarity Index for Image Quality Assessment , 2011, IEEE Transactions on Image Processing.

[15]  T. J. Mitchell,et al.  Bayesian Variable Selection in Linear Regression , 1988 .

[16]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[17]  Yuhang Liu,et al.  Frame-Based Variational Bayesian Learning for Independent or Dependent Source Separation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Mohammad Reza Mohammadi,et al.  Non-negative sparse decomposition based on constrained smoothed ℓ0 norm , 2014, Signal Process..

[19]  Franz Pernkopf,et al.  Sparse nonnegative matrix factorization with ℓ0-constraints , 2012, Neurocomputing.

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  M. Halperin.  Estimation in the Truncated Normal Distribution , 1952 .

[22]  Vincent Y. F. Tan,et al.  Online Nonnegative Matrix Factorization With Outliers , 2016, IEEE Transactions on Signal Processing.

[23]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[24]  Nicolas Dobigeon,et al.  Nonlinear Hyperspectral Unmixing With Robust Nonnegative Matrix Factorization , 2014, IEEE Transactions on Image Processing.

[25]  Zhigang Luo,et al.  Online Nonnegative Matrix Factorization With Robust Stochastic Approximation , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Lucien Wald,et al.  Data Fusion. Definitions and Architectures - Fusion of Images of Different Spatial Resolutions , 2002 .

[27]  Xi Chen,et al.  Spectral unmixing using nonnegative matrix factorization with smoothed L0 norm constraint , 2009, International Symposium on Multispectral Image Processing and Pattern Recognition.

[28]  Andrzej Cichocki,et al.  Non-Negative Matrix Factorization , 2020 .

[29]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.