Reversible Jump MCMC for Non-Negative Matrix Factorization

We present a fully Bayesian approach to NonNegative Matrix Factorisation (NMF) by developing a Reversible Jump Markov Chain Monte Carlo (RJMCMC) method which provides full posteriors over the matrix components. In addition the NMF model selection issue is addressed, for the first time, as our RJMCMC procedure provides the posterior distribution over the matrix dimensions and therefore the number of components in the NMF model. A comparative analysis is provided with the Bayesian Information Criterion (BIC) and model selection employing estimates of the marginal likelihood. An illustrative synthetic example is provided using blind mixtures of images. This is then followed by a large scale study of the recovery of component spectra from multiplexed Raman readouts. The power and flexibility of the Bayesian methodology and the proposed RJMCMC procedure to objectively assess differing model structures and infer the corresponding plausible component spectra for this complex data is demonstrated convincingly.

[1]  Simon J. Godsill,et al.  A Bayesian Approach for Blind Separation of Sparse Sources , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[2]  Duncan Graham,et al.  Biosensing using silver nanoparticles and surface enhanced resonance Raman scattering. , 2006, Chemical communications.

[3]  A. Pettitt,et al.  Marginal likelihood estimation via power posteriors , 2008 .

[4]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[5]  D. Brie,et al.  Separation of Non-Negative Mixture of Non-Negative Sources Using a Bayesian Approach and MCMC Sampling , 2006, IEEE Transactions on Signal Processing.

[6]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[7]  G. Roberts,et al.  Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions , 2003 .

[8]  S. Godsill.  On the Relationship Between Markov chain Monte Carlo Methods for Model Uncertainty , 2001 .

[9]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[10]  Xiao-Li Meng,et al.  Simulating Normalizing Constants: From Importance Sampling to Bridge Sampling to Path Sampling , 1998 .

[11]  Mark A. Girolami,et al.  Estimating Bayes factors via thermodynamic integration and population MCMC , 2009, Comput. Stat. Data Anal..

[12]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[13]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[14]  M. Newton,et al.  Estimating the Integrated Likelihood via Posterior Simulation Using the Harmonic Mean Identity , 2006 .

[15]  Stephen P. Brooks,et al.  Markov Chain Monte Carlo Convergence Assessment via Two-Way Analysis of Variance , 2000 .

[16]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[17]  Duncan Graham,et al.  Evaluation of surface-enhanced resonance Raman scattering for quantitative DNA analysis. , 2004, Analytical chemistry.

[18]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[19]  Michael A. West,et al.  BAYESIAN MODEL ASSESSMENT IN FACTOR ANALYSIS , 2004 .

[20]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo Methods for Computing Bayes Factors , 2001 .

[21]  S. Walker,et al.  Sampling Truncated Normal, Beta, and Gamma Densities , 2001 .