Gamma Markov Random fields for audio source modelling

Audio processing tasks, such as source separation or denoising, require the construction of realistic models that reflect physical properties of audio signals. In this paper, we modelled the variances of time-frequency coefficients of audio signals with Gamma Markov random fields (GMRFs) so that the dependencies between coefficients are captured. There is positive correlation between consecutive variance variables in this model and the strength of this correlation is determined by the coupling hyperparameters. Inference can be carried out using the Gibbs sampler or variational Bayes because the model is conditionally conjugate. However, the optimisation of the hyperparameters is not straightforward because of the intractable normalising constant. In this work, we used this model in denoising and single-channel source separation problems. The hyperparameters of the model are optimised using contrastive divergence and inference is performed using the Gibbs sampler

[1]  Aapo Hyvärinen,et al.  Estimation of Non-Normalized Statistical Models by Score Matching , 2005, J. Mach. Learn. Res..

[2]  N. Mitianoudis,et al.  Simple mixture model for sparse overcomplete ICA , 2004 .

[3]  Miguel Á. Carreira-Perpiñán,et al.  On Contrastive Divergence Learning , 2005, AISTATS.

[4]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[5]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[6]  K. Jarrod Millman,et al.  Learning Sparse Codes with a Mixture-of-Gaussians Prior , 1999, NIPS.

[7]  Mark A. Girolami,et al.  A Variational Method for Learning Sparse and Overcomplete Representations , 2001, Neural Computation.

[8]  S. Mase,et al.  Consistency of the Maximum Pseudo-Likelihood Estimator of Continuous State Space Gibbsian Processes , 1995 .

[9]  Aapo Hyvärinen,et al.  Consistency of Pseudolikelihood Estimation of Fully Visible Boltzmann Machines , 2006, Neural Computation.

[10]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[11]  J. Palmer Variational and scale mixture representations of non -Gaussian densities for estimation in the Bayesian linear model: Sparse coding, independent component analysis, and minimum entropy segmentation , 2006 .

[12]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[13]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Simon J. Godsill,et al.  Variational and stochastic inference for Bayesian source separation , 2007, Digit. Signal Process..

[15]  Aapo Hyvärinen,et al.  Some extensions of score matching , 2007, Comput. Stat. Data Anal..

[16]  Simon J. Godsill,et al.  A Bayesian Approach for Blind Separation of Sparse Sources , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[17]  Simon J. Godsill,et al.  Bayesian computational methods for sparse audio and music processing , 2007, 2007 15th European Signal Processing Conference.

[18]  Rainer Martin,et al.  Speech enhancement based on minimum mean-square error estimation and supergaussian priors , 2005, IEEE Transactions on Speech and Audio Processing.

[19]  S. Godsill,et al.  Bayesian variable selection and regularization for time–frequency surface estimation , 2004 .

[20]  Ali Taylan Cemgil,et al.  Conjugate Gamma Markov Random Fields for Modelling Nonstationary Sources , 2007, ICA.

[21]  Bhaskar D. Rao,et al.  Variational EM Algorithms for Non-Gaussian Latent Variable Models , 2005, NIPS.

[22]  J. Besag Statistical Analysis of Non-Lattice Data , 1975 .

[23]  Christopher K. I. Williams,et al.  An analysis of contrastive divergence learning in gaussian boltzmann machines , 2002 .

[24]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[25]  Bhaskar D. Rao,et al.  Modeling and Estimation of Dependent Subspaces with Non-radially Symmetric and Skewed Densities , 2007, ICA.

[26]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .