Blind separation of sparse sources using variational EM

In this paper, we tackle the general linear instantaneous model (possibly underdetermined and noisy) using the assumption of sparsity of the sources on a given dictionary. We model the sparsity of expansion coefficients with a Student t prior. The conjugate-exponential characterisation of the t distribution as an infinite mixture of scaled Gaussians enables us to derive an efficient variational expectation maximisation algorithm (V-EM). The resulting deterministic algorithm has superior properties in terms of computation time and achieves a separation performance comparable in quality to alternative methods based on Markov Chain Monte Carlo (MCMC).

[1]  Terrence J. Sejnowski,et al.  Learning Overcomplete Representations , 2000, Neural Computation.

[2]  Wim Wiegerinck,et al.  Variational Approximations between Mean Field Theory and the Junction Tree Algorithm , 2000, UAI.

[3]  Rémi Gribonval.  Sparse decomposition of stereo signals with Matching Pursuit and application to blind separation of more than two sources from a stereo mixture , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[4]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[5]  N. Mitianoudis,et al.  Simple mixture model for sparse overcomplete ICA , 2004 .

[6]  Zoubin Ghahramani,et al.  Propagation Algorithms for Variational Bayesian Learning , 2000, NIPS.

[7]  Simon J. Godsill,et al.  A Bayesian Approach for Blind Separation of Sparse Sources , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[8]  Simon J. Godsill,et al.  Bayesian Approach for Blind Separation of Underdetermined Mixtures of Sparse Sources , 2004, ICA.

[9]  Barak A. Pearlmutter,et al.  Blind source separation by sparse decomposition , 2000, SPIE Defense + Commercial Sensing.

[10]  S. Godsill,et al.  Bayesian variable selection and regularization for time–frequency surface estimation , 2004 .

[12]  M. West.  On scale mixtures of normal distributions , 1987 .

[13]  Özgür Yilmaz,et al.  Blind separation of disjoint orthogonal signals: demixing N sources from 2 mixtures , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[14]  K. Jarrod Millman,et al.  Learning Sparse Codes with a Mixture-of-Gaussians Prior , 1999, NIPS.

[15]  Mark A. Girolami,et al.  A Variational Method for Learning Sparse and Overcomplete Representations , 2001, Neural Computation.

[16]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[17]  Terrence J. Sejnowski,et al.  Blind source separation of more sources than mixtures using overcomplete representations , 1999, IEEE Signal Processing Letters.