An Expectation-Maximization Algorithm for Blind Separation of Noisy Mixtures Using Gaussian Mixture Model

In this paper, we propose a new expectation-maximization (EM) algorithm, named GMM-EM, to blind separation of noisy instantaneous mixtures, in which the non-Gaussianity of independent sources is exploited by modeling their distribution using the Gaussian mixture model (GMM). The compatibility between the incomplete-data structure of the GMM and the hidden variable nature of the source separation problem leads to an efficient hierarchical learning and alternative method for estimating the sources and the mixing matrix. In comparison with conventional blind source separation algorithms, the proposed GMM-EM algorithm has superior performance for the separation of noisy mixtures due to the fact that the covariance matrix of the additive Gaussian noise is treated as a parameter. Furthermore, the GMM-EM algorithm works well in underdetermined cases by incorporating any prior information one may have and jointly estimating the mixing matrix and source signals in a Bayesian framework. Systematic simulations with both synthetic and real speech signals are used to show the advantage of the proposed algorithm over conventional independent component analysis techniques, such as FastICA, especially for noisy and/or underdetermined mixtures. Moreover, it can even achieve similar performance to a recent technique called null space component analysis with less computational complexity.

[1]  P. Tseng Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[2]  Hang Zhang,et al.  Blind separation of non-stationary sources using continuous density hidden Markov models , 2013, Digit. Signal Process..

[3]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[4]  Hang Zhang,et al.  Generalized generating function with tucker decomposition and alternating least squares for underdetermined blind identification , 2013, EURASIP J. Adv. Signal Process..

[5]  Shun-ichi Amari,et al.  Adaptive blind signal processing-neural network approaches , 1998, Proc. IEEE.

[6]  Chin-Hui Lee,et al.  Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains , 1994, IEEE Trans. Speech Audio Process..

[7]  Hichem Snoussi,et al.  Bayesian blind separation of generalized hyperbolic processes in noisy and underdeterminate mixtures , 2006, IEEE Transactions on Signal Processing.

[8]  Wen-Liang Hwang,et al.  Null space component analysis for noisy blind source separation , 2015, Signal Process..

[9]  Jie Yang,et al.  Temporally correlated source separation based on variational Kalman smoother , 2008, Digit. Signal Process..

[10]  Adel Belouchrani Jean-Fran,et al.  Maximum likelihood source separation for discrete sources , 2007 .

[11]  Chi Hau Chen,et al.  A Gaussian mixture model for underdetermined independent component analysis , 2006, Signal Process..

[12]  Zhi-Quan Luo,et al.  A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization , 2012, SIAM J. Optim..

[13]  Lieven De Lathauwer,et al.  Fourth-Order Cumulant-Based Blind Identification of Underdetermined Mixtures , 2007, IEEE Transactions on Signal Processing.

[14]  Jinwen Ma,et al.  Asymptotic Convergence Rate of the EM Algorithm for Gaussian Mixtures , 2000, Neural Computation.

[15]  Joseph Tabrikian,et al.  MIMO-AR System Identification and Blind Source Separation for GMM-Distributed Sources , 2009, IEEE Transactions on Signal Processing.

[16]  Yingtao Jiang,et al.  Blind source separation with time series variational Bayes expectation maximization algorithm , 2012, Digit. Signal Process..

[17]  Hang Zhang,et al.  Blind Separation of Complex Sources Using Generalized Generating Function , 2013, IEEE Signal Processing Letters.

[18]  Wen-Liang Hwang,et al.  Null Space Pursuit: An Operator-based Approach to Adaptive Signal Separation , 2010, IEEE Transactions on Signal Processing.

[19]  Lieven De Lathauwer,et al.  Blind Identification of Underdetermined Mixtures by Simultaneous Matrix Diagonalization , 2008, IEEE Transactions on Signal Processing.

[20]  Jean-Luc Gauvain,et al.  Speaker adaptation based on MAP estimation of HMM parameters , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[21]  H. Snoussi,et al.  Unsupervised learning for source separation with mixture of Gaussians prior for sources and Gaussian prior for mixture coefficients , 2001, Neural Networks for Signal Processing XI: Proceedings of the 2001 IEEE Signal Processing Society Workshop (IEEE Cat. No.01TH8584).

[22]  Chin-Hui Lee,et al.  Bayesian adaptive learning of the parameters of hidden Markov model for speech recognition , 1995, IEEE Trans. Speech Audio Process..

[23]  Joseph Tabrikian,et al.  Blind Separation of Independent Sources Using Gaussian Mixture Model , 2007, IEEE Transactions on Signal Processing.

[24]  Jeff A. Bilmes,et al.  A gentle tutorial of the em algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models , 1998 .

[25]  Pierre Comon,et al.  Blind Identification of Underdetermined Mixtures Based on the Characteristic Function: The Complex Case , 2011, IEEE Transactions on Signal Processing.

[26]  Tobias Rydén,et al.  EM versus Markov chain Monte Carlo for estimation of hidden Markov models: a computational perspective , 2008 .

[27]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[28]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[29]  Eric Moulines,et al.  Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[30]  Laurent Albera,et al.  Blind Underdetermined Mixture Identification by Joint Canonical Decomposition of HO Cumulants , 2010, IEEE Transactions on Signal Processing.

[31]  Hang Zhang,et al.  A Bayesian Approach to Blind Separation of Mixed Discrete Sources by Gibbs Sampling , 2011, UIC.

[32]  Wen-Qing Huang,et al.  Image segmentation using temporal-spatial information in dynamic scenes , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[33]  Andrew R. Barron,et al.  Mixture Density Estimation , 1999, NIPS.

[34]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[35]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[36]  Michael I. Jordan,et al.  On Convergence Properties of the EM Algorithm for Gaussian Mixtures , 1996, Neural Computation.

[37]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[38]  Kevin H. Knuth,et al.  Informed source separation: A Bayesian tutorial , 2013, 2005 13th European Signal Processing Conference.

[39]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[40]  C. Yoo,et al.  Underdetermined Blind Source Separation Based on Generalized Gaussian Distribution , 2006, 2006 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing.

[41]  Pierre Comon,et al.  Blind identification of under-determined mixtures based on the characteristic function , 2006, Signal Process..