Blind Separation of Independent Sources Using Gaussian Mixture Model

In this paper, two novel algorithms for blind separation of noiseless instantaneous linear mixture of independent sources are presented. The proposed algorithms exploit nonGaussianity of the independent sources by modeling their distribution using the Gaussian mixture model (GMM). The first proposed method is based on the maximum-likelihood (ML) estimator. According to this method, the sensors distribution parameters are estimated via the expectation-maximization (EM) algorithm for GMM parameter estimation and the separation matrix is estimated by applying nonorthogonal joint diagonalization of the estimated GMM covariance matrices. The second proposed method is also a ML-based approach. According to this method, the distribution parameters of the prewhitened sensors are estimated via the EM algorithm for GMM parameter estimation and a unitary separation matrix is estimated by applying orthogonal joint diagonalization of the estimated GMM covariance matrices. It is shown that estimation of the sensors distribution parameters amounts to obtaining a tight lower bound on the log-likelihood of the separation matrix, and that the joint diagonalization of the estimated GMM covariance matrices amounts to maximization of the obtained tight lower bound. Simulations demonstrate that the proposed methods outperform state-of-the-art blind source separation techniques in terms of interference-to-signal ratio.

[1]  H. Akaike A new look at the statistical model identification , 1974 .

[2]  Huaiyu Zhu On Information and Sufficiency , 1997 .

[3]  Terrence J. Sejnowski,et al.  ICA Mixture Models for Unsupervised Classification of Non-Gaussian Classes and Automatic Context Switching in Blind Signal Separation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[5]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[6]  Hagai Attias,et al.  Independent Factor Analysis with Temporally Structured Sources , 1999, NIPS.

[7]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[8]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[9]  Jean-Franois Cardoso High-Order Contrasts for Independent Component Analysis , 1999, Neural Computation.

[10]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[11]  Jorma Rissanen,et al.  The Minimum Description Length Principle in Coding and Modeling , 1998, IEEE Trans. Inf. Theory.

[12]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[13]  N. Mitianoudis,et al.  Simple mixture model for sparse overcomplete ICA , 2004 .

[14]  T. Ens,et al.  Blind signal separation : statistical principles , 1998 .

[15]  W. Gautschi,et al.  An algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form , 1986 .

[16]  Andrew R. Barron,et al.  Mixture Density Estimation , 1999, NIPS.

[17]  Max Welling,et al.  A Constrained EM Algorithm for Independent Component Analysis , 2001, Neural Computation.

[18]  Eric Moulines,et al.  Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[20]  Ben J. A. Kröse,et al.  Efficient Greedy Learning of Gaussian Mixture Models , 2003, Neural Computation.

[21]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[22]  Dinh-Tuan Pham,et al.  Blind separation of instantaneous mixtures of nonstationary sources , 2001, IEEE Trans. Signal Process..

[23]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .