An Expectation–Maximization Method for Spatio–Temporal Blind Source Separation Using an AR-MOG Source Model

In this paper, we develop a maximum-likelihood (ML) spatio-temporal blind source separation (BSS) algorithm, where the temporal dependencies are explained by assuming that each source is an autoregressive (AR) process and the distribution of the associated independent identically distributed (i.i.d.) innovations process is described using a mixture of Gaussians. Unlike most ML methods, the proposed algorithm takes into account both spatial and temporal information, optimization is performed using the expectation-maximization (EM) method, the source model is adapted to maximize the likelihood, and the update equations have a simple, analytical form. The proposed method, which we refer to as autoregressive mixture of Gaussians (AR-MOG), outperforms nine other methods for artificial mixtures of real audio. We also show results for using AR-MOG to extract the fetal cardiac signal from real magnetocardiographic (MCG) data.

[1]  Dinh-Tuan Pham,et al.  Markovian source separation , 2003, IEEE Trans. Signal Process..

[2]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[3]  Simone G. O. Fiori,et al.  Probability Density Estimation Using Adaptive Activation Function Neurons , 2004, Neural Processing Letters.

[4]  Jose C. Principe,et al.  A unifying criterion for blind source separation and decorrelation: simultaneous diagonalization of correlation matrices , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[5]  Aapo Hyvärinen,et al.  Independent Component Analysis for Time-dependent Stochastic Processes , 1998 .

[6]  K. Sekihara.  A Graphical Model for Estimating Stimulus-Evoked Brain Responses in Noisy MEG data with Large Background Brain Activity , 2005 .

[7]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[8]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[9]  Dirk Van Compernolle,et al.  Signal separation by symmetric adaptive decorrelation: stability, convergence, and uniqueness , 1995, IEEE Trans. Signal Process..

[10]  Richard M. Everson,et al.  Hidden Markov Independent Component Analysis , 2000 .

[11]  L. Xu.  Independent Component Analysis and Extensions with Noise and Time: A Bayesian Ying-Yang Learning Perspective , 2003 .

[12]  Eric Moulines,et al.  Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[13]  Dinh-Tuan Pham,et al.  Mutual information approach to blind separation of stationary sources , 2002, IEEE Trans. Inf. Theory.

[14]  T. Hughes,et al.  Signals and systems , 2006, Genome Biology.

[15]  José Millet-Roig,et al.  Spatiotemporal blind source separation approach to atrial activity estimation in atrial tachyarrhythmias , 2005, IEEE Transactions on Biomedical Engineering.

[16]  R. Liu,et al.  AMUSE: a new blind identification algorithm , 1990, IEEE International Symposium on Circuits and Systems.

[17]  Hichem Snoussi,et al.  Bayesian Unsupervised Learning for Source Separation with Mixture of Gaussians Prior , 2004, J. VLSI Signal Process..

[18]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[19]  Nikos A. Vlassis,et al.  Efficient source adaptivity in independent component analysis , 2001, IEEE Trans. Neural Networks.

[20]  Shun-ichi Amari,et al.  Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information , 1997, Neural Computation.

[21]  C. Fyfe,et al.  Generalised independent component analysis through unsupervised learning with emergent Bussgang properties , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[22]  Andrzej Cichocki,et al.  A robust approach to independent component analysis of signals with high-level noise measurements , 2003, IEEE Trans. Neural Networks.

[23]  Simone G. O. Fiori,et al.  Hybrid independent component analysis by adaptive LUT activation function neurons , 2002, Neural Networks.

[24]  Barak A. Pearlmutter,et al.  A Context-Sensitive Generalization of ICA , 1996 .

[25]  Lei Xu,et al.  BYY harmony learning, independent state space, and generalized APT financial analyses , 2001, IEEE Trans. Neural Networks.

[26]  S Comani,et al.  Fetal magnetocardiographic mapping using independent component analysis. , 2004, Physiological measurement.

[27]  Lars Kai Hansen,et al.  CICAAR: Convolutive ICA with an Auto-regressive Inverse Model , 2004, ICA.

[28]  Geoffrey E. Hinton,et al.  A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants , 1998, Learning in Graphical Models.

[29]  R N Vigário,et al.  Extraction of ocular artefacts from EEG using independent component analysis. , 1997, Electroencephalography and clinical neurophysiology.

[30]  R. Liu,et al.  An extended fourth order blind identification algorithm in spatially correlated noise , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[31]  Vwani P. Roychowdhury,et al.  Independent component analysis based on nonparametric density estimation , 2004, IEEE Transactions on Neural Networks.

[32]  Richard M. Everson,et al.  Hidden Markov independent components for biosignal analysis , 2000 .

[33]  Barak A. Pearlmutter,et al.  Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA , 1996, NIPS.

[34]  Ramesh A. Gopinath,et al.  An EM algorithm for convolutive independent component analysis , 2002, Neurocomputing.

[35]  P. Philips,et al.  JADETD : COMBINING HIGHER-ORDER STATISTICS AND TEMPORALINFORMATION FOR BLIND SOURCE SEPARATION ( WITH NOISE ) , 1999 .

[36]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[37]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[38]  Shun-ichi Amari,et al.  Independent component analysis by the information-theoretic approach with mixture of densities , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[39]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[40]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Sub-Gaussian and Super-Gaussian Sources , 1999, Neural Comput..

[41]  J.E. Mazo,et al.  Digital communications , 1985, Proceedings of the IEEE.

[42]  Meir Feder,et al.  Multi-channel signal separation by decorrelation , 1993, IEEE Trans. Speech Audio Process..

[43]  P. Tichavský,et al.  Efficient variant of algorithm fastica for independent component analysis attaining the cramer-RAO lower bound , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[44]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[45]  Ali Mohammad-Djafari,et al.  A mean field approximation approach to blind source separation with L p priors , 2005 .

[46]  Hagai Attias,et al.  Independent Factor Analysis with Temporally Structured Sources , 1999, NIPS.

[47]  Hagai Attias,et al.  A graphical model for estimating stimulus-evoked brain responses from magnetoencephalography data with large background brain activity , 2006, NeuroImage.

[48]  Shun-ichi Amari,et al.  Learned parametric mixture based ICA algorithm , 1998, Neurocomputing.

[49]  Danilo P. Mandic,et al.  AN ONLINE ALGORITHM FOR BLIND EXTRACTION OF SOURCES WITH DIFFERENT DYNAMICAL STRUCTURES , 2003 .

[50]  XuLei.  BYY harmony learning, independent state space, and generalized APT financial analyses , 2001 .

[51]  Hagai Attias,et al.  An EM Method for Spatio-temporal Blind Source Separation Using an AR-MOG Source Model , 2006, ICA.

[52]  Sergio Cruces,et al.  On a new blind signal extraction algorithm: different criteria and stability analysis , 2002, IEEE Signal Processing Letters.

[53]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[54]  Deniz Erdogmus,et al.  An analysis of entropy estimators for blind source separation , 2006, Signal Process..

[55]  Yoram Baram,et al.  Multidimensional density shaping by sigmoids , 1996, IEEE Trans. Neural Networks.

[56]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[57]  Andreas Ziehe,et al.  TDSEP { an e(cid:14)cient algorithm for blind separation using time structure , 1998 .

[58]  Reinhold Orglmeister,et al.  A contextual blind separation of delayed and convolved sources , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[59]  Aapo Hyvärinen,et al.  A unifying model for blind separation of independent sources , 2005, Signal Process..

[60]  M.G. Bellanger,et al.  Digital processing of speech signals , 1980, Proceedings of the IEEE.

[61]  Hagai Attias.  Blind source separation by dynamic graphical models , 1999 .

[62]  R. A. Choudrey.  FLEXIBLE BAYESIAN INDEPENDENT COMPONENT ANALYSIS FOR BLIND SOURCE SEPARATION , 2001 .

[63]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[64]  P. M. Grant,et al.  Digital communications. 3rd ed , 2009 .

[65]  Lang Tong,et al.  Indeterminacy and identifiability of blind identification , 1991 .

[66]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[67]  Shun-ichi Amari,et al.  Neural Learning in Structured Parameter Spaces - Natural Riemannian Gradient , 1996, NIPS.

[68]  Lucas C. Parra,et al.  Blind Source Separation via Generalized Eigenvalue Decomposition , 2003, J. Mach. Learn. Res..

[69]  Stephen J. Roberts,et al.  Bayesian ICA with hidden Markov model sources , 2003 .

[70]  Andrzej Cichocki,et al.  Flexible Independent Component Analysis , 2000, J. VLSI Signal Process..

[71]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[72]  Ali Mohammad-Djafari,et al.  A mean field approximation approach to blind source separation with Lp priors , 2005, 2005 13th European Signal Processing Conference.

[73]  Bayesian Ying Yang System and Theory as A Uni ed Statistical Learning Approach III Models and Algorithms for Dependence Reduction Data Dimension Reduction ICA and Supervised Learning , 1997 .

[74]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[75]  Erkki Oja,et al.  Efficient Variant of Algorithm FastICA for Independent Component Analysis Attaining the CramÉr-Rao Lower Bound , 2006, IEEE Transactions on Neural Networks.

[76]  H Liang,et al.  Adaptive independent component analysis of multichannel electrogastrograms. , 2001, Medical engineering & physics.

[77]  H. Snoussi,et al.  Unsupervised learning for source separation with mixture of Gaussians prior for sources and Gaussian prior for mixture coefficients , 2001, Neural Networks for Signal Processing XI: Proceedings of the 2001 IEEE Signal Processing Society Workshop (IEEE Cat. No.01TH8584).