Convolutive blind source separation by minimizing mutual information between segments of signals

A method to perform convolutive blind source separation of super-Gaussian sources by minimizing the mutual information between segments of output signals is presented. The proposed approach is essentially an implementation of an idea previously proposed by Pham. The formulation of mutual information in the proposed criterion makes use of a nonparametric estimator of Renyi's /spl alpha/-entropy, which becomes Shannon's entropy in the limit as /spl alpha/ approaches 1. Since /spl alpha/ can be any number greater than 0, this produces a family of criteria having an infinite number of members. Interestingly, it appears that Shannon's entropy cannot be used for convolutive source separation with this type of estimator. In fact, only one value of /spl alpha/ appears to be appropriate, namely /spl alpha/=2, which corresponds to Renyi's quadratic entropy. Four experiments are included to show the efficacy of the proposed criterion.

[1]  Ruey-Wen Liu,et al.  Blind equalization of MIMO-FIR Channels driven by white but higher order colored source signals , 2002, IEEE Trans. Inf. Theory.

[2]  Scott C. Douglas,et al.  Blind separation of acoustical mixtures without time-domain deconvolution or decorrelation , 2001, Neural Networks for Signal Processing XI: Proceedings of the 2001 IEEE Signal Processing Society Workshop (IEEE Cat. No.01TH8584).

[3]  Kari Torkkola,et al.  Blind separation of convolved sources based on information maximization , 1996, Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop.

[4]  C. Jutten,et al.  Blind source separation of convolutive mixtures by maximization of fourth-order cumulants: the non i.i.d. case , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[5]  Inbar Fijalkow,et al.  A globally convergent approach for blind MIMO adaptive deconvolution , 2001, IEEE Trans. Signal Process..

[6]  Shoko Araki,et al.  Geometrically constraint ICA for convolutive mixtures of sound , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[7]  P. Comon,et al.  Contrasts for multichannel blind deconvolution , 1996, IEEE Signal Processing Letters.

[8]  Walter Kellermann,et al.  A generalization of blind source separation algorithms for convolutive mixtures based on second-order statistics , 2005, IEEE Transactions on Speech and Audio Processing.

[9]  J. Pesquet,et al.  Generalized contrasts for multichannel blind deconvolution of linear systems , 1997, IEEE Signal Processing Letters.

[10]  Jitendra K. Tugnait.  On blind separation of convolutive mixtures of independent linear signals in unknown additive noise , 1998, IEEE Trans. Signal Process..

[11]  Henrik Sahlin,et al.  Separation of real-world signals , 1998, Signal Process..

[12]  Te-Won Lee,et al.  Blind Separation of Delayed and Convolved Sources , 1996, NIPS.

[13]  Karim Abed-Meraim,et al.  Blind separation of convolutive mixtures using joint block diagonalization , 2001, Proceedings of the Sixth International Symposium on Signal Processing and its Applications (Cat.No.01EX467).

[14]  Jitendra K. Tugnait.  Adaptive blind separation of convolutive mixtures of independent linear signals , 1999, Signal Process..

[15]  Shoko Araki,et al.  Time domain blind source separation of non-stationary convolved signals by utilizing geometric beamforming , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[16]  Gert Cauwenberghs,et al.  BLIND SEPARATION OF LINEAR CONVOLUTIVE MIXTURES USING ORTHOGONAL FILTER BANKS , 2001 .

[17]  Kari Torkkola,et al.  Blind separation of delayed sources based on information maximization , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[18]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[19]  Philippe Loubaton,et al.  Adaptive subspace algorithm for blind separation of independent sources in convolutive mixture , 2000, IEEE Trans. Signal Process..

[20]  Reinhold Orglmeister,et al.  A contextual blind separation of delayed and convolved sources , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[21]  Christian Jutten,et al.  Blind source separation for convolutive mixtures , 1995, Signal Process..

[22]  M. Davies,et al.  NEW FIXED-POINT ICA ALGORITHMS FOR CONVOLVED MIXTURES , 2001 .

[23]  James P. Reilly,et al.  BLIND SOURCE SEPARATION ALGORITHM FOR MIMO CONVOLUTIVE MIXTURES , 2001 .

[24]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[25]  Jose C. Principe,et al.  A unifying criterion for blind source separation and decorrelation: simultaneous diagonalization of correlation matrices , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[26]  Bertram E. Shi,et al.  IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS — I : REGULAR PAPERS , VOL . ? ? , NO . ? ? , ? ? ? ? , 2007 .

[27]  Lucas C. Parra,et al.  Convolutive blind separation of non-stationary sources , 2000, IEEE Trans. Speech Audio Process..

[28]  Joos Vandewalle,et al.  An algebraic approach to blind MIMO identification , 2000 .

[29]  James P. Reilly,et al.  Blind source separation of convolved sources by joint approximate diagonalization of cross-spectral density matrices , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[30]  Reinhold Orglmeister,et al.  Blind source separation of real world signals , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[31]  Zhang Xinhua,et al.  Study on blind separation of underwater acoustic signals , 2000, WCC 2000 - ICSP 2000. 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000.

[32]  Nikolaos Mitianoudis,et al.  Audio source separation of convolutive mixtures , 2003, IEEE Trans. Speech Audio Process..

[33]  M. Girolami.  Symmetric adaptive maximum likelihood estimation for noise cancellation and signal separation , 1997 .

[34]  Shoko Araki,et al.  Fundamental limitation of frequency domain blind source separation for convolutive mixture of speech , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[35]  Frank Ehlers,et al.  Blind separation of convolutive mixtures and an application in automatic speech recognition in a noisy environment , 1997, IEEE Trans. Signal Process..

[36]  Adriana Dapena,et al.  A novel frequency domain approach for separating convolutive mixtures of temporally-white signals , 2003, Digit. Signal Process..

[37]  Francoise Briolle,et al.  Blind separation of sources applied to convolutive mixtures in shallow water , 1997, Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics.

[38]  Deniz Erdogmus,et al.  Blind source separation using Renyi's -marginal entropies , 2002, Neurocomputing.

[39]  A. Doucet,et al.  Bayesian blind marginal separation of convolutively mixed discrete sources , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[40]  H. Attias,et al.  Blind source separation and deconvolution by dynamic component analysis , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[41]  Holger Broman,et al.  Source separation using a criterion based on second-order statistics , 1998, IEEE Trans. Signal Process..

[42]  Yujiro Inouye,et al.  Super-exponential algorithms for multichannel blind deconvolution , 2000, IEEE Trans. Signal Process..

[43]  Yannick Deville,et al.  Self-adaptive separation of convolutively mixed signals with a recursive structure. Part I: Stability analysis and optimization of asymptotic behaviour , 1999, Signal Process..

[44]  Adriana Dapena,et al.  A SIMPLIFIED FREQUENCY-DOMAIN APPROACH FOR BLIND SEPARATION OF CONVOLUTIVE MIXTURES , 2001 .

[45]  Christopher V. Alvino,et al.  Geometric source separation: merging convolutive source separation with geometric beamforming , 2001, Neural Networks for Signal Processing XI: Proceedings of the 2001 IEEE Signal Processing Society Workshop (IEEE Cat. No.01TH8584).

[46]  Xiaoan Sun,et al.  A NATURAL GRADIENT CONVOLUTIVE BLIND SOURCE SEPARATION ALGORITHM FOR SPEECH MIXTURES , 2001 .

[47]  G. Kokkinakis,et al.  Blind signal separation and speech recognition in the frequency domain , 1999, ICECS'99. Proceedings of ICECS '99. 6th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.99EX357).

[48]  Shoji Makino,et al.  Blind source separation of convolutive mixtures , 2006, SPIE Defense + Commercial Sensing.

[49]  L. Parra,et al.  Convolutive blind source separation based on multiple decorrelation , 1998, Neural Networks for Signal Processing VIII. Proceedings of the 1998 IEEE Signal Processing Society Workshop (Cat. No.98TH8378).

[50]  Douglas L. Jones,et al.  Blind source separation of nonstationary convolutively mixed signals , 2000, Proceedings of the Tenth IEEE Workshop on Statistical Signal and Array Processing (Cat. No.00TH8496).

[51]  Kiyotoshi Matsuoka,et al.  Stability analyses of information-theoretic blind separation algorithms in the case where the sources are nonlinear processes , 2002, IEEE Trans. Signal Process..

[52]  José Carlos Príncipe,et al.  Blind separation of convolutive mixtures , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[53]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[54]  Jose C. Principe,et al.  An information-theoretic approach to sonar automatic target recognition , 2003 .

[55]  Meir Feder,et al.  Multi-channel signal separation by decorrelation , 1993, IEEE Trans. Speech Audio Process..

[56]  Andrzej Cichocki,et al.  Multichannel blind deconvolution of non-minimum phase systems using information backpropagation , 1999, ICONIP'99. ANZIIS'99 & ANNES'99 & ACNN'99. 6th International Conference on Neural Information Processing. Proceedings (Cat. No.99EX378).

[57]  T. Oliveira e Silva,et al.  On the equivalence between Gamma and Laguerre filters , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[58]  Jose C. Principe,et al.  Neural and adaptive systems , 2000 .

[59]  Russell H. Lambert,et al.  Blind separation of multiple speakers in a multipath environment , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[60]  C. Fancourt,et al.  The coherence function in blind source separation of convolutive mixtures of non-stationary signals , 2001, Neural Networks for Signal Processing XI: Proceedings of the 2001 IEEE Signal Processing Society Workshop (IEEE Cat. No.01TH8584).

[61]  Deniz Erdoğmuş,et al.  Blind source separation using Renyi's mutual information , 2001, IEEE Signal Processing Letters.

[62]  Dorothea Kolossa,et al.  REAL TIME SEPARATION OF CONVOLUTIVE MIXTURES , 2001 .

[63]  Noboru Ohnishi,et al.  Real world blind separation of convolved speech signals , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[64]  Nobuhiko Kitawaki,et al.  Combined approach of array processing and independent component analysis for blind separation of acoustic signals , 2003, IEEE Trans. Speech Audio Process..

[65]  Nathalie Delfosse,et al.  Adaptive blind separation of convolutive mixtures , 1996, Conference Record of The Twenty-Ninth Asilomar Conference on Signals, Systems and Computers.

[66]  Luis Castedo,et al.  SEPARATION OF CONVOLUTIVE MIXTURES OF TEMPORALLY-WHITE SIGNALS: A NOVEL FREQUENCY-DOMAIN APPROACH , 2001 .

[67]  Paris Smaragdis,et al.  Blind separation of convolved mixtures in the frequency domain , 1998, Neurocomputing.

[68]  Ehud Weinstein,et al.  Criteria for multichannel signal separation , 1994, IEEE Trans. Signal Process..

[69]  Marc Moonen,et al.  Deterministic blind modulation-induced source separation for digital wireless communications , 2001, IEEE Trans. Signal Process..

[70]  Deniz Erdoğmuş,et al.  Online entropy manipulation: stochastic information gradient , 2003, IEEE Signal Processing Letters.

[71]  Dennis R. Morgan,et al.  A multiresolution approach to blind separation of speech signals in a reverberant environment , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[72]  Andrzej Cichocki,et al.  Relationships between instantaneous blind source separation and multichannel blind deconvolution , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[73]  Eric Moulines,et al.  Subspace methods for the blind identification of multichannel FIR filters , 1995, IEEE Trans. Signal Process..

[74]  John C. Platt,et al.  Networks for the Separation of Sources that Are Superimposed and Delayed , 1991, NIPS.

[75]  Pierre Comon,et al.  BLIND SEPARATION OF CONVOLUTIVE MIXTURES A CONTRAST-BASED JOINT DIAGONALIZATION APPROACH , 2001 .

[76]  George S. Moschytz,et al.  An FFT-based algorithm for multichannel blind deconvolution , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[77]  Birger Kollmeier,et al.  Amplitude Modulation Decorrelation For Convolutive Blind Source Separation , 2000 .

[78]  Yannick Deville,et al.  Self-adaptive separation of convolutively mixed signals with a recursive structure. Part II: Theoretical extensions and application to synthetic and real signals , 1999, Signal Process..

[79]  Koutras.  CONTINUOUS SPEECH RECOGNITION IN A MULTI-SIMULTANEOUS-SPEAKER ENVIRONMENT USING DECORRELATION FILTERING IN THE FREQUENCY DOMAIN , 1998 .

[80]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[81]  A. Belouchrani,et al.  Jacobi-like algorithm for blind signal separation of convolutive mixtures , 2001 .

[82]  James P. Reilly,et al.  Blind signal separation for convolutive mixing environments using spatial-temporal processing , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[83]  B. Kollmeier,et al.  Convolutive blind source separation of speech signals based on amplitude modulation decorrelation , 2000 .

[84]  Hagai Attias,et al.  Blind Source Separation and Deconvolution: The Dynamic Component Analysis Algorithm , 1998, Neural Computation.

[85]  Ehud Weinstein,et al.  Multichannel signal separation: methods and analysis , 1996, IEEE Trans. Signal Process..

[86]  Paris Smaragdis,et al.  Efficient blind separation of convolved sound mixtures , 1997, Proceedings of 1997 Workshop on Applications of Signal Processing to Audio and Acoustics.

[87]  É. Moulines,et al.  Subspace method for blind identification of multichannel FIR systems in noise field with unknown spatial covariance , 1997, IEEE Signal Processing Letters.

[88]  Takayuki Nagai,et al.  Frequency domain multi-channel speech separation and its applications , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[89]  Allan Kardec Barros,et al.  Real world blind separation of convolved non-stationary signals , 1999 .

[90]  N. Charkani,et al.  Stability analysis and optimization of time-domain convolutive source separation algorithms , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[91]  W. Hachem,et al.  ON THE IDENTIFICATION OF CERTAIN NOISY FIR CONVOLUTIVE MIXTURES , 1996 .

[92]  Dinh-Tuan Pham,et al.  Mutual information approach to blind separation of stationary sources , 2002, IEEE Trans. Inf. Theory.

[93]  Saeid Sanei,et al.  Penalty function-based joint diagonalization approach for convolutive blind separation of nonstationary sources , 2005, IEEE Transactions on Signal Processing.

[94]  Jose C. Principe,et al.  Simultaneous Diagonalization in the Frequency Domain (SDIF) for Source Separation , 2000 .

[95]  Lucas C. Parra,et al.  On-line Convolutive Blind Source Separation of Non-Stationary Signals , 2000, J. VLSI Signal Process..

[96]  Yannick Deville,et al.  A convolutive source separation method with self-optimizing non-linearities , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[97]  Roland Gautier,et al.  Blind separation of convolutive mixtures using second and fourth order moments , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[98]  Dirk Van Compernolle,et al.  Signal separation by symmetric adaptive decorrelation: stability, convergence, and uniqueness , 1995, IEEE Trans. Signal Process..

[99]  Deniz Erdogmus,et al.  Sequential Feature Extraction Using Information-Theoretic Learning , 2001 .

[100]  Ruey-Wen Liu,et al.  Blind Deconvolution Algorithms for MIMO-FIR Systems Driven by Fourth-order Colored Signals , 2001 .

[101]  Barak A. Pearlmutter,et al.  Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA , 1996, NIPS.

[102]  E. S. Warner,et al.  Single-channel blind signal separation of filtered MPSK signals , 2003 .

[103]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[104]  Philippe Loubaton,et al.  On the convolutive mixture source separation by the decorrelation approach , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[105]  Athina P. Petropulu,et al.  Frequency-domain contrast functions for separation of convolutive mixtures , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[106]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis of Complex Valued Signals , 2000, Int. J. Neural Syst..

[107]  Scott C. Douglas,et al.  Adaptive paraunitary filter banks for contrast-based multichannel blind deconvolution , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).