Temporally correlated source separation using variational Bayesian learning approach

Basic blind source separation (BSS) algorithms did not adopt time information of signals. They assumed that each source was independent and identically distributed (i.i.d.). In the paper, we propose to use time structure and prior information of sources in order to improve separation. Modeling source by generalized autoregressive (GAR) process, we can tackle the problem of temporally correlated source separation using variational Bayesian (VB) learning approach. The advantages of our proposed algorithm are that (i) it makes full use of time structure of sources; (ii) it can separate different type of sources in noisy environment; (iii) it can avoid overfitting in separation. Experimental results demonstrate that our algorithm outperforms VB separation algorithm based on i.i.d. source model and second-order statistical decorrelation algorithm.

[1]  S. Mallat.  A wavelet tour of signal processing , 1998 .

[2]  S. Roberts,et al.  Variational Bayes for non-Gaussian autoregressive models , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[3]  Peter J. W. Rayner,et al.  Digital Audio Restoration: A Statistical Model Based Approach , 1998 .

[4]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[5]  H. Attias.  Independent Component Analysis: ICA, graphical models and variational methods , 2001 .

[6]  Erkki Oja,et al.  Adaptive Algorithm for Blind Separation from Noisy Time-Varying Mixtures , 2001, Neural Computation.

[7]  Juha Karhunen,et al.  An Unsupervised Ensemble Learning Method for Nonlinear Dynamic State-Space Models , 2002, Neural Computation.

[8]  Harri Valpola,et al.  Denoising Source Separation , 2005, J. Mach. Learn. Res..

[9]  Christian Jutten,et al.  Wavelet De-noising for Blind Source Separation in Noisy Mixtures , 2004, ICA.

[10]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[11]  Christopher M. Bishop,et al.  Neural Network for Pattern Recognition , 1995 .

[12]  William D. Penny,et al.  Variational Bayes for generalized autoregressive models , 2002, IEEE Trans. Signal Process..

[13]  Hagai Attias,et al.  Independent Factor Analysis , 1999, Neural Computation.

[14]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[15]  Hagai Attias,et al.  Independent Factor Analysis with Temporally Structured Sources , 1999, NIPS.

[16]  Stephen J. Roberts,et al.  An ensemble learning approach to independent component analysis , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[17]  Byung-Gook Lee,et al.  An EM-based approach for parameter enhancement with an application to speech signals , 1995, Signal Process..

[18]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[19]  J. W. Miskin,et al.  Ensemble Learning for Blind Source Separation , 2001 .

[20]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[21]  S. D. Gray,et al.  Filtering of colored noise for speech enhancement and coding , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[22]  Dinh-Tuan Pham,et al.  Markovian source separation , 2003, IEEE Trans. Signal Process..

[23]  Richard M. Everson,et al.  Independent Component Analysis: Principles and Practice , 2001 .

[24]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[25]  Christian Jutten,et al.  Markovian Source Separation in Post-nonlinear Mixtures , 2004, ICA.

[26]  S. Roberts,et al.  Variational Bayes for 1-dimensional Mixture Models , 2000 .

[27]  A. Cichocki,et al.  Robust whitening procedure in blind source separation context , 2000 .

[28]  Andreas Ziehe,et al.  TDSEP { an e(cid:14)cient algorithm for blind separation using time structure , 1998 .

[29]  Alan V. Oppenheim,et al.  Parameter estimation for autoregressive Gaussian-mixture processes: the EMAX algorithm , 1998, IEEE Trans. Signal Process..

[30]  X. Zhuang,et al.  Gaussian mixture density modeling of non-Gaussian source for autoregressive process , 1995, IEEE Trans. Signal Process..

[31]  S. Roberts,et al.  Bayesian methods for autoregressive models , 2000, Neural Networks for Signal Processing X. Proceedings of the 2000 IEEE Signal Processing Society Workshop (Cat. No.00TH8501).

[32]  Serge Dégerine,et al.  Second-order blind separation of sources based on canonical partial innovations , 2000, IEEE Trans. Signal Process..

[33]  Mohamed Najim,et al.  Speech enhancement as a realisation issue , 2002, Signal Process..