Survey of sparse and non‐sparse methods in source separation

Source separation arises in a variety of signal processing applications, ranging from speech processing to medical image analysis. The separation of a superposition of multiple signals is accomplished by taking into account the structure of the mixing process and by making assumptions about the sources. When the information about the mixing process and sources is limited, the problem is called ‘blind’. By assuming that the sources can be represented sparsely in a given basis, recent research has demonstrated that solutions to previously problematic blind source separation problems can be obtained. In some cases, solutions are possible to problems intractable by previous non‐sparse methods. Indeed, sparse methods provide a powerful approach to the separation of linear mixtures of independent data. This paper surveys the recent arrival of sparse blind source separation methods and the previously existing non‐sparse methods, providing insights and appropriate hooks into theliterature along the way. © 2005 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 15;18–33;2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.20035

[1]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[2]  B. Widrow,et al.  Adaptive noise cancelling: Principles and applications , 1975 .

[3]  M. DeWeese,et al.  Binary Spiking in Auditory Cortex , 2003, The Journal of Neuroscience.

[4]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[5]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[6]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[7]  John C. Platt,et al.  Networks for the Separation of Sources that Are Superimposed and Delayed , 1991, NIPS.

[8]  Ralph Linsker,et al.  An Application of the Principle of Maximum Information Preservation to Linear Systems , 1988, NIPS.

[9]  Barak A. Pearlmutter,et al.  A Context-Sensitive Generalization of ICA , 1996 .

[10]  Richard M. Everson,et al.  Independent Component Analysis: Principles and Practice , 2001 .

[11]  Paris Smaragdis,et al.  Blind separation of convolved mixtures in the frequency domain , 1998, Neurocomputing.

[12]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[13]  Tzyy-Ping Jung,et al.  Analyzing and Visualizing Single-Trial Event-Related Potentials , 1998, NIPS.

[14]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[15]  Barak A. Pearlmutter,et al.  Progress in blind separation of magnetoencephalographic data , 2003, SPIE Defense + Commercial Sensing.

[16]  S. Rickard,et al.  REAL-TIME TIME-FREQUENCY BASED BLIND SOURCE SEPARATION , 2001 .

[17]  Yishay Mansour,et al.  An Information-Theoretic Analysis of Hard and Soft Assignment Methods for Clustering , 1997, UAI.

[18]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[19]  Yutaka Kaneda,et al.  Sound source segregation based on estimating incident angle of each frequency component of input signals acquired by multiple microphones , 2001 .

[20]  Dan Barry,et al.  Sound Source Separation: Azimuth Discrimination and Resynthesis , 2004 .

[21]  Barak A. Pearlmutter,et al.  Blind Source Separation via Multinode Sparse Representation , 2001, NIPS.

[22]  Barak A. Pearlmutter,et al.  Blind source separation of multichannel neuromagnetic responses , 2000, Neurocomputing.

[23]  Juan K. Lin,et al.  Feature extraction approach to blind source separation , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[24]  Deniz Erdogmus,et al.  Underdetermined blind source separation in a time-varying environment , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[25]  Kari Torkkola,et al.  Blind separation of convolved sources based on information maximization , 1996, Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop.

[26]  M. Hulle.  Clustering approach to square and non-square blind source separation , 1999 .

[27]  Özgür Yilmaz,et al.  Blind separation of disjoint orthogonal signals: demixing N sources from 2 mixtures , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[28]  Tuomas Virtanen,et al.  Sound Source Separation Using Sparse Coding with Temporal Continuity Objective , 2003, ICMC.

[29]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[30]  Henrik Sahlin,et al.  Separation of real-world signals , 1998, Signal Process..

[31]  Barak A. Pearlmutter,et al.  Monaural Source Separation Using Spectral Cues , 2004, ICA.

[32]  D. Ellis,et al.  Speech separation in humans and machines , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[33]  Michael Zibulevsky,et al.  Underdetermined blind source separation using sparse representations , 2001, Signal Process..

[34]  P. Vaidyanathan.  Multirate Systems And Filter Banks , 1992 .

[35]  Simone G. O. Fiori.  Notes on cost functions and estimators for 'Bussgang' adaptive blind equalization , 2002, Eur. Trans. Telecommun..

[36]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[37]  Andreas Ziehe,et al.  Unmixing Hyperspectral Data , 1999, NIPS.

[38]  Russell H. Lambert,et al.  A new method for source separation , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[39]  Mark D. Plumbley.  Algorithms for nonnegative independent component analysis , 2003, IEEE Trans. Neural Networks.

[40]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[41]  Pierre Divenyi.  Speech Separation by Humans and Machines , 2004 .

[42]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[43]  Paris Smaragdis,et al.  Non-negative Matrix Factor Deconvolution; Extraction of Multiple Sound Sources from Monophonic Inputs , 2004, ICA.

[44]  Andreas Ziehe,et al.  Artifact Reduction in Magnetoneurography Based on Time-Delayed Second Order Correlations , 1998 .

[45]  L. Vielva,et al.  UNDERDETERMINED BLIND SOURCE SEPARATION USING A PROBABILISTIC SOURCE SPARSITY MODEL , 2001 .

[46]  G. Darmois,et al.  Analyse générale des liaisons stochastiques: etude particulière de l'analyse factorielle linéaire , 1953 .

[47]  Scott Rickard,et al.  Blind separation of speech mixtures via time-frequency masking , 2004, IEEE Transactions on Signal Processing.

[48]  Peter Földiák,et al.  SPARSE CODING IN THE PRIMATE CORTEX , 2002 .

[49]  R. Lambert.  Multichannel blind deconvolution: FIR matrix algebra and separation of multipath mixtures , 1996 .

[50]  Dinh Tuan Pham,et al.  Separation of a mixture of independent sources through a maximum likelihood approach , 1992 .

[51]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[52]  S Makeig,et al.  Analysis of fMRI data by blind separation into independent spatial components , 1998, Human brain mapping.

[53]  Fabian J. Theis,et al.  A geometric algorithm for overcomplete linear ICA , 2004, Neurocomputing.

[54]  Kari Torkkola,et al.  Blind separation of delayed sources based on information maximization , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[55]  Allan Kardec Barros,et al.  Closely Arranged Directional Microphone for Source Separation: Effectiveness in Reduction of the Number of Taps and Preventing Factors , 2004, ICA.

[56]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[57]  L. Ljung,et al.  A Comment on Leakage in Adaptive Algorithms , 1992 .

[58]  Sam T. Roweis,et al.  One Microphone Source Separation , 2000, NIPS.

[59]  K.P. Kording,et al.  Learning of sparse auditory receptive fields , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[60]  Erkki Oja,et al.  Independent component approach to the analysis of EEG and MEG recordings , 2000, IEEE Transactions on Biomedical Engineering.

[61]  Pau Bofill,et al.  Underdetermined blind separation of delayed sound sources in the frequency domain , 2003, Neurocomputing.

[62]  John G. McWhirter,et al.  Blind signal separation of convolutive mixtures , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[63]  Van Hulle MM.  Kernel-Based Equiprobabilistic Topographic Map Formation. , 1998, Neural computation.

[64]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[65]  Philippe Loubaton,et al.  Second order blind equalization in multiple input multiple output FIR systems: a weighted least squares approach , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[66]  Adel Belouchrani Jean-Fran,et al.  Maximum likelihood source separation for discrete sources , 2007 .

[67]  Julian J. Bussgang,et al.  Crosscorrelation functions of amplitude-distorted gaussian signals , 1952 .

[68]  Yoram Baram,et al.  Multidimensional density shaping by sigmoids , 1996, IEEE Trans. Neural Networks.

[69]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[70]  Andrew D. Back,et al.  A First Application of Independent Component Analysis to Extracting Structure from Stock Returns , 1997, Int. J. Neural Syst..

[71]  Heung-Kyu Lee,et al.  BSS: a new approach for watermark attack , 2002, Fourth International Symposium on Multimedia Software Engineering, 2002. Proceedings..

[72]  Aapo Hyvärinen,et al.  Survey on Independent Component Analysis , 1999 .

[73]  Changkyu Choi.  REAL-TIME BINAURAL BLIND SOURCE SEPARATION , 2003 .

[74]  Albert S. Bregman,et al.  The Auditory Scene. (Book Reviews: Auditory Scene Analysis. The Perceptual Organization of Sound.) , 1990 .

[75]  Francisco Castells,et al.  Convolutive Acoustic Mixtures Approximation to an Instantaneous Model Using a Stereo Boundary Microphone Configuration , 2004, ICA.

[76]  Barak A. Pearlmutter,et al.  Soft-LOST: EM on a Mixture of Oriented Lines , 2004, ICA.

[77]  Jean-Francois Cardoso,et al.  INDEPENDENT COMPONENT ANALYSIS OF THE COSMIC MICROWAVE BACKGROUND , 2003 .

[78]  Russell H. Lambert,et al.  Blind separation of multiple speakers in a multipath environment , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[79]  Birger Kollmeier,et al.  Adaptive separation of acoustic sources for anechoic conditions: A constrained frequency domain approach , 2003, Speech Commun..

[80]  Lucas C. Parra,et al.  Blind Source Separation via Generalized Eigenvalue Decomposition , 2003, J. Mach. Learn. Res..

[81]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[82]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[83]  Mark D. Plumbley.  Algorithms for Non-Negative Independent Component Analysis , 2002 .

[84]  Barak A. Pearlmutter,et al.  Hard-LOST: modified k-means for oriented lines , 2004 .

[85]  Jean-Francois Cardoso,et al.  Eigen-structure of the fourth-order cumulant tensor with application to the blind source separation problem , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[86]  E. C. Cherry.  Some Experiments on the Recognition of Speech, with One and with Two Ears , 1953 .

[87]  Özgür Yilmaz,et al.  On the approximate W-disjoint orthogonality of speech , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[88]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[89]  Atsuyoshi Nakamura,et al.  On the Minimum l1-Norm Signal Recovery in Underdetermined Source Separation , 2004, ICA.

[90]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[91]  Arie Yeredor,et al.  BLIND SOURCE SEPARATION WITH PURE DELAY MIXTURES , 2001 .

[92]  Jean-Francois Cardoso,et al.  Blind signal separation: statistical principles , 1998, Proc. IEEE.

[93]  Terrence J. Sejnowski,et al.  Blind source separation of more sources than mixtures using overcomplete representations , 1999, IEEE Signal Processing Letters.

[94]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[95]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[96]  Barak A. Pearlmutter,et al.  Blind Source Separation by Sparse Decomposition in a Signal Dictionary , 2001, Neural Computation.

[97]  Andreas Ziehe,et al.  An approach to blind source separation based on temporal structure of speech signals , 2001, Neurocomputing.

[98]  Andreas Ziehe,et al.  Independent component analysis of noninvasively recorded cortical magnetic DC-fields in humans , 2000, IEEE Transactions on Biomedical Engineering.

[99]  Hiroaki Kitano,et al.  Real-time sound source localization and separation for robot audition , 2002, INTERSPEECH.

[100]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[101]  Roland Gautier,et al.  Blind separation of convolutive mixtures using second and fourth order moments , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[102]  V. Michael Bove,et al.  Blind Separation Of Real World Audio Signals Using Overdetermined Mixtures , 1999 .

[103]  Christian Jutten,et al.  Space or time adaptive signal processing by neural network models , 1987 .

[104]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .