On Independent Component Analysis for Multimedia Signals

We discuss the independent component problem within a context of multimedia applications. The literature o ers several independent component analysis schemes which can be applied in this context, and each have its own trade-o between exibility, complexity and computational e ort. The speci c applications investigated in this chapter comprise modeling of speech/sound, images, and text data. 1 Background Blind reconstruction of statistically independent source signals from linear mixtures is relevant to many signal processing contexts [1, 6, 8, 9, 22, 24, 36]. With reference to Principal Component Analysis the problem is often referred to as Independent Component Analysis (ICA). The source separation problem can be formulated as a likelihood formulation, see e.g., [7, 32, 35, 37]. The likelihood formulation is attractive for several reasons. First, it allows a principled discussion of the inevitable priors implicit in any separation scheme. The prior distribution of the source signals can take many forms and factorizes in the source index expressing the fact that we look for independent sources. Secondly, the likelihood approach allows for direct adaptation of the plethora of powerful schemes for parameter optimization, regularization, and evaluation of supervised learning algorithms. Finally, for the case of linear mixtures There are a number of very useful ICA Web pages providing links to theoretical analysis, implementations and applications. Follow links from the page http://eivind.imm.dtu.dk/ staff/lkhansen/ica.html

[1]  Peter W. Foltz,et al.  Learning Human-like Knowledge by Singular Value Decomposition: A Progress Report , 1997, NIPS.

[2]  G. Deco,et al.  An Information-Theoretic Approach to Neural Computing , 1997, Perspectives in Neural Computing.

[3]  Frank Ehlers,et al.  Blind separation of convolutive mixtures and an application in automatic speech recognition in a noisy environment , 1997, IEEE Trans. Signal Process..

[4]  A. J. Bell,et al.  A Unifying Information-Theoretic Framework for Independent Component Analysis , 2000 .

[5]  Lars Kai Hansen,et al.  Blind Separation of Noisy Mixtures , 1998 .

[6]  Michael S. Lewicki,et al.  ICA mixture models for image processing , 1999 .

[7]  H. Attias,et al.  Blind source separation and deconvolution by dynamic component analysis , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[8]  Erkki Oja Helsinki.  PCA, ICA, and Nonlinear Hebbian Learning , 1995 .

[9]  Lars Kai Hansen,et al.  Unsupervised learning and generalization , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[10]  Eric Moulines,et al.  Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[11]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[12]  Schuster,et al.  Separation of a mixture of independent signals using time delayed correlations. , 1994, Physical review letters.

[13]  Erkki Oja,et al.  Image feature extraction by sparse coding and independent component analysis , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[14]  Erkki Oja,et al.  Image Feature Extraction Using Independent Component Analysis , 1996 .

[15]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[16]  B. Olshausen.  Learning linear, sparse, factorial codes , 1996 .

[17]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[18]  Terrence J. Sejnowski,et al.  Blind source separation of more sources than mixtures using overcomplete representations , 1999, IEEE Signal Processing Letters.

[19]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[20]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[21]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[22]  Adel Belouchrani,et al.  Maximum Likelihood Source Separation By the Expectation-Maximization Technique: Deterministic and St , 1995 .

[23]  Hagai Attias,et al.  Blind Source Separation and Deconvolution: The Dynamic Component Analysis Algorithm , 1998, Neural Computation.

[24]  Susan T. Dumais,et al.  Richard Harshman Indexing by Latent Semantic Analysis , 1990 .

[25]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[26]  Paul A. Viola,et al.  Restructuring Sparse High Dimensional Data for Effective Retrieval , 1998, NIPS.

[27]  L. K. Hansen,et al.  Generalizable Patterns in Neuroimaging: How Many Principal Components? , 1999, NeuroImage.

[28]  Barak A. Pearlmutter,et al.  Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA , 1996, NIPS.