Least-Squares Methods for Blind Source Separation Based on Nonlinear PCA

In standard blind source separation, one tries to extract unknown source signals from their instantaneous linear mixtures by using a minimum of a priori information. We have recently shown that certain nonlinear extensions of principal component type neural algorithms can be successfully applied to this problem. In this paper, we show that a nonlinear PCA criterion can be minimized using least-squares approaches, leading to computationally efficient and fast converging algorithms. Several versions of this approach are developed and studied, some of which can be regarded as neural learning algorithms. A connection to the nonlinear PCA subspace rule is also shown. Experimental results are given, showing that the least-squares methods usually converge clearly faster than stochastic gradient algorithms in blind separation problems.

[1]  Colin Fyfe,et al.  Stochastic ICA Contrast Maximisation Using Oja's Nonlinear PCA Algorithm , 1997, Int. J. Neural Syst..

[2]  J. Karhunen,et al.  A bigradient optimization approach for robust PCA, MCA, and source separation , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[3]  Andrzej Cichocki,et al.  Robust neural networks with on-line learning for blind identification and blind separation of sources , 1996 .

[4]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[5]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[6]  Eric Moreau,et al.  High order contrasts for self-adaptive source separation criteria for complex source separation , 1996 .

[7]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[8]  G. Deco,et al.  An Information-Theoretic Approach to Neural Computing , 1997, Perspectives in Neural Computing.

[9]  Gary S. Wasserman,et al.  Extensions of principal component analysis for nonlinear feature extraction , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[10]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[11]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[12]  Simon Haykin,et al.  Adaptive filter theory (2nd ed.) , 1991 .

[13]  Juha Karhunen,et al.  Blind source separation using least-squares type adaptive algorithms , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[14]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[15]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[16]  Bin Yang,et al.  Asymptotic convergence analysis of the projection approximation subspace tracking algorithms , 1996, Signal Process..

[17]  Erkki Oja,et al.  The nonlinear PCA criterion in blind source separation: Relations with other approaches , 1998, Neurocomputing.

[18]  Juha Karhunen,et al.  Neural approaches to independent component analysis and source separation , 1996, ESANN.

[19]  Fa-Long Luo,et al.  Applied neural networks for signal processing , 1997 .

[20]  Mahmood R. Azimi-Sadjadi,et al.  Principal component extraction using recursive least squares learning , 1995, IEEE Trans. Neural Networks.

[21]  J. Mendel.  Lessons in Estimation Theory for Signal Processing, Communications, and Control , 1995 .

[22]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[23]  R. Lambert.  Multichannel blind deconvolution: FIR matrix algebra and separation of multipath mixtures , 1996 .

[24]  Lei Xu,et al.  Theories for unsupervised learning: PCA and its nonlinear extensions , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[25]  F. Palmieri,et al.  Hebbian learning and self-association in nonlinear neural networks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[26]  Erkki Oja,et al.  Independent component analysis by general nonlinear Hebbian-like learning rules , 1998, Signal Process..

[27]  P. Pajunen,et al.  Blind source separation and tracking using nonlinear PCA criterion: a least-squares approach , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[28]  Erkki Oja,et al.  Neural Independent Component Analysis - Approaches and Applications , 1998 .

[29]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[31]  P. Pajunen,et al.  Hierarchic Nonlinear PCA Algorithms for Neural Blind Source Separation , 1996 .

[32]  Juha Karhunen,et al.  Generalizations of principal component analysis, optimization problems, and neural networks , 1995, Neural Networks.

[33]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[34]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[35]  Juha Karhunen,et al.  On Neural Blind Separation with Noise Suppression and Redundancy Reduction , 1997, Int. J. Neural Syst..

[36]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.

[37]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[38]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[39]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[40]  Jie Zhu,et al.  Self-association and Hebbian learning in linear neural networks , 1995, IEEE Trans. Neural Networks.

[41]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..

[42]  Erkki Oja,et al.  Applications of neural blind separation to signal and image processing , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[43]  Terence D. Sanger,et al.  An Optimality Principle for Unsupervised Learning , 1988, NIPS.

[44]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.