A bigradient optimization approach for robust PCA, MCA, and source separation

The authors earlier derived neural principal or minor component learning algorithms and their robust extensions by optimizing a generalized variance criterion under orthonormality constraints. In this paper, the authors propose an alternative approach, where the stochastic learning algorithm is derived by optimizing two criteria simultaneously. This yields a new bigradient algorithm, which can be used in slightly different forms for PCA, MCA, and their robust extensions in either symmetric (subspace) or hierarchic modes. The algorithm is successfully applied to separation of independent sources from their linear mixture.

[1]  Miss A.O. Penney.  (b) , 1974, The New Yale Book of Quotations.

[2]  Y. Chauvin,et al.  Principal component analysis by gradient descent on a constrained linear Hebbian cell , 1989, International 1989 Joint Conference on Neural Networks.

[3]  P. Comon.  Separation Of Stochastic Processes , 1989, Workshop on Higher-Order Spectral Analysis.

[4]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[5]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[6]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[7]  Andrzej Cichocki,et al.  Robust estimation of principal components by using neural network learning algorithms , 1993 .

[8]  Eric Moreau,et al.  New self-adaptative algorithms for source separation based on contrast functions , 1993, [1993 Proceedings] IEEE Signal Processing Workshop on Higher-Order Statistics.

[9]  J. Karhunen.  Optimization criteria and nonlinear PCA neural networks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[10]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.

[11]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[12]  George Mathew,et al.  Orthogonal eigensubspace estimation using neural networks , 1994, IEEE Trans. Signal Process..

[13]  J. Karhunen,et al.  Neural Estimation of Basis Vectors in Independent Component Analysis , 1995 .

[14]  Alan L. Yuille,et al.  Robust principal component analysis by self-organizing rules based on statistical physics approach , 1995, IEEE Trans. Neural Networks.

[15]  Juha Karhunen,et al.  Generalizations of principal component analysis, optimization problems, and neural networks , 1995, Neural Networks.

[16]  Erkki Oja,et al.  Blind Separation of Sources Using Nonlinear PCA Type Learning Algorithms , 1995 .

[17]  Juha Karhunen,et al.  Nonlinear PCA type approaches for source separation and independent component analysis , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[18]  Erkki Oja,et al.  Robust fitting by nonlinear neural units , 1996, Neural Networks.

[19]  Juha Karhunen,et al.  A Unified Neural Bigradient Algorithm for robust PCA and MCA , 1996, Int. J. Neural Syst..