Principal and Independent Components in Neural Networks - Recent Developments

Nonlinear extensions of one-unit and multi-unit Principal Component Analysis (PCA) neural networks, introduced earlier by the authors, are reviewed. The networks and their nonlinear Hebbian learning rules are related to other signal expansions like Projection Pursuit (PP) and Independent Component Analysis (ICA). Separation results for mixtures of real world signals and images are given.

[1]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[2]  Juha Karhunen,et al.  Adaptive algorithms for estimating eigenvectors of correlation type matrices , 1984, ICASSP.

[3]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[4]  T. Kohonen,et al.  The subspace learning algorithm as a formalism for pattern recognition and neural networks , 1988, IEEE 1988 International Conference on Neural Networks.

[5]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[6]  Y. Chauvin,et al.  Principal component analysis by gradient descent on a constrained linear Hebbian cell , 1989, International 1989 Joint Conference on Neural Networks.

[7]  P. Comon.  Separation Of Stochastic Processes , 1989, Workshop on Higher-Order Spectral Analysis.

[8]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[9]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[10]  John C. Platt,et al.  Networks for the Separation of Sources that Are Superimposed and Delayed , 1991, NIPS.

[11]  Charles W. Therrien,et al.  Discrete Random Signals and Statistical Signal Processing , 1992 .

[12]  Gilles Burel,et al.  Blind separation of sources: A nonlinear neural algorithm , 1992, Neural Networks.

[13]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[14]  Mark D. Plumbley.  A Hebbian/anti-Hebbian network which optimizes information capacity by orthonormalizing the principal subspace , 1993 .

[15]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[16]  Mark D. Plumbley,et al.  Information Theory and Neural Networks , 1993 .

[17]  Eric Moreau,et al.  New self-adaptative algorithms for source separation based on contrast functions , 1993, [1993 Proceedings] IEEE Signal Processing Workshop on Higher-Order Statistics.

[18]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[19]  J. Karhunen.  Optimization criteria and nonlinear PCA neural networks , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[20]  Jean-Francois Cardoso,et al.  Adaptive Source Separation With Uniform Performance , 1994 .

[21]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.

[22]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[23]  Mohamad H. Hassoun,et al.  Nonlinear Hebbian rule: a statistical interpretation , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[24]  J. Karhunen,et al.  Neural Estimation of Basis Vectors in Independent Component Analysis , 1995 .

[25]  Gustavo Deco,et al.  Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures , 1995, Neural Networks.

[26]  S. Klinke,et al.  Exploratory Projection Pursuit , 1995 .

[27]  Terrence J. Sejnowski,et al.  Blind separation and blind deconvolution: an information-theoretic approach , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[28]  Juha Karhunen,et al.  Generalizations of principal component analysis, optimization problems, and neural networks , 1995, Neural Networks.

[29]  Kiyotoshi Matsuoka,et al.  A neural net for blind separation of nonstationary signals , 1995, Neural Networks.

[30]  J. Karhunen,et al.  A bigradient optimization approach for robust PCA, MCA, and source separation , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[31]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..