Optimization criteria and nonlinear PCA neural networks

In this paper, the relationships of various nonlinear extensions of principal component analysis (PCA) to optimization are considered. Standard PCA arises as an optimal solution to several different information representation problems. It is claimed that basically this follows from the fact that PCA solution utilizes second-order statistics only. If the optimization problems are generalized for nonquadratic criteria so that higher order statistics are taken into account, their solutions will in general be different. The solutions define, in a natural way, several nonlinear extensions of PCA and give a solid foundation to them. The respective gradient type neural algorithms are discussed.<<ETX>>

[1]  T. W. Anderson,et al.  An Introduction to Multivariate Statistical Analysis , 1959 .

[2]  Harold W. Sorenson,et al.  Parameter estimation: Principles and problems , 1980 .

[3]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[4]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[5]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[6]  Ralph Linsker,et al.  Local Synaptic Learning Rules Suffice to Maximize Mutual Information in a Linear Network , 1992, Neural Computation.

[7]  Gilles Burel,et al.  Blind separation of sources: A nonlinear neural algorithm , 1992, Neural Networks.

[8]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[9]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[10]  Andrzej Cichocki,et al.  Robust estimation of principal components by using neural network learning algorithms , 1993 .

[11]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[12]  Todd K. Leen,et al.  Fast nonlinear dimension reduction , 1993, IEEE International Conference on Neural Networks.

[13]  Mark D. Plumbley.  Efficient information transfer and anti-Hebbian neural networks , 1993, Neural Networks.

[14]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.