Feedforward neural networks for principal components extraction

Abstract In recent times, an upsurge of interest in the study of artificial neural networks apt to computing a principal component extraction has been observed. The present work is devoted to the description and performance analysis (by means of computer simulations) of some neural networks of such a kind. The main conclusion reached is that, while the first principal component is almost always efficiently obtained, lesser components tends to be sloppily approximated. All the nets considered here share the interesting feature of being endowed with a feedforward connectivity, together with an Hebbian law of synaptic weights’ adjustment. Their potential usefulness as modular tools, to be inserted in more complex models of psychological and neurological functions has been suggested. There is, however, for the time being no clear evidence supporting any real biological implementation of these simple computational architectures.

[1]  Sandro Nicole,et al.  Effects of IAHP on pattern recall and the synchronization of firing , 1992 .

[2]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[3]  Juha Karhunen,et al.  Representation and separation of signals using nonlinear PCA type learning , 1994, Neural Networks.

[4]  G. Golub,et al.  Tracking a few extreme singular values and vectors in signal processing , 1990, Proc. IEEE.

[5]  I. Jolliffe.  Principal Component Analysis , 2002 .

[6]  Juha Karhunen,et al.  Generalizations of principal component analysis, optimization problems, and neural networks , 1995, Neural Networks.

[7]  J. Rubner,et al.  A Self-Organizing Network for Principal-Component Analysis , 1989 .

[8]  Zhaoping Li,et al.  Understanding Retinal Color Coding from First Principles , 1992, Neural Computation.

[9]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[10]  Erkki Oja,et al.  Neural Networks, Principal Components, and Subspaces , 1989, Int. J. Neural Syst..

[11]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[12]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[13]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[14]  Anil K. Jain,et al.  Artificial neural networks and statistical pattern recognition : old and new connections , 1991 .

[15]  L. Shen,et al.  Linear Algebra , 1968 .

[16]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[17]  J. Ariel Sirat,et al.  A Fast Neural Algorithm for Principal Component Analysis and Singular Value Decomposition , 1991, Int. J. Neural Syst..

[18]  Kurt Hornik,et al.  Convergence analysis of local feature extraction algorithms , 1992, Neural Networks.

[19]  Anil K. Jain,et al.  Artificial Neural Networks and Statistical Pattern Recognition , 1991 .

[20]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[21]  M. Tatsuoka.  Multivariate Analysis Techniques for Educational and Psychological Research , 1971 .

[22]  James A. Anderson,et al.  Neurocomputing (vol. 2): directions for research , 1990 .

[23]  P. Foldiak,et al.  Adaptive network for optimal linear feature extraction , 1989, International 1989 Joint Conference on Neural Networks.

[24]  S. Kaplan.  The Physiology of Thought , 1950 .

[25]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[26]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[27]  Leslie S. Smith,et al.  The principal components of natural images , 1992 .

[28]  Erkki Oja,et al.  Subspace methods of pattern recognition , 1983 .

[29]  Stephen D. Collins.  Neurocomputing 2 , 1992, Neurology.

[30]  R. Baddeley,et al.  A statistical analysis of natural images matches psychophysically derived orientation tuning curves , 1991, Proceedings of the Royal Society of London. Series B: Biological Sciences.