Cross-correlation neural network models for the smallest singular component of general matrix

Abstract In this paper, we provide the theoretical foundation for a novel neural model to solve the smallest singular component of general matrix, on the basis of an extension of the Hebbian rule and a modification of cross-coupled Hebbian rule. This model can efficiently extract the singular-value component of the cross-correlation matrix of two stochastic signals. By Lasalle’s invariance principle and Lyapunov’s indirect method, we study the global asymptotic convergence of the networks to the first singular vectors of the cross-correlation matrix or non-squares matrix. A comparative study on the related neural networks shows that this neural network is efficient for computing the smallest singular component of general matrix. The novel model may have useful applications in solving the total least-squares problems in adaptive signal processing and image compressing.

[1]  Sun-Yuan Kung,et al.  A neural network learning algorithm for adaptive principal component extraction (APEX) , 1990, International Conference on Acoustics, Speech, and Signal Processing.

[2]  Hong Chen,et al.  An on-line unsupervised learning machine for adaptive feature extraction , 1993, 1993 IEEE International Symposium on Circuits and Systems.

[3]  Sun-Yuan Kung,et al.  Cross-correlation neural network models , 1994, IEEE Trans. Signal Process..

[4]  Ralph Linsker,et al.  Self-organization in a perceptual network , 1988, Computer.

[5]  Terence D. Sanger,et al.  An Optimality Principle for Unsupervised Learning , 1988, NIPS.

[6]  J. Hale,et al.  Ordinary Differential Equations , 2019, Fundamentals of Numerical Mathematics for Physicists and Engineers.

[7]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[8]  G. Pflug.  Stochastic Approximation Methods for Constrained and Unconstrained Systems - Kushner, HJ.; Clark, D.S. , 1980 .

[9]  J. P. Lasalle.  The stability of dynamical systems , 1976 .

[10]  Kurt Hornik,et al.  Convergence analysis of local feature extraction algorithms , 1992, Neural Networks.

[11]  Terence D. Sanger,et al.  Optimal unsupervised learning in a single-layer linear feedforward neural network , 1989, Neural Networks.

[12]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[13]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[14]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[15]  J. Rubner,et al.  A Self-Organizing Network for Principal-Component Analysis , 1989 .

[16]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[17]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.