A solution of speed-stability problem in noncoupled neural network algorithms

A major and common problem, which is the so-called speed-stability problem, exists in most noncoupled neural network based algorithms for principal (PCA) and minor (MCA) component analysis. Coupled PCA or MCA algorithms, in which the principal or minor eigenvector and the corresponding eigenvalue of a covariance matrix are estimated in coupled equations simultaneously, can mitigate the speed-stability problem. Or in other words, coupled algorithms perform better, i.e., converge faster and more stable, than noncoupled algorithms. Until now, a large number of noncoupled learning algorithms have been proposed and analyzed, but unfortunately, there are only few of coupled learning algorithms can be found in existing literatures. To address issue, in this paper, we propose a method of solving the speed-stability problem to improve the convergence speed and stability of noncoupled algorithms, which is actually done by converting the noncoupled algorithms to the coupled ones. Based on the proposed method, we can obtain lots of coupled algorithms from the existing noncoupled algorithms, and all of them do not have the speed-stability problem thus perform better than their noncoupled versions.

[1]  Steve Bartelmaos,et al.  Fast adaptive algorithms for minor component analysis using Householder transformation , 2011, Digit. Signal Process..

[2]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[3]  Fa-Long Luo,et al.  A minor subspace analysis algorithm , 1997, IEEE Trans. Neural Networks.

[4]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[5]  E. Oja,et al.  Independent Component Analysis , 2013 .

[6]  Giuseppe Nisticò,et al.  3D Reconstruction of Coronal Loops by the Principal Component Analysis , 2013, Entropy.

[7]  Zhang Yi,et al.  A Modified Oja–Xu MCA Learning Algorithm and Its Convergence Analysis , 2007, IEEE Transactions on Circuits and Systems II: Express Briefs.

[8]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[9]  Ralf Möller,et al.  Coupled principal component analysis , 2004, IEEE Transactions on Neural Networks.

[10]  Sabine Van Huffel,et al.  The MCA EXIN neuron for the minor component analysis , 2002, IEEE Trans. Neural Networks.

[11]  Peter Filzmoser,et al.  Robust Sparse Principal Component Analysis , 2013, Technometrics.

[12]  S. Amari,et al.  A self-stabilized minor subspace rule , 1998, IEEE Signal Processing Letters.

[13]  Lei Xu.  Least MSE reconstruction for self-organization. II. Further theoretical and experimental studies on one-layer nets , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[14]  Ralf Möller,et al.  A Self-Stabilizing Learning Rule for Minor Component Analysis , 2004, Int. J. Neural Syst..

[15]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[16]  Zheng Bao,et al.  Robust recursive least squares learning algorithm for principal component analysis , 2000, IEEE Trans. Neural Networks Learn. Syst..

[17]  Soura Dasgupta,et al.  Adaptive estimation of eigensubspace , 1995, IEEE Trans. Signal Process..

[18]  Sun-Yuan Kung,et al.  Principal Component Neural Networks: Theory and Applications , 1996 .

[19]  Shyang Chang,et al.  An adaptive learning algorithm for principal component analysis , 1995, IEEE Trans. Neural Networks.

[20]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[21]  Zhang Yi,et al.  A neural networks learning algorithm for minor component analysis and its convergence analysis , 2008, Neurocomputing.

[22]  Gene H. Golub,et al.  Matrix computations , 1983 .

[23]  Zheng Bao,et al.  A neural network learning for adaptively extracting cross-correlation features between two high-dimensional data streams , 2004, IEEE Transactions on Neural Networks.

[24]  Zongming Ma.  Sparse Principal Component Analysis and Iterative Thresholding , 2011, 1112.2432.

[25]  Zheng Bao,et al.  Total least mean squares algorithm , 1998, IEEE Trans. Signal Process..

[26]  Zhang Yi,et al.  Adaptive multiple minor directions extraction in parallel using a PCA neural network , 2010, Theor. Comput. Sci..

[27]  Shan Ouyang,et al.  Quasi-Newton algorithm for adaptive minor component extraction , 2002 .