An Adaptive Self-Stabilizing Algorithm for Minor Generalized Eigenvector Extraction and Its Convergence Analysis

Generalized eigendecomposition, which extracts the generalized eigenvector from a matrix pencil, is a powerful tool and has been widely used in many fields, such as data classification and blind source separation. First, to extract the minor generalized eigenvector (MGE), we propose a deterministic discrete-time (DDT) system. Unlike some existing systems, the proposed DDT system does not need to normalize the weight vector in each iteration, since the weight vectors in the proposed DDT system are self-stabilizing. Second, we propose an adaptive algorithm corresponding to the proposed DDT system. Moreover, we study the dynamic behavior and convergence properties of the proposed DDT system and prove that the weight vector must converge to the direction of the MGE of a matrix pencil under some mild conditions. Numerical simulations show that the proposed algorithm has a better performance in terms of convergence speed and estimation accuracy than some existing algorithms. Finally, we conduct two experiments on real data sets to demonstrate its practicability.

[1]  Chongzhao Han,et al.  Convergence analysis of deterministic discrete time system of a unified self-stabilizing algorithm for PCA and MCA , 2012, Neural Networks.

[2]  Shifei Ding,et al.  An overview on nonparallel hyperplane support vector machine algorithms , 2013, Neural Computing and Applications.

[3]  Shun-ichi Amari,et al.  Unified stabilization approach to principal and minor components extraction algorithms , 2001, Neural Networks.

[4]  Karim Abed-Meraim,et al.  Minor subspace tracking using MNS technique , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[5]  Samir Attallah,et al.  A Fast Adaptive Algorithm for the Generalized Symmetric Eigenvalue Problem , 2008, IEEE Signal Processing Letters.

[6]  Angelika Bunse-Gerstner,et al.  An algorithm for the symmetric generalized eigenvalue problem , 1984 .

[7]  José Carlos Príncipe,et al.  Fast RLS-Like Algorithm for Generalized Eigendecomposition and its Applications , 2004, J. VLSI Signal Process..

[8]  Chongzhao Han,et al.  A Unified Self-Stabilizing Neural Network Algorithm for Principal and Minor Components Extraction , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Ngai Wong,et al.  Symmetric tensor decomposition by an iterative eigendecomposition algorithm , 2014, J. Comput. Appl. Math..

[10]  Ralf Möller,et al.  A Self-Stabilizing Learning Rule for Minor Component Analysis , 2004, Int. J. Neural Syst..

[11]  Jian Yang,et al.  Weighted Rule Based Adaptive Algorithm for Simultaneously Extracting Generalized Eigenvectors , 2011, IEEE Transactions on Neural Networks.

[12]  Lijun Liu,et al.  Dual subspace learning via geodesic search on Stiefel manifold , 2014, Int. J. Mach. Learn. Cybern..

[13]  Marcos Raydan,et al.  Residual algorithm for large-scale positive definite generalized eigenvalue problems , 2010, Comput. Optim. Appl..

[14]  Xiangyu Kong,et al.  A Novel Unified and Self-Stabilizing Algorithm for Generalized Eigenpairs Extraction , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Xiangyu Kong,et al.  Convergence Analysis of Möller Algorithm for Estimating Minor Component , 2014, Neural Processing Letters.

[16]  Yuan-Hai Shao,et al.  Improved Generalized Eigenvalue Proximal Support Vector Machine , 2013, IEEE Signal Processing Letters.

[17]  赵仪.  RLS-based adaptive algorithms for generalized eigen-decomposition , 2006 .

[18]  Giansalvo Cirrincione,et al.  Neural-Based Orthogonal Data Fitting: The EXIN Neural Networks , 2010 .

[19]  Michael D. Zoltowski,et al.  Self-organizing algorithms for generalized eigen-decomposition , 1997, IEEE Trans. Neural Networks.

[20]  Pedro J. Zufiria,et al.  On the discrete-time dynamics of the basic Hebbian neural network node , 2002, IEEE Trans. Neural Networks.

[21]  Isao Yamada,et al.  Adaptive Normalized Quasi-Newton Algorithms for Extraction of Generalized Eigen-Pairs and Their Convergence Analysis , 2013, IEEE Transactions on Signal Processing.

[22]  Isao Yamada,et al.  An adaptive extraction of generalized eigensubspace by using exact nested orthogonal complement structure , 2013, Multidimens. Syst. Signal Process..

[23]  Olvi L. Mangasarian,et al.  Multisurface proximal support vector machine classification via generalized eigenvalues , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Jian Yang,et al.  Fast Adaptive Extraction Algorithm for Multiple Principal Generalized Eigenvectors , 2013, Int. J. Intell. Syst..

[25]  Zhang Yi,et al.  A Globally Convergent MC Algorithm With an Adaptive Learning Rate , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Isao Yamada,et al.  Necessary and sufficient conditions for convergence of the DDT systems of the Normalized PAST algorithms , 2014, Signal Process..

[27]  Y. Nakatsukasa,et al.  Perturbation of Partitioned Hermitian Generalized Eigenvalue Problem , 2010 .

[28]  Zhansheng Duan,et al.  Adaptive Generalized Eigen-Pairs Extraction Algorithms and Their Convergence Analysis , 2016, IEEE Transactions on Signal Processing.

[29]  Isao Yamada,et al.  A unified convergence analysis of Normalized PAST algorithms for estimating principal and minor components , 2013, Signal Process..

[30]  Amir Valizadeh,et al.  A constrained optimization approach for an adaptive generalized subspace tracking algorithm , 2010, Comput. Electr. Eng..

[31]  Da-Zheng Feng,et al.  Adaptive Quasi-Newton Algorithm for Source Extraction via CCA Approach , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Jian Yang,et al.  Weighted non-linear criterion-based adaptive generalised eigendecomposition , 2013, IET Signal Process..

[33]  Karim Abed-Meraim,et al.  Low complexity adaptive algorithms for Principal and Minor Component Analysis , 2013, Digit. Signal Process..

[34]  Zhang Yi,et al.  Adaptive multiple minor directions extraction in parallel using a PCA neural network , 2010, Theor. Comput. Sci..

[35]  Mao Ye,et al.  A few online algorithms for extracting minor generalized eigenvectors , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[36]  Toshihisa Tanaka,et al.  Fast Generalized Eigenvector Tracking Based on the Power Method , 2009, IEEE Signal Processing Letters.

[37]  Wang Shougen,et al.  An algorithm for Ax = lBx with symmetric and positive-definite A and B , 1991 .

[38]  Ana Maria Tomé,et al.  The generalized eigendecomposition approach to the blind source separation problem , 2006, Digit. Signal Process..

[39]  Rong Wang,et al.  A Novel Orthonormalization Matrix Based Fast and Stable DPM Algorithm for Principal and Minor Subspace Tracking , 2012, IEEE Transactions on Signal Processing.

[40]  Xiangyu Kong,et al.  Unified and Coupled Self-Stabilizing Algorithms for Minor and Principal Eigen-pairs Extraction , 2017, Neural Processing Letters.

[41]  Gene H. Golub,et al.  Matrix computations , 1983 .