Unified and Self-Stabilized Parallel Algorithm for Multiple Generalized Eigenpairs Extraction

Generalized eigenvalue decomposition has many advantages when it is applied in modern signal processing. Compared with other methods, neural network model-based algorithms provide an efficient way to solve such problems online. Generalized feature extraction algorithms based on neural network models have been described in the literature. However, the majority of the existing algorithms can only extract the principal generalized eigenvector(s) or eigensubspace. To extract principal and minor generalized eigenvectors from two vector sequences, in this paper, two different information criteria are proposed, and a unified algorithm for the extraction of multiple components in a parallel way by simply altering the sign is derived based on these information criteria, which is feasible for generalized principal and minor component analysis. Moreover, all the corresponding principal and minor generalized eigenvalues can be extracted simultaneously because the desired equilibrium point depends on these values. Thus, the proposed algorithm can perform multiple generalized eigenpair extraction. The proposed algorithm possesses four properties: unification, self-stability, parallel extraction and generalized eigenpair extraction, that few of the existing algorithms can encompass. The global convergence and self-stability property of the proposed algorithm are proved through the Lyapunov method and ordinary differential equation method, respectively. The proposed algorithm has a fast convergence speed, high precision and strong tracking ability. Finally, numerical examples and applications are explored to further demonstrate the efficiency of the proposed algorithm.

[1]  Wang Shougen,et al.  An algorithm for Ax = lBx with symmetric and positive-definite A and B , 1991 .

[2]  Steve Bartelmaos,et al.  Fast adaptive algorithms for minor component analysis using Householder transformation , 2011, Digit. Signal Process..

[3]  Mark D. Plumbley Lyapunov functions for convergence of principal component algorithms , 1995, Neural Networks.

[4]  Haifeng Chen,et al.  Failure Detection in Large-Scale Internet Services by Principal Subspace Mapping , 2007, IEEE Transactions on Knowledge and Data Engineering.

[5]  Xiangyu Kong,et al.  A Novel Unified and Self-Stabilizing Algorithm for Generalized Eigenpairs Extraction , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Ralf Möller,et al.  A Self-Stabilizing Learning Rule for Minor Component Analysis , 2004, Int. J. Neural Syst..

[7]  Jian Yang,et al.  Weighted Rule Based Adaptive Algorithm for Simultaneously Extracting Generalized Eigenvectors , 2011, IEEE Transactions on Neural Networks.

[8]  Zheng Bao,et al.  Adaptive minor component extraction with modular structure , 2001, IEEE Trans. Signal Process..

[9]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..

[10]  D. R. Dennis R. Morgan,et al.  Adaptive algorithms for solving generalized eigenvalue signal enhancement problems , 2004, Signal Process..

[11]  Da-Zheng Feng,et al.  Adaptive Quasi-Newton Algorithm for Source Extraction via CCA Approach , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Xiangyu Kong,et al.  A Generalized Information Criterion for Generalized Minor Component Extraction , 2017, IEEE Transactions on Signal Processing.

[13]  George Mathew,et al.  A quasi-Newton adaptive algorithm for generalized symmetric eigenvalue problem , 1996, IEEE Trans. Signal Process..

[14]  Jian Yang,et al.  Weighted non-linear criterion-based adaptive generalised eigendecomposition , 2013, IET Signal Process..

[15]  B. Parlett The Symmetric Eigenvalue Problem , 1981 .

[16]  José Carlos Príncipe,et al.  Fast RLS-Like Algorithm for Generalized Eigendecomposition and its Applications , 2004, J. VLSI Signal Process..

[17]  Jian Yang,et al.  Fast Adaptive Extraction Algorithm for Multiple Principal Generalized Eigenvectors , 2013, Int. J. Intell. Syst..

[18]  Chongzhao Han,et al.  On the Discrete-Time Dynamics of a Class of Self-Stabilizing MCA Extraction Algorithms , 2010, IEEE Transactions on Neural Networks.

[19]  Konstantinos N. Plataniotis,et al.  Face recognition using LDA-based algorithms , 2003, IEEE Trans. Neural Networks.

[20]  Zhang Yi,et al.  A unified learning algorithm to extract principal and minor components , 2009, Digit. Signal Process..

[21]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[22]  Chongzhao Han,et al.  A Dual Purpose Principal and Minor Subspace Gradient Flow , 2012, IEEE Transactions on Signal Processing.

[23]  Shan Ouyang,et al.  Quasi-Newton algorithm for adaptive minor component extraction , 2002 .

[24]  Isao Yamada,et al.  Adaptive Normalized Quasi-Newton Algorithms for Extraction of Generalized Eigen-Pairs and Their Convergence Analysis , 2013, IEEE Transactions on Signal Processing.

[25]  Michael D. Zoltowski,et al.  Self-organizing algorithms for generalized eigen-decomposition , 1997, IEEE Trans. Neural Networks.

[26]  Zhansheng Duan,et al.  Adaptive Generalized Eigen-Pairs Extraction Algorithms and Their Convergence Analysis , 2016, IEEE Transactions on Signal Processing.

[27]  Joshua E. S. Socolar,et al.  Nonlinear Dynamical Systems , 2006 .

[28]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[29]  Ralf Möller,et al.  Coupled principal component analysis , 2004, IEEE Transactions on Neural Networks.

[30]  Toshihisa Tanaka,et al.  Fast Generalized Eigenvector Tracking Based on the Power Method , 2009, IEEE Signal Processing Letters.

[31]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[32]  S.Y. Kung,et al.  Adaptive Principal component EXtraction (APEX) and applications , 1994, IEEE Trans. Signal Process..

[33]  Toshihisa Tanaka,et al.  Acoustic Beamforming with Maximum SNR Criterion and Efficient Generalized Eigenvector Tracking , 2014, PCM.

[34]  Mohammed A. Hasan,et al.  Low-rank approximations with applications to principal singular component learning systems , 2008, 2008 47th IEEE Conference on Decision and Control.

[35]  Samir Attallah,et al.  A Fast Adaptive Algorithm for the Generalized Symmetric Eigenvalue Problem , 2008, IEEE Signal Processing Letters.

[36]  Jian Yang,et al.  RLS-based adaptive algorithms for generalized eigen-decomposition , 2006, IEEE Transactions on Signal Processing.

[37]  Sabine Van Huffel,et al.  The MCA EXIN neuron for the minor component analysis , 2002, IEEE Trans. Neural Networks.