Unified and Coupled Self-Stabilizing Algorithms for Minor and Principal Eigen-pairs Extraction

Neural network algorithms on principal component analysis (PCA) and minor component analysis (MCA) are of importance in signal processing. Unified (dual purpose) algorithm is capable of both PCA and MCA, thus it is valuable for reducing the complexity and the cost of hardware implementations. Coupled algorithm can mitigate the speed-stability problem which exists in most noncoupled algorithms. Though unified algorithm and coupled algorithm have these advantages compared with single purpose algorithm and noncoupled algorithm, respectively, there are only few of unified algorithms and coupled algorithms have been proposed. Moreover, to the best of the authors’ knowledge, there is no algorithm which is both unified and coupled has been proposed. In this paper, based on a novel information criterion, we propose two self-stabilizing algorithms which are both unified and coupled. In the derivation of our algorithms, it is easier to obtain the results compared with traditional methods, because it is not needed to calculate the inverse Hessian matrix. Experiment results show that the proposed algorithms perform better than existing coupled algorithms and unified algorithms.

[1]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[2]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[3]  Hui Wang,et al.  Neural Network Implementations for PCA and Its Extensions , 2012 .

[4]  Tianping Chen,et al.  Modified Oja's Algorithms For Principal Subspace and Minor Subspace extraction , 1997, Neural Processing Letters.

[5]  Xiangyu Kong,et al.  A self-stabilizing MSA algorithm in high-dimension data stream , 2010, Neural Networks.

[6]  Claude Delpha,et al.  Incipient fault detection and diagnosis based on Kullback-Leibler divergence using principal component analysis: Part II , 2015, Signal Process..

[7]  Chongzhao Han,et al.  A Unified Self-Stabilizing Neural Network Algorithm for Principal and Minor Components Extraction , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[8]  Isao Yamada,et al.  A unified convergence analysis of Normalized PAST algorithms for estimating principal and minor components , 2013, Signal Process..

[9]  Ralf Möller,et al.  Coupled principal component analysis , 2004, IEEE Transactions on Neural Networks.

[10]  Nanda Kambhatla,et al.  Dimension Reduction by Local Principal Component Analysis , 1997, Neural Computation.

[11]  D. Diallo,et al.  Faults diagnosis and detection using principal component analysis and Kullback-Leibler divergence , 2012, IECON 2012 - 38th Annual Conference on IEEE Industrial Electronics Society.

[12]  Shan Ouyang,et al.  Quasi-Newton algorithm for adaptive minor component extraction , 2002 .

[13]  Bernd Fritzke,et al.  The LBG-U Method for Vector Quantization – an Improvement over LBG Inspired from Neural Networks , 1997, Neural Processing Letters.

[14]  Konstantinos I. Diamantaras,et al.  Applying PCA neural models for the blind separation of signals , 2009, Neurocomputing.

[15]  Juha Karhunen,et al.  Principal component neural networks — Theory and applications , 1998, Pattern Analysis and Applications.

[16]  Mohammed A. Hasan,et al.  Natural gradient for minor component extraction , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[17]  Chen Tian-ping.  Online Algorithm of Coupled Principal(Minor) Component Analysis , 2006 .

[18]  Shyang Chang,et al.  An adaptive learning algorithm for principal component analysis , 1995, IEEE Trans. Neural Networks.

[19]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[20]  Ralf Möller,et al.  A Self-Stabilizing Learning Rule for Minor Component Analysis , 2004, Int. J. Neural Syst..

[21]  Isao Yamada,et al.  Adaptive Normalized Quasi-Newton Algorithms for Extraction of Generalized Eigen-Pairs and Their Convergence Analysis , 2013, IEEE Transactions on Signal Processing.

[22]  Fa-Long Luo,et al.  A Minor Component Analysis Algorithm , 1997, Neural Networks.

[23]  Zhang Yi,et al.  A stable MCA learning algorithm , 2008, Comput. Math. Appl..

[24]  Zhang Yi,et al.  A neural networks learning algorithm for minor component analysis and its convergence analysis , 2008, Neurocomputing.

[25]  Fa-Long Luo,et al.  A generalized learning algorithm of minor component , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[26]  J. Griffiths.  Adaptive array processing. A tutorial , 1983 .

[27]  Yingbo Hua,et al.  Fast subspace tracking and neural network learning by a novel information criterion , 1998, IEEE Trans. Signal Process..

[28]  Demba Diallo,et al.  Incipient fault detection and diagnosis based on Kullback-Leibler divergence using Principal Component Analysis: Part I , 2014, Signal Process..

[29]  M. Swamy,et al.  Learning algorithm for total least-squares adaptive signal processing , 1992 .

[30]  Richard Klemm,et al.  Adaptive airborne MTI: an auxiliary channel approach , 1987 .

[31]  Jian Yang,et al.  Fast Adaptive Extraction Algorithm for Multiple Principal Generalized Eigenvectors , 2013, Int. J. Intell. Syst..

[32]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[33]  Lindsay I. Smith,et al.  A tutorial on Principal Components Analysis , 2002 .

[34]  Gene H. Golub,et al.  Matrix computations , 1983 .

[35]  Zheng Bao,et al.  Adaptive minor component extraction with modular structure , 2001, IEEE Trans. Signal Process..

[36]  H. Abdi,et al.  Principal component analysis , 2010 .

[37]  Zhang Yi,et al.  A globally convergent learning algorithm for PCA neural networks , 2005, Neural Computing & Applications.

[38]  Mohammed A. Hasan,et al.  Self-Normalizing Dual Systems for Minor and Principal Component Extraction , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[39]  Sabine Van Huffel,et al.  The MCA EXIN neuron for the minor component analysis , 2002, IEEE Trans. Neural Networks.

[40]  Chongzhao Han,et al.  Convergence analysis of deterministic discrete time system of a unified self-stabilizing algorithm for PCA and MCA , 2012, Neural Networks.

[41]  Jian Yang,et al.  Dynamic transition embedding for image feature extraction and recognition , 2011, Neural Computing and Applications.

[42]  Rong Wang,et al.  Interference cancellation in GPS receiver using noise subspace tracking algorithm , 2011, Signal Process..

[43]  Qin Lin,et al.  A unified algorithm for principal and minor components extraction , 1998, Neural Networks.

[44]  Zhang Yi,et al.  A unified learning algorithm to extract principal and minor components , 2009, Digit. Signal Process..

[45]  Chongzhao Han,et al.  A Dual Purpose Principal and Minor Subspace Gradient Flow , 2012, IEEE Transactions on Signal Processing.

[46]  Mao Ye,et al.  A Class of Self-Stabilizing MCA Learning Algorithms , 2006, IEEE Transactions on Neural Networks.

[47]  Iven M. Y. Mareels,et al.  A dual purpose principal and minor component flow , 2005, Syst. Control. Lett..

[48]  Shun-ichi Amari,et al.  Unified stabilization approach to principal and minor components extraction algorithms , 2001, Neural Networks.

[49]  Zhang Yi,et al.  A Family of Fuzzy Learning Algorithms for Robust Principal Component Analysis Neural Networks , 2010, IEEE Transactions on Fuzzy Systems.

[50]  Chongzhao Han,et al.  On the Discrete-Time Dynamics of a Class of Self-Stabilizing MCA Extraction Algorithms , 2010, IEEE Transactions on Neural Networks.

[51]  M.N.S. Swamy,et al.  Neural Networks and Statistical Learning , 2013 .

[52]  Isao Yamada,et al.  An adaptive extraction of generalized eigensubspace by using exact nested orthogonal complement structure , 2013, Multidimens. Syst. Signal Process..