Neural Networks for Minor Component Analysis

The minor subspace (MS) is a subspace spanned by all the eigenvectors associated with the minor eigenvalues of the autocorrelation matrix of a high-dimensional vector sequence. The MS, also called the noise subspace (NS), has been extensively used in array signal processing. The NS tracking is a primary requirement in many real-time signal processing applications such as the adaptive direction-of-arrival (DOA) estimation, the data compression in data communications, the solution of a total least squares problem in adaptive signal processing, and the feature extraction technique for a high-dimensional data sequence.

[1]  L. Gleser.  Estimation in a Multivariate "Errors in Variables" Regression Model: Large Sample Results , 1981 .

[2]  Ralf Möller,et al.  A Self-Stabilizing Learning Rule for Minor Component Analysis , 2004, Int. J. Neural Syst..

[3]  George V. Moustakides,et al.  Fast and Stable Subspace Tracking , 2008, IEEE Transactions on Signal Processing.

[4]  Giansalvo Cirrincione,et al.  Linear system identification using the TLS EXIN neuron , 1999, Neurocomputing.

[5]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[6]  K. Abed-Meraim,et al.  Fast algorithms for subspace tracking , 2001, IEEE Signal Processing Letters.

[7]  Peter Strobach.  Square-root QR inverse iteration for tracking the minor subspace , 2000, IEEE Trans. Signal Process..

[8]  Mao Ye,et al.  A Class of Self-Stabilizing MCA Learning Algorithms , 2006, IEEE Transactions on Neural Networks.

[9]  Karim Abed-Meraim,et al.  Low-cost adaptive algorithm for noise subspace estimation , 2002 .

[10]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[11]  Chongzhao Han,et al.  On the Discrete-Time Dynamics of a Class of Self-Stabilizing MCA Extraction Algorithms , 2010, IEEE Transactions on Neural Networks.

[12]  Gene H. Golub,et al.  Some modified matrix eigenvalue problems , 1973, Milestones in Matrix Computation.

[13]  Fa-Long Luo,et al.  Applied neural networks for signal processing , 1997 .

[14]  Rong Wang,et al.  A Novel Orthonormalization Matrix Based Fast and Stable DPM Algorithm for Principal and Minor Subspace Tracking , 2012, IEEE Transactions on Signal Processing.

[15]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[16]  Jar-Ferr Yang,et al.  Adaptive eigensubspace algorithms for direction or frequency estimation and tracking , 1988, IEEE Trans. Acoust. Speech Signal Process..

[17]  Pedro J. Zufiria,et al.  On the discrete-time dynamics of the basic Hebbian neural network node , 2002, IEEE Trans. Neural Networks.

[18]  George V. Moustakides,et al.  Blind adaptive channel estimation in ofdm systems , 2006, IEEE Transactions on Wireless Communications.

[19]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[20]  Y. Hua,et al.  Orthogonal Oja algorithm , 2000, IEEE Signal Processing Letters.

[21]  Shun-ichi Amari,et al.  Unified stabilization approach to principal and minor components extraction algorithms , 2001, Neural Networks.

[22]  Zhang Yi,et al.  A neural networks learning algorithm for minor component analysis and its convergence analysis , 2008, Neurocomputing.

[23]  Roland Badeau,et al.  Yet another subspace tracker , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[24]  Xiangyu Kong,et al.  A self-stabilizing MSA algorithm in high-dimension data stream , 2010, Neural Networks.

[25]  Zhang Yi,et al.  On the discrete time dynamics of a self-stabilizing MCA learning algorithm , 2008, Math. Comput. Model..

[26]  Zhang Yi,et al.  Convergence analysis of a simple minor component analysis algorithm , 2007, Neural Networks.

[27]  Zheng Bao,et al.  Total least mean squares algorithm , 1998, IEEE Trans. Signal Process..

[28]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[29]  Roland Badeau,et al.  Fast and Stable YAST Algorithm for Principal and Minor Subspace Tracking , 2008, IEEE Transactions on Signal Processing.

[30]  Ching-Tai Chiang,et al.  On the inflation method in adaptive noise-subspace estimator , 1999, IEEE Trans. Signal Process..

[31]  M. Swamy,et al.  Learning algorithm for total least-squares adaptive signal processing , 1992 .

[32]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[33]  Lu Yang,et al.  Adaptive Noise Subspace Estimation Algorithm Suitable for VLSI Implementation , 2009, IEEE Signal Processing Letters.

[34]  Steve Bartelmaos,et al.  Principal and Minor Subspace Tracking: Algorithms & Stability Analysis , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[35]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[36]  Shun-ichi Amari,et al.  Sequential Extraction of Minor Components , 2001, Neural Processing Letters.

[37]  Fa-Long Luo,et al.  A Minor Component Analysis Algorithm , 1997, Neural Networks.

[38]  Samir Attallah,et al.  The generalized Rayleigh's quotient adaptive noise subspace algorithm: a householder transformation-based implementation , 2006, IEEE Transactions on Circuits and Systems II: Express Briefs.

[39]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics (Revised Edition) , 1999 .

[40]  Guisheng Liao,et al.  Adaptive step-size minor component extraction algorithm , 1999 .

[41]  D. M. Kammen,et al.  Quadrature and the development of orientation selective cortical cells by Hebb rules , 1989, Biological Cybernetics.

[42]  Giansalvo Cirrincione,et al.  Against the convergence of the minor component analysis neurons , 1999, IEEE Trans. Neural Networks.

[43]  Zhang Yi,et al.  A stable MCA learning algorithm , 2008, Comput. Math. Appl..

[44]  Lennart Ljung,et al.  Analysis of recursive stochastic algorithms , 1977 .

[45]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[46]  Lu Yang,et al.  Analysis of Orthogonality Error Propagation for FRANS and HFRANS Algorithms , 2008, IEEE Transactions on Signal Processing.

[47]  Fa-Long Luo,et al.  Neural network approach to the TLS linear prediction frequency estimation problem , 1996, Neurocomputing.

[48]  Roland Badeau,et al.  Yast Algorithm for Minor Subspace Tracking , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[49]  Sabine Van Huffel,et al.  The MCA EXIN neuron for the minor component analysis , 2002, IEEE Trans. Neural Networks.

[50]  Zhang Yi,et al.  Dynamics of Generalized PCA and MCA Learning Algorithms , 2007, IEEE Transactions on Neural Networks.

[51]  Soura Dasgupta,et al.  Adaptive estimation of eigensubspace , 1995, IEEE Trans. Signal Process..

[52]  S. Amari,et al.  A self-stabilized minor subspace rule , 1998, IEEE Signal Processing Letters.

[53]  Y. Hua,et al.  Fast orthonormal PAST algorithm , 2000, IEEE Signal Processing Letters.

[54]  Zhang Yi,et al.  Convergence Analysis of an Effective MCA Learning Algorithm , 2005, 2005 International Conference on Neural Networks and Brain.

[55]  Wei Xing Zheng,et al.  Neural network learning algorithms for tracking minor subspace in high-dimensional data stream , 2005, IEEE Transactions on Neural Networks.

[56]  Karim Abed-Meraim,et al.  An Efficient & Stable Algorithm for Minor Subspace Tracking and Stability Analysis , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[57]  Juha Karhunen,et al.  A Unified Neural Bigradient Algorithm for robust PCA and MCA , 1996, Int. J. Neural Syst..

[58]  G. Golub.  Some modified eigenvalue problems , 1971 .

[59]  Chongzhao Han,et al.  Modified gradient algorithm for total least square filtering , 2006, Neurocomputing.

[60]  Qingfu Zhang,et al.  A class of learning algorithms for principal component analysis and minor component analysis , 2000, IEEE Trans. Neural Networks Learn. Syst..

[61]  M. Swamy,et al.  A constrained anti-Hebbian learning algorithm for total least-squares estimation with applications to adaptive FIR and IIR filtering , 1994 .

[62]  George Mathew,et al.  Orthogonal eigensubspace estimation using neural networks , 1994, IEEE Trans. Signal Process..

[63]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[64]  Zheng Bao,et al.  Adaptive minor component extraction with modular structure , 2001, IEEE Trans. Signal Process..

[65]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[66]  Qin Lin,et al.  A unified algorithm for principal and minor components extraction , 1998, Neural Networks.

[67]  J. P. Lasalle.  The stability of dynamical systems , 1976 .

[68]  Peter Strobach.  The fast recursive row-Householder subspace tracking algorithm , 2009, Signal Process..

[69]  Peter Strobach.  The QS-Householder Sliding Window Bi-SVD Subspace Tracker , 2009, IEEE Transactions on Signal Processing.

[70]  Roland Badeau,et al.  Fast approximated power iteration subspace tracking , 2005, IEEE Transactions on Signal Processing.

[71]  G. Williamson,et al.  Stable simplified gradient algorithms for total least squares filtering , 2000, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154).

[72]  Andrzej Cichocki,et al.  Simplified neural networks for solving linear least squares and total least squares problems in real time , 1994, IEEE Trans. Neural Networks.

[73]  V. Pisarenko.  The Retrieval of Harmonics from a Covariance Function , 1973 .