On a Class of Orthonormal Algorithms for Principal and Minor Subspace Tracking

This paper elaborates on a new class of orthonormal power-based algorithms for fast estimation and tracking of the principal or minor subspace of a vector sequence. The proposed algorithms are closely related to the natural power method that has the fastest convergence rate among many power-based methods such as the Oja method, the projection approximation subspace tracking (PAST) method, and the novel information criterion (NIC) method. A common feature of the proposed algorithms is the exact orthonormality of the weight matrix at each iteration. The orthonormality is implemented in a most efficient way. Besides the property of orthonormality, the new algorithms offer, as compared to other power based algorithms, a better numerical stability and a linear computational complexity.

[1]  Craig S. MacInnes.  Fast, accurate subspace tracking using operator restriction analysis , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[2]  Wei-Yong Yan,et al.  Global convergence of Oja's subspace algorithm for principal component extraction , 1998, IEEE Trans. Neural Networks.

[3]  S.Y. Kung,et al.  Adaptive Principal component EXtraction (APEX) and applications , 1994, IEEE Trans. Signal Process..

[4]  Eric Moulines,et al.  Subspace methods for the blind identification of multichannel FIR filters , 1995, IEEE Trans. Signal Process..

[5]  Scott C. Douglas,et al.  Self-stabilized gradient algorithms for blind source separation with orthogonality constraints , 2000, IEEE Trans. Neural Networks Learn. Syst..

[6]  John B. Moore,et al.  Global analysis of Oja's flow for neural networks , 1994, IEEE Trans. Neural Networks.

[7]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..

[8]  Messaoud Benidir,et al.  The propagator method for source bearing estimation , 1995, Signal Process..

[9]  Yingbo Hua,et al.  Fast subspace tracking and neural network learning by a novel information criterion , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[10]  Ghassan Kawas Kaleh,et al.  Spatio-temporal blind adaptive multiuser detection , 2000, IEEE Trans. Commun..

[11]  Karim Abed-Meraim,et al.  A New Look at the Power Method for Fast Subspace Tracking , 1999, Digit. Signal Process..

[12]  E. Oja.  Simplified neuron model as a principal component analyzer , 1982, Journal of mathematical biology.

[13]  Qin Lin,et al.  A unified algorithm for principal and minor components extraction , 1998, Neural Networks.

[14]  G. Golub,et al.  Tracking a few extreme singular values and vectors in signal processing , 1990, Proc. IEEE.

[15]  Anuj Srivastava,et al.  A Bayesian approach to geometric subspace estimation , 2000, IEEE Trans. Signal Process..

[16]  G. W. Stewart,et al.  An updating algorithm for subspace tracking , 1992, IEEE Trans. Signal Process..

[17]  S. Amari,et al.  A self-stabilized minor subspace rule , 1998, IEEE Signal Processing Letters.

[18]  Thierry Chonavel,et al.  Fast adaptive eigenvalue decomposition: a maximum likelihood approach , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  Tony Gustafsson,et al.  A class of subspace tracking algorithms based on approximation of the noise-subspace , 2000, IEEE Trans. Signal Process..

[20]  S. Douglas.  Numerically-robust adaptive subspace tracking using Householder transformations , 2000, Proceedings of the 2000 IEEE Sensor Array and Multichannel Signal Processing Workshop. SAM 2000 (Cat. No.00EX410).

[21]  Y. Hua,et al.  Orthogonal Oja algorithm , 2000, IEEE Signal Processing Letters.

[22]  Constantinos B. Papadias.  Globally convergent algorithms for blind source separation , 2000, 2000 10th European Signal Processing Conference.

[23]  H. Vincent Poor,et al.  Blind Multiuser Detection: A Subspace Approach , 1998, IEEE Trans. Inf. Theory.

[24]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[25]  Y. Hua,et al.  Fast orthonormal PAST algorithm , 2000, IEEE Signal Processing Letters.

[26]  Gene H. Golub,et al.  Matrix computations , 1983 .

[27]  K. Abed-Meraim,et al.  Fast algorithms for subspace tracking , 2001, IEEE Signal Processing Letters.