A Stable Dual Purpose Adaptive Algorithm for Subspace Tracking on Noncompact Stiefel Manifold

Starting from an extended Rayleigh quotient defined on the noncompact Stiefel manifold, in this paper, we present a novel dual purpose subspace flows for subspace tracking. The proposed algorithm can switch from principal subspace to minor subspace tracking with a simple sign change of its stepsize parameter. More interestingly, the proposed dual purpose gradient system behaves the same invariant property as that of the well-known Chen-Amari-Lin system. The stability of the discrete version of the proposed subspace flow is guaranteed by an additional added stabilizing term. No tunable parameter is required for the proposed algorithm as opposed to the modified Oja algorithm. The strengths of the proposed algorithm is demonstrated using a defacto benchmark example.

[1]  Robert E. Mahony,et al.  A Grassmann-Rayleigh Quotient Iteration for Computing Invariant Subspaces , 2002, SIAM Rev..

[2]  Tianping Chen,et al.  Modified Oja's Algorithms For Principal Subspace and Minor Subspace extraction , 1997, Neural Processing Letters.

[3]  Lei Xu,et al.  Least mean square error reconstruction principle for self-organizing neural-nets , 1993, Neural Networks.

[4]  Qin Lin,et al.  A unified algorithm for principal and minor components extraction , 1998, Neural Networks.

[5]  Chongzhao Han,et al.  A Dual Purpose Principal and Minor Subspace Gradient Flow , 2012, IEEE Transactions on Signal Processing.

[6]  Iven M. Y. Mareels,et al.  A dual purpose principal and minor component flow , 2005, Syst. Control. Lett..

[7]  Y. Hua,et al.  Orthogonal Oja algorithm , 2000, IEEE Signal Processing Letters.

[8]  Shun-ichi Amari,et al.  Unified stabilization approach to principal and minor components extraction algorithms , 2001, Neural Networks.

[9]  U. Helmke,et al.  Optimization and Dynamical Systems , 1994, Proceedings of the IEEE.

[10]  John Moody,et al.  Learning rate schedules for faster stochastic gradient search , 1992, Neural Networks for Signal Processing II Proceedings of the 1992 IEEE Workshop.

[11]  S. Amari,et al.  A self-stabilized minor subspace rule , 1998, IEEE Signal Processing Letters.

[12]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[13]  Jean Pierre Delmas,et al.  Subspace Tracking for Signal Processing , 2010 .