A Self-Stabilizing Neural Algorithm for Total Least Squares Filtering

A neural approach for solving the total least square (TLS) problem is presented in the paper. It is based on a linear neuron with a self-stabilizing neural algorithm, capable of resolving the TLS problem present in the parameter estimation of an adaptive FIR filters for system identification, where noisy errors affect not only the observation vector but also the data matrix. The learning rule is analyzed mathematically and the condition to guarantee the stability of algorithm is educed. The computer simulations are given to illustrate that the neural approach is self-stabilizing and considerably outperforms the existing TLS methods when a larger learning factor is used or the signal-noise-rate is lower.

[1]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[2]  Shun-ichi Amari,et al.  Unified stabilization approach to principal and minor components extraction algorithms , 2001, Neural Networks.

[3]  Pedro J. Zufiria,et al.  On the discrete-time dynamics of the basic Hebbian neural network node , 2002, IEEE Trans. Neural Networks.

[4]  G. Williamson,et al.  Stable simplified gradient algorithms for total least squares filtering , 2000, Conference Record of the Thirty-Fourth Asilomar Conference on Signals, Systems and Computers (Cat. No.00CH37154).

[5]  Andrzej Cichocki,et al.  Simplified neural networks for solving linear least squares and total least squares problems in real time , 1994, IEEE Trans. Neural Networks.

[6]  Fa-Long Luo,et al.  Neural network approach to the TLS linear prediction frequency estimation problem , 1996, Neurocomputing.

[7]  M. Swamy,et al.  Learning algorithm for total least-squares adaptive signal processing , 1992 .

[8]  Sabine Van Huffel,et al.  The total least squares problem , 1993 .

[9]  Ralf Möller,et al.  A Self-Stabilizing Learning Rule for Minor Component Analysis , 2004, Int. J. Neural Syst..

[10]  Wei Xing Zheng,et al.  Neural network learning algorithms for tracking minor subspace in high-dimensional data stream , 2005, IEEE Transactions on Neural Networks.

[11]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[12]  Fa-Long Luo,et al.  A Minor Component Analysis Algorithm , 1997, Neural Networks.

[13]  Sabine Van Huffel,et al.  The MCA EXIN neuron for the minor component analysis , 2002, IEEE Trans. Neural Networks.

[14]  Zhang Yi,et al.  Dynamics of Generalized PCA and MCA Learning Algorithms , 2007, IEEE Transactions on Neural Networks.

[15]  Fa-Long Luo,et al.  Applied neural networks for signal processing , 1997 .

[16]  V. Pisarenko.  The Retrieval of Harmonics from a Covariance Function , 1973 .

[17]  Erkki Oja,et al.  Modified Hebbian learning for curve and surface fitting , 1992, Neural Networks.

[18]  M. Swamy,et al.  A constrained anti-Hebbian learning algorithm for total least-squares estimation with applications to adaptive FIR and IIR filtering , 1994 .

[19]  Zhang Yi,et al.  On the discrete time dynamics of a self-stabilizing MCA learning algorithm , 2008, Math. Comput. Model..

[20]  Zhang Yi,et al.  A neural networks learning algorithm for minor component analysis and its convergence analysis , 2008, Neurocomputing.

[21]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[22]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 1991 .

[23]  L. Gleser.  Estimation in a Multivariate "Errors in Variables" Regression Model: Large Sample Results , 1981 .

[24]  Mao Ye,et al.  A Class of Self-Stabilizing MCA Learning Algorithms , 2006, IEEE Transactions on Neural Networks.

[25]  Giansalvo Cirrincione,et al.  Linear system identification using the TLS EXIN neuron , 1999, Neurocomputing.

[26]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[27]  G. Golub.  Some modified eigenvalue problems , 1971 .

[28]  Chongzhao Han,et al.  Modified gradient algorithm for total least square filtering , 2006, Neurocomputing.

[29]  Tamer Basar,et al.  Analysis of Recursive Stochastic Algorithms , 2001 .

[30]  Qingfu Zhang,et al.  A class of learning algorithms for principal component analysis and minor component analysis , 2000, IEEE Trans. Neural Networks Learn. Syst..

[31]  Zhang Yi,et al.  Convergence Analysis of an Effective MCA Learning Algorithm , 2005, 2005 International Conference on Neural Networks and Brain.

[32]  S. Amari,et al.  A self-stabilized minor subspace rule , 1998, IEEE Signal Processing Letters.

[33]  J. P. Lasalle.  The stability of dynamical systems , 1976 .

[34]  Giansalvo Cirrincione,et al.  Against the convergence of the minor component analysis neurons , 1999, IEEE Trans. Neural Networks.

[35]  Zhang Yi,et al.  A stable MCA learning algorithm , 2008, Comput. Math. Appl..

[36]  Zhang Yi,et al.  Convergence analysis of a simple minor component analysis algorithm , 2007, Neural Networks.

[37]  Zheng Bao,et al.  Total least mean squares algorithm , 1998, IEEE Trans. Signal Process..

[38]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .