Stochastic Analysis of the LMS Algorithm for System Identification With Subspace Inputs

This paper studies the behavior of the low-rank least mean squares (LMS) adaptive algorithm for the general case in which the input transformation may not capture the exact input subspace. It is shown that the Independence Theory and the independent additive noise model are not applicable to this case. A new theoretical model for the weight mean and fluctuation behaviors is developed which incorporates the correlation between successive data vectors (as opposed to the Independence Theory model). The new theory is applied to a network echo cancellation scheme which uses partial-Haar input vector transformations. Comparison of the new model predictions with Monte Carlo simulations shows good-to-excellent agreement, certainly much better than predicted by the Independence Theory based model available in the literature.

[1]  Dimitris G. Manolakis,et al.  Statistical and Adaptive Signal Processing , 2000 .

[2]  Hans-Jürgen Butterweck.  A steady-state analysis of the LMS adaptive algorithm without use of the independence assumption , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[3]  M. Melamed.  Detection , 2021, SETI: Astronomy as a Contact Sport.

[4]  Markus Rupp,et al.  Overcoming the independence assumption in LMS filtering , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[5]  M. Honig,et al.  Adaptive techniques for multiuser CDMA receivers , 2000, IEEE Signal Processing Magazine.

[6]  Odile Macchi,et al.  Adaptive Processing: The Least Mean Squares Approach with Applications in Transmission , 1995 .

[7]  Scott C. Douglas,et al.  Exact expectation analysis of the LMS adaptive filter , 1995, IEEE Trans. Signal Process..

[8]  P. Strobach.  Low-rank adaptive filters , 1996, IEEE Trans. Signal Process..

[9]  Paulo Sergio Ramirez,et al.  Fundamentals of Adaptive Filtering , 2002 .

[10]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[11]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[12]  E. Eweda,et al.  Second-order convergence analysis of stochastic adaptive linear filtering , 1983 .

[13]  S. Tsujii,et al.  Convergence analysis of gradient adaptive algorithms for arbitrary inputs without independence assumption , 1992, [Proceedings 1992] IEEE International Conference on Systems Engineering.

[14]  Scott C. Douglas,et al.  Exact expectation analysis of the LMS adaptive filter for correlated Gaussian input data , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[15]  J. Scott Goldstein,et al.  Reduced-rank adaptive filtering , 1997, IEEE Trans. Signal Process..

[16]  Louis L. Scharf,et al.  A Multistage Representation of the Wiener Filter Based on Orthogonal Projections , 1998, IEEE Trans. Inf. Theory.

[17]  Magnus Jansson,et al.  Reduced rank linear regression and weighted low rank approximations , 2006, IEEE Transactions on Signal Processing.

[18]  Scott C. Douglas,et al.  Exact expectation analysis of the LMS adaptive filter without the independence assumption , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[19]  Mariane R. Petraglia,et al.  Performance analysis of an adaptive filter structure employing wavelets and sparse subfilters , 2000, 2000 10th European Signal Processing Conference.

[20]  J. Príncipe,et al.  Sparse channel estimation with regularization method using convolution inequality for entropy , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[21]  Dinko Begusic,et al.  Analysis of low rank transform domain adaptive filtering algorithm , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[22]  Bhaskar D. Rao,et al.  Statistical analysis of subspace-based estimation of reduced-rank linear regressions , 2002, IEEE Trans. Signal Process..

[23]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[24]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[25]  Neil J. Bershad,et al.  Fast coupled adaptation for sparse impulse responses using a partial haar transform , 2005, IEEE Transactions on Signal Processing.

[26]  Simon Haykin,et al.  Adaptive filter theory (2nd ed.) , 1991 .

[27]  M. Sondhi,et al.  The history of echo cancellation , 2006, IEEE Signal Processing Magazine.