On the Convergence Analysis of the Normalized LMS and the Normalized Least Mean M-Estimate Algorithms

This paper studies the convergence behaviors of the normalized least mean square (NLMS) and the normalized least mean M-estimate (NLMM) algorithms. Our analysis is obtained by extending the framework of Bershad [6], [7], which were previously reported for the NLMS algorithm with Gaussian inputs. Due to the difficulties in evaluating certain expectations involved, in [6], [7] the behaviors of the NLMS algorithm for general eigenvalue distributions of input autocorrelation matrix were not fully analyzed. In this paper, using an extension of Price's theorem to mixture Gaussian distributions and by introducing certain special integral functions, closed-form results of these expectations are obtained which allow us to interpret the convergence performance of both the NLMS and the NLMM algorithms in Contaminated Gaussian noise. The validity of the proposed analysis is verified through computer simulations.

[1]  J. Nagumo,et al.  A learning method for system identification , 1967, IEEE Transactions on Automatic Control.

[2]  B. Widrow,et al.  Stationary and nonstationary learning characteristics of the LMS adaptive filter , 1976, Proceedings of the IEEE.

[3]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[4]  Ehud Weinstein,et al.  Convergence analysis of LMS filters with uncorrelated Gaussian data , 1985, IEEE Trans. Acoust. Speech Signal Process..

[5]  Neil J. Bershad,et al.  Analysis of the normalized LMS algorithm with Gaussian inputs , 1986, IEEE Trans. Acoust. Speech Signal Process..

[6]  N. Bershad.  Behavior of the ε-normalized LMS algorithm with Gaussian inputs , 1987, IEEE Trans. Acoust. Speech Signal Process..

[7]  D. Ruppert.  Robust Statistics: The Approach Based on Influence Functions , 1987 .

[8]  J. Foley,et al.  A note on the convergence analysis of LMS adaptive filters with Gaussian data , 1988, IEEE Trans. Acoust. Speech Signal Process..

[9]  Arie Feuer,et al.  Convergence and performance analysis of the normalized LMS algorithm with uncorrelated Gaussian data , 1988, IEEE Trans. Inf. Theory.

[10]  V. John Mathews,et al.  Performance analysis of adaptive filters equipped with the dual sign algorithm , 1991, IEEE Trans. Signal Process..

[11]  Peter M. Clarkson,et al.  A class of order statistic LMS algorithms , 1992, IEEE Trans. Signal Process..

[12]  Dirk T. M. Slock,et al.  On the convergence behavior of the LMS and the normalized LMS algorithms , 1993, IEEE Trans. Signal Process..

[13]  Markus Rupp,et al.  The behavior of LMS and NLMS algorithms in the presence of spherically invariant processes , 1993, IEEE Trans. Signal Process..

[14]  Ali H. Sayed,et al.  Time-domain feedback analysis of adaptive gradient algorithms via the small gain theorem , 1995, Optics & Photonics.

[15]  J. Weng,et al.  Adaptive nonlinear RLS algorithm for robust filtering in impulse noise , 1997, Proceedings of 1997 IEEE International Symposium on Circuits and Systems. Circuits and Systems in the Information Age ISCAS '97.

[16]  Gerald Recktenwald,et al.  Numerical Methods with MATLAB : Implementations and Applications , 2000 .

[17]  Tung-Sang Ng,et al.  Fast least mean M-estimate algorithms for robust adaptive filtering in impulse noise , 2000, 2000 10th European Signal Processing Conference.

[18]  Ali H. Sayed,et al.  A unified approach to the steady-state and tracking analyses of adaptive filters , 2001, IEEE Trans. Signal Process..

[19]  Tareq Y. Al-Naffouri,et al.  Transient analysis of data-normalized adaptive filters , 2003, IEEE Trans. Signal Process..

[20]  Shin'ichi Koike.  Performance Analysis of the Normalized LMS Algorithm for Complex-Domain Adaptive Filters in the Presence of Impulse Noise at Filter Input , 2006, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..