The limiting behavior of LMS

A realization-oriented analysis is given of the gradient noise misadjustment and lag misadjustment performance of the LMS (least-mean-square) algorithm. New formulas are given for both of these components of excess mean-square error. It is shown that the traditional formula for lag misadjustment needs to be modified by adding further terms due to gradient noise and noise variance. To perform the analysis, it is necessary to study the convergence (with probability one) of the noise-free, fixed-parameter LMS algorithm. Convergence is found under simple conditions that improve on those previously obtained. >

[1]  O. Macchi.  Optimization of adaptive identification for time-varying filters , 1984, The 23rd IEEE Conference on Decision and Control.

[2]  P. Billingsley,et al.  Convergence of Probability Measures , 1969 .

[3]  Albert Benveniste.  Design of adaptive algorithms for the tracking of time‐varying systems , 1987 .

[4]  B. Anderson,et al.  Performance of adaptive estimation algorithms in dependent random environments , 1980 .

[5]  Ehud Weinstein,et al.  Convergence analysis of LMS filters with uncorrelated Gaussian data , 1985, IEEE Trans. Acoust. Speech Signal Process..

[6]  A. Benveniste,et al.  A measure of the tracking capability of recursive stochastic algorithms with constant gains , 1982 .

[7]  E. Eweda,et al.  Second-order convergence analysis of stochastic adaptive linear filtering , 1983 .

[8]  B. Widrow,et al.  Stationary and nonstationary learning characteristics of the LMS adaptive filter , 1976, Proceedings of the IEEE.

[9]  B. Anderson,et al.  Lyapunov techniques for the exponential stability of linear difference equations with random coefficients , 1980 .

[10]  F. Kozin,et al.  On almost sure convergence of adaptive algorithms , 1986 .

[11]  Bernard Widrow,et al.  Adaptive Signal Processing , 1985 .

[12]  V. Solo,et al.  The Second Order Properties of a Time Series Recursion , 1981 .

[13]  R. Bitmead.  Convergence in distribution of LMS-type adaptive parameter estimates , 1983 .

[14]  Bruce E. Hajek,et al.  Review of 'Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory' (Kushner, H.J.; 1984) , 1985, IEEE Transactions on Information Theory.