Convergence properties of LMS adaptive estimators with unbounded dependent inputs

This paper presents limit theorems for the behaviour of adaptive estimators using the LMS algorithm when the driving or input sequence is a member of a broad class of random processes which are not necessarily almost surely bounded and are dependent over time. Convergence in distribution of the estimates is established in the stationary case while general nonstationary tracking is characterised in the nonstationary case. These results follow from the exponential convergence of the homogeneous algorithm which in turn follows from a strong limit theorem for infinite products of ergodic and mixing sequences of matrices.

[1]  H. Furstenberg.  Noncommuting random products , 1963 .

[2]  U. Grenander.  Probabilities on Algebraic Structures , 1964 .

[3]  A. Gersho.  Adaptive equalization of highly dispersive channels for data transmission , 1969 .

[4]  Radu Theodorescu,et al.  Random processes and learning , 1969 .

[5]  Walter Philipp,et al.  The central limit problem for mixing sequences of random variables , 1969 .

[6]  THOMAS P. DANIELL.  Adaptive Estimation with Mutually Correlated Training Sequences , 1970, IEEE Trans. Syst. Sci. Cybern..

[7]  Lee D. Davisson.  Steady-state error in adaptive mean-square minimization , 1970, IEEE Trans. Inf. Theory.

[8]  J. Mendel,et al.  Discrete Techniques of Parameter Estimation: The Equation Error Formulation , 1973 .

[9]  Jae-Kyoon Kim,et al.  Adaptive linear estimation for stationary M-dependent processes , 1975, IEEE Trans. Inf. Theory.

[10]  C. C. Neaderhouser.  Limit Theorems for Multiply Indexed Mixing Random Variables, with Application to Gibbs Random Fields , 1978 .

[11]  B. Widrow,et al.  Stationary and nonstationary learning characteristics of the LMS adaptive filter , 1976, Proceedings of the IEEE.

[12]  V. L. Girko,et al.  A Limit Theorem for Products of Random Matrices , 1976 .

[13]  C. Johnson.  Input matching, error augmentation, self-tuning, and output error identification: Algorithmic similarities in discrete adaptive model following , 1979, 1979 18th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[14]  Khalid Sayood,et al.  On the "Desired behavior" of adaptive signal processing algorithms , 1979, ICASSP.

[15]  Alan Weiss,et al.  Digital adaptive filters: Conditions for convergence, rates of convergence, effects of noise and errors arising from the implementation , 1979, IEEE Trans. Inf. Theory.

[16]  B. Anderson,et al.  Lyapunov techniques for the exponential stability of linear difference equations with random coefficients , 1980 .

[17]  B. Anderson,et al.  Performance of adaptive estimation algorithms in dependent random environments , 1980 .

[18]  Richard M. Johnstone,et al.  Robust Lyapunov stability results and adaptive systems , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[19]  Ralph K. Cavin,et al.  Analysis of error-gradient adaptive linear estimators for a class of stationary dependent processes , 1982, IEEE Trans. Inf. Theory.

[20]  C. Richard Johnson,et al.  Exponential convergence of adaptive identification and control algorithms , 1981, Autom..