Stochastic analysis of the LMS algorithm with a saturation nonlinearity following the adaptive filter output

This paper presents a statistical analysis of the least mean square (LMS) algorithm with a zero-memory scaled error function nonlinearity following the adaptive filter output. This structure models saturation effects in active noise and active vibration control systems when the acoustic transducers are driven by large amplitude signals. The problem is first defined as a nonlinear signal estimation problem and the mean-square error (MSE) performance surface is studied. Analytical expressions are obtained for the optimum weight vector and the minimum achievable MSE as functions of the saturation. These results are useful for adaptive algorithm design and evaluation. The LMS algorithm behavior with saturation is analyzed for Gaussian inputs and slow adaptation. Deterministic nonlinear recursions are obtained for the time-varying mean weight and MSE behavior. Simplified results are derived for white inputs and small step sizes. Monte Carlo simulations display excellent agreement with the theoretical predictions, even for relatively large step sizes. The new analytical results accurately predict the effect of saturation on the LMS adaptive filter behavior.

[1]  Simon Haykin,et al.  Adaptive filter theory (2nd ed.) , 1991 .

[2]  Scott C. Douglas,et al.  Exact expectation analysis of the LMS adaptive filter , 1995, IEEE Trans. Signal Process..

[3]  Maria Huhtala,et al.  Random Variables and Stochastic Processes , 2021, Matrix and Tensor Decompositions in Signal Processing.

[4]  Jean-Marc Vesin,et al.  Stochastic analysis of gradient adaptive identification of nonlinear systems with memory for Gaussian data and noisy input and output measurements , 1999, IEEE Trans. Signal Process..

[5]  N. Bershad.  On weight update saturation nonlinearities in LMS adaptation , 1990, IEEE Trans. Acoust. Speech Signal Process..

[6]  Scott D. Snyder,et al.  Active control of vibration using a neural network , 1995, IEEE Trans. Neural Networks.

[7]  J. J. Shynk,et al.  Steady-state analysis of a single-layer perceptron based on a system identification model with bias terms , 1991 .

[8]  Neil J. Bershad,et al.  On error-saturation nonlinearities in LMS adaptation , 1988, IEEE Trans. Acoust. Speech Signal Process..

[9]  D. Duttweiler.  Adaptive filter performance with nonlinearities in the correlation multiplier , 1982 .

[10]  Roberto Cristi,et al.  On the optimal weight vector of a perceptron with Gaussian data and arbitrary nonlinearity , 1993, IEEE Trans. Signal Process..

[11]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[12]  E. Eweda,et al.  Analysis and design of a signed regressor LMS algorithm for stationary and nonstationary adaptive filtering with correlated Gaussian data , 1990 .

[13]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[14]  Neil J. Bershad.  On the optimum data nonlinearity in LMS adaptation , 1986, IEEE Trans. Acoust. Speech Signal Process..

[15]  Teresa H. Y. Meng,et al.  Normalized data nonlinearities for LMS adaptation , 1994, IEEE Trans. Signal Process..

[16]  S. Koike.  Convergence analysis of a data echo canceller with a stochastic gradient adaptive FIR filter using the sign algorithm , 1995, IEEE Trans. Signal Process..

[17]  Sen M. Kuo,et al.  Active Noise Control Systems: Algorithms and DSP Implementations , 1996 .

[18]  T. Claasen,et al.  Comparison of the convergence of two algorithms for adaptive FIR digital filters , 1981 .

[19]  Teresa H. Y. Meng,et al.  Stochastic gradient adaptation under general error criteria , 1994, IEEE Trans. Signal Process..

[20]  José Carlos M. Bermudez,et al.  Mean weight behavior of the filtered-X LMS algorithm , 2000, IEEE Trans. Signal Process..

[21]  José Carlos M. Bermudez,et al.  A nonlinear analytical model for the quantized LMS algorithm-the arbitrary step size case , 1996, IEEE Trans. Signal Process..

[22]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[23]  Audra E. Kosh,et al.  Linear Algebra and its Applications , 1992 .