Analysis of stochastic gradient tracking of time-varying polynomial Wiener systems

This paper presents analytical and Monte Carlo results for a stochastic gradient adaptive scheme that tracks a time-varying polynomial Wiener (1958) system [i.e., a linear time-invariant (LTI) filter with memory followed by a time-varying memoryless polynomial nonlinearity]. The adaptive scheme consists of two phases: (1) estimation of the LTI memory using the LMS algorithm and (2) tracking the time-varying polynomial-type nonlinearity using a second coupled gradient search for the polynomial coefficients. The time-varying polynomial nonlinearity causes a time-varying scaling for the optimum Wiener filter for Phase 1. These time variations are removed for Phase 2 using a novel coupling scheme to Phase 1. The analysis for Gaussian data includes recursions for the mean behavior of the LMS algorithm for estimating and tracking the optimum Wiener filter for Phase 1 for several different time-varying polynomial nonlinearities and recursions for the mean behavior of the stochastic gradient algorithm for Phase 2. The polynomial coefficients are shown to be accurately tracked. Monte Carlo simulations confirm the theoretical predictions and support the underlying statistical assumptions.

[1]  Jean-Marc Vesin,et al.  Analysis of stochastic gradient identification of polynomial nonlinear systems with memory , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[2]  N. Wiener,et al.  Nonlinear Problems in Random Theory , 1964 .

[3]  Mohamed Ibnkahla,et al.  Statistical analysis of a two-layer backpropagation algorithm used for modeling nonlinear memoryless channels: the single neuron case , 1997, IEEE Trans. Signal Process..

[4]  Nicholas Kalouptsidis,et al.  Adaptive system identification and signal processing algorithms , 1993 .

[5]  Sheng Chen,et al.  Practical identification of NARMAX models using radial basis functions , 1990 .

[6]  John J. Shynk,et al.  Statistical analysis of the single-layer backpropagation algorithm. II. MSE and classification performance , 1993, IEEE Trans. Signal Process..

[7]  S. Thomas Alexander,et al.  Adaptive Signal Processing , 1986, Texts and Monographs in Computer Science.

[8]  J. J. Shynk,et al.  Steady-state analysis of a single-layer perceptron based on a system identification model with bias terms , 1991 .

[9]  Julian J. Bussgang,et al.  Crosscorrelation functions of amplitude-distorted gaussian signals , 1952 .

[10]  Sang-Sik Ahn,et al.  Convergence of the delayed normalized LMS algorithm with decreasing step size , 1996, IEEE Trans. Signal Process..

[11]  L. Ljung,et al.  Necessary and sufficient conditions for stability of LMS , 1997, IEEE Trans. Autom. Control..

[12]  Bernard Mulgrew,et al.  Nonlinear system identification and prediction using orthogonal functions , 1997, IEEE Trans. Signal Process..

[13]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[14]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[15]  Scott C. Douglas,et al.  Exact expectation analysis of the LMS adaptive filter , 1995, IEEE Trans. Signal Process..

[16]  S. Billings,et al.  Identification of nonlinear systems using the Wiener model , 1977 .

[17]  V. John Mathews.  Adaptive Volterra filters using orthogonal structures , 1996, IEEE Signal Processing Letters.

[18]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[19]  Jacques Sombrin,et al.  Neural network modeling and identification of nonlinear channels with memory: algorithms, applications, and analytic models , 1998, IEEE Trans. Signal Process..

[20]  Neil J. Bershad,et al.  Stochastic analysis of adaptive gradient identification of Wiener-Hammerstein systems for Gaussian inputs , 2000, IEEE Trans. Signal Process..

[21]  Jean-Marc Vesin,et al.  Stochastic analysis of gradient adaptive identification of nonlinear systems with memory for Gaussian data and noisy input and output measurements , 1999, IEEE Trans. Signal Process..

[22]  Stephen A. Billings,et al.  Non-linear system identification using neural networks , 1990 .

[23]  John J. Shynk,et al.  Statistical analysis of the single-layer backpropagation algorithm. I. mean weight behavior , 1993, IEEE Trans. Signal Process..