Analysis of conjugate gradient algorithms for adaptive filtering

We describe and analyze two approaches to the implementation of the Conjugate Gradient algorithm for adaptive filtering. In particular their convergence rate and misadjustment are compared. A new analysis approach in the z-domain is used in order to find the asymptotic performance, and stability bounds are established. The behavior of the algorithms in finite word-length computation are described and dynamic range considerations are discussed. It is shown that, close to steady-state, the algorithms' behaviors are similar to the Steepest Descent algorithm, where the stalling phenomenon is also observed. Using 16-bit fixed-point number representation, our simulations show that the algorithms are numerically stable.

[1]  Tamal Bose,et al.  Conjugate gradient method in adaptive bilinear filtering , 1995, IEEE Trans. Signal Process..

[2]  Alan N. Willson,et al.  Adaptive spectral estimation using the conjugate gradient algorithm , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[3]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[4]  Robert J. Plemmons,et al.  FFT-based RLS in signal processing , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[5]  E. M. Dowling,et al.  Conjugate gradient eigenstructure tracking for adaptive spectral estimation , 1995, IEEE Trans. Signal Process..

[6]  David F. Shanno,et al.  Conjugate Gradient Methods with Inexact Searches , 1978, Math. Oper. Res..

[7]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[8]  Henry Stark,et al.  Probability, Random Processes, and Estimation Theory for Engineers , 1995 .

[9]  John G. Proakis,et al.  Introduction to Digital Signal Processing , 1988 .

[10]  Gregory E. Bottomley,et al.  A novel approach for stabilizing recursive least squares filters , 1991, IEEE Trans. Signal Process..

[11]  Gene H. Golub,et al.  Matrix computations , 1983 .

[12]  O. Axelsson.  Iterative solution methods , 1995 .

[13]  A.N. Willson,et al.  A roundoff error analysis of the normalized LMS algorithm , 1995, Conference Record of The Twenty-Ninth Asilomar Conference on Signals, Systems and Computers.

[14]  S. Haykin,et al.  Adaptive Filter Theory , 1986 .

[15]  Simon Haykin,et al.  Adaptive filter theory (2nd ed.) , 1991 .

[16]  Magnus R. Hestenes,et al.  Conjugate Direction Methods in Optimization , 1980 .

[17]  M. Srinath,et al.  Conjugate gradient techniques for adaptive filtering , 1992 .

[18]  M. Srinath,et al.  Conjugate gradient techniques for adaptive filtering , 1989, IEEE International Symposium on Circuits and Systems,.

[19]  W. K. Jenkins,et al.  Preconditioned conjugate gradient methods for adaptive filtering , 1991, 1991., IEEE International Sympoisum on Circuits and Systems.

[20]  C. Caraiscos,et al.  A roundoff error analysis of the LMS adaptive algorithm , 1984 .

[21]  Alan N. Willson,et al.  Adaptive filtering using modified conjugate gradient , 1995, 38th Midwest Symposium on Circuits and Systems. Proceedings.