Bayesian step least mean squares algorithm for Gaussian signals

Selecting the step of the least mean squares (LMS) algorithm is an old problem. This study uses a new approach to address this problem resulting in a new algorithm with excellent system identification performance. The LMS algorithm, with time-varying step, size can be shown to be equivalent to the Kalman filter in some conditions. This is as long as the state noise of the Kalman filter and the step size of the LMS algorithm are chosen carefully. The Kalman filter is the optimum linear estimator (Bayesian) given the state and the measurement noise covariance matrices, but these matrices are not always known. This work considers the case where these matrices are not known, in the special cases that the Kalman filter reduces to the LMS. This results in an algorithm to select the step-size of the LMS algorithm with few priors. The optimum step size can be calculated using estimates of the probability density function (PDF) of the coefficient estimation error variance (qw ) and measurement noise variance (qv ). The PDFs can be estimated from the data using Bayes’ rule and assuming Gaussian reference and measurement noise signals. The resulting algorithm to determine qw and qv is a second small Kalman filter, and the outputs of this filter (means and covariances) are used to determine the expected value of the step.

[1]  Roland Maas,et al.  The NLMS Algorithm with Time-Variant Optimum Stepsize Derived from a Bayesian Network Perspective , 2014, IEEE Signal Processing Letters.

[2]  Raymond H. Kwong,et al.  A variable step size LMS algorithm , 1992, IEEE Trans. Signal Process..

[3]  Yuanqing Xia,et al.  An adaptive Kalman filter estimating process noise covariance , 2017, Neurocomputing.

[4]  R. Mehra.  On the identification of variances and adaptive Kalman filtering , 1970 .

[5]  P. S. Maybeck,et al.  MMAE Detection of Interference/Jamming and Spoofing in a DGPS-Aided Inertial System , 1998 .

[6]  Henning Puder,et al.  Step-size control for acoustic echo cancellation filters - an overview , 2000, Signal Process..

[7]  Jacob Benesty,et al.  A Connection Between the Kalman Filter and an Optimized LMS Algorithm for Bilinear Forms , 2018, Algorithms.

[8]  Haiquan Zhao,et al.  Combined regularization parameter for normalized LMS algorithm and its performance analysis , 2019, Signal Process..

[9]  Asiya Sulthana,et al.  An Efficient Kalman Noise Canceller for Cardiac Signal Analysis in Modern Telecardiology Systems , 2018, IEEE Access.

[10]  Jiashu Zhang,et al.  Mean-square-deviation analysis of probabilistic LMS algorithm , 2019, Digit. Signal Process..

[11]  Alberto Carini,et al.  Optimal Variable Step-Size NLMS Algorithms With Auxiliary Noise Power Scheduling for Feedforward Active Noise Control , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[12]  Jacob Benesty,et al.  An optimized NLMS algorithm for system identification , 2016, Signal Process..

[13]  Yuanqing Xia,et al.  Kalman Filter With Recursive Covariance Estimation—Sequentially Estimating Process Noise Covariance , 2014, IEEE Transactions on Industrial Electronics.

[14]  D. Magill.  Optimal adaptive estimation of sampled stochastic processes , 1965 .

[15]  T. Moon.  The expectation-maximization algorithm , 1996, IEEE Signal Process. Mag..

[16]  Ali H. Sayed,et al.  Variable step-size NLMS and affine projection algorithms , 2004, IEEE Signal Processing Letters.

[17]  Tyseer Aboulnasr,et al.  A robust variable step-size LMS-type algorithm: analysis and simulations , 1997, IEEE Trans. Signal Process..

[18]  Jacob Benesty,et al.  A Nonparametric VSS NLMS Algorithm , 2006, IEEE Signal Processing Letters.

[19]  José Carlos M. Bermudez,et al.  A switched variable step size NLMS adaptive filter , 2020, Digit. Signal Process..

[20]  Vinay A. Bavdekar,et al.  Identification of process and measurement noise covariance for state and parameter estimation using extended Kalman filter , 2011 .

[21]  A. H. Mohamed,et al.  Adaptive Kalman Filtering for INS/GPS , 1999 .