Least mean square algorithms with Markov regime-switching limit

We analyze the tracking performance of the least mean square (LMS) algorithm for adaptively estimating a time varying parameter that evolves according to a finite state Markov chain. We assume the Markov chain jumps infrequently between the finite states at the same rate of change as the LMS algorithm. We derive mean square estimation error bounds for the tracking error of the LMS algorithm using perturbed Lyapunov function methods. Then combining results in two-time-scale Markov chains with weak convergence methods for stochastic approximation, we derive the limit dynamics satisfied by continuous-time interpolation of the estimates. Unlike most previous analyzes of stochastic approximation algorithms, the limit we obtain is a system of ordinary differential equations with regime switching controlled by a continuous-time Markov chain. Next, to analyze the rate of convergence, we take a continuous-time interpolation of a scaled sequence of the error sequence and derive its diffusion limit. Somewhat remarkably, for correlated regression vectors we obtain a jump Markov diffusion. Finally, two novel examples of the analysis are given for state estimation of hidden Markov models (HMMs) and adaptive interference suppression in wireless code division multiple access (CDMA) networks.

[1]  T. Rydén,et al.  Consistent Estimation of Linear and Non‐linear Autoregressive Models with Markov Regime , 1998 .

[2]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[3]  H. Vincent Poor,et al.  Code-aided interference suppression for DS/CDMA communications. I. Interference suppression capability , 1997, IEEE Trans. Commun..

[4]  Neri Merhav,et al.  Hidden Markov processes , 2002, IEEE Trans. Inf. Theory.

[5]  Qing Zhang,et al.  Continuous-Time Markov Chains and Applications , 1998 .

[6]  Gang George Yin,et al.  Singularly Perturbed Discrete-Time Markov Chains , 2000, SIAM J. Appl. Math..

[7]  Andrew L. Rukhin,et al.  Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach , 2001, Technometrics.

[8]  Brian D. O. Anderson,et al.  Asymptotic smoothing errors for hidden Markov models , 2000, IEEE Trans. Signal Process..

[9]  Tamer Basar,et al.  Analysis of Recursive Stochastic Algorithms , 2001 .

[10]  Laurence B. Milstein,et al.  Performance analysis of MMSE receivers for DS-CDMA in frequency-selective fading channels , 2000, IEEE Trans. Commun..

[11]  Jonathan D. Cryer,et al.  Time Series Analysis , 1986 .

[12]  W. Grassman.  Approximation and Weak Convergence Methods for Random Processes with Applications to Stochastic Systems Theory (Harold J. Kushner) , 1986 .

[13]  Gang George Yin,et al.  Recursive algorithms for estimation of hidden Markov models and autoregressive models with Markov regime , 2002, IEEE Trans. Inf. Theory.

[14]  H. H. Zeng,et al.  Blind equalization using the constant modulus algorithm , 1996, Proceedings of Third International Conference on Signal Processing (ICSP'96).

[15]  H. Vincent Poor,et al.  Integrated voice/data call admission control for wireless DS-CDMA systems , 2002, IEEE Trans. Signal Process..

[16]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[17]  Sergio Verdu,et al.  Multiuser Detection , 1998 .

[18]  Gang George Yin,et al.  Regime Switching Stochastic Approximation Algorithms with Application to Adaptive Discrete Stochastic Optimization , 2004, SIAM J. Optim..

[19]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[20]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[21]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[22]  Lei Guo.  Stability of recursive stochastic tracking algorithms , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[23]  Lennart Ljung,et al.  Adaptation and tracking in system identification - A survey , 1990, Autom..

[24]  George Yin,et al.  Discrete-time singularly perturbed Markov chains: aggregation, occupation measures, and switching diffusion limit , 2003, Advances in Applied Probability.

[25]  M. Juntti,et al.  LMMSE Receivers for DS-CDMA Systems in Frequency-Selective Fading Channels , 1999 .

[26]  H. Vincent Poor,et al.  Online activity detection in a multiuser environment using the matrix CUSUM Algorithm , 2002, IEEE Trans. Inf. Theory.

[27]  Lennart Ljung,et al.  Performance analysis of general tracking algorithms , 1995 .

[28]  Gopal K. Basak,et al.  Stability of a Random diffusion with linear drift , 1996 .

[29]  S. Ethier,et al.  Markov Processes: Characterization and Convergence , 2005 .

[30]  Gang George Yin,et al.  LMS algorithms for tracking slow Markov chains with applications to hidden Markov estimation and adaptive multiuser detection , 2005, IEEE Transactions on Information Theory.

[31]  Lennart Ljung,et al.  Recursive least‐squares and accelerated convergence in stochastic approximation schemes , 2001 .

[32]  Xuan Kong,et al.  Adaptive Signal Processing Algorithms: Stability and Performance , 1994 .

[33]  G. Yin,et al.  Adaptive filtering with averaging , 1995 .

[34]  Harold J. Kushner,et al.  Weak Convergence and Properties of Adaptive Asymptotic F ilters w ith 177 Constant Gains , 1998 .

[35]  Philip Schniter,et al.  Blind equalization using the constant modulus criterion: a review , 1998, Proc. IEEE.