An online sequential algorithm for the estimation of transition probabilities for jump Markov linear systems

This paper describes a new method to estimate the transition probabilities associated with a jump Markov linear system. The new algorithm uses stochastic approximation type recursions to minimize the Kullback-Leibler divergence between the likelihood function of the transition probabilities and the true likelihood function. Since the calculation of the likelihood function of the transition probabilities is impossible, an incomplete data paradigm, which has been previously applied to a similar problem for hidden Markov models, is used. The algorithm differs from the existing algorithms in that it assumes that the transition probabilities are deterministic quantities whereas the existing approaches consider them to be random variables with prior distributions.

[1]  X. R. Li,et al.  Online Bayesian estimation of transition probabilities for Markovian jump systems , 2004, IEEE Transactions on Signal Processing.

[2]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[3]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficientsY. Bar-Shalom , 1988 .

[4]  John B. Moore,et al.  On-line estimation of hidden Markov model parameters based on the Kullback-Leibler information measure , 1993, IEEE Trans. Signal Process..

[5]  J. C. Stiller,et al.  Online estimation of hidden Markov models , 1999, IEEE Signal Processing Letters.

[6]  Yaakov Bar-Shalom,et al.  Estimation and Tracking: Principles, Techniques, and Software , 1993 .

[7]  Kaddour Najim,et al.  Learning automata and stochastic optimization , 1997 .

[8]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[9]  Vikram Krishnamurthy,et al.  Expectation maximization algorithms for MAP estimation of jump Markov linear systems , 1999, IEEE Trans. Signal Process..

[10]  K. Ito,et al.  On State Estimation in Switching Environments , 1970 .

[11]  Arnaud Doucet,et al.  Stochastic sampling algorithms for state estimation of jump Markov linear systems , 2000, IEEE Trans. Autom. Control..

[12]  Y. Boers,et al.  Interacting multiple model particle filter , 2003 .

[13]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficients , 1988 .

[14]  X. Rong Li,et al.  Estimation of Markovian Jump Systems with Unknown Transition Probabilities through Bayesian Sampling , 2002, Numerical Methods and Application.

[15]  John B. Moore,et al.  On-line identification of hidden Markov models via recursive prediction error techniques , 1994, IEEE Trans. Signal Process..

[16]  O. Costa.  Linear minimum mean square error estimation for discrete-time Markovian jump linear systems , 1994, IEEE Trans. Autom. Control..

[17]  Vikram Krishnamurthy,et al.  An improvement to the interacting multiple model (IMM) algorithm , 2001, IEEE Trans. Signal Process..

[18]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[19]  John B. Moore,et al.  Adaptive Estimation of Hmm Transition Probabilities , 1996, Fourth International Symposium on Signal Processing and Its Applications.

[20]  Jitendra Tugnait,et al.  Adaptive estimation and identification for discrete systems with Markov jump parameters , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[21]  Ehud Weinstein,et al.  Sequential algorithms for parameter estimation based on the Kullback-Leibler information measure , 1990, IEEE Trans. Acoust. Speech Signal Process..

[22]  Amir Averbuch,et al.  Interacting Multiple Model Methods in Target Tracking: A Survey , 1988 .

[23]  Qing Zhang.  Optimal Filtering of Discrete-Time Hybrid Systems , 1999 .

[24]  Gang George Yin,et al.  Recursive algorithms for estimation of hidden Markov models and autoregressive models with Markov regime , 2002, IEEE Trans. Inf. Theory.

[25]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[26]  Arnaud Doucet,et al.  Particle filters for state estimation of jump Markov linear systems , 2001, IEEE Trans. Signal Process..

[27]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .