Online Bayesian estimation of transition probabilities for Markovian jump systems

Markovian jump systems (MJSs) evolve in a jump-wise manner by switching among simpler models, according to a finite Markov chain, whose parameters are commonly assumed known. This paper addresses the problem of state estimation of MJS with unknown transition probability matrix (TPM) of the embedded Markov chain governing the jumps. Under the assumption of a time-invariant but random TPM, an approximate recursion for the TPMs posterior probability density function (PDF) within the Bayesian framework is obtained. Based on this recursion, four algorithms for online minimum mean-square error (MMSE) estimation of the TPM are derived. The first algorithm (for the case of a two-state Markov chain) computes the MMSE estimate exactly, if the likelihood of the TPM is linear in the transition probabilities. Its computational load is, however, increasing with the data length. To limit the computational cost, three alternative algorithms are further developed based on different approximation techniques-truncation of high order moments, quasi-Bayesian approximation, and numerical integration, respectively. The proposed TPM estimation is naturally incorporable into a typical online Bayesian estimation scheme for MJS [e.g., generalized pseudo-Bayesian (GPB) or interacting multiple model (IMM)]. Thus, adaptive versions of MJS state estimators with unknown TPM are provided. Simulation results of TPM-adaptive IMM algorithms for a system with failures and maneuvering target tracking are presented.

[1]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[2]  Y. Sawaragi,et al.  Adaptive estimation for a linear system with interrupted observation , 1973 .

[3]  Demetrios Kazakos,et al.  Recursive estimation of prior probabilities using a mixture , 1977, IEEE Trans. Inf. Theory.

[4]  A. Haddad,et al.  State estimation under uncertain observations with unknown statistics , 1979, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[5]  A. F. Smith,et al.  A Quasi‐Bayes Sequential Procedure for Mixtures , 1978 .

[6]  Jitendra K. Tugnait,et al.  Adaptive estimation in linear systems with unknown Markovian noise statistics , 1980, IEEE Trans. Inf. Theory.

[7]  Jitendra Tugnait,et al.  Adaptive estimation and identification for discrete systems with Markov jump parameters , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[8]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[9]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[10]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficients , 1988 .

[11]  Amir Averbuch,et al.  Interacting Multiple Model Methods in Target Tracking: A Survey , 1988 .

[12]  Jerry M. Mendel,et al.  Optimal simultaneous detection and estimation of filtered discrete semi-Markov chains , 1988, IEEE Trans. Inf. Theory.

[13]  G. R. Dattatreya,et al.  Estimation of mixing probabilities in multiclass finite mixtures , 1990, IEEE Trans. Syst. Man Cybern..

[14]  Yaakov Bar-Shalom,et al.  Estimation and Tracking: Principles, Techniques, and Software , 1993 .

[15]  S. Port.  Theoretical Probability for Applications , 1993 .

[16]  Chin-Hui Lee,et al.  Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains , 1994, IEEE Trans. Speech Audio Process..

[17]  Chin-Hui Lee,et al.  Bayesian adaptive learning of the parameters of hidden Markov model for speech recognition , 1995, IEEE Trans. Speech Audio Process..

[18]  Yakov Bar-Shalom,et al.  Multitarget-Multisensor Tracking: Principles and Techniques , 1995 .

[19]  Samuel S. Blackman,et al.  Evaluation of IMM filtering for an air defense system application , 1995, Optics & Photonics.

[20]  X. Rong Li,et al.  Hybrid Estimation Techniques , 1996 .

[21]  Y. Bar-Shalom,et al.  Multiple-model estimation with variable structure , 1996, IEEE Trans. Autom. Control..

[22]  Chin-Hui Lee,et al.  On-line adaptive learning of the continuous density hidden Markov model based on approximate recursive Bayes estimate , 1997, IEEE Trans. Speech Audio Process..

[23]  Michael A. West,et al.  Bayesian forecasting and dynamic models (2nd ed.) , 1997 .

[24]  Vladimir Pavlovic,et al.  A dynamic Bayesian network approach to figure tracking using learned dynamic models , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[25]  Vikram Krishnamurthy,et al.  Expectation maximization algorithms for MAP estimation of jump Markov linear systems , 1999, IEEE Trans. Signal Process..

[26]  X. R. Li,et al.  Multiple-model estimation with variable structure. III. Model-group switching algorithm , 1999 .

[27]  X. R. Li,et al.  Multiple-model estimation with variable structure. IV. Design and evaluation of model-group switching algorithm , 1999 .

[28]  David D. Sworder,et al.  Estimation Problems in Hybrid Systems , 1999 .

[29]  Qiang Huo,et al.  On adaptive decision rules and decision parameter adaptation for automatic speech recognition , 2000, Proceedings of the IEEE.

[30]  X. Rong Li,et al.  Multiple-model estimation with variable structure. II. Model-set adaptation , 2000, IEEE Trans. Autom. Control..

[31]  Geoffrey E. Hinton,et al.  Variational Learning for Switching State-Space Models , 2000, Neural Computation.

[32]  Youmin Zhang,et al.  Multiple-model estimation with variable structure. V. Likely-model set algorithm , 2000, IEEE Trans. Aerosp. Electron. Syst..

[33]  X. Rong Li,et al.  Multiple-Model Estimation with Variable Structure—Part II: Model-Set Adaptation , 2000 .

[34]  Vikram Krishnamurthy,et al.  An improvement to the interacting multiple model (IMM) algorithm , 2001, IEEE Trans. Signal Process..

[35]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[36]  Robin J. Evans,et al.  Hidden Markov model multiarm bandits: a methodology for beam scheduling in multitarget tracking , 2001, IEEE Trans. Signal Process..

[37]  Arnaud Doucet,et al.  Particle filters for state estimation of jump Markov linear systems , 2001, IEEE Trans. Signal Process..

[38]  L. Bloomer,et al.  Are more models better?: the effect of the model transition matrix on the IMM filter , 2002, Proceedings of the Thirty-Fourth Southeastern Symposium on System Theory (Cat. No.02EX540).

[39]  X. R. Li,et al.  Chapter 10 Engineer ’ s Guide to Variable-Structure Multiple-Model Estimation for Tracking , 2022 .