Fast MCMC sampling for Markov jump processes and continuous time Bayesian networks

Markov jump processes and continuous time Bayesian networks are important classes of continuous time dynamical systems. In this paper, we tackle the problem of inferring unobserved paths in these models by introducing a fast auxiliary variable Gibbs sampler. Our approach is based on the idea of uniformization, and sets up a Markov chain over paths by sampling a finite set of virtual jump times and then running a standard hidden Markov model forward filtering-backward sampling algorithm over states at the set of extant and virtual jump times. We demonstrate significant computational benefits over a state-of-the-art Gibbs sampler on a number of continuous time Bayesian networks.

[1]  Nir Friedman,et al.  Gibbs Sampling in Factorized Continuous-Time Markov Processes , 2008, UAI.

[2]  L. Breuer.  Introduction to Stochastic Processes , 2022, Statistical Methods for Climate Scientists.

[3]  Daphne Koller,et al.  Continuous Time Bayesian Networks , 2012, UAI.

[4]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[5]  Guido Sanguinetti,et al.  Variational inference for Markov jump processes , 2007, NIPS.

[6]  Ryan P. Adams,et al.  Slice sampling covariance hyperparameters of latent Gaussian models , 2010, NIPS.

[7]  Jing Xu,et al.  Intrusion Detection using Continuous Time Bayesian Networks , 2010, J. Artif. Intell. Res..

[8]  Jing Xu,et al.  Continuous Time Bayesian Network Reasoning and Learning Engine , 2010, J. Mach. Learn. Res..

[9]  Rasmus Nielsen,et al.  Mapping mutations on phylogenies , 2005 .

[10]  A. Jensen,et al.  Markoff chains as an aid in the study of Markoff processes , 1953 .

[11]  Ryan P. Adams,et al.  Tractable nonparametric Bayesian inference in Poisson processes with Gaussian process intensities , 2009, ICML '09.

[12]  Asger Hobolth,et al.  SIMULATION FROM ENDPOINT-CONDITIONED, CONTINUOUS-TIME MARKOV CHAINS ON A FINITE STATE SPACE, WITH APPLICATIONS TO MOLECULAR EVOLUTION. , 2009, The annals of applied statistics.

[13]  Eric Horvitz,et al.  Continuous Time Bayesian Networks for Inferring Users’ Presence and Activities with Extensions for Modeling and Evaluation , 2003 .

[14]  P. Fearnhead,et al.  An exact Gibbs sampler for the Markov‐modulated Poisson process , 2006 .

[15]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[16]  Yu Fan,et al.  Sampling for Approximate Inference in Continuous Time Bayesian Networks , 2008, ISAIM.

[17]  D. Gillespie.  Exact Stochastic Simulation of Coupled Chemical Reactions , 1977 .

[18]  M. Plummer,et al.  CODA: convergence diagnosis and output analysis for MCMC , 2006 .

[19]  Radford M. Neal,et al.  Inferring State Sequences for Non-linear Systems with Embedded Hidden Markov Models , 2003, NIPS.

[20]  Nir Friedman,et al.  Mean Field Variational Approximation for Continuous-Time Bayesian Networks , 2009, J. Mach. Learn. Res..

[21]  Daphne Koller,et al.  Expectation Propagation for Continuous Time Bayesian Networks , 2005, UAI.