MCMC for continuous-time discrete-state systems

We propose a simple and novel framework for MCMC inference in continuous-time discrete-state systems with pure jump trajectories. We construct an exact MCMC sampler for such systems by alternately sampling a random discretization of time given a trajectory of the system, and then a new trajectory given the discretization. The first step can be performed efficiently using properties of the Poisson process, while the second step can avail of discrete-time MCMC techniques based on the forward-backward algorithm. We show the advantage of our approach compared to particle MCMC and a uniformization-based sampler.

[1]  R. Wolpert,et al.  Perfect simulation and moment properties for the Matérn type III process , 2010 .

[2]  R. Kohn,et al.  Markov chain Monte Carlo in conditionally Gaussian state space models , 1996 .

[3]  Erhan Çinlar,et al.  Introduction to stochastic processes , 1974 .

[4]  David Sonderman,et al.  Comparing Semi-Markov Processes , 1980, Math. Oper. Res..

[5]  Daryl J. Daley,et al.  An Introduction to the Theory of Point Processes , 2013 .

[6]  R. Nielsen.  Mapping mutations on phylogenies. , 2002, Systematic biology.

[7]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[8]  Darren J. Wilkinson,et al.  Bayesian inference for a discretely observed stochastic kinetic model , 2008, Stat. Comput..

[9]  J. Mateu,et al.  Likelihood Inference for Gibbs Processes in the Analysis of Spatial Point Patterns * , 2001 .

[10]  Eric Horvitz,et al.  Continuous Time Bayesian Networks for Inferring Users’ Presence and Activities with Extensions for Modeling and Evaluation , 2003 .

[11]  Mayank R. Mehta,et al.  Explicit-Duration Hidden Markov Model Inference of UP-DOWN States from Continuous Signals , 2011, PloS one.

[12]  M. Hill,et al.  The Intensity of Spatial Pattern in Plant Communities , 1973 .

[13]  Izzet Sahin.  A generalization of renewal processes , 1993, Oper. Res. Lett..

[14]  D. Kendall.  Stochastic Processes Occurring in the Theory of Queues and their Analysis by the Method of the Imbedded Markov Chain , 1953 .

[15]  Maurice G. Kendall,et al.  The Geographical Distribution of Crop Productivity in England , 1939 .

[16]  Yee Whye Teh,et al.  Gaussian process modulated renewal processes , 2011, NIPS.

[17]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[18]  B. Ripley.  Modelling Spatial Patterns , 1977 .

[19]  Radford M. Neal.  Probabilistic Inference Using Markov Chain Monte Carlo Methods , 2011 .

[20]  Ardavan Saeedi,et al.  Priors over Recurrent Continuous Time Processes , 2011, NIPS.

[21]  Yee Whye Teh,et al.  Bayesian Agglomerative Clustering with Coalescents , 2007, NIPS.

[22]  S. Asmussen,et al.  Applied Probability and Queues , 1989 .

[23]  Stephen G. Walker,et al.  Sampling the Dirichlet Mixture Model with Slices , 2006, Commun. Stat. Simul. Comput..

[24]  Charles J. Mode,et al.  Computational Methods for Renewal Theory and Semi-Markov Processes with Illustrative Examples , 1988 .

[25]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[26]  O. Kallenberg.  Foundations of Modern Probability , 2021, Probability Theory and Stochastic Modelling.

[27]  G. Roberts,et al.  Exact simulation of diffusions , 2005, math/0602523.

[28]  Robert J. Elliott,et al.  Option Pricing for Pure Jump Processes with Markov Switching Compensators , 2006, Finance Stochastics.

[29]  Radford M. Neal.  Slice Sampling , 2003, The Annals of Statistics.

[30]  Christophe Andrieu,et al.  A tutorial on adaptive MCMC , 2008, Stat. Comput..

[31]  H. D. Miller,et al.  The Theory Of Stochastic Processes , 1977, The Mathematical Gazette.

[32]  G. Roberts,et al.  Bayesian analysis for emerging infectious diseases , 2009 .

[33]  Yosihiko Ogata,et al.  On Lewis' simulation method for point processes , 1981, IEEE Trans. Inf. Theory.

[34]  W Feller,et al.  ON SEMI-MARKOV PROCESSES. , 1964, Proceedings of the National Academy of Sciences of the United States of America.

[35]  R. Waagepetersen,et al.  Modern Statistics for Spatial Point Processes * , 2007 .

[36]  Stuart J. Russell,et al.  Dynamic bayesian networks: representation, inference and learning , 2002 .

[37]  S. Frühwirth-Schnatter.  Data Augmentation and Dynamic Linear Models , 1994 .

[38]  P. A. W. Lewis,et al.  Simulation of Nonhomogeneous Poisson Processes with Degree-Two Exponential Polynomial Rate Function , 1979, Oper. Res..

[39]  R. Jarrett.  A note on the intervals between coal-mining disasters , 1979 .

[40]  R. Wolpert,et al.  Likelihood-based inference for Matérn type-III repulsive point processes , 2009, Advances in Applied Probability.

[41]  Ryan P. Adams,et al.  Slice sampling covariance hyperparameters of latent Gaussian models , 2010, NIPS.

[42]  Gunter Bolch,et al.  Queueing Networks and Markov Chains - Modeling and Performance Evaluation with Computer Science Applications, Second Edition , 1998 .

[43]  Ryan P. Adams,et al.  Elliptical slice sampling , 2009, AISTATS.

[44]  D. Cox.  Some Statistical Methods Connected with Series of Events , 1955 .

[45]  Liam Paninski,et al.  Statistical models for neural encoding, decoding, and optimal stimulus design. , 2007, Progress in brain research.

[46]  L. Mark Berliner,et al.  Bayesian Nonparametric Survival Analysis , 1988 .

[47]  S. L. Scott,et al.  The Markov Modulated Poisson Process and Markov Poisson Cascade with Applications to Web Traffic Modeling , 2003 .

[48]  Ryan P. Adams,et al.  Tractable nonparametric Bayesian inference in Poisson processes with Gaussian process intensities , 2009, ICML '09.

[49]  Jerald F. Lawless,et al.  A point-process model incorporating renewals and time trends, with application to repairable systems , 1996 .

[50]  Mark Berman,et al.  Inhomogeneous and modulated gamma processes , 1981 .

[51]  Yee Whye Teh,et al.  Spatial Normalized Gamma Processes , 2009, NIPS.

[52]  Guido Sanguinetti,et al.  Variational inference for Markov jump processes , 2007, NIPS.

[53]  Emery N. Brown,et al.  The Time-Rescaling Theorem and Its Application to Neural Spike Train Data Analysis , 2002, Neural Computation.

[54]  M. Eisen,et al.  Probability and its applications , 1975 .

[55]  Yee Whye Teh,et al.  Fast MCMC sampling for Markov jump processes and continuous time Bayesian networks , 2011, UAI.

[56]  Yee Whye Teh,et al.  Beam sampling for the infinite hidden Markov model , 2008, ICML '08.

[57]  Daphne Koller,et al.  Expectation Propagation for Continuous Time Bayesian Networks , 2005, UAI.

[58]  Vanessa Didelez,et al.  Graphical models for marked point processes based on local independence , 2007, 0710.5874.

[59]  Yee Whye Teh,et al.  Modelling Genetic Variations using Fragmentation-Coagulation Processes , 2011, NIPS.

[60]  A. Jensen,et al.  Markoff chains as an aid in the study of Markoff processes , 1953 .

[61]  Asger Hobolth,et al.  SIMULATION FROM ENDPOINT-CONDITIONED, CONTINUOUS-TIME MARKOV CHAINS ON A FINITE STATE SPACE, WITH APPLICATIONS TO MOLECULAR EVOLUTION. , 2009, The annals of applied statistics.

[62]  Nando de Freitas,et al.  New inference strategies for solving Markov Decision Processes using reversible jump MCMC , 2009, UAI.

[63]  Hervé Philippe,et al.  Uniformization for sampling realizations of Markov processes: applications to Bayesian implementations of codon substitution models , 2008, Bioinform..

[64]  Adrian Baddeley,et al.  spatstat: An R Package for Analyzing Spatial Point Patterns , 2005 .

[65]  D. Wilkinson.  Stochastic modelling for quantitative description of heterogeneous biological systems , 2009, Nature Reviews Genetics.

[66]  Robert E. Kass,et al.  A Spike-Train Probability Model , 2001, Neural Computation.

[67]  P. Fearnhead,et al.  An exact Gibbs sampler for the Markov‐modulated Poisson process , 2006 .

[68]  Sergei Zuyev.  Strong Markov Property of Poisson Processes and Slivnyak Formula , 2006 .

[69]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[70]  Yu Fan,et al.  Sampling for Approximate Inference in Continuous Time Bayesian Networks , 2008, ISAIM.

[71]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[72]  Lothar Breuer.  From Markov jump processes to spatial queues , 2003 .

[73]  Daphne Koller,et al.  Continuous Time Bayesian Networks , 2012, UAI.

[74]  James O. Berger,et al.  Bayesian Analysis for the Poly-Weibull Distribution , 1993 .

[75]  D. Gillespie.  Exact Stochastic Simulation of Coupled Chemical Reactions , 1977 .

[76]  M. Plummer,et al.  CODA: convergence diagnosis and output analysis for MCMC , 2006 .

[77]  Tom Parsons,et al.  Earthquake recurrence on the south Hayward fault is most consistent with a time dependent, renewal process , 2008 .

[78]  Radford M. Neal,et al.  Inferring State Sequences for Non-linear Systems with Embedded Hidden Markov Models , 2003, NIPS.

[79]  Nir Friedman,et al.  Gibbs Sampling in Factorized Continuous-Time Markov Processes , 2008, UAI.

[80]  Mark Berman,et al.  Approximating Point Process Likelihoods with Glim , 1992 .

[81]  Andrew Gelman,et al.  Handbook of Markov Chain Monte Carlo , 2011 .

[82]  Yu Fan,et al.  Learning Continuous-Time Social Network Dynamics , 2009, UAI.

[83]  Henk Tijms,et al.  Stochastic modelling and analysis: a computational approach , 1986 .

[84]  Frank D. Wood,et al.  Inference in Hidden Markov Models with Explicit State Duration Distributions , 2012, IEEE Signal Processing Letters.

[85]  Phillip James Edwin Peebles,et al.  Nature of the distribution of galaxies , 1974 .

[86]  Bo Henry Lindqvist.  Nonparametric Estimation of Time Trend for Repairable Systems Data , 2010 .

[87]  J. George Shanthikumar.  Uniformization and Hybrid Simulation/Analytic Models of Renewal Processes , 1986, Oper. Res..

[88]  Darren J Wilkinson,et al.  Bayesian parameter inference for stochastic biochemical network models using particle Markov chain Monte Carlo , 2011, Interface Focus.

[89]  B. Ripley.  Statistical inference for spatial processes , 1990 .

[90]  Stephen G. Walker,et al.  Bayesian nonparametric survival analysis via Levy driven Markov processes , 2004 .

[91]  Tomasz Burzykowski,et al.  Analysis of photon count data from single-molecule fluorescence experiments , 2003 .