Approximate inference in continuous time Gaussian-Jump processes

We present a novel approach to inference in conditionally Gaussian continuous time stochastic processes, where the latent process is a Markovian jump process. We first consider the case of jump-diffusion processes, where the drift of a linear stochastic differential equation can jump at arbitrary time points. We derive partial differential equations for exact inference and present a very efficient mean field approximation. By introducing a novel lower bound on the free energy, we then generalise our approach to Gaussian processes with arbitrary covariance, such as the non-Markovian RBF covariance. We present results on both simulated and real data, showing that the approach is very accurate in capturing latent dynamics and can be useful in a number of real data modelling tasks.

[1]  M. Opper,et al.  Chapter 1 Approximate inference for continuous-time Markov processes , 2009 .

[2]  Andreas Ruttor,et al.  Switching regulatory models of cellular stress response , 2009, Bioinform..

[3]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[4]  Gürol M. Süel,et al.  An excitable gene regulatory circuit induces transient cellular differentiation , 2006, Nature.

[5]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[6]  M. Barenco,et al.  Ranked prediction of p53 targets using hidden variable dynamic modeling , 2006, Genome Biology.

[7]  Nir Friedman,et al.  Mean Field Variational Approximation for Continuous-Time Bayesian Networks , 2009, J. Mach. Learn. Res..

[8]  Chiara Sabatti,et al.  Network component analysis: Reconstruction of regulatory signals in biological systems , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[9]  M. Lifshits Gaussian Random Functions , 1995 .

[10]  Neil D. Lawrence,et al.  Modelling transcriptional regulation using Gaussian Processes , 2006, NIPS.

[11]  Guido Sanguinetti,et al.  Variational inference for Markov jump processes , 2007, NIPS.

[12]  Andreas Ruttor,et al.  Efficient statistical inference for stochastic reaction processes. , 2009, Physical review letters.

[13]  Dan Cornford,et al.  Gaussian Process Approximations of Stochastic Differential Equations , 2007, Gaussian Processes in Practice.

[14]  Guido Sanguinetti,et al.  Learning combinatorial transcriptional dynamics from gene expression data , 2010, Bioinform..

[15]  C. Gardiner Handbook of Stochastic Methods , 1983 .

[16]  David Barber,et al.  Expectation Correction for Smoothed Inference in Switching Linear Dynamical Systems , 2006, J. Mach. Learn. Res..

[17]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[18]  Neil D. Lawrence,et al.  Latent Force Models , 2009, AISTATS.

[19]  Daphne Koller,et al.  Continuous Time Bayesian Networks , 2012, UAI.