Learning interpretable continuous-time models of latent stochastic dynamical systems

We develop an approach to learn an interpretable semi-parametric model of a latent continuous-time stochastic dynamical system, assuming noisy high-dimensional outputs sampled at uneven times. The dynamics are described by a nonlinear stochastic differential equation (SDE) driven by a Wiener process, with a drift evolution function drawn from a Gaussian process (GP) conditioned on a set of learnt fixed points and corresponding local Jacobian matrices. This form yields a flexible nonparametric model of the dynamics, with a representation corresponding directly to the interpretable portraits routinely employed in the study of nonlinear dynamical systems. The learning algorithm combines inference of continuous latent paths underlying observed data with a sparse variational description of the dynamical process. We demonstrate our approach on simulated data from different nonlinear dynamical systems.

[1]  Maneesh Sahani,et al.  Temporal alignment and latent Gaussian process factor inference in population spike trains , 2018, bioRxiv.

[2]  Neil D. Lawrence,et al.  Variational Gaussian Process Dynamical Systems , 2011, NIPS.

[3]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[4]  S. Kireev,et al.  Study of molecular iodine, iodate ions, iodide ions, and triiodide ions solutions absorption in the UV and visible light spectral bands , 2015 .

[5]  John P. Cunningham,et al.  Empirical models of spiking in neural populations , 2011, NIPS.

[6]  David Sussillo,et al.  FixedPointFinder: A Tensorflow toolbox for identifying and characterizing fixed points in recurrent neural networks , 2018, J. Open Source Softw..

[7]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[8]  Emre Ertin,et al.  Assumed density filtering for learning Gaussian process models , 2011, 2011 IEEE Statistical Signal Processing Workshop (SSP).

[9]  M. Sahani,et al.  Cortical control of arm movements: a dynamical systems perspective. , 2013, Annual review of neuroscience.

[10]  James Hensman,et al.  Identification of Gaussian Process State Space Models , 2017, NIPS.

[11]  David J. Fleet,et al.  Gaussian Process Dynamical Models , 2005, NIPS.

[12]  Guido Sanguinetti,et al.  Expectation propagation for continuous time stochastic processes , 2015, 1512.06098.

[13]  N. Ganapathisubramanian,et al.  Tristability in the iodate–As(III) chemical system arising from a model of stirring and mixing effects , 1991 .

[14]  Carl E. Rasmussen,et al.  State-Space Inference and Learning with Gaussian Processes , 2010, AISTATS.

[15]  Dan Cornford,et al.  Variational Inference for Diffusion Processes , 2007, NIPS.

[16]  Carl E. Rasmussen,et al.  Variational Gaussian Process State-Space Models , 2014, NIPS.

[17]  Maneesh Sahani,et al.  Empirical fixed point bifurcation analysis , 2018, ArXiv.

[18]  Dan Cornford,et al.  Gaussian Process Approximations of Stochastic Differential Equations , 2007, Gaussian Processes in Practice.

[19]  Uwe D. Hanebeck,et al.  Analytic moment-based Gaussian process filtering , 2009, ICML '09.

[20]  Andreas Ruttor,et al.  Approximate Bayes learning of stochastic differential equations. , 2017, Physical review. E.

[21]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[22]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[23]  David Sussillo,et al.  Opening the Black Box: Low-Dimensional Dynamics in High-Dimensional Recurrent Neural Networks , 2013, Neural Computation.

[24]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[25]  Marc Peter Deisenroth,et al.  Expectation Propagation in Gaussian Process Dynamical Systems , 2012, NIPS.