Generative ODE Modeling with Known Unknowns

In several crucial applications, domain knowledge is encoded by a system of ordinary differential equations (ODE). A motivating example is intensive care unit patients: The dynamics of some vital physiological variables such as heart rate, blood pressure and arterial compliance can be approximately described by a known system of ODEs. Typically, some of the ODE variables are directly observed while some are unobserved, and in addition many other variables are observed but not modeled by the ODE, for example body temperature. Importantly, the unobserved ODE variables are ``known-unknowns'': We know they exist and their functional dynamics, but cannot measure them directly, nor do we know the function tying them to all observed measurements. Estimating these known-unknowns is often highly valuable to physicians. Under this scenario we wish to: (i) learn the static parameters of the ODE generating each observed time-series (ii) infer the dynamic sequence of all ODE variables including the known-unknowns, and (iii) extrapolate the future of the ODE variables and the observations of the time-series. We address this task with a variational autoencoder incorporating the known ODE function, called GOKU-net for Generative ODE modeling with Known Unknowns. We test our method on videos of pendulums with unknown length, and a model of the cardiovascular system.

[1]  Dirk Husmeier,et al.  ODE parameter inference using adaptive gradient matching with Gaussian processes , 2013, AISTATS.

[2]  David Barber,et al.  Gaussian Processes for Bayesian Estimation in Ordinary Differential Equations , 2014, ICML.

[3]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[4]  Suchi Saria,et al.  Learning Treatment-Response Models from Multivariate Longitudinal Data , 2017, UAI.

[5]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[6]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[7]  Stefan Bauer,et al.  Scalable Variational Inference for Dynamical Systems , 2017, NIPS.

[8]  Markus Heinonen,et al.  Learning unknown ODE models with Gaussian processes , 2018, ICML.

[9]  T. G. Coleman,et al.  Circulation: overall regulation. , 1972, Annual review of physiology.

[10]  David Duvenaud,et al.  Latent Ordinary Differential Equations for Irregularly-Sampled Time Series , 2019, NeurIPS.

[11]  Joachim M. Buhmann,et al.  Disentangled State Space Representations , 2019, ArXiv.

[12]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[13]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[14]  R. Baker,et al.  Mechanistic models versus machine learning, a fight worth fighting for the biological community? , 2018, Biology Letters.

[15]  Uri Shalit,et al.  Structured Inference Networks for Nonlinear State Space Models , 2016, AAAI.

[16]  Neil D. Lawrence,et al.  Latent Force Models , 2009, AISTATS.

[17]  Florian Nadel,et al.  Stochastic Processes And Filtering Theory , 2016 .

[18]  Jason Yosinski,et al.  Hamiltonian Neural Networks , 2019, NeurIPS.

[19]  Gilles Clermont,et al.  From Inverse Problems in Mathematical Physiology to Quantitative Differential Diagnoses , 2007, PLoS Comput. Biol..

[20]  C. T. Kelley,et al.  Patient-specific modeling of cardiovascular and respiratory dynamics during hypercapnia. , 2013, Mathematical biosciences.

[21]  David Duvenaud,et al.  Neural Ordinary Differential Equations , 2018, NeurIPS.

[22]  Ernst Wit,et al.  Reproducing kernel Hilbert space based estimation of systems of ordinary differential equations , 2013, Pattern Recognit. Lett..

[23]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[24]  Barbara E. Engelhardt,et al.  Patient-Specific Effects of Medication Using Latent Force Models with Gaussian Processes , 2020, AISTATS.

[25]  Nico S. Gorbach,et al.  Fast Gaussian Process Based Gradient Matching for Parameter Identification in Systems of Nonlinear ODEs , 2018, AISTATS.

[26]  A. J. Lotka.  Contribution to the Theory of Periodic Reactions , 1909 .

[27]  David Sontag,et al.  Grounded Recurrent Neural Networks , 2017, ArXiv.

[28]  Bertrand Chapron,et al.  Learning Latent Dynamics for Partially-Observed Chaotic Systems , 2019, Chaos.

[29]  Mette S Olufsen,et al.  A practical approach to parameter estimation applied to model predicting heart rate regulation , 2013, Journal of mathematical biology.

[30]  J Geoffrey Chase,et al.  Minimal haemodynamic system model including ventricular interaction and valve dynamics. , 2004, Medical engineering & physics.

[31]  R. E. Kalman,et al.  Contributions to the Theory of Optimal Control , 1960 .

[32]  Philippe Wenk,et al.  ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems , 2019, AAAI.