ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems

Parameter inference in ordinary differential equations is an important problem in many applied sciences and in engineering, especially in a data-scarce setting. In this work, we introduce a novel generative modeling approach based on constrained Gaussian processes and leverage it to build a computationally and data efficient algorithm for state and parameter inference. In an extensive set of experiments, our approach outperforms the current state of the art for parameter inference both in terms of accuracy and computational cost. It also shows promising results for the much more challenging problem of model selection.

[1]  K. Emanuel,et al.  Optimal Sites for Supplementary Weather Observations: Simulation with a Small Model , 1998 .

[2]  T. Brubaker,et al.  Nonlinear Parameter Estimation , 1979 .

[3]  Mark A. Girolami,et al.  Bayesian ranking of biochemical system models , 2008, Bioinform..

[4]  Carl E. Rasmussen,et al.  Derivative Observations in Gaussian Process Models of Dynamic Systems , 2002, NIPS.

[5]  Neil D. Lawrence,et al.  Accelerating Bayesian Inference over Nonlinear Differential Equations with Gaussian Processes , 2008, NIPS.

[6]  Hulin Wu,et al.  Sparse Additive Ordinary Differential Equations for Dynamic Gene Regulatory Network Modeling , 2014, Journal of the American Statistical Association.

[7]  Jouni Hartikainen,et al.  Kalman filtering and smoothing solutions to temporal Gaussian process regression models , 2010, 2010 IEEE International Workshop on Machine Learning for Signal Processing.

[8]  Stefan Bauer,et al.  Scalable Variational Inference for Dynamical Systems , 2017, NIPS.

[9]  Nico S. Gorbach,et al.  Fast Gaussian Process Based Gradient Matching for Parameter Identification in Systems of Nonlinear ODEs , 2018, AISTATS.

[10]  Ali Shojaie,et al.  Network Reconstruction From High-Dimensional Ordinary Differential Equations , 2016, Journal of the American Statistical Association.

[11]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[12]  R. FitzHugh.  Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[13]  Karl J. Friston,et al.  Nonlinear Dynamic Causal Models for Fmri Nonlinear Dynamic Causal Models for Fmri Nonlinear Dynamic Causal Models for Fmri , 2022 .

[14]  Dirk Husmeier,et al.  ODE parameter inference using adaptive gradient matching with Gaussian processes , 2013, AISTATS.

[15]  Ernst Wit,et al.  Reproducing kernel Hilbert space based estimation of systems of ordinary differential equations , 2013, Pattern Recognit. Lett..

[16]  David Barber,et al.  Gaussian Processes for Bayesian Estimation in Ordinary Differential Equations , 2014, ICML.

[17]  J. Varah.  A Spline Least Squares Method for Numerical Parameter Estimation in Differential Equations , 1982 .

[18]  Alfred J. Lotka,et al.  The growth of mixed populations: Two species competing for a common food supply , 1978 .

[19]  Philippe Wenk,et al.  AReS and MaRS - Adversarial and MMD-Minimizing Regression for SDEs , 2019, ICML.

[20]  Dirk Husmeier,et al.  Fast inference in nonlinear dynamical systems using gradient matching , 2016, ICML 2016.

[21]  Mark A. Girolami,et al.  Bayesian ranking of biochemical system models , 2008, Bioinform..

[22]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[23]  J. NAGUMOt,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 2006 .

[24]  Markus Heinonen,et al.  Learning unknown ODE models with Gaussian processes , 2018, ICML.

[25]  Maurizio Filippone,et al.  Constraining the Dynamics of Deep Probabilistic Models , 2018, ICML.

[26]  Dirk Husmeier,et al.  Controversy in mechanistic modelling with Gaussian processes , 2015, ICML.