Fast Gaussian Process Based Gradient Matching for Parameter Identification in Systems of Nonlinear ODEs

Parameter identification and comparison of dynamical systems is a challenging task in many fields. Bayesian approaches based on Gaussian process regression over time-series data have been successfully applied to infer the parameters of a dynamical system without explicitly solving it. While the benefits in computational cost are well established, a rigorous mathematical framework has been missing. We offer a novel interpretation which leads to a better understanding and improvements in state-of-the-art performance in terms of accuracy for nonlinear dynamical systems.

[1]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[2]  S. Yoshizawa,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 1962, Proceedings of the IRE.

[3]  Dirk Husmeier,et al.  ODE parameter inference using adaptive gradient matching with Gaussian processes , 2013, AISTATS.

[4]  Xinxin Zhang,et al.  Inference of transcriptional regulation using multi-objective optimization with Gaussian Process , 2016, 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI).

[5]  David Barber,et al.  Gaussian Processes for Bayesian Estimation in Ordinary Differential Equations , 2014, ICML.

[6]  Joshua B. Tenenbaum,et al.  Structure Discovery in Nonparametric Regression through Compositional Kernel Search , 2013, ICML.

[7]  Stefan Bauer,et al.  Scalable Variational Inference for Dynamical Systems , 2017, NIPS.

[8]  Stefan Bauer,et al.  Model Selection for Gaussian Process Regression , 2017, GCPR.

[9]  Stefan Bauer,et al.  Efficient and Flexible Inference for Stochastic Systems , 2017, NIPS.

[10]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[11]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[12]  Ernst Wit,et al.  Reproducing kernel Hilbert space based estimation of systems of ordinary differential equations , 2013, Pattern Recognit. Lett..

[13]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[14]  B. Macdonald.  Statistical inference for ordinary differential equations using gradient matching , 2017 .

[15]  Neil D. Lawrence,et al.  Accelerating Bayesian Inference over Nonlinear Differential Equations with Gaussian Processes , 2008, NIPS.

[16]  Dirk Husmeier,et al.  Fast inference in nonlinear dynamical systems using gradient matching , 2016, ICML 2016.

[17]  Jiguo Cao,et al.  Parameter estimation for differential equations: a generalized smoothing approach , 2007 .

[18]  M. Schervish,et al.  On posterior consistency in nonparametric regression problems , 2007 .

[19]  David Duvenaud,et al.  Automatic model construction with Gaussian processes , 2014 .

[20]  R. FitzHugh.  Impulses and Physiological States in Theoretical Models of Nerve Membrane. , 1961, Biophysical journal.

[21]  Dirk Husmeier,et al.  Gradient Matching Methods for Computational Inference in Mechanistic Models for Systems Biology: A Review and Comparative Analysis , 2015, Front. Bioeng. Biotechnol..

[22]  Alfred J. Lotka,et al.  The growth of mixed populations: Two species competing for a common food supply , 1978 .

[23]  Michael P H Stumpf,et al.  Topological sensitivity analysis for systems biology , 2014, Proceedings of the National Academy of Sciences.

[24]  Philippe Wenk,et al.  ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems , 2019, AAAI.

[25]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[26]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.