Statistical inference for ordinary differential equations using gradient matching

A central objective of current systems biology research is explaining the interactions amongst components in biopathways. A standard approach is to view a biopathway as a network of biochemical reactions, which is modelled as a system of ordinary differential equations (ODEs). Conventional inference methods typically rely on searching the space of parameter values, and at each candidate, numerically solving the ODEs and comparing the output with that observed. After choosing an appropriate noise model, the form of the likelihood is defined, and a measure of similarity between the data signals and the signals described by the current set of ODE parameters can be calculated. This process is repeated, as part of either an iterative optimisation scheme or sampling procedure in order to estimate the parameters. However, the computational costs involved with repeatedly numerically solving the ODEs are usually high. Several authors have adopted approaches based on gradient matching, aiming to reduce this computational complexity. These approaches are based on the following two-step procedure. At the first step, interpolation is used to smooth the time series data, in order to avoid modelling noisy observations; in a second step, the kinetic parameters of the ODEs are either optimised or sampled, whilst minimising some metric measuring the difference between the slopes of the tangents to the interpolants, and the parameter-dependent time derivative from the ODEs. In this fashion, the ODEs never have to be numerically integrated, and the problem of inferring the typically unknown initial conditions of the system is removed, as it is not required for matching gradients. A downside to this two-step scheme is that the results of parameter inference are critically dependent on the quality of the initial interpolant. Alternatively, the ODEs can be allowed to regularise the interpolant and it has been demonstrated that it significantly improves the parameter inference accuracy and robustness with respect to noise. This thesis extends and develops methods of gradient matching for parameter inference and model selection in ODE systems in a systems biology context.

[1]  M. Muir Physical Chemistry , 1888, Nature.

[2]  J. Mercer Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[3]  S. Yoshizawa,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 1962, Proceedings of the IRE.

[4]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[5]  Alfred J. Lotka,et al.  The growth of mixed populations: Two species competing for a common food supply , 1978 .

[6]  G Duckett,et al.  Modeling the dynamics of cardiac action potentials. , 2000, Physical review letters.

[7]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[8]  Carl E. Rasmussen,et al.  Derivative Observations in Gaussian Process Models of Dynamic Systems , 2002, NIPS.

[9]  Thore Graepel,et al.  Solving Noisy Linear Operator Equations by Gaussian Processes: Application to Ordinary and Partial Differential Equations , 2003, ICML.

[10]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[11]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[12]  Jiguo Cao,et al.  Parameter estimation for differential equations: a generalized smoothing approach , 2007 .

[13]  Universitext An Introduction to Ordinary Differential Equations , 2006 .

[14]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[15]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[16]  Eleni Bazigou,et al.  Cell signaling and cancer , 2007, Genome Biology.

[17]  A. Pettitt,et al.  Marginal likelihood estimation via power posteriors , 2008 .

[18]  Hulin Wu,et al.  Parameter Estimation for Differential Equation Models Using a Framework of Measurement Error in Regression Models , 2008, Journal of the American Statistical Association.

[19]  Mark A. Girolami,et al.  Bayesian ranking of biochemical system models , 2008, Bioinform..

[20]  Neil D. Lawrence,et al.  Accelerating Bayesian Inference over Nonlinear Differential Equations with Gaussian Processes , 2008, NIPS.

[21]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[22]  S. Göktepe,et al.  Computational modeling of cardiac electrophysiology: A novel finite element approach , 2009 .

[23]  渡邊 澄夫 Algebraic geometry and statistical learning theory , 2009 .

[24]  Ryan P. Adams,et al.  Slice sampling covariance hyperparameters of latent Gaussian models , 2010, NIPS.

[25]  Sumio Watanabe,et al.  Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory , 2010, J. Mach. Learn. Res..

[26]  E. Choi,et al.  Pathological roles of MAPK signaling pathways in human diseases. , 2010, Biochimica et biophysica acta.

[27]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[28]  Hulin Wu,et al.  Sieve Estimation of Constant and Time-Varying Coefficients in Nonlinear Ordinary Differential Equation Models by Considering Both Numerical Error and Measurement Error. , 2010, Annals of statistics.

[29]  S. Zhou Bayesian Model Selection in terms of Kullback-Leibler discrepancy , 2011 .

[30]  Hongzhe Li,et al.  High-Dimensional ODEs Coupled With Mixed-Effects Modeling Techniques for Dynamic Gene Regulatory Network Identification , 2011, Journal of the American Statistical Association.

[31]  Bruce E. Hansen,et al.  Nonparametric Sieve Regression: Least Squares, Averaging Least Squares, and Cross-Validation , 2012 .

[32]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[33]  A. Millar,et al.  The clock gene circuit in Arabidopsis includes a repressilator with additional feedback loops , 2012, Molecular systems biology.

[34]  David Campbell,et al.  Smooth functional tempering for nonlinear differential equation models , 2012, Stat. Comput..

[35]  Ivan Vujačić,et al.  Inferring latent gene regulatory network kinetics , 2013, Statistical applications in genetics and molecular biology.

[37]  David Higdon,et al.  Gaussian Process Modeling of Derivative Curves , 2013, Technometrics.

[38]  Itai Dattner,et al.  Optimal rate of direct estimators in systems of ordinary differential equations linear in functions of the parameters , 2013 .

[39]  S. Vivekanandan,et al.  PROPOGATION OF ACTION POTENTIAL FOR HANSEN ’ S DISEASE AFFECTED NERVE MODEL USING FITZHUGH NAGUMO LIKE EXCITATION , 2013 .

[40]  Dirk Husmeier,et al.  ODE parameter inference using adaptive gradient matching with Gaussian processes , 2013, AISTATS.

[41]  David Barber,et al.  Gaussian Processes for Bayesian Estimation in Ordinary Differential Equations , 2014, ICML.

[42]  Michael P H Stumpf,et al.  Topological sensitivity analysis for systems biology , 2014, Proceedings of the National Academy of Sciences.

[43]  Ernst Wit,et al.  Reproducing kernel Hilbert space based estimation of systems of ordinary differential equations , 2013, Pattern Recognit. Lett..

[44]  Birgit Brüggemeier Improving the utility of Drosophila melanogaster for neurodegenerative disease research by modelling courtship behaviour patterns , 2014 .

[45]  Hulin Wu,et al.  Sparse Additive Ordinary Differential Equations for Dynamic Gene Regulatory Network Modeling , 2014, Journal of the American Statistical Association.

[46]  Dirk Husmeier,et al.  Controversy in mechanistic modelling with Gaussian processes , 2015, ICML.

[47]  Dirk Husmeier,et al.  Computational Inference in Systems Biology , 2015, IWBBIO.

[48]  Dirk Husmeier,et al.  Gradient Matching Methods for Computational Inference in Mechanistic Models for Systems Biology: A Review and Comparative Analysis , 2015, Front. Bioeng. Biotechnol..

[49]  Farhanahani Mahmud,et al.  FPGA Implementation for Cardiac Excitation-Conduction Simulation Based on FitzHugh-Nagumo Model , 2015 .

[50]  Ernst Wit,et al.  Bayesian Smooth-and-Match estimation of ordinary differential equations' parameters with quantifiable solution uncertainty , 2016 .

[51]  M. Filippone,et al.  Approximate parameter inference in systems biology using gradient matching: a comparative evaluation , 2016, Biomedical engineering online.