Estimation of ordinary differential equation models with discretization error quantification

We consider estimation of ordinary differential equation (ODE) models from noisy observations. For this problem, one conventional approach is to fit numerical solutions (e.g., Euler, Runge--Kutta) of ODEs to data. However, such a method does not account for the discretization error in numerical solutions and has limited estimation accuracy. In this study, we develop an estimation method that quantifies the discretization error based on data. The key idea is to model the discretization error as random variables and estimate their variance simultaneously with the ODE parameter. The proposed method has the form of iteratively reweighted least squares, where the discretization error variance is updated with the isotonic regression algorithm and the ODE parameter is updated by solving a weighted least squares problem using the adjoint system. Experimental results demonstrate that the proposed method improves estimation accuracy by accounting for the discretization error in a data-driven manner.

[1]  H. D. Brunk,et al.  Statistical inference under order restrictions : the theory and application of isotonic regression , 1973 .

[2]  Nicholas J. Higham,et al.  A New Approach to Probabilistic Rounding Error Analysis , 2019, SIAM J. Sci. Comput..

[3]  F. T. Wright,et al.  Order restricted statistical inference , 1988 .

[4]  E. Hairer,et al.  Geometric Numerical Integration: Structure Preserving Algorithms for Ordinary Differential Equations , 2004 .

[5]  Ben Calderhead,et al.  Probabilistic Linear Multistep Methods , 2016, NIPS.

[6]  S. Ito,et al.  Adjoint-based exact Hessian-vector multiplication using symplectic Runge-Kutta methods , 2019, ArXiv.

[7]  J. Kalbfleisch.  Statistical Inference Under Order Restrictions , 1975 .

[8]  Simeon Ola Fatunla.  LINEAR MULTISTEP METHODS , 1988 .

[9]  M. J. D. Powell,et al.  On search directions for minimization algorithms , 1973, Math. Program..

[10]  F. Krogh,et al.  Solving Ordinary Differential Equations , 2019, Programming for Computations - Python.

[11]  S. Moolgavkar,et al.  A Method for Computing Profile-Likelihood- Based Confidence Intervals , 1988 .

[12]  Luigi Grippo,et al.  On the convergence of the block nonlinear Gauss-Seidel method under convex constraints , 2000, Oper. Res. Lett..

[13]  Hulin Wu,et al.  Sieve Estimation of Constant and Time-Varying Coefficients in Nonlinear Ordinary Differential Equation Models by Considering Both Numerical Error and Measurement Error. , 2010, Annals of statistics.

[14]  J. M. Sanz-Serna,et al.  Symplectic integrators for Hamiltonian problems: an overview , 1992, Acta Numerica.

[15]  Søren Hauberg,et al.  Probabilistic Solutions to Differential Equations and their Application to Riemannian Statistics , 2013, AISTATS.

[16]  Ernst Hairer,et al.  Solving Ordinary Differential Equations I: Nonstiff Problems , 2009 .

[17]  M. Girolami,et al.  Bayesian Solution Uncertainty Quantification for Differential Equations , 2013 .

[18]  Mark A. Girolami,et al.  Bayesian Probabilistic Numerical Methods , 2017, SIAM Rev..

[19]  B. Leimkuhler,et al.  Simulating Hamiltonian Dynamics , 2005 .

[20]  C. Eeden.  Restricted Parameter Space Estimation Problems , 2006 .

[21]  Jesús María Sanz-Serna,et al.  Symplectic Runge-Kutta Schemes for Adjoint Equations, Automatic Differentiation, Optimal Control, and More , 2015, SIAM Rev..

[22]  Simo Särkkä,et al.  A probabilistic model for the numerical solution of initial value problems , 2016, Statistics and Computing.

[23]  S. Yoshizawa,et al.  An Active Pulse Transmission Line Simulating Nerve Axon , 1962, Proceedings of the IRE.

[24]  E. Hairer,et al.  Solving Ordinary Differential Equations II , 2010 .

[25]  Simo Särkkä,et al.  Probabilistic solutions to ordinary differential equations as nonlinear Bayesian filtering: a new perspective , 2018, Statistics and Computing.

[26]  Jenný Brynjarsdóttir,et al.  Learning about physical parameters: the importance of model discrepancy , 2014 .

[27]  Andrew M. Stuart,et al.  Statistical analysis of differential equations: introducing probability measures on numerical solutions , 2016, Statistics and Computing.

[28]  Philipp Hennig,et al.  Active Uncertainty Calibration in Bayesian ODE Solvers , 2016, UAI.

[29]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[30]  Robert G. Aykroyd,et al.  Bayesian Probabilistic Numerical Methods in Time-Dependent State Estimation for Industrial Hydrocyclone Equipment , 2017, Journal of the American Statistical Association.

[31]  Assyr Abdulle,et al.  Random time step probabilistic methods for uncertainty quantification in chaotic and geometric numerical integration , 2018, Statistics and Computing.

[32]  David Duvenaud,et al.  Probabilistic ODE Solvers with Runge-Kutta Means , 2014, NIPS.

[33]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[34]  T. Ferguson.  A Course in Large Sample Theory , 1996 .

[35]  Daniela Calvetti,et al.  Linear multistep methods, particle filtering and sequential Monte Carlo , 2013 .

[36]  T. J. Sullivan,et al.  Strong convergence rates of probabilistic integrators for ordinary differential equations , 2017, Statistics and Computing.

[37]  E. Hairer,et al.  Geometric numerical integration illustrated by the Störmer–Verlet method , 2003, Acta Numerica.

[38]  Ernst Hairer,et al.  Achieving Brouwer’s law with implicit Runge–Kutta methods , 2008 .

[39]  J. M. Sanz-Serna,et al.  Numerical Hamiltonian Problems , 1994 .

[40]  T. E. Hull,et al.  Comparing Numerical Methods for Ordinary Differential Equations , 1972 .

[41]  J. Neumann,et al.  Numerical inverting of matrices of high order. II , 1951 .

[42]  J. Butcher.  Numerical methods for ordinary differential equations , 2003 .

[43]  E. Lorenz.  Deterministic nonperiodic flow , 1963 .