Bayesian collocation tempering and generalized profiling for estimation of parameters from differential equation models

The widespread use of ordinary differential equation (ODE) models has long been underrepresented in the statistical literature. The most common methods for estimating parameters from ODE models are nonlinear least squares and an MCMC based method. Both of these methods depend on a likelihood involving the numerical solution to the ODE. The challenge faced by these methods is parameter spaces that are difficult to navigate, exacerbated by the wide variety of behaviours that a single ODE model can produce with respect to small changes in parameter values.%%%%In this work, two competing methods, generalized profile estimation and Bayesian collocation tempering are described. Both of these methods use a basis expansion to approximate the ODE solution in the likelihood, where the shape of the basis expansion, or data smooth, is guided by the ODE model. This approximation to the ODE, smooths out the likelihood surface, reducing restrictions on parameter movement.%%%%Generalized Profile Estimation maximizes the profile likelihood for the ODE parameters while profiling out the basis coefficients of the data smooth. The smoothing parameter determines the balance between fitting the data and the ODE model, and consequently is used to build a parameter cascade, reducing the dimension of the estimation problem. Generalized profile estimation is described with under a constraint to ensure the smooth follows known behaviour such as monotonicity or non-negativity.%%%%Bayesian collocation tempering, uses a sequence posterior densities with smooth approximations to the ODE solution. The level of the approximation is determined by the value of the smoothing parameter, which also determines the level of smoothness in the likelihood surface. In an algorithm similar to parallel tempering, parallel MCMC chains are run to sample from the sequence of posterior densities, while allowing ODE parameters to swap between chains. This method is introduced and tested against a variety of alternative Bayesian models, in terms of posterior variance and rate of convergence.%%%%The performance of generalized profile estimation and Bayesian collocation tempering are tested and compared using simulated data sets from the FitzHugh-Nagumo ODE system and real data from nylon production dynamics.