A Comparison of Methods for Quantifying Prediction Uncertainty in Systems Biology

Abstract The parameters of dynamical models of biological processes always possess some degree of uncertainty. This parameter uncertainty translates into an uncertainty of model predictions. The trajectories of unmeasured state variables are examples of such predictions. Quantifying the uncertainty associated with a given prediction is an important problem for model developers and users. However, the nonlinearity and complexity of most dynamical models renders it nontrivial. Here, we evaluate three state-of-the-art approaches for prediction uncertainty quantification using two models of different sizes and computational complexities. We discuss the trade-offs between applicability and statistical interpretability of the different methods, and provide guidelines for their application.

[1]  Gunnar Cedersund,et al.  Prediction Uncertainty Estimation Despite Unidentifiability: An Overview of Recent Developments , 2016 .

[2]  Jens Timmer,et al.  Likelihood based observability analysis and confidence intervals for predictions of dynamic models , 2011, BMC Systems Biology.

[3]  Ali Shahmohammadi,et al.  Sequential Model-Based A-Optimal Design of Experiments When the Fisher Information Matrix Is Noninvertible , 2019, Industrial & Engineering Chemistry Research.

[4]  Jan Hasenauer,et al.  PESTO: Parameter EStimation TOolbox , 2017, Bioinform..

[5]  Eva Balsa-Canto,et al.  A consensus approach for estimating the predictive accuracy of dynamic models in biology , 2015, Comput. Methods Programs Biomed..

[6]  David Henriques,et al.  MEIGO: an open-source software suite based on metaheuristics for global optimization in systems biology and bioinformatics , 2013, BMC Bioinformatics.

[7]  J. Timmer,et al.  Division of labor by dual feedback regulators controls JAK2/STAT5 signaling over broad ligand range , 2011, Molecular systems biology.

[8]  Dagmar Iber,et al.  Analyzing and constraining signaling networks: parameter estimation for the user. , 2012, Methods in molecular biology.

[9]  Eric Moulines,et al.  Adaptive parallel tempering algorithm , 2012, 1205.1076.

[10]  Jörg Stelling,et al.  Systems analysis of cellular networks under uncertainty , 2009, FEBS letters.

[11]  Christopher R Myers,et al.  Extracting Falsifiable Predictions from Sloppy Models , 2007, Annals of the New York Academy of Sciences.

[12]  Jan Hasenauer,et al.  Scalable Inference of Ordinary Differential Equation Models of Biochemical Processes. , 2017, Methods in molecular biology.

[13]  Jens Timmer,et al.  Fast integration-based prediction bands for ordinary differential equation models , 2016, Bioinform..

[14]  John F. MacGregor,et al.  ASQC Chemical Division Technical Conference 1971 Prize Winning Paper Some Problems Associated with the Analysis of Multiresponse Data , 1973 .

[15]  Ursula Klingmüller,et al.  Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood , 2009, Bioinform..