BioPreDyn-bench: benchmark problems for kinetic modelling in systems biology

Dynamic modelling is one of the cornerstones of systems biology. Many research efforts are currently being invested in the development and exploitation of large-scale kinetic models. The associated problems of parameter estimation (model calibration) and optimal experimental design are particularly challenging. The community has already developed many methods and software packages which aim to facilitate these tasks. However, there is a lack of suitable benchmark problems which allow a fair and systematic evaluation and comparison of these contributions. Here we present BioPreDyn-bench, a set of challenging parameter estimation problems which aspire to serve as reference test cases in this area. This set comprises six problems including medium and large-scale kinetic models of the bacterium E. coli, baker's yeast S. cerevisiae, the vinegar fly D. melanogaster, Chinese Hamster Ovary cells, and a generic signal transduction network. The level of description includes metabolism, transcription, signal transduction, and development. For each problem we provide (i) a basic description and formulation, (ii) implementations ready-to-run in several formats, (iii) computational results obtained with specific solvers, (iv) a basic analysis and interpretation. This suite of benchmark problems can be readily used to evaluate and compare parameter estimation methods. Further, it can also be used to build test problems for sensitivity and identifiability analysis, model reduction and optimal experimental design methods. The suite, including codes and documentation, can be freely downloaded from this http URL

[1]  Pedro Mendes,et al.  Yeast 5 – an expanded reconstruction of the Saccharomyces cerevisiae metabolic network , 2012, BMC Systems Biology.

[2]  Steffen Klamt,et al.  Transforming Boolean models to continuous models: methodology and application to T-cell receptor signaling , 2009, BMC Systems Biology.

[3]  Anton Crombach,et al.  Efficient Reverse-Engineering of a Developmental Gene Regulatory Network , 2012, PLoS Comput. Biol..

[4]  Rudiyanto Gunawan,et al.  Iterative approach to model identification of biological networks , 2005, BMC Bioinformatics.

[5]  Eva Balsa-Canto,et al.  Bioinformatics Applications Note Systems Biology Genssi: a Software Toolbox for Structural Identifiability Analysis of Biological Models , 2022 .

[6]  Dag Wedelin,et al.  Benchmarks for identification of ordinary differential equations from time series data , 2009, Bioinform..

[7]  Carmen G. Moles,et al.  Parameter estimation in biochemical pathways: a comparison of global optimization methods. , 2003, Genome research.

[8]  Eva Balsa-Canto,et al.  An iterative identification procedure for dynamic modeling of biochemical networks , 2010, BMC Systems Biology.

[9]  Johannes Jaeger,et al.  Gene Circuit Analysis of the Terminal Gap Gene huckebein , 2009, PLoS Comput. Biol..

[10]  Julio Saez-Rodriguez,et al.  Network topology and parameter estimation: from experimental design methods to gene regulatory network kinetics using a community based approach , 2014, BMC Systems Biology.

[11]  Marija Cvijovic,et al.  Kinetic models in industrial biotechnology - Improving cell factory performance. , 2014, Metabolic engineering.

[12]  Dario Floreano,et al.  Generating Realistic In Silico Gene Networks for Performance Assessment of Reverse Engineering Methods , 2009, J. Comput. Biol..

[13]  Hong Wang,et al.  Insights into the behaviour of systems biology models from dynamic sensitivity and identifiability analysis: a case study of an NF-kappaB signalling pathway. , 2006, Molecular bioSystems.

[14]  Eva Balsa-Canto,et al.  Parameter estimation and optimal experimental design. , 2008, Essays in biochemistry.

[15]  C. Chassagnole,et al.  Dynamic modeling of the central carbon metabolism of Escherichia coli. , 2002, Biotechnology and bioengineering.

[16]  Gunnar Cedersund,et al.  Conclusions via unique predictions obtained despite unidentifiability – new definitions and a general method , 2012, The FEBS journal.

[17]  F. Wurm.  Production of recombinant protein therapeutics in cultivated mammalian cells , 2004, Nature Biotechnology.

[18]  David H. Sharp,et al.  A connectionist model of development. , 1991, Journal of theoretical biology.

[19]  Julio R. Banga,et al.  Optimization in computational systems biology , 2008, BMC Systems Biology.

[20]  Hugh D. Spence,et al.  Minimum information requested in the annotation of biochemical models (MIRIAM) , 2005, Nature Biotechnology.

[21]  Julio R. Banga,et al.  A cooperative strategy for parameter estimation in large scale systems biology models , 2012, BMC Systems Biology.

[22]  Steffen Klamt,et al.  SBML qualitative models: a model representation format and infrastructure to foster interactions between qualitative modelling formalisms and tools , 2013, BMC Systems Biology.

[23]  Rudiyanto Gunawan,et al.  Parameter identifiability of power-law biochemical system models. , 2010, Journal of biotechnology.

[24]  U. Sauer,et al.  Advancing metabolic models with kinetic information. , 2014, Current opinion in biotechnology.

[25]  Judith B. Zaugg,et al.  Bacterial adaptation through distributed sensing of metabolic fluxes , 2010, Molecular systems biology.

[26]  Mudita Singhal,et al.  COPASI - a COmplex PAthway SImulator , 2006, Bioinform..

[27]  Natal A. W. van Riel,et al.  Dynamic modelling and analysis of biochemical networks: mechanism-based models and model-based experiments , 2006, Briefings Bioinform..

[28]  Melanie I. Stefan,et al.  BioModels Database: An enhanced, curated and annotated resource for published quantitative kinetic models , 2010, BMC Systems Biology.

[29]  Julio R. Banga,et al.  Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems , 2006, BMC Bioinformatics.

[30]  Todd Munson,et al.  Benchmarking optimization software with COPS 3.0. , 2001 .

[31]  Dario Floreano,et al.  GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods , 2011, Bioinform..

[32]  Joseph J. DiStefano,et al.  Dynamic Systems Biology Modeling and Simulation , 2015 .

[33]  Xiaohua Xia,et al.  On Identifiability of Nonlinear ODE Models and Applications in Viral Dynamics , 2011, SIAM Rev..

[34]  Julio R. Banga,et al.  Inference of complex biological networks: distinguishability issues and optimization-based solutions , 2011, BMC Systems Biology.

[35]  Julio R. Banga,et al.  An evolutionary method for complex-process optimization , 2010, Comput. Oper. Res..

[36]  N. V. van Riel.  Dynamic modelling and analysis of biochemical networks: mechanism-based models and model-based experiments. , 2006, Briefings in bioinformatics.

[37]  D. Ramkrishna,et al.  Modeling metabolic systems: the need for dynamics , 2013 .

[38]  E D Sontag.  For Differential Equations with r Parameters, 2r+1 Experiments Are Enough for Identification , 2003, J. Nonlinear Sci..

[39]  Maksat Ashyraliyev,et al.  Systems biology: parameter estimation for biochemical models , 2009, The FEBS journal.

[40]  Neil Swainston,et al.  Sustainable model building the role of standards and biological semantics. , 2011, Methods in enzymology.

[41]  David H. Sharp,et al.  Mechanism of eve stripe formation , 1995, Mechanisms of Development.

[42]  N. Monk,et al.  Bioattractors: dynamical systems theory and the evolution of regulatory processes , 2014, The Journal of physiology.

[43]  J. Banga,et al.  Structural Identifiability of Systems Biology Models: A Critical Comparison of Methods , 2011, PloS one.

[44]  P. Mendes,et al.  Large-Scale Metabolic Models: From Reconstruction to Differential Equations , 2013 .

[45]  Nikolaus Hansen,et al.  Benchmarking of Continuous Black Box Optimization Algorithms , 2012, Evolutionary Computation.

[46]  Evangelos Simeonidis,et al.  Flux balance analysis: a geometric perspective. , 2009, Journal of theoretical biology.

[47]  F. Doyle,et al.  A benchmark for methods in reverse engineering and model discrimination: problem formulation and solutions. , 2004, Genome research.

[48]  Hiroaki Kitano,et al.  The systems biology markup language (SBML): a medium for representation and exchange of biochemical network models , 2003, Bioinform..

[49]  Michael R. Brent,et al.  Benchmarking regulatory network reconstruction with GRENDEL , 2009, Bioinform..

[50]  Reinhard Laubenbacher,et al.  Comparison of Reverse‐Engineering Methods Using an in Silico Network , 2007, Annals of the New York Academy of Sciences.

[51]  Daniel E. Zak,et al.  Importance of input perturbations and stochastic gene expression in the reverse engineering of genetic regulatory networks: insights from an identifiability analysis of an in silico network. , 2003, Genome research.

[52]  Rudiyanto Gunawan,et al.  Incremental parameter estimation of kinetic metabolic network models , 2012, BMC Systems Biology.

[53]  Michel Dumontier,et al.  Controlled vocabularies and semantics in systems biology , 2011, Molecular systems biology.

[54]  Edda Klipp,et al.  Modular rate laws for enzymatic reactions: thermodynamics, elasticities and implementation , 2010, Bioinform..

[55]  Gaudenz Danuser,et al.  Linking data to models: data regression , 2006, Nature Reviews Molecular Cell Biology.

[56]  Eva Balsa-Canto,et al.  AMIGO, a toolbox for advanced model identification in systems biology using global optimization , 2011, Bioinform..

[57]  David Henriques,et al.  MEIGO: an open-source software suite based on metaheuristics for global optimization in systems biology and bioinformatics , 2013, BMC Bioinformatics.

[58]  Julio R. Banga,et al.  Reverse engineering and identification in systems biology: strategies, perspectives and challenges , 2014, Journal of The Royal Society Interface.

[59]  H. D. Jong,et al.  On the identifiability of metabolic network models , 2013, Journal of mathematical biology.

[60]  H. Lodish.  Molecular Cell Biology , 1986 .

[61]  David H. Sharp,et al.  Dynamic control of positional information in the early Drosophila embryo , 2004, Nature.

[62]  Beatriz Peñalver Bernabé,et al.  State–time spectrum of signal transduction logic models , 2012, Physical biology.

[63]  Eva Balsa-Canto,et al.  High-Confidence Predictions in Systems Biology Dynamic Models , 2014, PACBB.

[64]  G. Cedersund,et al.  Conservation laws and unidentifiability of rate expressions in biochemical models. , 2007, IET systems biology.

[65]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[66]  Eric Walter,et al.  Identification of Parametric Models: from Experimental Data , 1997 .

[67]  Pedro Mendes,et al.  Artificial gene networks for objective comparison of analysis algorithms , 2003, ECCB.

[68]  N A W van Riel,et al.  Parameter uncertainty in biochemical models described by ordinary differential equations. , 2013, Mathematical biosciences.

[69]  Eva Balsa-Canto,et al.  Reverse-Engineering Post-Transcriptional Regulation of Gap Genes in Drosophila melanogaster , 2013, PLoS Comput. Biol..