Dynamic Optimization with Particle Swarms (DOPS): a meta-heuristic for parameter estimation in biochemical models

BackgroundMathematical modeling is a powerful tool to analyze, and ultimately design biochemical networks. However, the estimation of the parameters that appear in biochemical models is a significant challenge. Parameter estimation typically involves expensive function evaluations and noisy data, making it difficult to quickly obtain optimal solutions. Further, biochemical models often have many local extrema which further complicates parameter estimation. Toward these challenges, we developed Dynamic Optimization with Particle Swarms (DOPS), a novel hybrid meta-heuristic that combined multi-swarm particle swarm optimization with dynamically dimensioned search (DDS). DOPS uses a multi-swarm particle swarm optimization technique to generate candidate solution vectors, the best of which is then greedily updated using dynamically dimensioned search.ResultsWe tested DOPS using classic optimization test functions, biochemical benchmark problems and real-world biochemical models. We performed T$\mathcal {T}$ = 25 trials with N$\mathcal {N}$ = 4000 function evaluations per trial, and compared the performance of DOPS with other commonly used meta-heuristics such as differential evolution (DE), simulated annealing (SA) and dynamically dimensioned search (DDS). On average, DOPS outperformed other common meta-heuristics on the optimization test functions, benchmark problems and a real-world model of the human coagulation cascade.ConclusionsDOPS is a promising meta-heuristic approach for the estimation of biochemical model parameters in relatively few function evaluations. DOPS source code is available for download under a MIT license at http://www.varnerlab.org.

[1]  K. Mann,et al.  Models of blood coagulation , 2000, Blood coagulation & fibrinolysis : an international journal in haemostasis and thrombosis.

[2]  Ajith Abraham,et al.  Swarm Intelligence: Foundations, Perspectives and Applications , 2006, Swarm Intelligent Systems.

[3]  James V. Beck,et al.  Parameter Estimation in Engineering and Science , 1977 .

[4]  Julio R. Banga,et al.  Optimization in computational systems biology , 2008, BMC Systems Biology.

[5]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[6]  Douglas B. Kell,et al.  Non-linear optimization of biochemical pathways: applications to metabolic engineering and parameter estimation , 1998, Bioinform..

[7]  M. Clerc,et al.  Particle Swarm Optimization , 2006 .

[8]  K. Mann,et al.  Thrombin formation. , 2003, Chest.

[9]  Declan G. Bates,et al.  Ultrasensitive Negative Feedback Control: A Natural Approach for the Design of Synthetic Controllers , 2016, PloS one.

[10]  Julio R. Banga,et al.  Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems , 2006, BMC Bioinformatics.

[11]  Kwang-Hyun Cho,et al.  Dynamics of biological systems: role of systems biology in medical research , 2006, Expert review of molecular diagnostics.

[12]  Julio R. Banga,et al.  Scatter search for chemical and bio-process optimization , 2007, J. Glob. Optim..

[13]  Aaron L. Fogelson,et al.  Coagulation under Flow: The Influence of Flow-Mediated Transport on the Initiation and Inhibition of Coagulation , 2006, Pathophysiology of Haemostasis and Thrombosis.

[14]  Julio R. Banga,et al.  A cooperative strategy for parameter estimation in large scale systems biology models , 2012, BMC Systems Biology.

[15]  D. Lauffenburger,et al.  Physicochemical modelling of cell signalling pathways , 2006, Nature Cell Biology.

[16]  Hong Sun,et al.  Smolign: A Spatial Motifs-Based Protein Multiple Structural Alignment Method , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[17]  Jing J. Liang,et al.  Dynamic multi-swarm particle swarm optimizer with local search for Large Scale Global Optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[18]  K. C. Jones,et al.  A Model for the Stoichiometric Regulation of Blood Coagulation* , 2002, The Journal of Biological Chemistry.

[19]  Eva Balsa-Canto,et al.  BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology , 2015, BMC Systems Biology.

[20]  Eva Balsa-Canto,et al.  High-Confidence Predictions in Systems Biology Dynamic Models , 2014, PACBB.

[21]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[22]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[23]  Keith A. Woodbury,et al.  Inverse problems and parameter estimation: integration of measurements and analysis , 1998 .

[24]  Saso Dzeroski,et al.  Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis , 2011, BMC Systems Biology.

[25]  Yongwimon Lenbury,et al.  Mathematical modeling and application of genetic algorithm to parameter estimation in signal transduction: Trafficking and promiscuous coupling of G-protein coupled receptors , 2008, Comput. Biol. Medicine.

[26]  H. Iba,et al.  Inferring Gene Regulatory Networks using Differential Evolution with Local Search Heuristics , 2007, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[27]  Feng-Sheng Wang,et al.  Evolutionary optimization with data collocation for reverse engineering of biological networks , 2005, Bioinform..

[28]  Maksat Ashyraliyev,et al.  Systems biology: parameter estimation for biochemical models , 2009, The FEBS journal.

[29]  K. Mann,et al.  The dynamics of thrombin formation. , 2003, Arteriosclerosis, thrombosis, and vascular biology.

[30]  Ronald B Ponn,et al.  Simple mediastinal cysts: resect them all? , 2003, Chest.

[31]  Kumbakonam R. Rajagopal,et al.  A Model Incorporating Some of the Mechanical and Biochemical Factors Underlying Clot Formation and Dissolution in Flowing Blood , 2003 .

[32]  Peter Young,et al.  Parameter estimation for continuous-time models - A survey , 1979, Autom..

[33]  Jonathan M. Garibaldi,et al.  Parameter Estimation Using Metaheuristics in Systems Biology: A Comprehensive Review , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[34]  Natal A. W. van Riel,et al.  Dynamic modelling and analysis of biochemical networks: mechanism-based models and model-based experiments , 2006, Briefings Bioinform..

[35]  Jeffrey D. Varner,et al.  JuPOETs: a constrained multiobjective optimization approach to estimate biochemical model ensembles in the Julia programming language , 2016, bioRxiv.

[36]  Hong-Bin Shen,et al.  OptiFel: A Convergent Heterogeneous Particle Swarm Optimization Algorithm for Takagi–Sugeno Fuzzy Modeling , 2014, IEEE Transactions on Fuzzy Systems.

[37]  Scott L. Diamond,et al.  Systems Biology of Coagulation Initiation: Kinetics of Thrombin Generation in Resting and Activated Human Blood , 2010, PLoS Comput. Biol..

[38]  Michael E Phelps,et al.  Systems Biology and New Technologies Enable Predictive and Preventative Medicine , 2004, Science.

[39]  S. Diamond,et al.  Systems biology of coagulation , 2013, Journal of thrombosis and haemostasis : JTH.

[40]  Bryan A. Tolson,et al.  Dynamically dimensioned search algorithm for computationally efficient watershed model calibration , 2007 .

[41]  Xin-She Yang,et al.  A literature survey of benchmark functions for global optimisation problems , 2013, Int. J. Math. Model. Numer. Optimisation.

[42]  H. Kitano Systems Biology: A Brief Overview , 2002, Science.

[43]  Feng-Sheng Wang,et al.  Hybrid Differential Evolution for Problems of Kinetic Parameter Estimation and Dynamic Optimization of an Ethanol Fermentation Process , 2001 .

[44]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[45]  Dale E. Seborg,et al.  A review of process identification and parameter estimation techniques , 1971 .

[46]  Gaudenz Danuser,et al.  Linking data to models: data regression , 2006, Nature Reviews Molecular Cell Biology.

[47]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[48]  E. Vogler,et al.  Contact activation of blood-plasma coagulation. , 2009, Biomaterials.

[49]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[50]  P. Mendes,et al.  Large-Scale Metabolic Models: From Reconstruction to Differential Equations , 2013 .

[51]  Deyan Luan,et al.  Computationally Derived Points of Fragility of a Human Cascade Are Consistent with Current Therapeutic Strategies , 2007, PLoS Comput. Biol..

[52]  J. J. Moré,et al.  Levenberg--Marquardt algorithm: implementation and theory , 1977 .

[53]  K. Mann,et al.  What is all that thrombin for? , 2003, Journal of thrombosis and haemostasis : JTH.

[54]  Carmen G. Moles,et al.  Parameter estimation in biochemical pathways: a comparison of global optimization methods. , 2003, Genome research.

[55]  Christodoulos A. Floudas,et al.  Deterministic Global Optimization in Nonlinear Optimal Control Problems , 2000, J. Glob. Optim..

[56]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.