Biochemical parameter estimation vs. benchmark functions: A comparative study of optimization performance and representation design

Abstract Computational Intelligence methods, which include Evolutionary Computation and Swarm Intelligence, can efficiently and effectively identify optimal solutions to complex optimization problems by exploiting the cooperative and competitive interplay among their individuals. The exploration and exploitation capabilities of these meta-heuristics are typically assessed by considering well-known suites of benchmark functions, specifically designed for numerical global optimization purposes. However, their performances could drastically change in the case of real-world optimization problems. In this paper, we investigate this issue by considering the Parameter Estimation (PE) of biochemical systems, a common computational problem in the field of Systems Biology. In order to evaluate the effectiveness of various meta-heuristics in solving the PE problem, we compare their performance by considering a set of benchmark functions and a set of synthetic biochemical models characterized by a search space with an increasing number of dimensions. Our results show that some state-of-the-art optimization methods – able to largely outperform the other meta-heuristics on benchmark functions – are characterized by considerably poor performances when applied to the PE problem. We also show that a limiting factor of these optimization methods concerns the representation of the solutions: indeed, by means of a simple semantic transformation, it is possible to turn these algorithms into competitive alternatives. We corroborate this finding by performing the PE of a model of metabolic pathways in red blood cells. Overall, in this work we state that classic benchmark functions cannot be fully representative of all the features that make real-world optimization problems hard to solve. This is the case, in particular, of the PE of biochemical systems. We also show that optimization problems must be carefully analyzed to select an appropriate representation, in order to actually obtain the performance promised by benchmark results.

[1]  Michèle Sebag,et al.  Extending Population-Based Incremental Learning to Continuous Search Spaces , 1998, PPSN.

[2]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[3]  Eberhard O. Voit,et al.  150 Years of the Mass Action Law , 2015, PLoS Comput. Biol..

[4]  Xin-She Yang,et al.  A literature survey of benchmark functions for global optimisation problems , 2013, Int. J. Math. Model. Numer. Optimisation.

[5]  Riccardo Poli,et al.  Evolving Problems to Learn About Particle Swarm Optimizers and Other Search Algorithms , 2006, IEEE Transactions on Evolutionary Computation.

[6]  Roger W. Elliott.  Kiviat-graphs as a means for displaying performance data for on-line retrieval systems , 1976, J. Am. Soc. Inf. Sci..

[7]  Neema Jamshidi,et al.  Mass action stoichiometric simulation models: incorporating kinetics and regulation into stoichiometric models. , 2010, Biophysical journal.

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Ajith Abraham,et al.  Turbulent Particle Swarm Optimization Using Fuzzy Parameter Tuning , 2009, Foundations of Computational Intelligence.

[10]  Kenneth W. Kolence,et al.  Software unit profiles & Kiviat figures , 1973, PERV.

[11]  David H. Wolpert,et al.  What makes an optimization problem hard? , 1995, Complex..

[12]  Marco S. Nobile,et al.  The impact of particles initialization in PSO: Parameter estimation as a case in point , 2015, 2015 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB).

[13]  Fei He,et al.  Sensitivity analysis and robust experimental design of a signal transduction pathway system , 2008 .

[14]  Andries Petrus Engelbrecht,et al.  Inertia weight control strategies for particle swarm optimization , 2016, Swarm Intelligence.

[15]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[16]  J. Kämpf,et al.  A comparison of global optimization algorithms with standard benchmark functions and real-world applications using EnergyPlus , 2009 .

[17]  Carmen G. Moles,et al.  Parameter estimation in biochemical pathways: a comparison of global optimization methods. , 2003, Genome research.

[18]  Giancarlo Mauri,et al.  Proactive Particles in Swarm Optimization: A self-tuning algorithm based on Fuzzy Logic , 2015, 2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[19]  Giancarlo Mauri,et al.  Fuzzy Self-Tuning PSO: A settings-free algorithm for global optimization , 2017, Swarm Evol. Comput..

[20]  Daniela Besozzi,et al.  Reaction-Based Models of Biochemical Networks , 2016, CiE.

[21]  H. Kitano.  Systems Biology: A Brief Overview , 2002, Science.

[22]  Johannes M. Dieterich,et al.  Empirical review of standard benchmark functions using evolutionary global optimization , 2012, ArXiv.

[23]  Changhe Li,et al.  A Generalized Approach to Construct Benchmark Problems for Dynamic Optimization , 2008, SEAL.

[24]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[25]  Guanrong Chen,et al.  Evolving benchmark functions using kruskal-wallis test , 2018, GECCO.

[26]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[27]  Fabrice Rossi,et al.  Mean Absolute Percentage Error for regression models , 2016, Neurocomputing.

[28]  Andreas Zell,et al.  Modeling metabolic networks in C . glutamicum : a comparison of rate laws in combination with various parameter optimization strategies , 2009 .

[29]  Marco S. Nobile,et al.  Computational Strategies for a System-Level Understanding of Metabolism , 2014, Metabolites.

[30]  Marcos L. Corazza,et al.  A comparison among stochastic optimization algorithms for parameter estimation of biochemical kinetic models , 2013, Appl. Soft Comput..

[31]  Jing J. Liang,et al.  Novel composition test functions for numerical global optimization , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[32]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[33]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[34]  Giancarlo Mauri,et al.  GPU-accelerated simulations of mass-action kinetics models with cupSODA , 2014, The Journal of Supercomputing.

[35]  Bernard Grossman,et al.  A Comparison of Global Optimization Methods for the Design of a High-speed Civil Transport , 2001, J. Glob. Optim..

[36]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[37]  Ming Yang,et al.  An Open Framework for Constructing Continuous Optimization Problems , 2019, IEEE Transactions on Cybernetics.

[38]  S. Bhat,et al.  Modeling and analysis of mass-action kinetics , 2009, IEEE Control Systems.

[39]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[40]  Risto Miikkulainen,et al.  A Probabilistic Reformulation of No Free Lunch: Continuous Lunches Are Not Free , 2016, Evolutionary Computation.

[41]  Yang Lou,et al.  On constructing alternative benchmark suite for evolutionary algorithms , 2019, Swarm Evol. Comput..

[42]  Olivier Teytaud,et al.  Continuous lunches are free! , 2007, GECCO '07.

[43]  Giancarlo Mauri,et al.  cupSODA: A CUDA-Powered Simulator of Mass-Action Kinetics , 2013, PaCT.

[44]  Marco S. Nobile,et al.  Proactive Particles in Swarm Optimization: A settings-free algorithm for real-parameter single objective optimization problems , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[45]  Martin Pelikan,et al.  An introduction and survey of estimation of distribution algorithms , 2011, Swarm Evol. Comput..

[46]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[47]  Marco S. Nobile,et al.  Dilation Functions in Global Optimization , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[48]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[49]  Hao Wang,et al.  Towards a theory-guided benchmarking suite for discrete black-box optimization heuristics: profiling (1 + λ) EA variants on onemax and leadingones , 2018, GECCO.

[50]  Marcus Gallagher,et al.  Towards improved benchmarking of black-box optimization algorithms using clustering problems , 2016, Soft Comput..

[51]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[52]  Giancarlo Mauri,et al.  Computational Intelligence for Parameter Estimation of Biochemical Systems , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[53]  R. Eberhart,et al.  Fuzzy adaptive particle swarm optimization , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[54]  Marco S. Nobile,et al.  Reboot strategies in particle swarm optimization and their impact on parameter estimation of biochemical systems , 2017, 2017 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB).

[55]  Marco Beccuti,et al.  Efficient and Settings-Free Calibration of Detailed Kinetic Metabolic Models with Enzyme Isoforms Characterization , 2018, CIBB.

[56]  Olivier Teytaud,et al.  Continuous Lunches Are Free Plus the Design of Optimal Optimization Algorithms , 2010, Algorithmica.

[57]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[58]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..