Evolution Control for parallel ANN-assisted simulation-based optimization application to Tuberculosis Transmission Control

Abstract In many optimal design searches, the function to optimize is a simulator that is computationally expensive. While current High Performance Computing (HPC) methods are not able to solve such problems efficiently, parallelism can be coupled with approximate models (surrogates or meta-models) that imitate the simulator in timely fashion to achieve better results. This combined approach reduces the number of simulations thanks to surrogate use whereas the remaining evaluations are handled by supercomputers. While the surrogates’ ability to limit computational times is very attractive, integrating them into the overarching optimization process can be challenging. Indeed, it is critical to address the major trade-off between the quality (precision) and the efficiency (execution time) of the resolution. In this article, we investigate Evolution Controls (ECs) which are strategies that define the alternation between the simulator and the surrogate within the optimization process. We propose a new EC based on the prediction uncertainty obtained from Monte Carlo Dropout (MCDropout), a technique originally dedicated to quantifying uncertainty in deep learning. Investigations of such uncertainty-aware ECs remain uncommon in surrogate-assisted evolutionary optimization. In addition, we use parallel computing in a complementary way to address the high computational burden. Our new strategy is implemented in the context of a pioneering application to Tuberculosis Transmission Control. The reported results show that the MCDropout-based EC coupled with massively parallel computing outperforms strategies previously proposed in the field of surrogate-assisted optimization.

[1]  Edgar Tello-Leal,et al.  A Review of Surrogate Assisted Multiobjective Evolutionary Algorithms , 2016, Comput. Intell. Neurosci..

[2]  Dario Izzo,et al.  esa/pagmo2: pagmo 2.10 , 2018 .

[3]  Enrique Alba,et al.  Parallel metaheuristics: recent advances and new trends , 2012, Int. Trans. Oper. Res..

[4]  E. McBryde,et al.  Strategic Planning for Tuberculosis Control in the Republic of Fiji , 2019, Tropical medicine and infectious disease.

[5]  Carlos A. Coello Coello,et al.  Comparison of metamodeling techniques in evolutionary algorithms , 2017, Soft Comput..

[6]  Christine A. Shoemaker,et al.  Parallel Stochastic Global Optimization Using Radial Basis Functions , 2009, INFORMS J. Comput..

[7]  Juan J. Alonso,et al.  AIAA 2004 – 1758 Design of a Low-Boom Supersonic Business Jet Using Evolutionary Algorithms and an Adaptive Unstructured Mesh Method , 2004 .

[8]  Khaled Rasheed,et al.  A Survey of Fitness Approximation Methods Applied in Evolutionary Algorithms , 2010 .

[9]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[10]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[11]  Cliff C. Kerr,et al.  Maximizing the impact of malaria funding through allocative efficiency: using the right interventions in the right locations , 2017, Malaria Journal.

[12]  Yarin Gal,et al.  Uncertainty in Deep Learning , 2016 .

[13]  Qingfu Zhang,et al.  A Batched Scalable Multi-Objective Bayesian Optimization Algorithm , 2018, ArXiv.

[14]  Anirban Chaudhuri,et al.  Parallel surrogate-assisted global optimization with expensive functions – a survey , 2016 .

[15]  Stephen J. Leary,et al.  A parallel updating scheme for approximating and optimizing high fidelity computer simulations , 2004 .

[16]  El-Ghazali Talbi,et al.  Metaheuristics - From Design to Implementation , 2009 .

[17]  Yuansheng Cheng,et al.  Pseudo expected improvement criterion for parallel EGO algorithm , 2017, J. Glob. Optim..

[18]  John Doherty,et al.  Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems , 2017, IEEE Transactions on Cybernetics.

[19]  Nirupam Chakraborti,et al.  A data-driven surrogate-assisted evolutionary algorithm applied to a many-objective blast furnace optimization problem , 2017 .

[20]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[21]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[22]  Helio J. C. Barbosa,et al.  On Similarity-Based Surrogate Models for Expensive Single- and Multi-objective Evolutionary Optimization , 2010 .

[23]  Bryan Glaz,et al.  Surrogate based optimization of helicopter rotor blades for vibration reduction in forward flight , 2006 .

[24]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[25]  El-Ghazali Talbi,et al.  GPU-based island model for evolutionary algorithms , 2010, GECCO '10.

[26]  Luis F. Gonzalez,et al.  A Generic Framework for the Design Optimisation of Multidisciplinary UAV Intelligent Systems using Evolutionary Computing , 2006 .

[27]  Romain Ragonnet,et al.  Scenario Analysis for Programmatic Tuberculosis Control in Western Province, Papua New Guinea. , 2016, American journal of epidemiology.

[28]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[29]  M. Keeling,et al.  Modeling Infectious Diseases in Humans and Animals , 2007 .

[30]  M Elizabeth Halloran,et al.  Epidemiological benefits of more-effective tuberculosis vaccines, drugs, and diagnostics , 2009, Proceedings of the National Academy of Sciences.

[31]  Ky Khac Vu,et al.  Surrogate-based methods for black-box optimization , 2017, Int. Trans. Oper. Res..

[32]  Eduardo Gotuzzo,et al.  Rapid molecular detection of tuberculosis and rifampin resistance. , 2010, The New England journal of medicine.

[33]  Dragan Pamucar,et al.  Planning the City Logistics Terminal Location by Applying the Green p-Median Model and Type-2 Neurofuzzy Network , 2016, Comput. Intell. Neurosci..

[34]  Hugo Jair Escalante,et al.  A hybrid surrogate-based approach for evolutionary multi-objective optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[35]  Franck Cappello,et al.  Grid'5000: A Large Scale And Highly Reconfigurable Experimental Grid Testbed , 2006, Int. J. High Perform. Comput. Appl..

[36]  Meng-Sing Liou,et al.  Multi-Objective Optimization of Transonic Compressor Blade Using Evolutionary Algorithm , 2005 .

[37]  Fuli Wang,et al.  A modified global optimization method based on surrogate model and its application in packing profile optimization of injection molding process , 2010 .

[38]  Robert Ivor John,et al.  A parallel surrogate-assisted multi-objective evolutionary algorithm for computationally expensive optimization problems , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[39]  Kevin Tucker,et al.  Response surface approximation of pareto optimal front in multi-objective optimization , 2004 .

[40]  Christine A. Shoemaker,et al.  Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems , 2014, J. Glob. Optim..

[41]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[42]  R. Regis Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points , 2014 .

[43]  Romain Ragonnet,et al.  Modular programming for tuberculosis control, the “AuTuMN” platform , 2017, BMC Infectious Diseases.

[44]  Qing Li,et al.  Multiobjective optimization for crash safety design of vehicles using stepwise regression model , 2008 .

[45]  Ying Tan,et al.  Multiobjective Infill Criterion Driven Gaussian Process-Assisted Particle Swarm Optimization of High-Dimensional Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[46]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[47]  Bithin Datta,et al.  Multi-objective management of saltwater intrusion in coastal aquifers using genetic programming and modular neural network based surrogate models. , 2010 .

[48]  Jack Dongarra,et al.  Numerical Linear Algebra for High-Performance Computers , 1998 .

[49]  Régis Duvigneau,et al.  Hybrid genetic algorithms and artificial neural networks for complex design optimization in CFD , 2004 .

[50]  Andy J. Keane,et al.  Multi-Objective Optimization Using Surrogates , 2010 .

[51]  Johann Sienz,et al.  Particle swarm algorithm with adaptive constraint handling and integrated surrogate model for the management of petroleum fields , 2015, Appl. Soft Comput..

[52]  Taimoor Akhtar,et al.  Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection , 2016, J. Glob. Optim..

[53]  Nouredine Melab,et al.  An Adaptive Evolution Control based on Confident Regions for Surrogate-assisted Optimization , 2018, 2018 International Conference on High Performance Computing & Simulation (HPCS).

[54]  Kalyanmoy Deb,et al.  An Evolutionary Multi-objective Adaptive Meta-modeling Procedure Using Artificial Neural Networks , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.