Efficient Multi-Objective Optimization through Population-based Parallel Surrogate Search

Multi-Objective Optimization (MOO) is very difficult for expensive functions because most current MOO methods rely on a large number of function evaluations to get an accurate solution. We address this problem with surrogate approximation and parallel computation. We develop an MOO algorithm MOPLS-N for expensive functions that combines iteratively updated surrogate approximations of the objective functions with a structure for efficiently selecting a population of $N$ points so that the expensive objectives for all points are simultaneously evaluated on $N$ processors in each iteration. MOPLS incorporates Radial Basis Function (RBF) approximation, Tabu Search and local candidate search around multiple points to strike a balance between exploration, exploitation and diversification during each algorithm iteration. Eleven test problems (with 8 to 24 decision variables and two real-world watershed problems are used to compare performance of MOPLS to ParEGO, GOMORS, Borg, MOEA/D, and NSGA-III on a limited budget of evaluations with between 1 (serial) and 64 processors. MOPLS in serial is better than all non-RBF serial methods tested. Parallel speedup of MOPLS is higher than all other parallel algorithms with 16 and 64 processors. With both algorithms on 64 processors MOPLS is at least 2 times faster than NSGA-III on the watershed problems.

[1]  Hugo Jair Escalante,et al.  A hybrid surrogate-based approach for evolutionary multi-objective optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[2]  Christine A. Shoemaker,et al.  SO-MI: A surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems , 2013, Comput. Oper. Res..

[3]  Taimoor Akhtar,et al.  Multi objective optimization of computationally expensive multi-modal functions with RBF surrogates and multi-rule selection , 2016, J. Glob. Optim..

[4]  Qingfu Zhang,et al.  Expensive Multiobjective Optimization by MOEA/D With Gaussian Process Model , 2010, IEEE Transactions on Evolutionary Computation.

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  Saúl Zapotecas Martínez,et al.  A Memetic Algorithm with Non Gradient-Based Local Search Assisted by a Meta-model , 2010, PPSN.

[7]  Kyriakos C. Giannakoglou,et al.  A multi-objective metamodel-assisted memetic algorithm with strength-based local refinement , 2009 .

[8]  Francisco Herrera,et al.  An Evolutionary Multiobjective Model and Instance Selection for Support Vector Machines With Pareto-Based Ensembles , 2017, IEEE Transactions on Evolutionary Computation.

[9]  George Kourakos,et al.  Development of a multi-objective optimization algorithm using surrogate models for coastal aquifer management. , 2013 .

[10]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point Based Nondominated Sorting Approach, Part II: Handling Constraints and Extending to an Adaptive Approach , 2014, IEEE Transactions on Evolutionary Computation.

[11]  Patrick M. Reed,et al.  Borg: An Auto-Adaptive Many-Objective Evolutionary Computing Framework , 2013, Evolutionary Computation.

[12]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[13]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[14]  Patrick M. Reed,et al.  Large-scale parallelization of the Borg multiobjective evolutionary algorithm to enhance the management of complex environmental systems , 2015, Environ. Model. Softw..

[15]  Shigeru Obayashi,et al.  Efficient global optimization (EGO) for multi-objective problem and data mining , 2005, 2005 IEEE Congress on Evolutionary Computation.

[16]  Kalyanmoy Deb,et al.  Multi-objective optimization using evolutionary algorithms , 2001, Wiley-Interscience series in systems and optimization.

[17]  Kalyanmoy Deb,et al.  An Evolutionary Many-Objective Optimization Algorithm Using Reference-Point-Based Nondominated Sorting Approach, Part I: Solving Problems With Box Constraints , 2014, IEEE Transactions on Evolutionary Computation.

[18]  Kalyanmoy Deb,et al.  An Evolutionary Multi-objective Adaptive Meta-modeling Procedure Using Artificial Neural Networks , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[19]  D. H. Marks,et al.  A review and evaluation of multiobjective programing techniques , 1975 .

[20]  C. Shoemaker,et al.  Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization , 2013 .

[21]  Lothar Thiele,et al.  The Hypervolume Indicator Revisited: On the Design of Pareto-compliant Indicators Via Weighted Integration , 2007, EMO.

[22]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[23]  Saúl Zapotecas Martínez,et al.  Combining surrogate models and local search for dealing with expensive multi-objective optimization problems , 2013, 2013 IEEE Congress on Evolutionary Computation.

[24]  Christine A. Shoemaker,et al.  Cannonsville Reservoir Watershed SWAT2000 model development, calibration and validation , 2007 .

[25]  Christine A. Shoemaker,et al.  Estimation of plume distribution for carbon sequestration using parameter estimation with limited monitoring data , 2013 .

[26]  Nicola Beume,et al.  SMS-EMOA: Multiobjective selection based on dominated hypervolume , 2007, Eur. J. Oper. Res..

[27]  Carlos A. Coello Coello,et al.  Use of Radial Basis Functions and Rough Sets for Evolutionary Multi-Objective Optimization , 2007, 2007 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making.

[28]  Wilfrido Gómez-Flores,et al.  On the selection of surrogate models in evolutionary optimization algorithms , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[29]  Joshua D. Knowles,et al.  ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems , 2006, IEEE Transactions on Evolutionary Computation.

[30]  Kevin Leyton-Brown,et al.  Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[31]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[32]  Qingfu Zhang,et al.  Multiobjective optimization Test Instances for the CEC 2009 Special Session and Competition , 2009 .

[33]  Wolfgang Ponweiser,et al.  Multiobjective Optimization on a Limited Budget of Evaluations Using Model-Assisted -Metric Selection , 2008, PPSN.

[34]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[35]  Bryan A. Tolson,et al.  Review of surrogate modeling in water resources , 2012 .

[36]  Qingfu Zhang,et al.  Multiobjective Optimization Problems With Complicated Pareto Sets, MOEA/D and NSGA-II , 2009, IEEE Transactions on Evolutionary Computation.

[37]  Martin D. Buhmann,et al.  Radial Basis Functions , 2021, Encyclopedia of Mathematical Geosciences.

[38]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[39]  Jeffrey G. Arnold,et al.  Soil and Water Assessment Tool Theoretical Documentation Version 2009 , 2011 .

[40]  Rommel G. Regis,et al.  Multi-objective constrained black-box optimization using radial basis function surrogates , 2016, J. Comput. Sci..

[41]  Rituparna Datta,et al.  A surrogate-assisted evolution strategy for constrained multi-objective optimization , 2016, Expert Syst. Appl..

[42]  Johannes Bader,et al.  Hypervolume-based search for multiobjective optimization: Theory and methods , 2010 .

[43]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[44]  Tom Dhaene,et al.  Efficient Multi-Objective Simulation-Driven Antenna Design Using Co-Kriging , 2014, IEEE Transactions on Antennas and Propagation.

[45]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[46]  Christine A. Shoemaker,et al.  Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems , 2014, J. Glob. Optim..

[47]  Hao Wang,et al.  A Multicriteria Generalization of Bayesian Global Optimization , 2016, Advances in Stochastic and Deterministic Global Optimization.

[48]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[49]  Juliane Müller,et al.  SOCEMO: Surrogate Optimization of Computationally Expensive Multiobjective Problems , 2017, INFORMS J. Comput..

[50]  Bernd Bischl,et al.  Model-Based Multi-objective Optimization: Taxonomy, Multi-Point Proposal, Toolbox and Benchmark , 2015, EMO.

[51]  Kalyanmoy Deb,et al.  Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems , 1999, Evolutionary Computation.

[52]  Chen Wang,et al.  Multiobjective adaptive surrogate modeling‐based optimization for parameter estimation of large, complex geophysical models , 2016 .

[53]  Bernd Bischl,et al.  Multi-objective parameter configuration of machine learning algorithms using model-based optimization , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[54]  Jasper A Vrugt,et al.  Improved evolutionary optimization from genetically adaptive multimethod search , 2007, Proceedings of the National Academy of Sciences.

[55]  Shigeru Obayashi,et al.  Expected Improvement of Penalty-Based Boundary Intersection for Expensive Multiobjective Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[56]  Avi Ostfeld,et al.  State of the Art for Genetic Algorithms and Beyond in Water Resources Planning and Management , 2010 .

[57]  Kevin L. Priddy,et al.  Artificial neural networks - an introduction , 2005, Tutorial text series.

[58]  Marios K. Karakasis,et al.  On the use of metamodel-assisted, multi-objective evolutionary algorithms , 2006 .