A surrogate-based cooperative optimization framework for computationally expensive black-box problems

Most parallel surrogate-based optimization algorithms focus only on the mechanisms for generating multiple updating points in each cycle, and rather less attention has been paid to producing them through the cooperation of several algorithms. For this purpose, a surrogate-based cooperative optimization framework is here proposed. Firstly, a class of parallel surrogate-based optimization algorithms is developed, based on the idea of viewing the infill sampling criterion as a bi-objective optimization problem. Each algorithm of this class is called a Sequential Multipoint Infill Sampling Algorithm (SMISA) and is the combination resulting from choosing a surrogate model, an exploitation measure, an exploration measure and a multi-objective optimization approach to its solution. SMISAs are the basic algorithms on which collaboration mechanisms are established. Many SMISAs can be defined, and the focus has been on scalar approaches for bi-objective problems such as the $$\varepsilon $$ ε -constrained method, revisiting the Parallel Constrained Optimization using Response Surfaces (CORS-RBF) method and the Efficient Global Optimization with Pseudo Expected Improvement (EGO-PEI) algorithm as instances of SMISAs. In addition, a parallel version of the Lower Confidence Bound-based (LCB) algorithm is given as a member within the SMISA class. Secondly, we propose a cooperative optimization framework between the SMISAs. The cooperation between SMISAs occurs in two ways: (1) they share solutions and their objective function values to update their surrogate models and (2) they use the sampled points obtained from different SMISAs to guide their own search process. Some convergence results for this cooperative framework are given under weak conditions. A numerical comparison between EGO-PEI, Parallel CORS-RBF and a cooperative method using both, named CPEI, shows that CPEI improves the performance of the baseline algorithms. The numerical results were derived from 17 analytic tests and they show the reduction of wall-clock time with respect to the increase in the number of processors.

[1]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[2]  M. Patriksson,et al.  A method for simulation based optimization using radial basis functions , 2010 .

[3]  Layne T. Watson,et al.  Efficient global optimization algorithm assisted by multiple surrogate techniques , 2012, Journal of Global Optimization.

[4]  Luis Jiménez,et al.  A cooperative Brain Storm Optimization Algorithm , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[5]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[6]  Edgar Tello-Leal,et al.  A Review of Surrogate Assisted Multiobjective Evolutionary Algorithms , 2016, Comput. Intell. Neurosci..

[7]  Zhonghua Han,et al.  Efficient aerodynamic shape optimization of transonic wings using a parallel infilling strategy and surrogate models , 2016, Structural and Multidisciplinary Optimization.

[8]  Christine A. Shoemaker,et al.  Parallel radial basis function methods for the global optimization of expensive functions , 2007, Eur. J. Oper. Res..

[9]  Haiyan Liu,et al.  Cooperative Coevolution with Formula-Based Variable Grouping for Large-Scale Global Optimization , 2018, Evolutionary Computation.

[10]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[11]  Wilfrido Gómez-Flores,et al.  On the selection of surrogate models in evolutionary optimization algorithms , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[12]  William J. Welch,et al.  Computer experiments and global optimization , 1997 .

[13]  Taimoor Akhtar,et al.  SOP: parallel surrogate global optimization with Pareto center selection for computationally expensive single objective problems , 2016, J. Glob. Optim..

[14]  Xiaodong Li,et al.  DG2: A Faster and More Accurate Differential Grouping for Large-Scale Black-Box Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[15]  Ying Tan,et al.  Multiobjective Infill Criterion Driven Gaussian Process-Assisted Particle Swarm Optimization of High-Dimensional Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[16]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[17]  Yuansheng Cheng,et al.  Pseudo expected improvement criterion for parallel EGO algorithm , 2017, J. Glob. Optim..

[18]  James M. Parr,et al.  Infill sampling criteria for surrogate-based optimization with constraint handling , 2012 .

[19]  Bernd Bischl,et al.  MOI-MBO: Multiobjective Infill for Parallel Model-Based Optimization , 2014, LION.

[20]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[21]  R. Jagannathan.  On Some Properties of Programming Problems in Parametric form Pertaining to Fractional Programming , 1966 .

[22]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[23]  M. E. Johnson,et al.  Minimax and maximin distance designs , 1990 .

[24]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[25]  Christine A. Shoemaker,et al.  Improved Strategies for Radial basis Function Methods for Global Optimization , 2007, J. Glob. Optim..

[26]  Timoteo Carletti,et al.  A Surrogate-Assisted Cooperative Co-evolutionary Algorithm Using Recursive Differential Grouping as Decomposition Strategy , 2019, 2019 IEEE Congress on Evolutionary Computation (CEC).

[27]  Caroline Sainvitu,et al.  Multi-point infill sampling strategies exploiting multiple surrogate models , 2019, GECCO.

[28]  Ricardo García-Ródenas,et al.  A Memetic Chaotic Gravitational Search Algorithm for unconstrained global optimization problems , 2019, Appl. Soft Comput..

[29]  R. Ródenas,et al.  Extensions of Dinkelbach's algorithm for solving non-linear fractional programming problems , 1999 .

[30]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[31]  Christine A. Shoemaker,et al.  Constrained Global Optimization of Expensive Black Box Functions Using Radial Basis Functions , 2005, J. Glob. Optim..

[32]  Stephen J. Leary,et al.  A parallel updating scheme for approximating and optimizing high fidelity computer simulations , 2004 .

[33]  Bernd Bischl,et al.  Multi-objective parameter configuration of machine learning algorithms using model-based optimization , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[34]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[35]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[36]  Xiaodong Li,et al.  A Competitive Divide-and-Conquer Algorithm for Unconstrained Large-Scale Black-Box Optimization , 2016, ACM Trans. Math. Softw..

[37]  Kenny Q. Ye,et al.  Algorithmic construction of optimal symmetric Latin hypercube designs , 2000 .

[38]  Anirban Chaudhuri,et al.  Parallel surrogate-assisted global optimization with expensive functions – a survey , 2016 .

[39]  Jennifer A. Scott,et al.  A Sparse Symmetric Indefinite Direct Solver for GPU Architectures , 2016, ACM Trans. Math. Softw..

[40]  D. Ginsbourger,et al.  Kriging is well-suited to parallelize optimization , 2010 .

[41]  Christine A. Shoemaker,et al.  Parallel Stochastic Global Optimization Using Radial Basis Functions , 2009, INFORMS J. Comput..

[42]  Kamy Sepehrnoori,et al.  Tie-Simplex-Based Phase-Behavior Modeling in an IMPEC Reservoir Simulator , 2014 .

[43]  D. Krige.  A statistical approach to some basic mine valuation problems on the Witwatersrand, by D.G. Krige, published in the Journal, December 1951 : introduction by the author , 1951 .

[44]  Saúl Zapotecas Martínez,et al.  MOEA/D assisted by rbf networks for expensive multi-objective optimization problems , 2013, GECCO '13.

[45]  Ky Khac Vu,et al.  Surrogate-based methods for black-box optimization , 2017, Int. Trans. Oper. Res..