Truncation-learning-driven surrogate assisted social learning particle swarm optimization for computationally expensive problem

Abstract Surrogate-assisted evolutionary optimization greatly slashes the computational burden of evolutionary algorithms for computationally expensive problems. However, new issues arise concerning the compatibility and fault tolerance of surrogates, evolutionary learning operators, and problem property. To this end, this paper proposes a truncation-learning-driven surrogate assisted social learning particle swarm optimizer (TL-SSLPSO) to coordinate these three ingredients. For avoiding and correcting the deceptions induced by the low confidence exemplars due to the surrogate in behavior learning, TL-SSLPSO equally segments the iterative population into multiple sub-populations with different fitness levels and selects exemplars from the randomly selected high-level sub-populations for the behavior learning of low-level sub-population, while truncating the behavior learning of the highest-level sub-population composed of some of the best approximated or real evaluated particles and retaining the sub-population directly to the next generation. Besides, a greedy sampling strategy is employed to find promising solutions with better fitness versus the global best to complement the truncation learning. Extensive experiments on twenty-four widely used benchmark problems and a stepped cantilever beam design problem with 17 steps are conducted to assess the effectiveness of cooperation between truncation learning and greedy sampling and comparisons with several state-of-the-art algorithms demonstrate the superiority of the proposed method.

[1]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[2]  John Doherty,et al.  Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems , 2017, IEEE Transactions on Cybernetics.

[3]  Layne T. Watson,et al.  Efficient global optimization algorithm assisted by multiple surrogate techniques , 2012, Journal of Global Optimization.

[4]  Ricardo M. Paiva,et al.  A Comparison of Surrogate Models in the Framework of an MDO Tool for Wing Design , 2009 .

[5]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[6]  Andy J. Keane,et al.  Combining Global and Local Surrogate Models to Accelerate Evolutionary Optimization , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[7]  Bernhard Sendhoff,et al.  Comparing neural networks and Kriging for fitness approximation in evolutionary optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[8]  Ying Tan,et al.  Surrogate-assisted hierarchical particle swarm optimization , 2018, Inf. Sci..

[9]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[10]  Weiming Shen,et al.  A Surrogate-Assisted Multiswarm Optimization Algorithm for High-Dimensional Computationally Expensive Problems , 2020, IEEE Transactions on Cybernetics.

[11]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[12]  Yaochu Jin,et al.  Surrogate-Assisted Multicriteria Optimization: Complexities, Prospective Solutions, and Business Case , 2017 .

[13]  Bernhard Sendhoff,et al.  Fitness Approximation In Evolutionary Computation - a Survey , 2002, GECCO.

[14]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[15]  Liang Gao,et al.  A multi-point sampling method based on kriging for global optimization , 2017 .

[16]  Georges G. E. Gielen,et al.  GASPAD: A General and Efficient mm-Wave Integrated Circuit Synthesis Method Based on Surrogate Model Assisted Evolutionary Algorithm , 2014, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[17]  Christine A. Shoemaker,et al.  A Stochastic Radial Basis Function Method for the Global Optimization of Expensive Functions , 2007, INFORMS J. Comput..

[18]  Marco Savoia,et al.  Coupling Response Surface and Differential Evolution for Parameter Identification Problems , 2015, Comput. Aided Civ. Infrastructure Eng..

[19]  Carlos A. Coello Coello,et al.  Comparison of metamodeling techniques in evolutionary algorithms , 2017, Soft Comput..

[20]  Yan Wang,et al.  An active learning radial basis function modeling method based on self-organization maps for simulation-based design problems , 2017, Knowl. Based Syst..

[21]  Yaochu Jin,et al.  A social learning particle swarm optimization algorithm for scalable optimization , 2015, Inf. Sci..

[22]  Chen Jiang,et al.  Two-layer adaptive surrogate-assisted evolutionary algorithm for high-dimensional computationally expensive problems , 2019, J. Glob. Optim..

[23]  Régis Duvigneau,et al.  Low cost PSO using metamodels and inexact pre-evaluation: Application to aerodynamic shape design , 2009 .

[24]  Kaisa Miettinen,et al.  A Surrogate-Assisted Reference Vector Guided Evolutionary Algorithm for Computationally Expensive Many-Objective Optimization , 2018, IEEE Transactions on Evolutionary Computation.

[25]  Zhigang Ren,et al.  Surrogate model assisted cooperative coevolution for large scale optimization , 2018, Applied Intelligence.

[26]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[27]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[28]  Jianchao Zeng,et al.  Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems , 2017, IEEE Transactions on Evolutionary Computation.

[29]  Haitao Liu,et al.  An adaptive sampling approach for Kriging metamodeling by maximizing expected prediction error , 2017, Comput. Chem. Eng..

[30]  Bernd Bischl,et al.  MOI-MBO: Multiobjective Infill for Parallel Model-Based Optimization , 2014, LION.

[31]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[32]  Bryan A. Tolson,et al.  Numerical assessment of metamodelling strategies in computationally intensive optimization , 2012, Environ. Model. Softw..

[33]  Kambiz Haji Hajikolaei,et al.  High Dimensional Model Representation With Principal Component Analysis , 2014 .

[34]  Ying Tan,et al.  A generation-based optimal restart strategy for surrogate-assisted social learning particle swarm optimization , 2019, Knowl. Based Syst..

[35]  Selen Cremaschi,et al.  Adaptive sequential sampling for surrogate model generation with artificial neural networks , 2014, Comput. Chem. Eng..

[36]  Yong Wang,et al.  Global and Local Surrogate-Assisted Differential Evolution for Expensive Constrained Optimization Problems With Inequality Constraints , 2019, IEEE Transactions on Cybernetics.

[37]  Tianyou Chai,et al.  Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems , 2019, IEEE Transactions on Cybernetics.

[38]  Petros Koumoutsakos,et al.  Accelerating evolutionary algorithms with Gaussian process fitness function models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[39]  M. Powell.  Recent research at Cambridge on radial basis functions , 1999 .

[40]  Yaochu Jin,et al.  Demonstrator selection in a social learning particle swarm optimizer , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[41]  Liang Gao,et al.  Sequential approximation optimization assisted particle swarm optimization for expensive problems , 2019, Appl. Soft Comput..

[42]  Haitao Liu,et al.  Sampling Strategies and Metamodeling Techniques for Engineering Design: Comparison and Application , 2016 .

[43]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[44]  C. Shoemaker,et al.  Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization , 2013 .

[45]  Haitao Liu,et al.  A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design , 2017, Structural and Multidisciplinary Optimization.

[46]  Ke Tang,et al.  Classification- and Regression-Assisted Differential Evolution for Computationally Expensive Problems , 2012, Journal of Computer Science and Technology.

[47]  Ahmed Kattan,et al.  Surrogate Genetic Programming: A semantic aware evolutionary search , 2015, Inf. Sci..

[48]  Ying Tan,et al.  Multiobjective Infill Criterion Driven Gaussian Process-Assisted Particle Swarm Optimization of High-Dimensional Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[49]  Deyi Xue,et al.  Performance Comparison of Metamodeling Methods From the Perspective of Sample Quality Merits , 2009, DAC 2009.

[50]  Dan Guo,et al.  Data-Driven Evolutionary Optimization: An Overview and Case Studies , 2019, IEEE Transactions on Evolutionary Computation.

[51]  R. Regis.  Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points , 2014 .

[52]  M. J. D. Powell,et al.  Radial basis function methods for interpolation to functions of many variables , 2001, HERCMA.

[53]  Zuomin Dong,et al.  Trends, features, and tests of common and recently introduced global optimization methods , 2010 .

[54]  An Chen,et al.  Enhancing cooperative coevolution for large scale optimization by adaptively constructing surrogate models , 2018, GECCO.

[55]  Aimin Zhou,et al.  A Multioperator Search Strategy Based on Cheap Surrogate Models for Evolutionary Optimization , 2015, IEEE Transactions on Evolutionary Computation.

[56]  G. Gary Wang,et al.  Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions , 2010 .

[57]  Yang Yu,et al.  A two-layer surrogate-assisted particle swarm optimization algorithm , 2014, Soft Computing.

[58]  Guangyao Li,et al.  Variable stiffness composite material design by using support vector regression assisted efficient global optimization method , 2017 .

[59]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[60]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[61]  Xin Yao,et al.  Meta-Heuristic Algorithms in Car Engine Design: A Literature Survey , 2015, IEEE Transactions on Evolutionary Computation.

[62]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[63]  Qingfu Zhang,et al.  A multi-fidelity surrogate-model-assisted evolutionary algorithm for computationally expensive optimization problems , 2016, J. Comput. Sci..

[64]  Kambiz Haji Hajikolaei,et al.  Decomposition for large-scale global optimization based on quantified variable correlations uncovered by metamodelling , 2015 .

[65]  Hirotaka Nakayama,et al.  Meta-Modeling in Multiobjective Optimization , 2008, Multiobjective Optimization.

[66]  J. Havinga,et al.  Sequential improvement for robust optimization using an uncertainty measure for radial basis functions , 2017 .

[67]  Bernhard Sendhoff,et al.  On Evolutionary Optimization with Approximate Fitness Functions , 2000, GECCO.

[68]  Xinyu Li,et al.  Surrogate-guided differential evolution algorithm for high dimensional expensive problems , 2019, Swarm Evol. Comput..

[69]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[70]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[71]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[72]  Xin Yao,et al.  Classification-assisted Differential Evolution for computationally expensive problems , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[73]  T. W. Layne,et al.  A Comparison of Approximation Modeling Techniques: Polynomial Versus Interpolating Models , 1998 .

[74]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[75]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[76]  Rommel G. Regis,et al.  Particle swarm with radial basis function surrogates for expensive black-box optimization , 2014, J. Comput. Sci..

[77]  Jeng-Shyang Pan,et al.  A new fitness estimation strategy for particle swarm optimization , 2013, Inf. Sci..

[78]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[79]  Fred H. Lesh,et al.  Multi-dimensional least-squares polynomial curve fitting , 1959, CACM.

[80]  Junheung Park,et al.  Meta-modeling using generalized regression neural network and particle swarm optimization , 2017, Appl. Soft Comput..

[81]  Antonio Bolufé Röhler,et al.  Measuring the curse of dimensionality and its effects on particle swarm optimization and differential evolution , 2014, Applied Intelligence.