A Classifier-Assisted Level-Based Learning Swarm Optimizer for Expensive Optimization

Surrogate-assisted evolutionary algorithms (SAEAs) have become one popular method to solve complex and computationally expensive optimization problems. However, most existing SAEAs suffer from performance degradation with the dimensionality increasing. To solve this issue, this paper proposes a classifier-assisted level-based learning swarm optimizer on the basis of the level-based learning swarm optimizer (LLSO) and the gradient boosting gradient classifier (GBC) to improve the robustness and scalability of SAEAs. Particularly, the level-based learning strategy in LLSO has a tight correspondence with the classification characteristic by setting the number of levels in LLSO to be the same as the number of classes in GBC. Together, the classification results feedback the distribution of promising candidates to accelerate the evolution of the optimizer, while the evolved population helps improve the accuracy of the classifier. To select informative and valuable candidates for real evaluations, we devise a L1-exploitation strategy to extensively exploit promising areas. Then, the candidate selection is conducted between the predicted L1 offspring and the already real-evaluated L1 individuals based on their Euclidean distances. Extensive experiments on commonly-used benchmark functions demonstrate that the proposed optimizer can achieve competitive or better performance with a very small training dataset compared with three state-of-the-art SAEAs.

[1]  Lamjed Ben Said,et al.  Steady state IBEA assisted by MLP neural networks for expensive multi-objective optimization problems , 2014, GECCO.

[2]  Jeng-Shyang Pan,et al.  A new fitness estimation strategy for particle swarm optimization , 2013, Inf. Sci..

[3]  Bernhard Sendhoff,et al.  Structure optimization of neural networks for evolutionary design optimization , 2005, Soft Comput..

[4]  Ying Tan,et al.  Surrogate-assisted hierarchical particle swarm optimization , 2018, Inf. Sci..

[5]  Zhe Song,et al.  Anticipatory Control of Wind Turbines With Data-Driven Predictive Models , 2009, IEEE Transactions on Energy Conversion.

[6]  Tapabrata Ray,et al.  A Surrogate Assisted Approach for Single-Objective Bilevel Optimization , 2017, IEEE Transactions on Evolutionary Computation.

[7]  Saúl Zapotecas Martínez,et al.  MOEA/D assisted by rbf networks for expensive multi-objective optimization problems , 2013, GECCO '13.

[8]  Yong Wang,et al.  Global and Local Surrogate-Assisted Differential Evolution for Expensive Constrained Optimization Problems With Inequality Constraints , 2019, IEEE Transactions on Cybernetics.

[9]  Yew-Soon Ong,et al.  A study on polynomial regression and Gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[10]  Zhang Jun,et al.  Scheduling Workflows With Composite Tasks: A Nested Particle Swarm Optimization Approach , 2022, IEEE Transactions on Services Computing.

[11]  Qingfu Zhang,et al.  A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems , 2014, IEEE Transactions on Evolutionary Computation.

[12]  Jaime G. Carbonell,et al.  Approaches to machine learning , 1984, J. Am. Soc. Inf. Sci..

[13]  Dan Guo,et al.  Small data driven evolutionary multi-objective optimization of fused magnesium furnaces , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[14]  Dan Guo,et al.  Data-Driven Evolutionary Optimization: An Overview and Case Studies , 2019, IEEE Transactions on Evolutionary Computation.

[15]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[16]  Feng Zhao,et al.  A Cooperative Co-Evolutionary Approach to Large-Scale Multisource Water Distribution Network Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[17]  Tianlong Gu,et al.  Ant Colony Optimization for the Control of Pollutant Spreading on Social Networks , 2020, IEEE Transactions on Cybernetics.

[18]  Xinyu Li,et al.  Efficient Generalized Surrogate-Assisted Evolutionary Algorithm for High-Dimensional Expensive Problems , 2020, IEEE Transactions on Evolutionary Computation.

[19]  Tapabrata Ray,et al.  A Multiple Surrogate Assisted Decomposition-Based Evolutionary Algorithm for Expensive Multi/Many-Objective Optimization , 2019, IEEE Transactions on Evolutionary Computation.

[20]  Yang Yu,et al.  A two-layer surrogate-assisted particle swarm optimization algorithm , 2014, Soft Computing.

[21]  J. Friedman Greedy function approximation: A gradient boosting machine. , 2001 .

[22]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[23]  J. Friedman Stochastic gradient boosting , 2002 .

[24]  Handing Wang,et al.  Data-Driven Surrogate-Assisted Multiobjective Evolutionary Optimization of a Trauma System , 2016, IEEE Transactions on Evolutionary Computation.

[25]  Ke Tang,et al.  Classification- and Regression-Assisted Differential Evolution for Computationally Expensive Problems , 2012, Journal of Computer Science and Technology.

[26]  Jianchao Zeng,et al.  Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems , 2017, IEEE Transactions on Evolutionary Computation.

[27]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[28]  Andrew Kusiak,et al.  Optimization of Wind Turbine Performance With Data-Driven Models , 2010, IEEE Transactions on Sustainable Energy.

[29]  António Gaspar-Cunha,et al.  A Multi-Objective Evolutionary Algorithm Using Neural Networks to Approximate Fitness Evaluations , 2005, Int. J. Comput. Syst. Signals.

[30]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[31]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[32]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms in Engineering Applications , 1997, Springer Berlin Heidelberg.

[33]  Marco Dorigo,et al.  Ant colony optimization theory: A survey , 2005, Theor. Comput. Sci..

[34]  Jun Zhang,et al.  A Distributed Swarm Optimizer With Adaptive Communication for Large-Scale Optimization , 2020, IEEE Transactions on Cybernetics.

[35]  Luc De Raedt,et al.  Active Learning for High Throughput Screening , 2008, Discovery Science.

[36]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[37]  Hakan Cevikalp,et al.  Hyperdisk based large margin classifier , 2013, Pattern Recognit..

[38]  Yang Wang,et al.  Cost-sensitive boosting for classification of imbalanced data , 2007, Pattern Recognit..

[39]  M. Stein Large sample properties of simulations using latin hypercube sampling , 1987 .

[40]  John Doherty,et al.  Hierarchical Surrogate-Assisted Evolutionary Multi-Scenario Airfoil Shape Optimization , 2018, 2018 IEEE Congress on Evolutionary Computation (CEC).

[41]  John Doherty,et al.  Offline Data-Driven Evolutionary Optimization Using Selective Surrogate Ensembles , 2019, IEEE Transactions on Evolutionary Computation.

[42]  Nirupam Chakraborti,et al.  A data-driven surrogate-assisted evolutionary algorithm applied to a many-objective blast furnace optimization problem , 2017 .

[43]  Yang Wang,et al.  A Novel Evolutionary Sampling Assisted Optimization Method for High-Dimensional Expensive Problems , 2019, IEEE Transactions on Evolutionary Computation.

[44]  Rommel G. Regis,et al.  Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions , 2014, IEEE Transactions on Evolutionary Computation.

[45]  Michèle Sebag,et al.  A mono surrogate for multiobjective optimization , 2010, GECCO '10.

[46]  Handing Wang,et al.  A Random Forest-Assisted Evolutionary Algorithm for Data-Driven Constrained Multiobjective Combinatorial Optimization of Trauma Systems , 2020, IEEE Transactions on Cybernetics.

[47]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[48]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[49]  Michèle Sebag,et al.  Comparison-Based Optimizers Need Comparison-Based Surrogates , 2010, PPSN.

[50]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[51]  John Doherty,et al.  Committee-Based Active Learning for Surrogate-Assisted Particle Swarm Optimization of Expensive Problems , 2017, IEEE Transactions on Cybernetics.