An adaptive-topology ensemble algorithm for engineering optimization problems

Modern engineering design optimization often relies on computer simulations to evaluate candidate designs, a scenario which formulates the problem of optimizing a computationally expensive black-box functions. In such problems, there will often exist candidate designs which cause the simulation to fail, and this can degrade the optimization effectiveness. To address this issue, this paper proposes a new optimization algorithm which incorporates classifiers into the optimization search. The classifiers predict which candidate design are expected to cause the simulation to fail, and their prediction is used to bias the search towards valid designs, namely, for which the simulation is expected to succeed. However, the effectiveness of this approach strongly depends on the type of metamodels and classifiers being used, but due to the high cost of evaluating the simulation-based objective function it may be impractical to identify by numerical experiments the most suitable types of each. Leveraging on these issues, the proposed algorithm offers two main contributions: (a) it uses ensembles of both metamodels and classifiers to benefit from a diversity of predictions of different metamodels and classifiers, and (b) to improve the search effectiveness, it continuously adapts the ensembles’ topology during the search. The performance of the proposed algorithm was evaluated using an engineering problem of airfoil shape optimization. Performance analysis of the proposed algorithm using an engineering problem of airfoil shape optimization shows that: (a) incorporating classifiers into the search was an effective approach to handle simulation failures (b) using ensembles of metamodels and classifiers, and updating their topology during the search, improved the search effectiveness in comparison to using a single metamodel and classifier, and (c) it is beneficial to update the topology of the metamodel ensemble in all problem types, and it is beneficial to update the classifier ensemble topology in problems where simulation failures are prevalent.

[1]  Antonio Filippone,et al.  Flight Performance of Fixed- and Rotary-Wing Aircraft , 2006 .

[2]  Katya Scheinberg,et al.  A derivative free optimization algorithm in practice , 1998 .

[3]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[4]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[5]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[6]  Erwie Zahara,et al.  Hybrid Nelder-Mead simplex search and particle swarm optimization for constrained engineering design problems , 2009, Expert Syst. Appl..

[7]  Peter J. Fleming,et al.  The MATLAB genetic algorithm toolbox , 1995 .

[8]  Haym Hirsh,et al.  A genetic algorithm for continuous design space search , 1997, Artif. Intell. Eng..

[9]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[10]  Kazuhiro Izui,et al.  Handling Undefined Vectors in Expensive Optimization Problems , 2010, EvoApplications.

[11]  Bernhard Sendhoff,et al.  Reducing Fitness Evaluations Using Clustering Techniques and Neural Network Ensembles , 2004, GECCO.

[12]  Petros Koumoutsakos,et al.  Accelerating evolutionary algorithms with Gaussian process fitness function models , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[13]  R. M. Hicks,et al.  Wing Design by Numerical Optimization , 1977 .

[14]  M. Golberg,et al.  Improved multiquadric approximation for partial differential equations , 1996 .

[15]  C. Poloni,et al.  Hybridization of a multi-objective genetic algorithm, a neural network and a classical optimizer for a complex design problem in fluid dynamics , 2000 .

[16]  R. Haftka,et al.  Ensemble of surrogates , 2007 .

[17]  N. M. Alexandrov,et al.  A trust-region framework for managing the use of approximation models in optimization , 1997 .

[18]  E. Acar.  Various approaches for constructing an ensemble of metamodels using local measures , 2010 .

[19]  Chi-Keong Goh,et al.  Computational Intelligence in Expensive Optimization Problems , 2010 .

[20]  Bernhard Sendhoff,et al.  A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation , 2007, GECCO '07.

[21]  W. R. Madych,et al.  Miscellaneous error bounds for multiquadric and related interpolators , 1992 .

[22]  David J. Sheskin,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 1997 .

[23]  Thomas Bäck,et al.  Metamodel-Assisted Evolution Strategies , 2002, PPSN.

[24]  Feng Liu,et al.  Comparisons of Three Geometric Representations of Airfoils for Aerodynamic Optimization , 2003 .

[25]  Niko Kotilainen,et al.  A Memetic-Neural Approach to Discover Resources in P2P Networks , 2008, Recent Advances in Evolutionary Computation for Combinatorial Optimization.

[26]  Daniel J Poole,et al.  Free-form aerodynamic wing optimization using mathematically-derived design variables , 2015 .

[27]  Timothy W. Simpson,et al.  Sampling Strategies for Computer Experiments: Design and Analysis , 2001 .

[28]  Timothy W. Simpson,et al.  Axisymmetric Vehicle Nose Shape Optimization , 2008 .

[29]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[30]  Katya Scheinberg,et al.  Introduction to derivative-free optimization , 2010, Math. Comput..

[31]  Yoel Tenne,et al.  A Versatile Surrogate-Assisted Memetic Algorithm for Optimization of Computationally Expensive Functions and its Engineering Applications , 2008 .

[32]  J. Dennis,et al.  An efficient class of direct search surrogate methods for solving expensive optimization problems with CPU-time-related functions , 2012 .

[33]  A. Ratle.  Optimal sampling strategies for learning a fitness model , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[34]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[35]  Katya Scheinberg,et al.  On the convergence of derivative-free methods for unconstrained optimization , 1997 .

[36]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[37]  Allan Zhong,et al.  Construction of surrogate model ensembles with sparse data , 2007, 2007 IEEE Congress on Evolutionary Computation.

[38]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[39]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[40]  JianJun Wang,et al.  Ensemble of metamodels with Recursive arithmetic average , 2010, 2010 The 2nd International Conference on Industrial Mechatronics and Automation.

[41]  Chee Keong Kwoh,et al.  Feasibility Structure Modeling: An Effective Chaperone for Constrained Memetic Algorithms , 2010, IEEE Transactions on Evolutionary Computation.

[42]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..