Trust-region based adaptive radial basis function algorithm for global optimization of expensive constrained black-box problems

Abstract It has been a very challenging task to develop efficient and robust techniques to solve real-world engineering optimization problems due to the unknown function properties, complex constraints and severely limited computational budget. To address this issue, TARBF algorithm (trust-region based adaptive radial basis function interpolation) for solving expensive constrained black-box optimization problems is proposed in this paper. The approach successfully decomposes the original optimization problem into a sequence of sub-problems approximated by radial basis functions in a series of trust regions. Then, the solution of each sub-problem becomes the starting point for the next iteration. According to the values of objective and constraint functions, an effective online normalization technique is further developed to adaptively improve the model accuracy in the trust region, where the surrogate is updated iteratively. Averagely, TARBF has the ability to robustly solve the 21 G-problems (CEC’2006) and 4 engineering problems within 535.69 and 234.44 function evaluations, respectively. The comparison results with other state-of-the-art metamodel-based algorithms prove that TARBF is a convergent, efficient and accurate paradigm. Moreover, the sophisticated trust region strategy developed in TARBF, which is a major contribution to the field of the efficient constrained optimization, has the capability to facilitate an effective balance of exploration and exploitation for solving constrained black-box optimization problems.

[1]  Z. Dong,et al.  Metamodelling and search using space exploration and unimodal region elimination for design optimization , 2010 .

[2]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[3]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[4]  Afonso Celso de Castro Lemonge,et al.  A surrogate assisted differential evolution to solve constrained optimization problems , 2017, 2017 IEEE Latin American Conference on Computational Intelligence (LA-CCI).

[5]  Connie M. Borror,et al.  Response Surface Methodology: A Retrospective and Literature Survey , 2004 .

[6]  Thomas Bäck,et al.  Self-adjusting parameter control for surrogate-assisted constrained optimization under limited budgets , 2017, Appl. Soft Comput..

[7]  Zuomin Dong,et al.  Hybrid surrogate-based optimization using space reduction (HSOSR) for expensive black-box functions , 2018, Appl. Soft Comput..

[8]  V. Toropov,et al.  Implementation of Discrete Capability into the Enhanced Multipoint Approximation Method for Solving Mixed Integer-Continuous Optimization Problems , 2016 .

[9]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[10]  Mohammed Azmi Al-Betar,et al.  Island artificial bee colony for global optimization , 2020, Soft Computing.

[11]  Herbert Hamers,et al.  Constrained optimization involving expensive function evaluations: A sequential approach , 2005, Eur. J. Oper. Res..

[12]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[13]  Rommel G. Regis,et al.  Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions , 2011, Comput. Oper. Res..

[14]  Guangyao Li,et al.  Hybrid metamodel-based design space management method for expensive problems , 2017 .

[15]  Mohammed Azmi Al-Betar,et al.  Island flower pollination algorithm for global optimization , 2019, The Journal of Supercomputing.

[16]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[17]  K. Yamazaki,et al.  Sequential Approximate Optimization using Radial Basis Function network for engineering optimization , 2011 .

[18]  K. Gupta,et al.  Metamodel-Based Optimization for Problems With Expensive Objective and Constraint Functions , 2011 .

[19]  T. Ray,et al.  A framework for design optimization using surrogates , 2005 .

[20]  Chengyang Liu,et al.  Efficient strategies for constrained black-box optimization by intrinsically linear approximation (CBOILA) , 2020, Engineering with Computers.

[21]  Ponnuthurai N. Suganthan,et al.  An improved differential evolution algorithm using efficient adapted surrogate model for numerical optimization , 2018, Inf. Sci..

[22]  Christine A. Shoemaker,et al.  ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions , 2008, SIAM J. Sci. Comput..

[23]  Erwie Zahara,et al.  Hybrid Nelder-Mead simplex search and particle swarm optimization for constrained engineering design problems , 2009, Expert Syst. Appl..

[24]  Juliane Müller,et al.  GOSAC: global optimization with surrogate approximation of constraints , 2017, J. Glob. Optim..

[25]  Fang Liu,et al.  A novel selection evolutionary strategy for constrained optimization , 2013, Inf. Sci..

[26]  Liang Shi,et al.  ASAGA: an adaptive surrogate-assisted genetic algorithm , 2008, GECCO '08.

[27]  Takashi Okamoto,et al.  Constrained optimization using the quasi-chaotic optimization method with the exact penalty function and the sequential quadratic programming , 2011, 2011 IEEE International Conference on Systems, Man, and Cybernetics.

[28]  Anirban Chaudhuri,et al.  Parallel surrogate-assisted global optimization with expensive functions – a survey , 2016 .

[29]  Jianqiao Chen,et al.  A surrogate-based particle swarm optimization algorithm for solving optimization problems with expensive black box functions , 2013 .

[30]  Shengli Xu,et al.  Constrained global optimization via a DIRECT-type constraint-handling technique and an adaptive metamodeling strategy , 2017 .

[31]  Yizhong Wu,et al.  A Kriging-based constrained global optimization algorithm for expensive black-box functions with infeasible initial points , 2017, J. Glob. Optim..

[32]  Yang Yu,et al.  A two-layer surrogate-assisted particle swarm optimization algorithm , 2014, Soft Computing.

[33]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[34]  Zuomin Dong,et al.  SCGOSR: Surrogate-based constrained global optimization using space reduction , 2018, Appl. Soft Comput..

[35]  R. Regis Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points , 2014 .

[36]  Thomas Philip Runarsson,et al.  Constrained Evolutionary Optimization by Approximate Ranking and Surrogate Models , 2004, PPSN.

[37]  M. Powell A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation , 1994 .

[38]  Xinyu Shao,et al.  An efficient surrogate-assisted particle swarm optimization algorithm for high-dimensional expensive problems , 2019, Knowl. Based Syst..

[39]  Vassili Toropov,et al.  Mid-range metamodel assembly building based on linear regression for large scale optimization problems , 2012 .

[40]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[41]  Rommel G. Regis,et al.  Particle swarm with radial basis function surrogates for expensive black-box optimization , 2014, J. Comput. Sci..

[42]  Stefan M. Wild,et al.  CONORBIT: constrained optimization by radial basis function interpolation in trust regions† , 2017, Optim. Methods Softw..

[43]  Bertil E. Damato,et al.  On the use of multi-objective evolutionary algorithms for survival analysis , 2007, Biosyst..

[44]  Ann-Britt Ryberg,et al.  Metamodel-Based Multidisciplinary Design Optimization for Automotive Applications , 2012 .

[45]  Byeongdo Kim,et al.  Comparison study on the accuracy of metamodeling technique for non-convex functions , 2009 .

[46]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[47]  Bernard Widrow,et al.  The basic ideas in neural networks , 1994, CACM.

[48]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[49]  A. Basudhar,et al.  Constrained efficient global optimization with support vector machines , 2012, Structural and Multidisciplinary Optimization.

[50]  Bilal H. Abed-alguni Island-based Cuckoo Search with Highly Disruptive Polynomial Mutation , 2019 .

[51]  Bilal H. Abed-alguni,et al.  Island-based whale optimisation algorithm for continuous optimisation problems , 2019, Int. J. Reason. based Intell. Syst..

[52]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[53]  J. Sampson Adaptation in Natural and Artificial Systems (John H. Holland) , 1976 .

[54]  T. Simpson,et al.  Analysis of support vector regression for approximation of complex engineering analyses , 2005, DAC 2003.

[55]  S. Rippa,et al.  Numerical Procedures for Surface Fitting of Scattered Data by Radial Functions , 1986 .

[56]  A. Jahangirian,et al.  A surrogate assisted evolutionary optimization method with application to the transonic airfoil design , 2010 .

[57]  Eric Jones,et al.  SciPy: Open Source Scientific Tools for Python , 2001 .

[58]  Christine A. Shoemaker,et al.  Global Convergence of Radial Basis Function Trust Region Derivative-Free Algorithms , 2011, SIAM J. Optim..

[59]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[60]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .