A model-independent adaptive sequential sampling technique based on response nonlinearity estimation

Sequential adaptive sampling strategies attempt to efficiently and accurately generate surrogate model (SM) training datasets by limiting the number of required function evaluations. Among these techniques, those that are model independent have attracted attention during the last decade because they do not use a global SM to supervise the sampling process, thereby relieving the user from the burden of selecting an appropriate SM formulation at the beginning of the sampling process. In this study, we propose a new model-independent sequential adaptive sampling technique called nearest neighbors adaptive sampling (NNAS). The NNAS formulation introduces a refinement metric based on local linear models that leads to a quasilinear dependency of the algorithm complexity with respect to the number of samples. Additionally, the exploration-refinement balance is achieved by a stochastic Pareto-ranking-based selection criterion that attempts to simultaneously maximize both the exploration and refinement. NNAS is tested on ten two-dimensional and one ten-dimensional analytic functions, a five-dimensional engineering test case based on the XRotor solver (Drela and Youngren 2014 ), and a two-response two-dimensional analytic problem. The results show that NNAS is more computationally efficient than comparable methods without causing a relevant increase in sample requirement.

[1]  Mengjie Zhang,et al.  Pareto front feature selection based on artificial bee colony optimization , 2018, Inf. Sci..

[2]  Haitao Liu,et al.  A Robust Error-Pursuing Sequential Sampling Approach for Global Metamodeling Based on Voronoi Diagram and Cross Validation , 2014 .

[3]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[4]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[5]  S. Rippa,et al.  Numerical Procedures for Surface Fitting of Scattered Data by Radial Functions , 1986 .

[6]  Y. Censor Pareto optimality in multiobjective problems , 1977 .

[7]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[8]  Dirk Gorissen,et al.  Space-filling sequential design strategies for adaptive surrogate modelling , 2009, SOCO 2009.

[9]  Haitao Liu,et al.  A survey of adaptive sampling for global metamodeling in support of simulation-based complex engineering design , 2017, Structural and Multidisciplinary Optimization.

[10]  Ruichen Jin,et al.  On Sequential Sampling for Global Metamodeling in Engineering Design , 2002, DAC 2002.

[11]  Christian B Allen,et al.  Comparison of Adaptive Sampling Methods for Generation of Surrogate Aerodynamic Models , 2013 .

[12]  Thomas H. Lee,et al.  The Design of CMOS Radio-Frequency Integrated Circuits: RF CIRCUITS THROUGH THE AGES , 2003 .

[13]  Mrinal K. Sen,et al.  Hybrid Gaussian-cubic radial basis functions for scattered data interpolation , 2015, Computational Geosciences.

[14]  Brian J. German,et al.  Adaptive Sampling with Adaptive Surrogate Model Selection for Computer Experiment Applications , 2017 .

[15]  Ying Ma,et al.  An Adaptive Bayesian Sequential Sampling Approach for Global Metamodeling , 2016 .

[16]  Timothy W. Simpson,et al.  Sampling Strategies for Computer Experiments: Design and Analysis , 2001 .

[17]  Ernesto P. Adorio,et al.  MVF - Multivariate Test Functions Library in C for Unconstrained Global Optimization , 2005 .

[18]  Atharv Bhosekar,et al.  Advances in surrogate based modeling, feasibility analysis, and optimization: A review , 2018, Comput. Chem. Eng..

[19]  Qing Li,et al.  Multiobjective optimization for crash safety design of vehicles using stepwise regression model , 2008 .

[20]  Daniel W. Apley,et al.  Lifted Brownian Kriging Models , 2017, Technometrics.

[21]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[22]  B. Yegnanarayana,et al.  Artificial Neural Networks , 2004 .

[23]  Dirk Gorissen,et al.  A Novel Hybrid Sequential Design Strategy for Global Surrogate Modeling of Computer Experiments , 2011, SIAM J. Sci. Comput..

[24]  Dirk Gorissen,et al.  Sequential modeling of a low noise amplifier with neural networks and active learning , 2009, Neural Computing and Applications.

[25]  G. Gary Wang,et al.  Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions , 2010 .

[26]  Jack P. C. Kleijnen,et al.  Application-driven sequential designs for simulation experiments: Kriging metamodelling , 2004, J. Oper. Res. Soc..

[27]  Brian J. German,et al.  Performance assessment of a cross-validation sampling strategy with active surrogate model selection , 2019, Structural and Multidisciplinary Optimization.

[28]  R. L. Hardy Multiquadric equations of topography and other irregular surfaces , 1971 .

[29]  Bo Wang,et al.  Determination of realistic worst imperfection for cylindrical shells using surrogate model , 2013 .

[30]  Brian J. German,et al.  Comparison of Adaptive Design Space Exploration Methods Applied to S-Duct CFD Simulation , 2016 .

[31]  T. J. Mitchell,et al.  Bayesian design and analysis of computer experiments: Use of derivatives in surface prediction , 1993 .

[32]  M. Tarter Inverse cumulative approximation and applications. , 1968, Biometrika.

[33]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[34]  Malek Ben Salem,et al.  Universal Prediction Distribution for Surrogate Models , 2015, SIAM/ASA J. Uncertain. Quantification.

[35]  Xiuli Shen,et al.  An improved support vector regression using least squares method , 2018 .

[36]  Yiming Zhang,et al.  General Surrogate Adaptive Sampling using Interquartile Range for Design Space Exploration , 2019, AIAA Scitech 2019 Forum.

[37]  Jack P. C. Kleijnen,et al.  Stochastic Intrinsic Kriging for Simulation Metamodelling , 2015 .

[38]  Alberto Lovison,et al.  Adaptive sampling with a Lipschitz criterion for accurate metamodeling , 2010 .

[39]  Tom Dhaene,et al.  A Fuzzy Hybrid Sequential Design Strategy for Global Surrogate Modeling of High-Dimensional Computer Experiments , 2015, SIAM J. Sci. Comput..

[40]  Hui Zhou,et al.  An active learning variable-fidelity metamodelling approach based on ensemble of metamodels and objective-oriented sequential sampling , 2016 .

[41]  Timothy W. Simpson,et al.  Sampling Strategies for Computer Experiments , 2001 .

[42]  Franz Aurenhammer,et al.  Voronoi diagrams—a survey of a fundamental geometric data structure , 1991, CSUR.

[43]  S. Gunn Support Vector Machines for Classification and Regression , 1998 .

[44]  D. Mavris,et al.  Performance Analysis of Different Positions of Leading Edge Tubercles on a Wind Turbine Blade , 2018 .

[45]  William I. Notz,et al.  Sequential adaptive designs in computer experiments for response surface model fit , 2008 .

[46]  Chen Jiang,et al.  A two-stage support vector regression assisted sequential sampling approach for global metamodeling , 2018 .

[47]  V. Klee On the complexity ofd- dimensional Voronoi diagrams , 1979 .

[48]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[49]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[50]  Daniel N. Wilke,et al.  Simple Intuitive Multi-objective ParalLElization of Efficient Global Optimization: SIMPLE-EGO , 2017 .

[51]  Taimoor Akhtar,et al.  An adaptive population-based candidate search algorithm with surrogates for global multi objective optimization of expensive functions , 2019 .

[52]  Wagdi G. Habashi,et al.  Clarification: Edge-Based Finite Element Modeling of Magnetogasdynamic-Based Propulsion Systems , 2020 .

[53]  Herman Lam,et al.  Adaptive Sampling with Varying Sampling Cost for Design Space Exploration , 2019, AIAA Journal.

[54]  Alex A. Gorodetsky,et al.  Mercer kernels and integrated variance experimental design: connections between Gaussian process regression and polynomial approximation , 2015, SIAM/ASA J. Uncertain. Quantification.