A review on design of experiments and surrogate models in aircraft real-time and many-query aerodynamic analyses

Abstract Full scale aerodynamic wind tunnel testing, numerical simulation of high dimensional (full-order) aerodynamic models or flight testing are some of the fundamental but complex steps in the various design phases of recent civil transport aircrafts. Current aircraft aerodynamic designs have increase in complexity (multidisciplinary, multi-objective or multi-fidelity) and need to address the challenges posed by the nonlinearity of the objective functions and constraints, uncertainty quantification in aerodynamic problems or the restrained computational budgets. With the aim to reduce the computational burden and generate low-cost but accurate models that mimic those full order models at different values of the design variables, Recent progresses have witnessed the introduction, in real-time and many-query analyses, of surrogate-based approaches as rapid and cheaper to simulate models. In this paper, a comprehensive and state-of-the art survey on common surrogate modeling techniques and surrogate-based optimization methods is given, with an emphasis on models selection and validation, dimensionality reduction, sensitivity analyses, constraints handling or infill and stopping criteria. Benefits, drawbacks and comparative discussions in applying those methods are described. Furthermore, the paper familiarizes the readers with surrogate models that have been successfully applied to the general field of fluid dynamics, but not yet in the aerospace industry. Additionally, the review revisits the most popular sampling strategies used in conducting physical and simulation-based experiments in aircraft aerodynamic design. Attractive or smart designs infrequently used in the field and discussions on advanced sampling methodologies are presented, to give a glance on the various efficient possibilities to a priori sample the parameter space. Closing remarks foster on future perspectives, challenges and shortcomings associated with the use of surrogate models by aircraft industrial aerodynamicists, despite their increased interest among the research communities.

[1]  Benjamin Stamm,et al.  EFFICIENT GREEDY ALGORITHMS FOR HIGH-DIMENSIONAL PARAMETER SPACES WITH APPLICATIONS TO EMPIRICAL INTERPOLATION AND REDUCED BASIS METHODS ∗ , 2014 .

[2]  Alberto D. Pascual-Montano,et al.  A survey of dimensionality reduction techniques , 2014, ArXiv.

[3]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[4]  P. Beran,et al.  Reduced-order modeling: new approaches for computational physics , 2004 .

[5]  Ahsan Kareem,et al.  Modeling Nonlinear Systems by Volterra Series , 2010 .

[6]  Stefan Görtz,et al.  Improving variable-fidelity surrogate modeling via gradient-enhanced kriging and a generalized hybrid bridge function , 2013 .

[7]  James J. Filliben,et al.  Taguchi’s Orthogonal Arrays Are Classical Designs of Experiments , 1991, Journal of research of the National Institute of Standards and Technology.

[8]  Ken Badcock,et al.  Transonic Aeroelastic Stability Analysis Using a Kriging-Based Schur Complement Formulation , 2011 .

[9]  R. Haftka,et al.  A Stopping Criterion for Surrogate Based Optimization using EGO , 2013 .

[10]  Carl D. Sorensen,et al.  A general approach for robust optimal design , 1993 .

[11]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[12]  K. Willcox Unsteady Flow Sensing and Estimation via the Gappy Proper Orthogonal Decomposition , 2004 .

[13]  O. L. Davies,et al.  Design and analysis of industrial experiments , 1954 .

[14]  Aziz Hamdouni,et al.  Experimental and numerical POD study of the Coanda effect used to reduce self-sustained tones , 2004 .

[15]  J. Marsden,et al.  A subspace approach to balanced truncation for model reduction of nonlinear control systems , 2002 .

[16]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[17]  Kevin Burrage,et al.  Populations of Models, Experimental Designs and Coverage of Parameter Space by Latin Hypercube and Orthogonal Sampling , 2015, ICCS.

[18]  Selden B. Crary,et al.  Design of Computer Experiments for Metamodel Generation , 2002 .

[19]  Tom Dhaene,et al.  Performance study of multi-fidelity gradient enhanced kriging , 2015 .

[20]  Kai-Tai Fang,et al.  A new approach in constructing orthogonal and nearly orthogonal arrays , 2000 .

[21]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[22]  Min-Qian Liu,et al.  An algorithmic approach to constructing mixed-level orthogonal and near-orthogonal arrays , 2008, Comput. Stat. Data Anal..

[23]  Karen Willcox,et al.  Surrogate and reduced-order modeling: a comparison of approaches for large-scale statistical inverse problems [Chapter 7] , 2010 .

[24]  A. Sudjianto,et al.  An Efficient Algorithm for Constructing Optimal Design of Computer Experiments , 2005, DAC 2003.

[25]  V. Klein,et al.  Determination of airplane model structure from flight data using splines and stepwise regression , 1983 .

[26]  Ralf Zimmermann,et al.  Gradient-enhanced surrogate modeling based on proper orthogonal decomposition , 2013, J. Comput. Appl. Math..

[27]  Brian D. Ripley,et al.  Statistical aspects of neural networks , 1993 .

[28]  Virgil L. Anderson,et al.  Design of experiments : a no-name approach , 1993 .

[29]  P. Sagaut,et al.  Building Efficient Response Surfaces of Aerodynamic Functions with Kriging and Cokriging , 2008 .

[30]  David Herrero Pérez,et al.  Kriging-based infill sampling criterion for constraint handling in multi-objective optimization , 2016, J. Glob. Optim..

[31]  Warren F. Kuhfeld,et al.  Some new orthogonal arrays , 2005 .

[32]  Bertrand Clarke,et al.  Principles and Theory for Data Mining and Machine Learning , 2009 .

[33]  W. Silva,et al.  Identification of Nonlinear Aeroelastic Systems Based on the Volterra Theory: Progress and Opportunities , 2005 .

[34]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[35]  Qiang Du,et al.  Centroidal Voronoi Tessellations: Applications and Algorithms , 1999, SIAM Rev..

[36]  Philip D. Wasserman,et al.  Advanced methods in neural computing , 1993, VNR computer library.

[37]  R. L. Rechtschaffner Saturated Fractions of 2n and 3n Factorial Designs , 1967 .

[38]  C. Wu,et al.  Probability-based Latin hypercube designs for slid-rectangular regions , 2010 .

[39]  Klaus Becker,et al.  An industrial view on numerical simulation for aircraft aerodynamic design , 2011 .

[40]  James M. Parr,et al.  Infill sampling criteria for surrogate-based optimization with constraint handling , 2012 .

[41]  David Amsallem,et al.  Fast local reduced basis updates for the efficient reduction of nonlinear systems with hyper-reduction , 2015, Adv. Comput. Math..

[42]  Richard P. Dwight,et al.  Flow simulation and shape optimization for aircraft design , 2007 .

[43]  R. Haftka,et al.  Multiple surrogates: how cross-validation errors can help us to obtain the best predictor , 2009 .

[44]  George E. Karniadakis,et al.  Gappy data: To Krig or not to Krig? , 2006, J. Comput. Phys..

[45]  Stefan Görtz,et al.  Improved extrapolation of steady turbulent aerodynamics using a non-linear POD-based reduced order model , 2012, The Aeronautical Journal (1968).

[46]  Jack P. C. Kleijnen,et al.  Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping , 2008, Eur. J. Oper. Res..

[47]  Karen Willcox,et al.  Parametric reduced-order models for probabilistic analysis of unsteady aerodynamic applications , 2007 .

[48]  Fabio Vetrano,et al.  Assessment of Strategies for Interpolating POD Based Reduced Order Models and Application to Aeroelasticity , 2012 .

[49]  Moncef Gabbouj,et al.  Evolutionary artificial neural networks by multi-dimensional particle swarm optimization , 2009, Neural Networks.

[50]  Michael S. Eldred,et al.  Formulations for Surrogate-Based Optimization Under Uncertainty , 2002 .

[51]  Antony Jameson,et al.  Aerodynamic design via control theory , 1988, J. Sci. Comput..

[52]  Nicolas R. Gauger,et al.  Adjoint approaches in aerodynamic shape optimization and MDO context , 2006 .

[53]  Peter Shirley,et al.  Discrepancy as a Quality Measure for Sample Distributions , 1991, Eurographics.

[54]  Jay I. Myung,et al.  Global model analysis by parameter space partitioning. , 2019, Psychological review.

[55]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[56]  Jack P. C. Kleijnen,et al.  Kriging interpolation in simulation: a survey , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[57]  T. P. Ryan,et al.  Modern Experimental Design , 2007 .

[58]  Raphael T. Haftka,et al.  Multi-fidelity design of stiffened composite panel with a crack , 2002 .

[59]  Steven K. Thompson,et al.  Adaptive Cluster Sampling , 1990 .

[60]  Donald E. Myers,et al.  Basic Linear Geostatistics , 1998, Technometrics.

[61]  Lih-Yuan Deng,et al.  Orthogonal Arrays: Theory and Applications , 1999, Technometrics.

[62]  G. Gary Wang,et al.  Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions , 2010 .

[63]  Gianluigi Rozza,et al.  Model Order Reduction in Fluid Dynamics: Challenges and Perspectives , 2014 .

[64]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[65]  Philip E. Gill,et al.  Practical optimization , 1981 .

[66]  G. Box,et al.  Some New Three Level Designs for the Study of Quantitative Variables , 1960 .

[67]  Qiqi Wang,et al.  Residual Minimizing Model Interpolation for Parameterized Nonlinear Dynamical Systems , 2010, SIAM J. Sci. Comput..

[68]  Andy J. Keane,et al.  Computational Approaches for Aerospace Design: The Pursuit of Excellence , 2005 .

[69]  Y. Maday,et al.  Reduced Basis Techniques for Stochastic Problems , 2010, 1004.0357.

[70]  Ralf Zimmermann,et al.  Simplified Cross-Correlation Estimation For Multi-Fidelity Surrogate Cokriging Models , 2010 .

[71]  Jack P. C. Kleijnen,et al.  Application-driven sequential designs for simulation experiments: Kriging metamodelling , 2004, J. Oper. Res. Soc..

[72]  Robert F. Stengel,et al.  Identification of aerodynamic coefficients using computational neural networks , 1992 .

[73]  H. Wackernagel,et al.  Mapping temperature using kriging with external drift: Theory and an example from scotland , 1994 .

[74]  J. Kiefer,et al.  Optimum Designs in Regression Problems , 1959 .

[75]  R. Haftka Combining global and local approximations , 1991 .

[76]  N. Nguyen,et al.  An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations , 2004 .

[77]  Aimo A. Törn,et al.  Global Optimization , 1999, Science.

[78]  R. Zimmermann,et al.  Interpolation-based reduced-order modelling for steady transonic flows via manifold learning , 2014 .

[79]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[80]  Jian Sun,et al.  Support-Vector-Machine-Based Reduced-Order Model for Limit Cycle Oscillation Prediction of Nonlinear Aeroelastic System , 2012 .

[81]  Dirk P. Kroese,et al.  Handbook of Monte Carlo Methods , 2011 .

[82]  Janet S. Peterson,et al.  The Reduced Basis Method for Incompressible Viscous Flow Calculations , 1989 .

[83]  Sidney Addelman,et al.  trans-Dimethanolbis(1,1,1-trifluoro-5,5-dimethylhexane-2,4-dionato)zinc(II) , 2008, Acta crystallographica. Section E, Structure reports online.

[84]  Rahul Mukerjee,et al.  A Modern Theory Of Factorial Designs , 2006 .

[85]  Bento Silva de Mattos,et al.  Aerodynamic Coefficient Prediction of Transport Aircraft Using Neural Network , 2006 .

[86]  Florent Renac,et al.  Local and Global Search Methods for Design in Aeronautics. , 2011 .

[87]  R. Caflisch Monte Carlo and quasi-Monte Carlo methods , 1998, Acta Numerica.

[88]  Edward J. Dudewicz,et al.  Modern Design and Analysis of Discrete-Event Computer Simulations , 1985 .

[89]  Taehyoun Kim,et al.  Frequency-Domain Karhunen -Loeve Method and Its Application to Linear Dynamic Systems , 1998 .

[90]  Alex Smola,et al.  Kernel methods in machine learning , 2007, math/0701907.

[91]  Gunzburger,et al.  Advances in Studies and Applications of Centroidal Voronoi Tessellations , 2010 .

[92]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[93]  M. Y. M. Ahmed,et al.  Surrogate-Based Aerodynamic Design Optimization: Use of Surrogates in Aerodynamic Design Optimization , 2009 .

[94]  Pierre Sagaut,et al.  Comparison of Gradient-Based and Gradient-Enhanced Response-Surface-Based Optimizers , 2010 .

[95]  Peter Benner,et al.  A Robust Algorithm for Parametric Model Order Reduction , 2007 .

[96]  Danny C. Sorensen,et al.  Nonlinear Model Reduction via Discrete Empirical Interpolation , 2010, SIAM J. Sci. Comput..

[97]  James R. Carr,et al.  Cokriging—a computer program , 1985 .

[98]  M. E. Johnson,et al.  Minimax and maximin distance designs , 1990 .

[99]  Dick den Hertog,et al.  Maximin Latin Hypercube Designs in Two Dimensions , 2007, Oper. Res..

[100]  Slawomir Koziel,et al.  Surrogate-based modeling and optimization : applications in engineering , 2013 .

[101]  Bertrand Iooss,et al.  Numerical studies of space-filling designs: optimization of Latin Hypercube Samples and subprojection properties , 2013, J. Simulation.

[102]  Jacob K. White,et al.  Model order reduction for nonlinear dynamical systems based on trajectory piecewise-linear approximations , 2006 .

[103]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[104]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[105]  David Ryckelynck Hyper‐reduction of mechanical models involving internal variables , 2009 .

[106]  Julien Bect,et al.  Sequential search based on kriging: convergence analysis of some algorithms , 2011, ArXiv.

[107]  B. Iooss,et al.  A Review on Global Sensitivity Analysis Methods , 2014, 1404.2405.

[108]  Junqiang Bai,et al.  Stationary flow fields prediction of variable physical domain based on proper orthogonal decomposition and kriging surrogate model , 2015 .

[109]  R. Plackett,et al.  THE DESIGN OF OPTIMUM MULTIFACTORIAL EXPERIMENTS , 1946 .

[110]  Bertrand Iooss,et al.  Numerical Study of the Metamodel Validation Process , 2009, 2009 First International Conference on Advances in System Simulation.

[111]  Walter A. Silva,et al.  Reduced-Order Models Based on Linear and Nonlinear Aerodynamic Impulse Responses , 1999 .

[112]  G. Box,et al.  On the Experimental Attainment of Optimum Conditions , 1951 .

[113]  Paul Bratley,et al.  Algorithm 659: Implementing Sobol's quasirandom sequence generator , 1988, TOMS.

[114]  Henry P. Wynn,et al.  Maximum entropy sampling , 1987 .

[115]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[116]  Michael S. Eldred,et al.  OVERVIEW OF MODERN DESIGN OF EXPERIMENTS METHODS FOR COMPUTATIONAL SIMULATIONS , 2003 .

[117]  Jyh-Ching Juang,et al.  Estimation of aerodynamic coefficients using neural networks , 1993 .

[118]  Emiliano Iuliano,et al.  Application of Surrogate-based Global Optimization to Aerodynamic Design , 2016 .

[119]  Bento Silva de Mattos,et al.  Aerodynamic Coefficient Prediction of Airfoils Using Neural Networks , 2008 .

[120]  Timothy W. Simpson,et al.  Metamodeling in Multidisciplinary Design Optimization: How Far Have We Really Come? , 2014 .

[121]  Heike Faßbender,et al.  Projection-Based Model Order Reduction for Steady Aerodynamics , 2013 .

[122]  Juan J. Alonso,et al.  Investigation of non-linear projection for POD based reduced order models for Aerodynamics , 2001 .

[123]  K. Willcox,et al.  Constrained multifidelity optimization using model calibration , 2012, Structural and Multidisciplinary Optimization.

[124]  Timothy W. Simpson,et al.  On the Use of Kriging Models to Approximate Deterministic Computer Models , 2004, DAC 2004.

[125]  R. Pinnau Model Reduction via Proper Orthogonal Decomposition , 2008 .

[126]  Stefan Görtz,et al.  Alternative Cokriging Method for Variable-Fidelity Surrogate Modeling , 2012 .

[127]  Robert Haimes,et al.  Multifidelity Optimization for Variable-Complexity Design , 2006 .

[128]  Richard DeLoach,et al.  The Modern Design of Experiments for Configuration Aerodynamics: A Case Study , 2006 .

[129]  Kari Karhunen,et al.  Über lineare Methoden in der Wahrscheinlichkeitsrechnung , 1947 .

[130]  J. C. Wang,et al.  Nearly orthogonal arrays with mixed levels and small runs , 1992 .

[131]  J. Peraire,et al.  Balanced Model Reduction via the Proper Orthogonal Decomposition , 2002 .

[132]  R. H. Myers Classical and modern regression with applications , 1986 .

[133]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[134]  Zhong-Hua Han,et al.  Support Vector Regression-based Multidisciplinary Design Optimization in Aircraft Conceptual Design , 2013 .

[135]  Peter Benner,et al.  Model Order Reduction for Linear and Nonlinear Systems: A System-Theoretic Perspective , 2014 .

[136]  Pierre-Alain Boucard,et al.  Variable-fidelity modeling of structural analysis of assemblies , 2016, J. Glob. Optim..

[137]  Luc Pronzato,et al.  Design of computer experiments: space filling and beyond , 2011, Statistics and Computing.

[138]  I K Fodor,et al.  A Survey of Dimension Reduction Techniques , 2002 .

[139]  George S. Dulikravich,et al.  Multi-Winglets: Multi-Objective Optimization of Aerodynamic Shapes , 2015 .

[140]  Layne T. Watson,et al.  Pitfalls of using a single criterion for selecting experimental designs , 2008 .

[141]  J. Gentle Random number generation and Monte Carlo methods , 1998 .

[142]  Kwang-Yong Kim,et al.  Multiple surrogate modeling for axial compressor blade shape optimization , 2008 .

[143]  Karen Willcox,et al.  A Bayesian-Based Approach to Multifidelity Multidisciplinary Design Optimization , 2010 .

[144]  C. Pain,et al.  Non‐intrusive reduced‐order modelling of the Navier–Stokes equations based on RBF interpolation , 2015 .

[145]  Charbel Farhat,et al.  Nonlinear Model Reduction for CFD Problems Using Local Reduced Order Bases , 2012 .

[146]  Manuel Laguna,et al.  Fine-Tuning of Algorithms Using Fractional Experimental Designs and Local Search , 2006, Oper. Res..

[147]  J. Alonso,et al.  Managing Gradient Inaccuracies while Enhancing Optimal Shape Design Methods , 2013 .

[148]  Zhenghong Gao,et al.  Research on multi-fidelity aerodynamic optimization methods , 2013 .

[149]  Sholom M. Weiss,et al.  Computer Systems That Learn , 1990 .

[150]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[151]  Charbel Farhat,et al.  Nonlinear model order reduction based on local reduced‐order bases , 2012 .

[152]  Fred J. Hickernell,et al.  Goodness-of-fit statistics, discrepancies and robust designs , 1999 .

[153]  Y. Zhu,et al.  A method for exact calculation of the discrepancy of low-dimensional finite point sets I , 1993 .

[154]  D. Ryckelynck,et al.  A priori hyperreduction method: an adaptive approach , 2005 .

[155]  Alexander H. G. Rinnooy Kan,et al.  Bayesian stopping rules for multistart global optimization methods , 1987, Math. Program..

[156]  C. Shoemaker,et al.  Sensitivity Analysis for Computationally Expensive Models using Optimization and Objective-oriented Surrogate Approximations , 2014, 1410.7291.

[157]  Stefan Görtz,et al.  Hierarchical Kriging Model for Variable-Fidelity Surrogate Modeling , 2012 .

[158]  F. Chinesta,et al.  A Short Review in Model Order Reduction Based on Proper Generalized Decomposition , 2018 .

[159]  Aslak Tveito,et al.  Elements of Scientific Computing , 2010 .

[160]  Atsuyuki Okabe,et al.  Spatial Tessellations: Concepts and Applications of Voronoi Diagrams , 1992, Wiley Series in Probability and Mathematical Statistics.

[161]  Richard P. Dwight,et al.  Aerodynamic Shape Optimization Using the Discrete Adjoint of the Navier-Stokes Equations: Applications towards Complex 3D Configurations , 2009 .

[162]  Yudong Zhang,et al.  A Comprehensive Survey on Particle Swarm Optimization Algorithm and Its Applications , 2015 .

[163]  Michael Goldstein,et al.  Constructing partial prior specifications for models of complex physical systems , 1998 .

[164]  C. Lemieux Monte Carlo and Quasi-Monte Carlo Sampling , 2009 .

[165]  J. S. Hunter,et al.  The 2 k—p Fractional Factorial Designs Part I , 2000, Technometrics.

[166]  Stefan Görtz,et al.  Non-Linear POD-based Reduced Order Models for Steady Turbulent Aerodynamics , 2010 .

[167]  Weng Kee Wong,et al.  A unified approach to the construction of minimax designs , 1992 .

[168]  Bernhard Schölkopf,et al.  Learning with kernels , 2001 .

[169]  James V. Bondar Universal optimality of experimental designs: definitions and a criterion , 1983 .

[170]  W. G. Hunter,et al.  Minimum Aberration 2k-p Designs , 1980 .

[171]  Christian B Allen,et al.  Aerodynamic Data Modeling using Multi-Criteria Adaptive Sampling , 2010 .

[172]  Matteo Franciolini,et al.  Adaptive design of experiments for efficient and accurate estimation of aerodynamic loads , 2017 .

[173]  Ali Elham,et al.  Winglet multi-objective shape optimization , 2014 .

[174]  Jack P. C. Kleijnen Design and Analysis of Simulation Experiments , 2007 .

[175]  Emiliano Iuliano,et al.  Adaptive Sampling Strategies for Surrogate-Based Aerodynamic Optimization , 2016 .

[176]  Tiangang Cui,et al.  Data‐driven model reduction for the Bayesian solution of inverse problems , 2014, 1403.4290.

[177]  L. Carro-Calvo,et al.  Constrained Single-Point Aerodynamic Shape Optimization of the DPW-W1 Wing Through Evolutionary Programming and Support Vector Machines , 2019 .

[178]  Paul G. Constantine,et al.  Active Subspaces - Emerging Ideas for Dimension Reduction in Parameter Studies , 2015, SIAM spotlights.

[179]  Thomas W. Lucas,et al.  Efficient Nearly Orthogonal and Space-Filling Latin Hypercubes , 2007, Technometrics.

[180]  N. M. Alexandrov,et al.  A trust-region framework for managing the use of approximation models in optimization , 1997 .

[181]  Serge Gratton,et al.  A surrogate management framework using rigorous trust-region steps , 2014, Optim. Methods Softw..

[182]  Hekmat Alighanbari,et al.  Application of Artificial Neural Networks in Aerodynamics Prediction of Low-Reynolds-Number Figure-Eight Motion of an Airfoil , 2010 .

[183]  A. Jahangirian,et al.  Aerodynamic shape optimization using efficient evolutionary algorithms and unstructured CFD solver , 2011 .

[184]  Michael S. Eldred,et al.  Second-Order Corrections for Surrogate-Based Optimization with Model Hierarchies , 2004 .

[185]  Slawomir Koziel,et al.  Multi-objective Airfoil Design Using Variable-Fidelity CFD Simulations and Response Surface Surrogates , 2014 .

[186]  C. Farhat,et al.  Efficient non‐linear model reduction via a least‐squares Petrov–Galerkin projection and compressive tensor approximations , 2011 .

[187]  J. Kiefer Optimum Experimental Designs V, with Applications to Systematic and Rotatable Designs , 1961 .

[188]  Zhong-Hua Han,et al.  Surrogate-Based Optimization , 2012, Engineering Design Optimization.

[189]  Pierre Ladevèze,et al.  On the verification of model reduction methods based on the proper generalized decomposition , 2011 .

[190]  Thomas J. Santner,et al.  Noncollapsing Space-Filling Designs for Bounded Nonrectangular Regions , 2012, Technometrics.

[191]  G. Heuvelink,et al.  A generic framework for spatial prediction of soil variables based on regression-kriging , 2004 .

[192]  Aaron A. Tucker,et al.  Application of Design of Experiments to Flight Test: A Case Study , 2010 .

[193]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[194]  J. Morlier,et al.  Improving kriging surrogates of high-dimensional design models by Partial Least Squares dimension reduction , 2016 .

[195]  Lawrence Sirovich,et al.  Karhunen–Loève procedure for gappy data , 1995 .

[196]  Jaijeet Roychowdhury,et al.  Model reduction via projection onto nonlinear manifolds, with applications to analog circuits and biochemical systems , 2008, ICCAD 2008.

[197]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[198]  Laurene V. Fausett,et al.  Fundamentals Of Neural Networks , 1994 .

[199]  L. Sirovich Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[200]  Christian B Allen,et al.  Evaluation of radial basis functions for CFD volume data interpolation , 2010 .

[201]  Piotr Breitkopf,et al.  Model reduction by CPOD and Kriging , 2010 .

[202]  William Line Proven Cost Savings by Using Modern Design of Experiments (MDOE) , 2010 .

[203]  Christian B Allen,et al.  Comparison of Adaptive Sampling Methods for Generation of Surrogate Aerodynamic Models , 2013 .

[204]  Karen Willcox,et al.  Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input Space , 2008, SIAM J. Sci. Comput..

[205]  Jerome Sacks,et al.  Choosing the Sample Size of a Computer Experiment: A Practical Guide , 2009, Technometrics.

[206]  Tiangang Cui,et al.  Multifidelity importance sampling , 2016 .

[207]  Zhong-Hua Han,et al.  Improving Adjoint-Based Aerodynamic Optimization via Gradient-Enhanced Kriging , 2012 .

[208]  R. DeLoach Improved Quality in Aerospace Testing Through the Modern Design of Experiments , 2000 .

[209]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[210]  Juan J. Alonso,et al.  Active Subspaces for Shape Optimization , 2014 .

[211]  Filip De Turck,et al.  Blind Kriging: Implementation and performance analysis , 2012, Adv. Eng. Softw..

[212]  Peter Guttorp,et al.  Karl Pearson and the Scandinavian School of Statistics , 2009 .

[213]  Runze Li,et al.  Uniform design for computer experiments and its optimal properties , 2006 .

[214]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[215]  Charbel Farhat,et al.  The GNAT method for nonlinear model reduction: Effective implementation and application to computational fluid dynamics and turbulent flows , 2012, J. Comput. Phys..

[216]  Yaochu Jin,et al.  Surrogate-assisted evolutionary computation: Recent advances and future challenges , 2011, Swarm Evol. Comput..

[217]  Andy J. Keane,et al.  Dimension Reduction for Aerodynamic Design Optimization , 2011 .

[218]  Thomas A. Brenner,et al.  Practical Aspects of the Implementation of Proper Orthogonal Decomposition , 2009 .

[219]  Ken Badcock,et al.  On the generation of flight dynamics aerodynamic tables by computational fluid dynamics , 2011 .

[220]  Danny C. Sorensen,et al.  A Posteriori Error Estimation for DEIM Reduced Nonlinear Dynamical Systems , 2014, SIAM J. Sci. Comput..

[221]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[222]  Andy J. Keane,et al.  Non-stationary kriging for design optimization , 2012 .

[223]  Bryan Glaz,et al.  Multiple-Surrogate Approach to Helicopter Rotor Blade Vibration Reduction , 2009 .

[224]  Zhong-Hua Han,et al.  Optimization of Active Flow Control over an Airfoil Using a Surrogate-Management Framework , 2010 .

[225]  Francisco Chinesta,et al.  Recent Advances and New Challenges in the Use of the Proper Generalized Decomposition for Solving Multidimensional Models , 2010 .

[226]  Pierre-Alain Boucard,et al.  The use of partially converged simulations in building surrogate models , 2014, Adv. Eng. Softw..

[227]  K. Morgan,et al.  Reduced order modelling for unsteady fluid flow using proper orthogonal decomposition and radial basis functions , 2013 .

[228]  Anthony C. Atkinson,et al.  Optimum Design of Experiments , 2004 .

[229]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[230]  Melanie Mitchell,et al.  Genetic algorithms: An overview , 1995, Complex..

[231]  Karen Willcox,et al.  Proper orthogonal decomposition extensions for parametric applications in compressible aerodynamics , 2003 .

[232]  Donald R. Jones,et al.  Global versus local search in constrained optimization of computer models , 1998 .

[233]  Karen Willcox,et al.  Gradient-based multifidelity optimisation for aircraft design using Bayesian model calibration , 2011, The Aeronautical Journal (1968).

[234]  Tien-Tsin Wong,et al.  Sampling with Hammersley and Halton Points , 1997, J. Graphics, GPU, & Game Tools.

[235]  V. Roshan Joseph,et al.  Space-filling designs for computer experiments: A review , 2016 .

[236]  Anthony T. C. Goh,et al.  Multivariate adaptive regression splines and neural network models for prediction of pile drivability , 2016 .

[237]  Hester Bijl,et al.  Speeding up Kriging through fast estimation of the hyperparameters in the frequency-domain , 2013, Comput. Geosci..

[238]  Ranjan Ganguli,et al.  Aerodynamic Derivative Calculation Using Radial Basis Function Neural Networks , 2017 .

[239]  Zhonghua Han,et al.  Efficient Uncertainty Quantification using Gradient-Enhanced Kriging , 2009 .

[240]  Charbel Farhat,et al.  An Online Method for Interpolating Linear Parametric Reduced-Order Models , 2011, SIAM J. Sci. Comput..

[241]  Slawomir Koziel,et al.  Surrogate-Based Modeling and Optimization , 2013 .

[242]  Leo Wai-Tsun Ng,et al.  Multifidelity approaches for optimization under uncertainty , 2014 .

[243]  Zhong-Hua Han,et al.  A New Cokriging Method for Variable-Fidelity Surrogate Modeling of Aerodynamic Data , 2010 .

[244]  David Amsallem,et al.  An adaptive and efficient greedy procedure for the optimal training of parametric reduced‐order models , 2015 .

[245]  Ken Badcock,et al.  Gust analysis using computational fluid dynamics derived reduced order models , 2017 .

[246]  Kenny Q. Ye,et al.  Algorithmic construction of optimal symmetric Latin hypercube designs , 2000 .

[247]  Christos Koukouvinos,et al.  On the use of three level orthogonal arrays in robust parameter design , 2006 .

[248]  V. Schulz,et al.  Comparing sampling strategies for aerodynamic Kriging surrogate models , 2012 .

[249]  Tuffin Bruno On the use of low discrepancy sequences in Monte Carlo methods , 1996 .

[250]  Leo Breiman,et al.  Discussion: Multivariate Adaptive Regression Splines , 1991 .

[251]  Jens-Dominik Müller,et al.  CAD‐based shape optimisation with CFD using a discrete adjoint , 2014 .

[252]  Simon Fong,et al.  Survey of Meta-Heuristic Algorithms for Deep Learning Training , 2016 .

[253]  Jacob K. White,et al.  A trajectory piecewise-linear approach to model order reduction and fast simulation of nonlinear circuits and micromachined devices , 2001, IEEE/ACM International Conference on Computer Aided Design. ICCAD 2001. IEEE/ACM Digest of Technical Papers (Cat. No.01CH37281).

[254]  Aziz Hamdouni,et al.  Reduced‐order modelling for solving linear and non‐linear equations , 2011 .

[255]  Charbel Farhat,et al.  Progressive construction of a parametric reduced‐order model for PDE‐constrained optimization , 2014, ArXiv.

[256]  Karen Willcox,et al.  A Survey of Projection-Based Model Reduction Methods for Parametric Dynamical Systems , 2015, SIAM Rev..

[257]  E. Jaynes Probability theory : the logic of science , 2003 .

[258]  Volker Schulz,et al.  Efficient Response Surface Methods Based on Generic Surrogate Models , 2012, SIAM J. Sci. Comput..

[259]  Charbel Farhat,et al.  Gradient-based constrained optimization using a database of linear reduced-order models , 2015, J. Comput. Phys..

[260]  Jack P. C. Kleijnen,et al.  State-of-the-Art Review: A User's Guide to the Brave New World of Designing Simulation Experiments , 2005, INFORMS J. Comput..

[261]  Bento Silva de Mattos,et al.  Wing-Fuselage Drag Prediction Using Artificial Neural Networks , 2012 .

[262]  Runze Li,et al.  Design and Modeling for Computer Experiments , 2005 .

[263]  David M. Steinberg,et al.  Comparison of designs for computer experiments , 2006 .

[264]  Andras Varga,et al.  Preface Special issue on “Order reduction of large-scale systems” , 2006 .

[265]  J. Alonso,et al.  Using gradients to construct cokriging approximation models for high-dimensional design optimization problems , 2002 .

[266]  Danny C. Sorensen,et al.  A State Space Error Estimate for POD-DEIM Nonlinear Model Reduction , 2012, SIAM J. Numer. Anal..

[267]  Sancho Salcedo-Sanz,et al.  Efficient aerodynamic design through evolutionary programming and support vector regression algorithms , 2012, Expert Syst. Appl..

[268]  C. F. Jeff Wu,et al.  Experiments: Planning, Analysis, and Parameter Design Optimization , 2000 .

[269]  Chenjie Gu,et al.  Model Order Reduction of Nonlinear Dynamical Systems , 2011 .

[270]  Hester Bijl,et al.  Reducing uncertainties in a wind-tunnel experiment using Bayesian updating , 2012 .

[271]  K. Willcox,et al.  Interpolation among reduced‐order matrices to obtain parameterized models for design, optimization and probabilistic analysis , 2009 .

[272]  K. A. Bush Orthogonal Arrays of Index Unity , 1952 .

[273]  H. V. D. Vorst,et al.  Model Order Reduction: Theory, Research Aspects and Applications , 2008 .

[274]  M. Anitescu,et al.  Polynomial Regression Approaches Using Derivative Information for Uncertainty Quantification , 2010 .

[275]  Mikael A. Langthjem,et al.  Multifidelity Response Surface Approximations for the Optimum Design of Diffuser Flows , 2001 .

[276]  A. Keane,et al.  Kriging Hyperparameter Tuning Strategies , 2008 .

[277]  Stefan Görtz,et al.  A Variable-Fidelity Modeling Method for Aero-Loads Prediction , 2010 .

[278]  Yongliang Chen Modeling of longitudinal unsteady aerodynamics at high angle-of-attack based on support vector machines , 2012, 2012 8th International Conference on Natural Computation.

[279]  Charbel Farhat,et al.  A Compact Proper Orthogonal Decomposition Basis for Optimization-Oriented Reduced-Order Models , 2008 .

[280]  Andy J. Keane,et al.  Enhancing infill sampling criteria for surrogate-based constrained optimization , 2012, J. Comput. Methods Sci. Eng..

[281]  H. Niederreiter,et al.  Monte Carlo and Quasi-Monte Carlo Methods in Scientific Computing , 1995 .

[282]  Bernhard Schölkopf,et al.  Support vector learning , 1997 .

[283]  Min-Qian Liu,et al.  Construction of nearly orthogonal Latin hypercube designs , 2013 .

[284]  Richard DeLoach,et al.  Applications of modern experiment design to wind tunnel testing at NASA Langley Research Center , 1998 .

[285]  G. Elfving Optimum Allocation in Linear Regression Theory , 1952 .

[286]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[287]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[288]  Aloke Dey,et al.  Fractional Factorial Plans , 1999 .

[289]  Gauthier Picard,et al.  Dynamic Design Space Partitioning for Optimization of an Integrated Thermal Protection System , 2013 .

[290]  Susan M. Sanchez,et al.  Very large fractional factorial and central composite designs , 2005, TOMC.

[291]  M. Rai Three-dimensional aerodynamic design using artificial neural networks , 2002 .

[292]  R. Haftka,et al.  Ensemble of surrogates , 2007 .

[293]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[294]  Alain Galli,et al.  Study of a Gas Reservoir Using the External Drift Method , 1987 .

[295]  Jack P. C. Kleijnen,et al.  An Overview of the Design and Analysis of Simulation Experiments for Sensitivity Analysis , 2005, Eur. J. Oper. Res..

[296]  Edwin R. van Dam,et al.  Two-Dimensional Minimax Latin Hypercube Designs , 2005, Discret. Appl. Math..

[297]  Eric O. Postma,et al.  Dimensionality Reduction: A Comparative Review , 2008 .

[298]  T. J. Mitchell,et al.  Bayesian Prediction of Deterministic Functions, with Applications to the Design and Analysis of Computer Experiments , 1991 .

[299]  J. Pereira,et al.  Surrogate models based on function and derivative values for aerodynamic global optimization , 2010 .

[300]  L Sirovich,et al.  Low-dimensional Procedure for the Characterization of Human Faces , 1986 .

[301]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[302]  Dave Higdon,et al.  Combining Field Data and Computer Simulations for Calibration and Prediction , 2005, SIAM J. Sci. Comput..

[303]  Halbert White,et al.  Artificial Neural Networks: Approximation and Learning Theory , 1992 .

[304]  Carlos Guedes Soares,et al.  Numerical study of algorithms for metamodel construction and validation , 2008 .

[305]  George S. Dulikravich,et al.  Aerodynamic data modeling using support vector machines , 2005 .

[306]  Carlos E. S. Cesnik,et al.  Reduced-Order Modeling of Unsteady Aerodynamics Across Multiple Mach Regimes , 2014 .

[307]  Anthony T. Patera,et al.  A Posteriori Error Bounds for the Empirical Interpolation Method , 2010 .

[308]  A. O'Hagan,et al.  Predicting the output from a complex computer code when fast approximations are available , 2000 .

[309]  Karen Willcox,et al.  Model reduction for large-scale CFD applications using the balanced proper orthogonal decomposition , 2005 .

[310]  Gianluigi Rozza,et al.  Reduced Order Methods for Modeling and Computational Reduction , 2013 .

[311]  Karen Willcox,et al.  Surrogate Modeling Approach to Support Real-Time Structural Assessment and Decision Making , 2015 .

[312]  D. Steinberg,et al.  Computer experiments: a review , 2010 .

[313]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007 .

[314]  Art B. Owen,et al.  Scrambling Sobol' and Niederreiter-Xing Points , 1998, J. Complex..

[315]  Bernard Grossman,et al.  Response Surface Models Combining Linear and Euler Aerodynamics for Supersonic Transport Design , 1999 .

[316]  W. G. Hunter,et al.  Minimum Aberration 2 k–p Designs , 1980 .

[317]  Chang-Xing Ma,et al.  Wrap-Around L2-Discrepancy of Random Sampling, Latin Hypercube and Uniform Designs , 2001, J. Complex..

[318]  V. R. Joseph,et al.  ORTHOGONAL-MAXIMIN LATIN HYPERCUBE DESIGNS , 2008 .

[319]  Boxin Tang,et al.  Nearly orthogonal arrays mappable into fully orthogonal arrays , 2014 .

[320]  Stefan Görtz,et al.  Model order reduction for steady aerodynamics of high-lift configurations , 2014 .

[321]  Jonathan E. Cooper,et al.  Parametric reduced-order model approach for simulation and optimization of aeroelastic systems with structural nonlinearities , 2016 .

[322]  Garret N. Vanderplaats,et al.  Numerical optimization techniques for engineering design , 1999 .

[323]  H. Faure Good permutations for extreme discrepancy , 1992 .

[324]  B. Lockwood,et al.  Gradient-based methods for uncertainty quantification in hypersonic flows , 2013 .

[325]  B. Bowerman Statistical Design and Analysis of Experiments, with Applications to Engineering and Science , 1989 .

[326]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[327]  Peter Winker,et al.  Centered L2-discrepancy of random sampling and Latin hypercube design, and construction of uniform designs , 2002, Math. Comput..

[328]  Charles Audet,et al.  A surrogate-model-based method for constrained optimization , 2000 .

[329]  David Amsallem,et al.  A posteriori error estimators for linear reduced‐order models using Krylov‐based integrators , 2015 .

[330]  Zhong-Hua Han,et al.  Variable-Fidelity and Reduced-Order Models for Aero Data for Loads Predictions , 2013 .

[331]  Agus Sudjianto,et al.  Blind Kriging: A New Method for Developing Metamodels , 2008 .

[332]  Russell M. Cummings,et al.  Sampling Strategies for Reduced-Order Modeling of Nonlinear and Unsteady Aerodynamics , 2014 .

[333]  Mariette Yvinec,et al.  Algorithmic geometry , 1998 .

[334]  Rahul Rai,et al.  Qualitative and Quantitative Sequential Sampling , 2006, DAC 2006.

[335]  T. J. Mitchell,et al.  Exploratory designs for computational experiments , 1995 .

[336]  Daniel M. Dunlavy,et al.  Formulations for Surrogate-Based Optimization with Data Fit, Multifidelity, and Reduced-Order Models , 2006 .

[337]  R. Haftka,et al.  Multiple Surrogates for the Shape Optimization of Bluff Body-Facilitated Mixing , 2005 .

[338]  Qiqi Wang,et al.  Erratum: Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces , 2013, SIAM J. Sci. Comput..

[339]  Sebastian Schöps,et al.  An algorithmic comparison of the Hyper-Reduction and the Discrete Empirical Interpolation Method for a nonlinear thermal problem , 2016, ArXiv.

[340]  J. Hiriart-Urruty,et al.  Comparison of public-domain software for black box global optimization , 2000 .

[341]  Charbel Farhat,et al.  Projection‐based model reduction for contact problems , 2015, 1503.01000.

[342]  Michael Goldstein,et al.  Small Sample Bayesian Designs for Complex High-Dimensional Models Based on Information Gained Using Fast Approximations , 2009, Technometrics.

[343]  D. Mavriplis Discrete Adjoint-Based Approach for Optimization Problems on Three-Dimensional Unstructured Meshes , 2007 .

[344]  Thomas Bäck,et al.  Evolutionary computation: comments on the history and current state , 1997, IEEE Trans. Evol. Comput..

[345]  Stefan Volkwein,et al.  Greedy Sampling Using Nonlinear Optimization , 2014 .

[346]  Lorenz T. Biegler,et al.  A trust-region framework for constrained optimization using reduced order modeling , 2013 .

[347]  Antoine Vandendorpe,et al.  Model reduction of linear systems : an interpolation point of view/ , 2004 .

[348]  Geoffrey Oxberry,et al.  Error bounds and analysis of proper orthogonal decomposition model reduction methods using snapshots from the solution and the time derivatives , 2015, 1501.02004.

[349]  Woong Je Sung,et al.  A neural network construction method for surrogate modeling of physics-based analysis , 2012 .

[350]  Murray Smith,et al.  Neural Networks for Statistical Modeling , 1993 .

[351]  M. Rumpfkeil,et al.  Design Optimization Utilizing Gradient/Hessian Enhanced Surrogate Model , 2010 .

[352]  I. Sobola,et al.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[353]  Harald Niederreiter,et al.  Random number generation and Quasi-Monte Carlo methods , 1992, CBMS-NSF regional conference series in applied mathematics.

[354]  Ameet Talwalkar,et al.  Foundations of Machine Learning , 2012, Adaptive computation and machine learning.

[355]  Cong Wang,et al.  Variable fidelity methods and surrogate modeling of critical loads on X-31 aircraft , 2013 .

[356]  Jan Albert Mulder,et al.  Online Aerodynamic Model Identification Using a Recursive Sequential Method for Multivariate Splines , 2013 .

[357]  Alexander I. J. Forrester,et al.  Multi-fidelity optimization via surrogate modelling , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[358]  Edmondo Minisci,et al.  HIGH DIMENSIONAL SENSITIVITY ANALYSIS USING SURROGATE MODELING AND HIGH DIMENSIONAL MODEL REPRESENTATION , 2015 .

[359]  Dimitri N. Mavris,et al.  An Overview of Design Challenges and Methods in Aerospace Engineering , 2011, CSDM.

[360]  Jack P. C. Kleijnen,et al.  Robustness of Kriging when interpolating in random simulation with heterogeneous variances: Some experiments , 2005, Eur. J. Oper. Res..

[361]  Richard DeLoach MDOE Perspectives on Wind Tunnel Testing Objectives , 2002 .

[362]  Marco Ratto,et al.  Sensitivity Analysis for Importance Assessment , 2002, Risk analysis : an official publication of the Society for Risk Analysis.

[363]  Liqian Peng,et al.  Nonlinear model reduction via a locally weighted POD method , 2016 .

[364]  Jack P. C. Kleijnen,et al.  Kriging for interpolation in random simulation , 2003, J. Oper. Res. Soc..

[365]  D. Myers Matrix formulation of co-kriging , 1982 .

[366]  Weihua Zhang,et al.  A faster optimization method based on support vector regression for aerodynamic problems , 2013 .

[367]  Markus P. Rumpfkeil,et al.  Optimization Under Uncertainty Using Gradients, Hessians, and Surrogate Models , 2012 .

[368]  Min-Qian Liu,et al.  CONSTRUCTION OF ORTHOGONAL AND NEARLY ORTHOGONAL LATIN HYPERCUBE DESIGNS FROM ORTHOGONAL DESIGNS , 2012 .

[369]  Carlos A. Coello Coello,et al.  Multi-objective airfoil shape optimization using a multiple-surrogate approach , 2012, 2012 IEEE Congress on Evolutionary Computation.

[370]  K. Glover All optimal Hankel-norm approximations of linear multivariable systems and their L, ∞ -error bounds† , 1984 .

[371]  Afzal Suleman,et al.  Comparison of Surrogate Models in a Multidisciplinary Optimization Framework for Wing Design , 2010 .

[372]  N. Maculan,et al.  Global optimization : from theory to implementation , 2006 .

[373]  Mark R. Opmeer,et al.  Model Order Reduction by Balanced Proper Orthogonal Decomposition and by Rational Interpolation , 2012, IEEE Transactions on Automatic Control.

[374]  Domenico Quagliarella,et al.  Proper Orthogonal Decomposition, surrogate modelling and evolutionary optimization in aerodynamic design , 2013 .

[375]  A. McBratney,et al.  Further results on prediction of soil properties from terrain attributes: heterotopic cokriging and regression-kriging , 1995 .

[376]  R. Iman,et al.  A distribution-free approach to inducing rank correlation among input variables , 1982 .

[377]  Jerome Sacks,et al.  Designs for Computer Experiments , 1989 .

[378]  M. Schetzen The Volterra and Wiener Theories of Nonlinear Systems , 1980 .

[379]  Charbel Farhat,et al.  Construction of Parametrically-Robust CFD-Based Reduced-Order Models for PDE-Constrained Optimization , 2013 .

[380]  Christiaan J. J. Paredis,et al.  A Rational Design Approach to Gaussian Process Modeling for Variable Fidelity Models , 2011, DAC 2011.

[381]  Benjamin Peherstorfer,et al.  Survey of multifidelity methods in uncertainty propagation, inference, and optimization , 2018, SIAM Rev..

[382]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[383]  F. Schoen,et al.  A stochastic technique for global optimization , 1991 .

[384]  Karen Willcox,et al.  Surrogate Modeling for Uncertainty Assessment with Application to Aviation Environmental System Models , 2010 .

[385]  A. A. Zhigli︠a︡vskiĭ,et al.  Stochastic Global Optimization , 2007 .

[386]  R. Dwight,et al.  Effect of Approximations of the Discrete Adjoint on Gradient-Based Optimization , 2006 .

[387]  Daniel J Poole,et al.  Comparison of Local and Global Constrained Aerodynamic Shape Optimization , 2014 .

[388]  J. Friedman Multivariate adaptive regression splines , 1990 .

[389]  Laurent Cordier,et al.  Calibration of POD reduced‐order models using Tikhonov regularization , 2009 .

[390]  R. Zimmermann A Locally Parametrized Reduced-Order Model for the Linear Frequency Domain Approach to Time-Accurate Computational Fluid Dynamics , 2014, SIAM J. Sci. Comput..

[391]  Richard DeLoach,et al.  Fractional Factorial Experiment Designs to Minimize Configuration Changes in Wind Tunnel Testing , 2002 .

[392]  Farrokh Mistree,et al.  Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization , 2001 .

[393]  Hongquan Xu,et al.  An Algorithm for Constructing Orthogonal and Nearly-Orthogonal Arrays With Mixed Levels and Small Runs , 2002, Technometrics.

[394]  Jun S. Liu,et al.  Monte Carlo strategies in scientific computing , 2001 .

[395]  Joel Brezillon,et al.  2D and 3D aerodynamic shape optimisation using the adjoint approach , 2004 .

[396]  T. Pulliam,et al.  Multipoint and Multi-Objective Aerodynamic Shape Optimization , 2002 .

[397]  Wei Chen,et al.  A non‐stationary covariance‐based Kriging method for metamodelling in engineering design , 2007 .

[398]  Karen Willcox,et al.  Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems , 2010, SIAM J. Sci. Comput..

[399]  Emiliano Iuliano,et al.  Aerodynamic shape optimization via non-intrusive POD-based surrogate modelling , 2013, 2013 IEEE Congress on Evolutionary Computation.

[400]  G. Matheron Principles of geostatistics , 1963 .

[401]  Christian P. Robert,et al.  The Bayesian choice : from decision-theoretic foundations to computational implementation , 2007 .

[402]  Randy R. Sitter,et al.  Orthogonal and nearly orthogonal designs for computer experiments , 2009 .

[403]  Jun Zheng,et al.  The Variable Fidelity Optimization for simulation-based design: A review , 2012, Proceedings of the 2012 IEEE 16th International Conference on Computer Supported Cooperative Work in Design (CSCWD).

[404]  George E. P. Box,et al.  Empirical Model‐Building and Response Surfaces , 1988 .

[405]  Slawomir Koziel,et al.  Simulation-Driven Aerodynamic Design Using Variable-Fidelity Models , 2015 .

[406]  Sancho Salcedo-Sanz,et al.  Aerodynamic Shape Design by Evolutionary Optimization and Support Vector Machines , 2016 .

[407]  Russell R. Barton,et al.  A review on design, modeling and applications of computer experiments , 2006 .

[408]  Seongim Choi,et al.  Variable-Fidelity Design Method Using Gradient-Enhanced Kriging Surrogate Model with Regression , 2014 .

[409]  Qing Wang,et al.  Unsteady aerodynamic modeling at high angles of attack using support vector machines , 2015 .

[410]  Tom Dhaene,et al.  Performance study of gradient-enhanced Kriging , 2015, Engineering with Computers.

[411]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[412]  Adrien Leygue,et al.  The Proper Generalized Decomposition for Advanced Numerical Simulations: A Primer , 2013 .

[413]  P. Holmes,et al.  Turbulence, Coherent Structures, Dynamical Systems and Symmetry , 1996 .

[414]  C. Mader,et al.  Computing Stability Derivatives and Their Gradients for Aerodynamic Shape Optimization , 2014 .

[415]  Isabelle Guyon,et al.  Automatic Capacity Tuning of Very Large VC-Dimension Classifiers , 1992, NIPS.

[416]  Miguel Á. Carreira-Perpiñán,et al.  A Review of Dimension Reduction Techniques , 2009 .

[417]  R. Dwight,et al.  Numerical sensitivity analysis for aerodynamic optimization: A survey of approaches , 2010 .

[418]  Jiju Antony,et al.  Design of experiments for engineers and scientists , 2003 .

[419]  H. J. Arnold Introduction to the Practice of Statistics , 1990 .

[420]  J. Peter,et al.  Comparison of surrogate models for turbomachinery design , 2008 .

[421]  Leifur Leifsson,et al.  Surrogate-Based Methods , 2011, Computational Optimization, Methods and Algorithms.

[422]  James E. Steck,et al.  Some applications of artificial neural networks in modeling of nonlinear aerodynamics and flight dynamics , 1997 .

[423]  Raphael T. Haftka,et al.  Surrogate Model-Based Optimization Framework: A Case Study in Aerospace Design , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[424]  A. Quarteroni,et al.  Reduced Basis Methods for Partial Differential Equations: An Introduction , 2015 .

[425]  Ricardo M. Paiva,et al.  A Comparison of Surrogate Models in the Framework of an MDO Tool for Wing Design , 2009 .

[426]  Siep Weiland,et al.  Missing Point Estimation in Models Described by Proper Orthogonal Decomposition , 2004, IEEE Transactions on Automatic Control.

[427]  A. Hay,et al.  Local improvements to reduced-order models using sensitivity analysis of the proper orthogonal decomposition , 2009, Journal of Fluid Mechanics.

[428]  C. Farhat,et al.  Design optimization using hyper-reduced-order models , 2015 .

[429]  Linda R. Petzold,et al.  Approved for public release; further dissemination unlimited Error Estimation for Reduced Order Models of Dynamical Systems ∗ , 2003 .

[430]  V. R. Joseph,et al.  Maximum projection designs for computer experiments , 2015 .

[431]  Ching-Shui Cheng Some Projection Properties of Orthogonal Arrays , 1995 .

[432]  A. Patera,et al.  A posteriori error bounds for reduced-basis approximations of parametrized parabolic partial differential equations , 2005 .

[433]  C. Farhat,et al.  Interpolation Method for Adapting Reduced-Order Models and Application to Aeroelasticity , 2008 .

[434]  Juan J. Alonso,et al.  Design of a Low-Boom Supersonic Business Jet Using Cokriging Approximation Models , 2002 .

[435]  David Ryckelynck,et al.  Multi-level A Priori Hyper-Reduction of mechanical models involving internal variables , 2010 .

[436]  C. R. Rao,et al.  Factorial Experiments Derivable from Combinatorial Arrangements of Arrays , 1947 .

[437]  P. Holmes,et al.  The Proper Orthogonal Decomposition in the Analysis of Turbulent Flows , 1993 .

[438]  Vinay Ramanath,et al.  From Small X to Large X: Assessment of Space -Filling Criteria for the Design and Analysis of Computer Experiments , 2006 .

[439]  Matthias H. Y. Tan,et al.  Minimax Designs for Finite Design Regions , 2013, Technometrics.

[440]  V. L. Anderson,et al.  Design of Experiments: A Realistic Approach , 1974 .

[441]  W. Rugh Nonlinear System Theory: The Volterra / Wiener Approach , 1981 .

[442]  Vito Volterra,et al.  Theory of Functionals and of Integral and Integro-Differential Equations , 2005 .

[443]  Vicente J. Romero,et al.  Comparison of pure and "Latinized" centroidal Voronoi tessellation against various other statistical sampling methods , 2006, Reliab. Eng. Syst. Saf..

[444]  Caroline Sainvitu,et al.  Adaptive sampling strategies for non‐intrusive POD‐based surrogates , 2013 .

[445]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[446]  Chien-Yu Peng,et al.  On the Choice of Nugget in Kriging Modeling for Deterministic Computer Experiments , 2014 .

[447]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[448]  Ronald L. Wasserstein,et al.  Monte Carlo: Concepts, Algorithms, and Applications , 1997 .

[449]  F. Pukelsheim Optimal Design of Experiments , 1993 .

[450]  Peter Z. G. Qian,et al.  Bayesian Hierarchical Modeling for Integrating Low-Accuracy and High-Accuracy Experiments , 2008, Technometrics.

[451]  Stefan Görtz,et al.  Non-linear reduced order models for steady aerodynamics , 2010, ICCS.

[452]  Russell R. Barton,et al.  Metamodels for simulation input-output relations , 1992, WSC '92.

[453]  D. D. Kosambi Statistics in Function Space , 2016 .

[454]  Ying Xiong Using Predictive Models in Engineering Design: Metamodeling, Uncertainty Quantification, and Model Validation , 2008 .

[455]  B. Moore Principal component analysis in linear systems: Controllability, observability, and model reduction , 1981 .