A multi-fidelity surrogate model based on support vector regression

Computational simulations with different fidelities have been widely used in engineering design and optimization. A high-fidelity (HF) model is generally more accurate but also more time-consuming than the corresponding low-fidelity (LF) model. To take advantage of both HF and LF models, a number of multi-fidelity surrogate (MFS) models based on different surrogate models (e.g., Kriging, response surface, and radial basis function) have been developed, but MFS models based on support vector regression are rarely reported. In this paper, a new MFS model based on support vector regression, which is named Co_SVR, is developed. In the proposed method, the HF and LF samples are mapped into a high-dimensional feature space through a kernel function, and then, a linear model is utilized to evaluate the relationship between inputs and outputs. The root mean square error ( RMSE ) of HF responses of interest is used to express the training error of Co_SVR, and a heuristic algorithm, grey wolf optimizer, is used to obtain the optimal parameters. For verification, the Co_SVR model is compared with four popular multi-fidelity surrogate models and four single-fidelity surrogate models through a number of numerical cases and a pressure relief valve design problem. The results show that Co_SVR provides competitive performance in both numerical cases and the practical case. Moreover, the effects of key factors (i.e., the correlation between HF and LF models, the cost ratio of HF to LF models, and the combination of HF and LF samples) on the performance of Co_SVR are also explored.

[1]  Eric D. Heggestad,et al.  Erratum to: Polynomial Regression with Response Surface Analysis: A Powerful Approach for Examining Moderation and Overcoming Limitations of Difference Scores , 2010 .

[2]  Jianmei Song,et al.  Multi-fidelity uncertainty propagation using polynomial chaos and Gaussian process modeling , 2019, Structural and Multidisciplinary Optimization.

[3]  Maolin Shi,et al.  Multidisciplinary design optimization of dental implant based on finite element method and surrogate models , 2017 .

[4]  G. Matheron Principles of geostatistics , 1963 .

[5]  Slawomir Koziel,et al.  Multi-Fidelity Local Surrogate Model for Computationally Efficient Microwave Component Design Optimization , 2019, Sensors.

[6]  Jean-Antoine Désidéri,et al.  Multifidelity surrogate modeling based on radial basis functions , 2017 .

[7]  Haitao Liu,et al.  Cope with diverse data structures in multi-fidelity modeling: A Gaussian process method , 2018, Eng. Appl. Artif. Intell..

[8]  Hans-Martin Gutmann,et al.  A Radial Basis Function Method for Global Optimization , 2001, J. Glob. Optim..

[9]  Weihong Zhang,et al.  Extended Co-Kriging interpolation method based on multi-fidelity data , 2018, Appl. Math. Comput..

[10]  Wei Sun,et al.  A radial basis function-based multi-fidelity surrogate model: exploring correlation between high-fidelity and low-fidelity models , 2019, Structural and Multidisciplinary Optimization.

[11]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[12]  Ralf Zimmermann,et al.  Theoretical investigations of the new Cokriging method for variable-fidelity surrogate modeling , 2018, Adv. Comput. Math..

[13]  Raphael T. Haftka,et al.  Remarks on multi-fidelity surrogates , 2016, Structural and Multidisciplinary Optimization.

[14]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[15]  Philip S. Beran,et al.  Multi-Fidelity Sparse Polynomial Chaos Surrogate Models for Flutter Database Generation , 2019, AIAA Scitech 2019 Forum.

[16]  Yang Yang,et al.  An Improved Co-Kriging Multi-fidelity Surrogate Modeling Method for Non-nested Sampling Data , 2019, International Journal of Mechanical Engineering and Robotics Research.

[17]  Eric D. Heggestad,et al.  Polynomial Regression with Response Surface Analysis: A Powerful Approach for Examining Moderation and Overcoming Limitations of Difference Scores , 2010 .

[18]  Achille Messac,et al.  An adaptive hybrid surrogate model , 2012, Structural and Multidisciplinary Optimization.

[19]  Liping Wang,et al.  A Strategy for Adaptive Sampling of Multi-fidelity Gaussian Process to Reduce Predictive Uncertainty , 2019, Volume 2B: 45th Design Automation Conference.

[20]  Zhao Jing,et al.  A cooperative radial basis function method for variable-fidelity surrogate modeling , 2017 .

[21]  David J. J. Toal,et al.  Some considerations regarding the use of multi-fidelity Kriging in the construction of surrogate models , 2015 .

[22]  Zhenzhou Lu,et al.  Multi-level multi-fidelity sparse polynomial chaos expansion based on Gaussian process regression , 2019, Computer Methods in Applied Mechanics and Engineering.

[23]  Liang Gao,et al.  Metamodeling for high dimensional design problems by multi-fidelity simulations , 2017 .

[24]  Yan Wang,et al.  A sequential multi-fidelity metamodeling approach for data regression , 2017, Knowl. Based Syst..

[25]  Sancho Salcedo-Sanz,et al.  Efficient aerodynamic design through evolutionary programming and support vector regression algorithms , 2012, Expert Syst. Appl..

[26]  Stefan Görtz,et al.  Alternative Cokriging Method for Variable-Fidelity Surrogate Modeling , 2012 .

[27]  Markus Olhofer,et al.  Multi-fidelity surrogate model approach to optimization , 2018, GECCO.

[28]  Jacek M. Zurada,et al.  Extraction of rules from artificial neural networks for nonlinear regression , 2002, IEEE Trans. Neural Networks.

[29]  R. Haftka,et al.  Review of multi-fidelity models , 2016, Advances in Computational Science and Engineering.

[30]  Slawomir Koziel,et al.  Multi-fidelity design optimization of transonic airfoils using physics-based surrogate modeling and shape-preserving response prediction , 2010, J. Comput. Sci..

[31]  Hui Zhou,et al.  An adaptive global variable fidelity metamodeling strategy using a support vector regression based scaling function , 2015, Simul. Model. Pract. Theory.

[32]  R. Haftka,et al.  Multifidelity Surrogate Based on Single Linear Regression , 2017, AIAA Journal.

[33]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[34]  Changqing Cheng,et al.  Multi-fidelity modeling in sequential design for stability identification in dynamic time-delay systems. , 2019, Chaos.

[35]  Alexander I. J. Forrester,et al.  Multi-fidelity optimization via surrogate modelling , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[36]  Liang Yan,et al.  Adaptive multi-fidelity polynomial chaos approach to Bayesian inference in inverse problems , 2018, J. Comput. Phys..

[37]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[38]  S. Koziel,et al.  Computationally Efficient Multi-Fidelity Bayesian Support Vector Regression Modeling of Planar Antenna Input Characteristics , 2013, IEEE Transactions on Antennas and Propagation.

[39]  A. O'Hagan,et al.  Predicting the output from a complex computer code when fast approximations are available , 2000 .

[40]  Nathalie Bartoli,et al.  Linear regression-based multifidelity surrogate for disturbance amplification in multiphase explosion , 2019, Structural and Multidisciplinary Optimization.

[41]  Hao Zhang,et al.  When Doesn't Cokriging Outperform Kriging? , 2015, 1507.08403.

[42]  Reinhard Radermacher,et al.  Cross-validation based single response adaptive design of experiments for Kriging metamodeling of deterministic computer simulations , 2013 .

[43]  Sondipon Adhikari,et al.  A Surrogate Based Multi-fidelity Approach for Robust Design Optimization , 2017 .

[44]  Xin Yang,et al.  Research on a Multi-Fidelity Surrogate Model Based Model Updating Strategy , 2018, Volume 13: Design, Reliability, Safety, and Risk.

[45]  Gang Sun,et al.  Application of deep learning based multi-fidelity surrogate model to robust aerodynamic design optimization , 2019, Aerospace Science and Technology.

[46]  P Perdikaris,et al.  Multi-fidelity modelling via recursive co-kriging and Gaussian–Markov random fields , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[47]  Daniele Venturi,et al.  Multi-fidelity Gaussian process regression for prediction of random fields , 2017, J. Comput. Phys..

[48]  Qingfu Zhang,et al.  A multi-fidelity surrogate-model-assisted evolutionary algorithm for computationally expensive optimization problems , 2016, J. Comput. Sci..

[49]  Liang Gao,et al.  Difference mapping method using least square support vector regression for variable-fidelity metamodelling , 2015 .