An enhanced Kriging surrogate modeling technique for high-dimensional problems

Abstract Surrogate modeling techniques are widely used to simulate the behavior of manufactured and engineering systems. The construction of such surrogate models may become intractable in cases when input spaces have high dimensions, because the large number of model responses is typically required to estimate model parameters. In this paper, we proposed a new Kriging modeling technique combined with dimension reduction method to address the issue. In the proposed method, the sliced inverse regression technique is utilized to achieve a dimension reduction by constructing a new projection vector which reduces the dimension of the original input vector without losing the essential information of the model response quantify of interest. In the dimension reduction subspace, a new correlation function of Kriging is constructed by means of the tensor product of several correlation functions with respect to each projection direction. The proposed method is especially promising for high-dimensional problems. In examples including finite element model (FEM) pertinent to low cycle fatigue life (LCF) of a aero-engine compressor disc, the enhanced Kriging is found to outperform several well-established surrogate models when small sample sizes are used.

[1]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[2]  Bruno Sudret,et al.  Global sensitivity analysis using low-rank tensor approximations , 2016, Reliab. Eng. Syst. Saf..

[3]  P. Hall.  On Projection Pursuit Regression , 1989 .

[4]  Nicolas Gayton,et al.  AK-MCS: An active learning reliability method combining Kriging and Monte Carlo Simulation , 2011 .

[5]  Dianne Cook,et al.  A projection pursuit index for large p small n data , 2010, Stat. Comput..

[6]  Yves Deville,et al.  DiceKriging, DiceOptim: Two R Packages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and Optimization , 2012 .

[7]  Paul G. Constantine,et al.  Global sensitivity metrics from active subspaces , 2015, Reliab. Eng. Syst. Saf..

[8]  Chun-Houh Chen,et al.  CAN SIR BE AS POPULAR AS MULTIPLE LINEAR REGRESSION , 2003 .

[9]  Ilias Bilionis,et al.  Deep UQ: Learning deep neural network surrogate models for high dimensional uncertainty quantification , 2018, J. Comput. Phys..

[10]  Edward J. Haug,et al.  Design Sensitivity Analysis of Structural Systems , 1986 .

[11]  I. Jolliffe.  Principal Component Analysis , 2002 .

[12]  Qiujing Pan,et al.  An efficient reliability method combining adaptive Support Vector Machine and Monte Carlo Simulation , 2017 .

[13]  Lu Wang,et al.  Modified GMDH-NN algorithm and its application for global sensitivity analysis , 2017, J. Comput. Phys..

[14]  Sébastien Da Veiga,et al.  Global sensitivity analysis with dependence measures , 2013, ArXiv.

[15]  Jun Li,et al.  Using polynomial chaos expansion for uncertainty and sensitivity analysis of bridge structures , 2019, Mechanical Systems and Signal Processing.

[16]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[17]  Qiujing Pan,et al.  Sliced inverse regression-based sparse polynomial chaos expansions for reliability analysis in high dimensions , 2017, Reliab. Eng. Syst. Saf..

[18]  Max D. Morris,et al.  Sampling plans based on balanced incomplete block designs for evaluating the importance of computer model inputs , 2006 .

[19]  Bing Li,et al.  A general theory for nonlinear sufficient dimension reduction: Formulation and estimation , 2013, 1304.0580.

[20]  Nilay Shah,et al.  The identification of model effective dimensions using global sensitivity analysis , 2011, Reliab. Eng. Syst. Saf..

[21]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[22]  Ning Li,et al.  Gaussian process regression for tool wear prediction , 2018 .

[23]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[24]  Bruno Sudret,et al.  Adaptive sparse polynomial chaos expansion based on least angle regression , 2011, J. Comput. Phys..

[25]  Z. Zhong,et al.  Application of mixed kernels function (MKF) based support vector regression model (SVR) for CO2 - Reservoir oil minimum miscibility pressure prediction , 2016 .

[26]  Hiroshi Motoda,et al.  Feature Extraction, Construction and Selection: A Data Mining Perspective , 1998 .

[27]  Su-Yun Huang,et al.  Nonlinear Dimension Reduction with Kernel Sliced Inverse Regression , 2009, IEEE Transactions on Knowledge and Data Engineering.

[28]  Vince D. Calhoun,et al.  A projection pursuit algorithm to classify individuals using fMRI data: Application to schizophrenia , 2008, NeuroImage.

[29]  Paul Bannister,et al.  Uncertainty quantification of squeal instability via surrogate modelling , 2015 .

[30]  John W. Tukey,et al.  A Projection Pursuit Algorithm for Exploratory Data Analysis , 1974, IEEE Transactions on Computers.

[31]  Sriram Narasimhan,et al.  A Gaussian process latent force model for joint input-state estimation in linear structural systems , 2019, Mechanical Systems and Signal Processing.

[32]  Yanping Wang,et al.  A new efficient simulation method based on Bayes' theorem and importance sampling Markov chain simulation to estimate the failure-probability-based global sensitivity measure , 2018 .

[33]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[34]  B. Sudret,et al.  Reliability analysis of high-dimensional models using low-rank tensor approximations , 2016, 1606.08577.

[35]  Lexin Li,et al.  Sparse Sliced Inverse Regression , 2006, Technometrics.

[36]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[37]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[38]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[39]  Iason Papaioannou,et al.  PLS-based adaptation for efficient PCE representation in high dimensions , 2019, J. Comput. Phys..

[40]  Stefano Marelli,et al.  UQLab: a framework for uncertainty quantification in MATLAB , 2014 .

[41]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[42]  Bing Li,et al.  Inverse regression-based uncertainty quantification algorithms for high-dimensional models: Theory and practice , 2016, J. Comput. Phys..

[43]  Nilay Shah,et al.  Global sensitivity analysis using sparse high dimensional model representations generated by the group method of data handling , 2016, Math. Comput. Simul..

[44]  R. Rackwitz.  Reliability analysis—a review and some perspectives , 2001 .

[45]  Paul Diaz,et al.  Sparse polynomial chaos expansions via compressed sensing and D-optimal design , 2017, Computer Methods in Applied Mechanics and Engineering.

[46]  Joe Wiart,et al.  A new surrogate modeling technique combining Kriging and polynomial chaos expansions - Application to uncertainty analysis in computational dosimetry , 2015, J. Comput. Phys..

[47]  Henry P. Wynn,et al.  Screening, predicting, and computer experiments , 1992 .

[48]  B. Iooss,et al.  Derivative based global sensitivity measures , 2014, 1412.2619.

[49]  S. Sundararajan,et al.  Predictive Approaches for Choosing Hyperparameters in Gaussian Processes , 1999, Neural Computation.

[50]  J. Friedman.  Exploratory Projection Pursuit , 1987 .

[51]  Hermann G. Matthies,et al.  Probabilistic optimization of engineering system with prescribed target design in a reduced parameter space , 2018, Computer Methods in Applied Mechanics and Engineering.

[52]  R. Platz,et al.  Uncertainty quantification in the mathematical modelling of a suspension strut using Bayesian inference , 2019, Mechanical Systems and Signal Processing.

[53]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[54]  David A. Landgrebe,et al.  Hyperspectral data analysis and supervised feature reduction via projection pursuit , 1999, IEEE Trans. Geosci. Remote. Sens..

[55]  Zhongming Jiang,et al.  High dimensional structural reliability with dimension reduction , 2017 .

[56]  Agus Sudjianto,et al.  Blind Kriging: A New Method for Developing Metamodels , 2008 .

[57]  Yan Shi,et al.  An expanded sparse Bayesian learning method for polynomial chaos expansion , 2019, Mechanical Systems and Signal Processing.

[58]  Qiqi Wang,et al.  Erratum: Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces , 2013, SIAM J. Sci. Comput..

[59]  D. Massart,et al.  Sequential projection pursuit using genetic algorithms for data mining of analytical data. , 2000, Analytical chemistry.

[60]  Johan Larsson,et al.  Exploiting active subspaces to quantify uncertainty in the numerical simulation of the HyShot II scramjet , 2014, J. Comput. Phys..

[61]  Zhenzhou Lu,et al.  Variable importance analysis: A comprehensive review , 2015, Reliab. Eng. Syst. Saf..

[62]  Jinsheng Cai,et al.  Surrogate-based aerodynamic shape optimization with the active subspace method , 2018, Structural and Multidisciplinary Optimization.

[63]  Alain Berro,et al.  Genetic algorithms and particle swarm optimization for exploratory projection pursuit , 2010, Annals of Mathematics and Artificial Intelligence.

[64]  Andrea Saltelli,et al.  Sensitivity Analysis for Importance Assessment , 2002, Risk analysis : an official publication of the Society for Risk Analysis.

[65]  Louis B. Rall,et al.  Automatic Differentiation: Techniques and Applications , 1981, Lecture Notes in Computer Science.

[66]  Zhenzhou Lu,et al.  AK-SYSi: an improved adaptive Kriging model for system reliability analysis with multiple failure modes by a refined U learning function , 2018, Structural and Multidisciplinary Optimization.

[67]  J. Oakley.  Estimating percentiles of uncertain computer code outputs , 2004 .

[68]  Stefano Marelli,et al.  Data-driven polynomial chaos expansion for machine learning regression , 2018, J. Comput. Phys..

[69]  Ilias Bilionis,et al.  Gaussian processes with built-in dimensionality reduction: Applications in high-dimensional uncertainty propagation , 2016, 1602.04550.

[70]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[71]  Søren Nymand Lophaven,et al.  DACE - A Matlab Kriging Toolbox , 2002 .

[72]  Ker-Chau Li,et al.  On almost Linearity of Low Dimensional Projections from High Dimensional Data , 1993 .

[73]  Tong Zhou,et al.  An efficient reliability method combining adaptive global metamodel and probability density evolution method , 2019, Mechanical Systems and Signal Processing.

[74]  Chen Jiang,et al.  An active failure-pursuing Kriging modeling method for time-dependent reliability analysis , 2019, Mechanical Systems and Signal Processing.

[75]  Tom Dhaene,et al.  Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling , 2011, Eur. J. Oper. Res..