Variance-based adaptive sequential sampling for Polynomial Chaos Expansion

Abstract This paper presents a novel adaptive sequential sampling method for building Polynomial Chaos Expansion surrogate models . The technique enables one-by-one extension of an experimental design while trying to obtain an optimal sample at each stage of the adaptive sequential surrogate model construction process. The proposed sequential sampling strategy selects from a pool of candidate points by trying to cover the design domain proportionally to their local variance contribution. The proposed criterion for the sample selection balances both exploitation of the surrogate model and exploration of the design domain. The adaptive sequential sampling technique can be used in tandem with any user-defined sampling method, and here was coupled with commonly used Latin Hypercube Sampling and advanced Coherence D-optimal sampling in order to present its general performance. The obtained numerical results confirm its superiority over standard non-sequential approaches in terms of surrogate model accuracy and estimation of the output variance.

[1]  Stefano Marelli,et al.  Sequential Design of Experiment for Sparse Polynomial Chaos Expansions , 2017, SIAM/ASA J. Uncertain. Quantification.

[2]  Charles Tong,et al.  Refinement strategies for stratified sampling methods , 2006, Reliab. Eng. Syst. Saf..

[3]  S. Marelli,et al.  An active-learning algorithm that combines sparse polynomial chaos expansions and bootstrap for structural reliability analysis , 2017, Structural Safety.

[4]  I. Sobol.  Uniformly distributed sequences with an additional uniform property , 1976 .

[5]  Hans Petter Langtangen,et al.  Chaospy: An open source tool for designing methods of uncertainty quantification , 2015, J. Comput. Sci..

[6]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[7]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[8]  Jan Eliáš,et al.  Modification of the Maximin and ϕp (Phi) Criteria to Achieve Statistically Uniform Distribution of Sampling Points , 2020, Technometrics.

[9]  J. Halton.  On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals , 1960 .

[10]  Michael D. Shields,et al.  REFINED LATINIZED STRATIFIED SAMPLING: A ROBUST SEQUENTIAL SAMPLE SIZE EXTENSION METHODOLOGY FOR HIGH-DIMENSIONAL LATIN HYPERCUBE AND STRATIFIED DESIGNS , 2016 .

[11]  A. Owen.  Controlling correlations in latin hypercube samples , 1994 .

[12]  Miroslav Vořechovský,et al.  Distance-based optimal sampling in a hypercube: Analogies to N-body systems , 2019, Adv. Eng. Softw..

[13]  Gregery T. Buzzard,et al.  Global sensitivity analysis using sparse grid interpolation and polynomial chaos , 2012, Reliab. Eng. Syst. Saf..

[14]  T. J. Mitchell,et al.  Exploratory designs for computational experiments , 1995 .

[15]  R. S. Anderssen,et al.  Concerning $\int_0^1 \cdots \int_0^1 {(x_1^2 + \cdots + x_k^2 )} ^{{1 / 2}} dx_1 \cdots ,dx_k $ and a Taylor Series Method , 1976 .

[16]  Xufeng Yang,et al.  Active learning method combining Kriging model and multimodal‐optimization‐based importance sampling for the estimation of small failure probability , 2020, International Journal for Numerical Methods in Engineering.

[17]  Elsie S. Valeroso,et al.  Comparison of Design Optimality Criteria of Reduced Models for Response Surface Designs in the Hypercube , 2001, Technometrics.

[18]  Alireza Doostan,et al.  Basis adaptive sample efficient polynomial chaos (BASE-PC) , 2017, J. Comput. Phys..

[19]  Giovanni Migliorati,et al.  Adaptive Approximation by Optimal Weighted Least-Squares Methods , 2018, SIAM J. Numer. Anal..

[20]  M. E. Johnson,et al.  Minimax and maximin distance designs , 1990 .

[21]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[22]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[23]  R. Grandhi,et al.  Polynomial Chaos Expansion with Latin Hypercube Sampling for Estimating Response Variability , 2003 .

[24]  Akil C. Narayan,et al.  A Generalized Sampling and Preconditioning Scheme for Sparse Approximation of Polynomial Chaos Expansions , 2016, SIAM J. Sci. Comput..

[25]  Christiane Lemieux,et al.  Generalized Halton sequences in 2008: A comparative study , 2009, TOMC.

[26]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[27]  Kyriakos C. Giannakoglou,et al.  A painless intrusive polynomial chaos method with RANS-based applications , 2019, Computer Methods in Applied Mechanics and Engineering.

[28]  Arthur Flexer,et al.  Choosing ℓp norms in high-dimensional spaces based on hub analysis , 2015, Neurocomputing.

[29]  Zhenzhou Lu,et al.  An efficient and robust adaptive sampling method for polynomial chaos expansion in sparse Bayesian learning framework , 2019, Computer Methods in Applied Mechanics and Engineering.

[30]  Paul Diaz,et al.  Sparse polynomial chaos expansions via compressed sensing and D-optimal design , 2017, Computer Methods in Applied Mechanics and Engineering.

[31]  Alan J. Miller,et al.  A Fedorov Exchange Algorithm for D-optimal Design , 1994 .

[32]  Zeping Wu,et al.  Efficient space-filling and near-orthogonality sequential Latin hypercube for computer experiments , 2017 .

[33]  Beibei Sun,et al.  An active learning reliability method with multiple kernel functions based on radial basis function , 2019, Structural and Multidisciplinary Optimization.

[34]  T. Ishigami,et al.  An importance quantification technique in uncertainty analysis for computer models , 1990, [1990] Proceedings. First International Symposium on Uncertainty Modeling and Analysis.

[35]  Michael D. Shields,et al.  Adaptive Monte Carlo analysis for strongly nonlinear stochastic systems , 2018, Reliab. Eng. Syst. Saf..

[36]  Alireza Doostan,et al.  Coherence motivated sampling and convergence analysis of least squares polynomial Chaos regression , 2014, 1410.1931.

[37]  K. Fang,et al.  Number-theoretic methods in statistics , 1993 .

[38]  Jonathan Sadeghi,et al.  OpenCossan 2.0: an efficient computational toolbox for risk, reliability and resilience analysis , 2018 .

[39]  C. Soares,et al.  Spectral stochastic finite element analysis for laminated composite plates , 2008 .

[40]  M. Vořechovský,et al.  EVALUATION OF PAIRWISE DISTANCES AMONG POINTS FORMING A REGULAR ORTHOGONAL GRID IN A HYPERCUBE , 2018, Journal of Civil Engineering and Management.

[41]  Bruno Sudret,et al.  Adaptive sparse polynomial chaos expansion based on least angle regression , 2011, J. Comput. Phys..

[42]  A. Cohen,et al.  Optimal weighted least-squares methods , 2016, 1608.00512.

[43]  Lambros S. Katafygiotis,et al.  Geometric insight into the challenges of solving high-dimensional reliability problems , 2008 .

[44]  O. Dykstra.  The Augmentation of Experimental Data to Maximize [X′X] , 1971 .

[45]  Kai Cheng,et al.  Active learning polynomial chaos expansion for reliability analysis by maximizing expected indicator function prediction error , 2020, International Journal for Numerical Methods in Engineering.

[46]  Michael D. Shields,et al.  The generalization of Latin hypercube sampling , 2015, Reliab. Eng. Syst. Saf..

[47]  Stefano Marelli,et al.  UQLab: a framework for uncertainty quantification in MATLAB , 2014 .

[48]  Shu Tezuka,et al.  Uniform Random Numbers , 1995 .

[49]  Charu C. Aggarwal,et al.  On the Surprising Behavior of Distance Metrics in High Dimensional Spaces , 2001, ICDT.

[50]  Polynomial chaos expansions for dependent random variables , 2019, Computer Methods in Applied Mechanics and Engineering.

[51]  Jan Masek,et al.  Distance-based optimal sampling in a hypercube: Energy potentials for high-dimensional and low-saturation designs , 2020, Adv. Eng. Softw..

[52]  Miroslav Vorechovský,et al.  Hierarchical Refinement of Latin Hypercube Samples , 2015, Comput. Aided Civ. Infrastructure Eng..

[53]  H. Niederreiter.  Low-discrepancy and low-dispersion sequences , 1988 .

[54]  Alireza Doostan,et al.  Compressive sampling of polynomial chaos expansions: Convergence analysis and sampling strategies , 2014, J. Comput. Phys..

[55]  Olivier P. Le Maître,et al.  Polynomial chaos expansion for sensitivity analysis , 2009, Reliab. Eng. Syst. Saf..

[56]  Wanying Yun,et al.  Active sparse polynomial chaos expansion for system reliability analysis , 2020, Reliab. Eng. Syst. Saf..

[57]  G. Biau,et al.  High-Dimensional \(p\)-Norms , 2013, 1311.0587.

[58]  Michael D. Shields,et al.  UQpy: A general purpose Python package and development environment for uncertainty quantification , 2020, J. Comput. Sci..

[59]  M. Stein.  Large sample properties of simulations using latin hypercube sampling , 1987 .

[60]  Drahomír Novák,et al.  Polynomial chaos expansion for surrogate modelling: Theory and software , 2018, Beton- und Stahlbetonbau.

[61]  Stefano Marelli,et al.  Sparse Polynomial Chaos Expansions: Literature Survey and Benchmark , 2021, SIAM/ASA J. Uncertain. Quantification.

[62]  B. Sudret,et al.  An adaptive algorithm to build up sparse polynomial chaos expansions for stochastic finite element analysis , 2010 .

[63]  A. Owen.  A Central Limit Theorem for Latin Hypercube Sampling , 1992 .

[64]  I. Sobol.  On the distribution of points in a cube and the approximate evaluation of integrals , 1967 .

[65]  Bruno Sudret,et al.  Global sensitivity analysis using polynomial chaos expansions , 2008, Reliab. Eng. Syst. Saf..

[66]  Xin-She Yang,et al.  A literature survey of benchmark functions for global optimisation problems , 2013, Int. J. Math. Model. Numer. Optimisation.

[67]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[68]  Danny Lathouwers,et al.  Grid and basis adaptive polynomial chaos techniques for sensitivity and uncertainty analysis , 2014, J. Comput. Phys..

[69]  Olivier Roustant,et al.  Calculations of Sobol indices for the Gaussian process metamodel , 2008, Reliab. Eng. Syst. Saf..

[70]  H. Niederreiter.  Point sets and sequences with small discrepancy , 1987 .

[71]  L. Pronzato.  Minimax and maximin space-filling designs: some properties and methods for construction , 2017 .

[72]  A. Doostan,et al.  Least squares polynomial chaos expansion: A review of sampling strategies , 2017, 1706.07564.

[73]  Nicolas Gayton,et al.  AK-MCS: An active learning reliability method combining Kriging and Monte Carlo Simulation , 2011 .

[74]  R. Lebrun,et al.  An innovating analysis of the Nataf transformation from the copula viewpoint , 2009 .

[75]  Fred J. Hickernell,et al.  A generalized discrepancy and quadrature error bound , 1998, Math. Comput..

[76]  Adam Hapij,et al.  Refined Stratified Sampling for efficient Monte Carlo based uncertainty quantification , 2015, Reliab. Eng. Syst. Saf..

[77]  N. Wiener.  The Homogeneous Chaos , 1938 .

[78]  F. Pillichshammer,et al.  Digital Nets and Sequences: Discrepancy Theory and Quasi-Monte Carlo Integration , 2010 .

[79]  Miroslav Vorechovský,et al.  Periodic version of the minimax distance criterion for Monte Carlo integration , 2020, Adv. Eng. Softw..

[80]  Tao Zhou,et al.  A Christoffel function weighted least squares algorithm for collocation approximations , 2014, Math. Comput..

[81]  F. J. Hickernell.  Lattice rules: how well do they measure up? in random and quasi-random point sets , 1998 .

[82]  Sameer B. Mulani,et al.  Adaptive weighted least-squares polynomial chaos expansion with basis adaptivity and sequential adaptive sampling , 2020 .