Spectral convergence of probability densities

The computation of probability density functions (PDF) using approximate maps (surrogate models) is a building block in such diverse fields as forward uncertainty quantification (UQ), sampling algorithms, learning, and inverse problems. In these settings, the probability measure of interest is induced by an unknown map from a known probability space to the real line, i.e., the measure of interest is a pushforward of another known measure. In computation, we do not know the true map, but only an approximate one. In the field of UQ, the generalized Polynomial Chaos (gPC) method is widely popular and yields excellent approximations of the map and its moment. But can the pushforward PDF be approximated with spectral accuracy as well? In this paper, we prove the first results of this kind. We provide convergence rates for PDFs using colocation and Galerkin gPC methods in all dimensions, guaranteeing exponential rates for analytic maps. In one dimension, we provide more refined results with stronger convergence rates, as well as an alternative proof strategy based on optimal-transport techniques.

[1]  S. Walker,et al.  Wasserstein upper bounds of the total variation for smooth densities , 2020 .

[2]  T. G. Coleman,et al.  Numerical Integration , 2019, Numerical Methods for Engineering An introduction using MATLAB® and computational electromagnetics examples.

[3]  J. Lang,et al.  POD-Galerkin Modeling and Sparse-Grid Collocation for a Natural Convection Problem with Stochastic Boundary Conditions , 2014 .

[4]  高等学校計算数学学報編輯委員会編.  高等学校計算数学学報 = Numerical mathematics , 1979 .

[5]  P. Revesz.  Interpolation and Approximation , 2010 .

[6]  A. Ditkowski,et al.  Loss of phase and universality of stochastic interactions between laser beams. , 2017, Optics express.

[7]  V. Bogachev,et al.  Triangular transformations of measures , 2005 .

[8]  Erich Novak,et al.  High dimensional polynomial interpolation on sparse grids , 2000, Adv. Comput. Math..

[9]  Henry P. Wynn,et al.  Approximation of probability density functions for SPDEs using truncated series expansions. , 2018, 1810.01028.

[10]  Dustin Tran,et al.  Hierarchical Implicit Models and Likelihood-Free Variational Inference , 2017, NIPS.

[11]  Houman Owhadi,et al.  Handbook of Uncertainty Quantification , 2017 .

[12]  Gadi Fibich,et al.  Density Estimation in Uncertainty Propagation Problems Using a Surrogate Model , 2018, SIAM/ASA J. Uncertain. Quantification.

[13]  Haiyong Wang,et al.  How fast does the best polynomial approximation converge than Legendre projection? , 2020, ArXiv.

[14]  Eric Nalisnick,et al.  Normalizing Flows for Probabilistic Modeling and Inference , 2019, J. Mach. Learn. Res..

[15]  Tim Wildey,et al.  Convergence of Probability Densities Using Approximate Models for Forward and Inverse Problems in Uncertainty Quantification , 2018, SIAM J. Sci. Comput..

[16]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[17]  Simon Tavener,et al.  Nonparametric Density Estimation for Randomly Perturbed Elliptic Problems I: Computational Methods, A Posteriori Analysis, and Adaptive Error Control , 2009, SIAM J. Sci. Comput..

[18]  Roger G. Ghanem,et al.  Basis adaptation in homogeneous chaos spaces , 2014, J. Comput. Phys..

[19]  G. Burton.  Sobolev Spaces , 2013 .

[20]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[21]  A. Quarteroni,et al.  Approximation results for orthogonal polynomials in Sobolev spaces , 1982 .

[22]  D. Xiu.  Numerical Methods for Stochastic Computations: A Spectral Method Approach , 2010 .

[23]  Jan S. Hesthaven,et al.  Spectral Methods for Time-Dependent Problems: Fourier spectral methods , 2007 .

[24]  Dongbin Xiu,et al.  High-Order Collocation Methods for Differential Equations with Random Inputs , 2005, SIAM J. Sci. Comput..

[25]  Amir Sagiv,et al.  The Wasserstein distances between pushed-forward measures with applications to uncertainty quantification , 2019, Communications in Mathematical Sciences.

[26]  F. Santambrogio.  Optimal Transport for Applied Mathematicians: Calculus of Variations, PDEs, and Modeling , 2015 .

[27]  HAIYONG WANG,et al.  On the convergence rates of Legendre approximation , 2011, Math. Comput..

[28]  R. Tempone,et al.  Stochastic Spectral Galerkin and Collocation Methods for PDEs with Random Coefficients: A Numerical Comparison , 2011 .

[29]  Stergios B. Fotopoulos,et al.  All of Nonparametric Statistics , 2007, Technometrics.

[30]  Jan S. Hesthaven,et al.  Uncertainty analysis for the steady-state flows in a dual throat nozzle , 2005 .

[31]  E. Tabak,et al.  DENSITY ESTIMATION BY DUAL ASCENT OF THE LOG-LIKELIHOOD ∗ , 2010 .

[32]  G. Karniadakis,et al.  An adaptive multi-element generalized polynomial chaos method for stochastic differential equations , 2005 .

[33]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[34]  Rami Katz,et al.  On Spectral Approximations with Nonstandard Weight Functions and Their Implementations to Generalized Chaos Expansions , 2019, J. Sci. Comput..

[35]  S. S. Vallender.  Calculation of the Wasserstein Distance Between Probability Distributions on the Line , 1974 .

[36]  Baskar Ganapathysubramanian,et al.  Sparse grid collocation schemes for stochastic natural convection problems , 2007, J. Comput. Phys..

[37]  Filippo Santambrogio,et al.  Optimal Transport for Applied Mathematicians , 2015 .

[38]  H. Bungartz,et al.  Sparse grids , 2004, Acta Numerica.

[39]  R. Ghanem,et al.  Uncertainty propagation using Wiener-Haar expansions , 2004 .

[40]  Rene F. Swarttouw,et al.  Orthogonal polynomials , 2020, NIST Handbook of Mathematical Functions.

[41]  Michael Griebel,et al.  Optimized general sparse grid approximation spaces for operator equations , 2009, Math. Comput..

[42]  Houman Owhadi,et al.  On the Brittleness of Bayesian Inference , 2013, SIAM Rev..

[43]  Fabio Nobile,et al.  Uncertainty Quantification of geochemical and mechanical compaction in layered sedimentary basins , 2017, 1703.03845.

[44]  Youssef Marzouk,et al.  Sparse approximation of triangular transports on bounded domains , 2020, ArXiv.

[45]  M. Ablowitz,et al.  Interacting nonlinear wave envelopes and rogue wave formation in deep water , 2014, 1407.5077.

[46]  Larry Wasserman,et al.  All of Statistics: A Concise Course in Statistical Inference , 2004 .

[47]  A. Gaeta,et al.  Loss of polarization of elliptically polarized collapsing beams , 2019, Physical Review A.

[48]  Shakir Mohamed,et al.  Variational Inference with Normalizing Flows , 2015, ICML.

[49]  C. Villani.  Topics in Optimal Transportation , 2003 .

[50]  O. L. Maître,et al.  Asynchronous Time Integration for Polynomial Chaos Expansion of Uncertain Periodic Dynamics , 2010 .

[51]  A. O'Hagan,et al.  Polynomial Chaos : A Tutorial and Critique from a Statistician ’ s Perspective , 2013 .

[52]  Michael S. Eldred,et al.  Sparse Pseudospectral Approximation Method , 2011, 1109.2936.

[53]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[54]  T. N. Sriram.  Asymptotics in Statistics–Some Basic Concepts , 2002 .

[55]  G. M.,et al.  Partial Differential Equations I , 2023, Applied Mathematical Sciences.

[56]  G. Stefanou.  The stochastic finite element method: Past, present and future , 2009 .

[57]  Chiara Piazzola,et al.  Uncertainty Quantification of Ship Resistance via Multi-Index Stochastic Collocation and Radial Basis Function Surrogates: A Comparison , 2020, AIAA AVIATION 2020 FORUM.