Compressed Sensing Approaches for Polynomial Approximation of High-Dimensional Functions

In recent years, the use of sparse recovery techniques in the approximation of high-dimensional functions has garnered increasing interest. In this work we present a survey of recent progress in this emerging topic. Our main focus is on the computation of polynomial approximations of high-dimensional functions on d-dimensional hypercubes. We show that smooth, multivariate functions possess expansions in orthogonal polynomial bases that are not only approximately sparse but possess a particular type of structured sparsity defined by so-called lower sets. This structure can be exploited via the use of weighted l1 minimization techniques, and, as we demonstrate, doing so leads to sample complexity estimates that are at most logarithmically dependent on the dimension d. Hence the curse of dimensionality – the bane of high-dimensional approximation – is mitigated to a significant extent. We also discuss several practical issues, including unknown noise (due to truncation or numerical error), and highlight a number of open problems and challenges.

[1]  Houman Owhadi,et al.  A non-adapted sparse approximation of PDEs with stochastic inputs , 2010, J. Comput. Phys..

[2]  Holger Rauhut,et al.  Compressive sensing Petrov-Galerkin approximation of high-dimensional parametric operator equations , 2014, Math. Comput..

[3]  H. Rauhut.  Random Sampling of Sparse Trigonometric Polynomials , 2005, math/0512642.

[4]  Tao Zhou,et al.  On Sparse Interpolation and the Design of Deterministic Interpolation Points , 2013, SIAM J. Sci. Comput..

[5]  W. Sickel,et al.  Approximation of Mixed Order Sobolev Functions on the d-Torus: Asymptotics, Preasymptotics, and d-Dependence , 2013, 1312.6386.

[6]  Guannan Zhang,et al.  Analysis of quasi-optimal polynomial approximations for parameterized PDEs with deterministic and stochastic coefficients , 2015, Numerische Mathematik.

[7]  Fabio Nobile,et al.  Analysis of Discrete $$L^2$$L2 Projection on Polynomial Spaces with Random Evaluations , 2014, Found. Comput. Math..

[8]  G. Migliorati,et al.  Multivariate Markov-type and Nikolskii-type inequalities for polynomials associated with downward closed multi-index sets , 2015, J. Approx. Theory.

[9]  Tao Zhou,et al.  A Christoffel function weighted least squares algorithm for collocation approximations , 2014, Math. Comput..

[10]  Holger Rauhut,et al.  Sparse Legendre expansions via l1-minimization , 2012, J. Approx. Theory.

[11]  Pierre Weiss,et al.  An Analysis of Block Sampling Strategies in Compressed Sensing , 2013, IEEE Transactions on Information Theory.

[12]  Ben Adcock,et al.  Infinite-dimensional $\ell^1$ minimization and function approximation from pointwise data , 2015, 1503.02352.

[13]  Fabio Nobile,et al.  Analysis of discrete least squares on multivariate polynomial spaces with evaluations at low-discrepancy point sets , 2015, J. Complex..

[14]  S. Foucart.  Stability and robustness of ℓ1-minimizations with Weibull matrices and redundant dictionaries , 2014 .

[15]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[16]  G. Szegő.  Zeros of orthogonal polynomials , 1939 .

[17]  Tao Zhou,et al.  Stochastic collocation on unstructured multivariate meshes , 2015, 1501.05891.

[18]  A. Cohen,et al.  Discrete Least-Squares Approximations over Optimized Downward Closed Polynomial Spaces in Arbitrary Dimension , 2015, 1610.07315.

[19]  Nathan A. Baker,et al.  Enhancing sparsity of Hermite polynomial expansions by iterative rotations , 2015, J. Comput. Phys..

[20]  Hoang Tran,et al.  Polynomial approximation via compressed sensing of high-dimensional functions on lower sets , 2016, Math. Comput..

[21]  Alexey Chernov,et al.  New explicit-in-dimension estimates for the cardinality of high-dimensional hyperbolic crosses and approximation of functions having mixed smoothness , 2013, J. Complex..

[22]  H. Rauhut,et al.  Interpolation via weighted $l_1$ minimization , 2013, 1308.0759.

[23]  Fabio Nobile,et al.  An Anisotropic Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data , 2008, SIAM J. Numer. Anal..

[24]  E. M. Wright,et al.  Adaptive Control Processes: A Guided Tour , 1961, The Mathematical Gazette.

[25]  Simona Perotto,et al.  A theoretical study of COmpRessed SolvING for advection-diffusion-reaction problems , 2017, Math. Comput..

[26]  Xiu Yang,et al.  Reweighted ℓ1ℓ1 minimization method for stochastic elliptic differential equations , 2013, J. Comput. Phys..

[27]  Max D. Gunzburger,et al.  Sparse Collocation Methods for Stochastic Interpolation and Quadrature , 2017 .

[28]  Fabio Nobile,et al.  A Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data , 2008, SIAM J. Numer. Anal..

[29]  R. DeVore,et al.  ANALYTIC REGULARITY AND POLYNOMIAL APPROXIMATION OF PARAMETRIC AND STOCHASTIC ELLIPTIC PDE'S , 2011 .

[30]  Emmanuel J. Candès,et al.  A Probabilistic and RIPless Theory of Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[31]  Omar M. Knio,et al.  Spectral Methods for Uncertainty Quantification , 2010 .

[32]  Gary Tang,et al.  Subsampled Gauss Quadrature Nodes for Estimating Polynomial Chaos Expansions , 2014, SIAM/ASA J. Uncertain. Quantification.

[33]  Fabio Nobile,et al.  Computers and Mathematics with Applications Convergence of Quasi-optimal Stochastic Galerkin Methods for a Class of Pdes with Random Coefficients , 2022 .

[34]  Ben Adcock,et al.  Correcting for unknown errors in sparse high-dimensional function approximation , 2019, Numerische Mathematik.

[35]  Albert Cohen,et al.  Sparse adaptive Taylor approximation algorithms for parametric and stochastic elliptic PDEs , 2011 .

[36]  Holger Rauhut,et al.  Multi-level Compressed Sensing Petrov-Galerkin discretization of high-dimensional parametric PDEs , 2017, 1701.01671.

[37]  Khachik Sargsyan,et al.  Enhancing ℓ1-minimization estimates of polynomial chaos expansions using basis selection , 2014, J. Comput. Phys..

[38]  Kyle A. Gallivan,et al.  A compressed sensing approach for partial differential equations with random input data , 2012 .

[39]  Albert Cohen,et al.  Breaking the curse of dimensionality in sparse polynomial approximation of parametric PDEs , 2015 .

[40]  Christoph Schwab,et al.  Regularity and generalized polynomial chaos approximation of parametric and random 2nd order hyperbolic partial differential equations , 2011 .

[41]  Albert Cohen,et al.  Approximation of high-dimensional parametric PDEs * , 2015, Acta Numerica.

[42]  Akil C. Narayan,et al.  A Generalized Sampling and Preconditioning Scheme for Sparse Approximation of Polynomial Chaos Expansions , 2016, SIAM J. Sci. Comput..

[43]  P. Wojtaszczyk,et al.  Stability and Instance Optimality for Gaussian Measurements in Compressed Sensing , 2010, Found. Comput. Math..

[44]  Clayton G. Webster,et al.  A Dynamically Adaptive Sparse Grid Method for Quasi-Optimal Interpolation of Multidimensional Analytic Functions , 2015, 1508.01125.

[45]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[46]  Michael P. Friedlander,et al.  Probing the Pareto Frontier for Basis Pursuit Solutions , 2008, SIAM J. Sci. Comput..

[47]  慧 廣瀬.  A Mathematical Introduction to Compressive Sensing , 2015 .

[48]  Albert Cohen,et al.  High-Dimensional Adaptive Sparse Polynomial Interpolation and Applications to Parametric PDEs , 2013, Foundations of Computational Mathematics.

[49]  Albert Cohen,et al.  On the Stability and Accuracy of Least Squares Approximations , 2011, Foundations of Computational Mathematics.

[50]  Clayton G. Webster.  Sparse grid stochastic collocation techniques for the numerical solution of partial differential equations with random input data , 2007 .

[51]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[52]  Yonina C. Eldar,et al.  Structured Compressed Sensing: From Theory to Applications , 2011, IEEE Transactions on Signal Processing.

[53]  Alireza Doostan,et al.  Compressive sampling of polynomial chaos expansions: Convergence analysis and sampling strategies , 2014, J. Comput. Phys..

[54]  Albert Cohen,et al.  Convergence Rates of Best N-term Galerkin Approximations for a Class of Elliptic sPDEs , 2010, Found. Comput. Math..

[55]  Christoph Schwab,et al.  REGULARITY AND GENERALIZED POLYNOMIAL CHAOS APPROXIMATION OF PARAMETRIC AND RANDOM SECOND-ORDER HYPERBOLIC PARTIAL DIFFERENTIAL EQUATIONS , 2012 .

[56]  Alireza Doostan,et al.  A weighted l1-minimization approach for sparse polynomial chaos expansions , 2013, J. Comput. Phys..

[57]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[58]  Giovanni Migliorati,et al.  Polynomial approximation by means of the random discrete L2 projection and application to inverse problems for PDEs with stochastic data , 2013 .

[59]  Yonina C. Eldar,et al.  Introduction to Compressed Sensing , 2022 .

[60]  Rémi Gribonval,et al.  Stable recovery of low-dimensional cones in Hilbert spaces: One RIP to rule them all , 2015, Applied and Computational Harmonic Analysis.

[61]  D. Xiu,et al.  STOCHASTIC COLLOCATION ALGORITHMS USING 𝓁 1 -MINIMIZATION , 2012 .

[62]  Ben Adcock,et al.  Robustness to Unknown Error in Sparse Regularization , 2017, IEEE Transactions on Information Theory.

[63]  R. DeVore,et al.  Analytic regularity and polynomial approximation of parametric and stochastic elliptic PDEs , 2010 .

[64]  Yuhang Chen,et al.  Stochastic collocation methods via $L_1$ minimization using randomized quadratures , 2016, 1602.00995.

[65]  Alireza Doostan,et al.  On polynomial chaos expansion via gradient-enhanced ℓ1-minimization , 2015, J. Comput. Phys..

[66]  David Gross,et al.  Recovering Low-Rank Matrices From Few Coefficients in Any Basis , 2009, IEEE Transactions on Information Theory.

[67]  Guannan Zhang,et al.  Stochastic finite element methods for partial differential equations with random input data* , 2014, Acta Numerica.

[68]  Ben Adcock,et al.  Infinite-Dimensional Compressed Sensing and Function Interpolation , 2015, Foundations of Computational Mathematics.

[69]  Ben Adcock,et al.  Generalized Sampling and Infinite-Dimensional Compressed Sensing , 2016, Found. Comput. Math..

[70]  Alireza Doostan,et al.  Coherence motivated sampling and convergence analysis of least squares polynomial Chaos regression , 2014, 1410.1931.

[71]  Ben Adcock,et al.  Compressed Sensing and Parallel Acquisition , 2016, IEEE Transactions on Information Theory.

[72]  A. Cohen,et al.  Optimal weighted least-squares methods , 2016, 1608.00512.

[73]  Hans-Joachim Bungartz,et al.  Acta Numerica 2004: Sparse grids , 2004 .

[74]  Albert Cohen,et al.  Discrete least squares polynomial approximation with random evaluations − application to parametric and stochastic elliptic PDEs , 2015 .