Approximation of High-Dimensional Rank One Tensors

Many real world problems are high-dimensional in that their solution is a function which depends on many variables or parameters. This presents a computational challenge since traditional numerical techniques are built on model classes for functions based solely on smoothness. It is known that the approximation of smoothness classes of functions suffers from the so-called ‘curse of dimensionality’. Avoiding this curse requires new model classes for real world functions that match applications. This has led to the introduction of notions such as sparsity, variable reduction, and reduced modeling. One theme that is particularly common is to assume a tensor structure for the target function. This paper investigates how well a rank one function f(x1,…,xd)=f1(x1)⋯fd(xd), defined on Ω=[0,1]d can be captured through point queries. It is shown that such a rank one function with component functions fj in $W^{r}_{\infty}([0,1])$ can be captured (in L∞) to accuracy O(C(d,r)N−r) from N well-chosen point evaluations. The constant C(d,r) scales like ddr. The queries in our algorithms have two ingredients, a set of points built on the results from discrepancy theory and a second adaptive set of queries dependent on the information drawn from the first set. Under the assumption that a point z∈Ω with nonvanishing f(z) is known, the accuracy improves to O(dN−r).

[1]  R. Schilling,et al.  Weak Order for the Discretization of the Stochastic Heat Equation Driven by Impulsive Noise , 2009, 0911.4681.

[2]  Wolfgang Dahmen,et al.  Super-Resolution Image Reconstruction by Nonlocal Means Applied to High-Angle Annular Darkfield Scanning Transmission Electron Microscopy (HAADF-STEM) , 2009 .

[3]  Stephan Dahlke,et al.  Adaptive wavelet methods and sparsity reconstruction for inverse heat conduction problems , 2010, Adv. Comput. Math..

[4]  Armin Iske,et al.  Curvature analysis of frequency modulated manifolds in dimensionality reduction , 2011 .

[5]  H. Bungartz,et al.  Sparse grids , 2004, Acta Numerica.

[6]  Armin Iske,et al.  Optimal representation of piecewise Hölder smooth bivariate functions by the Easy Path Wavelet Transform , 2013, J. Approx. Theory.

[7]  S. B. Stechkin Approximation of periodic functions , 1974 .

[8]  G. Teschke,et al.  A compressive Landweber iteration for solving ill-posed inverse problems , 2008 .

[9]  Christian Bender,et al.  Least-Squares Monte Carlo for Backward SDEs , 2012 .

[10]  Winfried Sickel,et al.  Best m-Term Approximation and Sobolev–Besov Spaces of Dominating Mixed Smoothness—the Case of Compact Embeddings , 2012 .

[11]  H. Yserentant,et al.  On the Complexity of the Electronic Schrödinger Equation , 2022 .

[12]  André Uschmajew,et al.  Local Convergence of the Alternating Least Squares Algorithm for Canonical Tensor Approximation , 2012, SIAM J. Matrix Anal. Appl..

[13]  M. Hansen,et al.  n-term approximation rates and Besov regularity for elliptic PDEs on polyhedral domains , 2012 .

[14]  Simen Kvaal,et al.  Multiconfigurational time-dependent Hartree method to describe particle loss due to absorbing boundary conditions , 2011, 1102.3899.

[15]  J. Hammersley MONTE CARLO METHODS FOR SOLVING MULTIVARIABLE PROBLEMS , 1960 .

[16]  Henryk Wozniakowski,et al.  The curse of dimensionality for the class of monotone functions and for the class of convex functions , 2011, J. Approx. Theory.

[17]  G. Plonka The Easy Path Wavelet Transform: A New Adaptive Wavelet Transform for Sparse Representation of Two-Dimensional Data , 2009 .

[18]  E. Ullmann,et al.  Preconditioning Stochastic Galerkin Saddle Point Problems , 2022 .

[19]  Petru A. Cioica,et al.  On the $L_q(L_p)$-regularity and Besov smoothness of stochastic parabolic equations on bounded Lipschitz domains , 2013, 1301.1180.

[20]  Dajana Conte,et al.  Mathematical Modelling and Numerical Analysis an Error Analysis of the Multi-configuration Time-dependent Hartree Method of Quantum Dynamics , 2022 .

[21]  Wolfgang Hackbusch,et al.  Tensorisation of vectors and their efficient convolution , 2011, Numerische Mathematik.

[22]  Wolfgang Hackbusch,et al.  An Introduction to Hierarchical (H-) Rank and TT-Rank of Tensors with Examples , 2011, Comput. Methods Appl. Math..

[23]  Hans-Jörg Starkloff,et al.  ON THE CONVERGENCE OF THE STOCHASTIC GALERKIN METHOD FOR RANDOM ELLIPTIC PARTIAL DIFFERENTIAL EQUATIONS , 2013 .

[24]  Erwan Faou,et al.  Computing Semiclassical Quantum Dynamics with Hagedorn Wavepackets , 2009, SIAM J. Sci. Comput..

[25]  Andreas Zeiser,et al.  Wavelet Approximation in Weighted Sobolev Spaces of Mixed Order with Applications to the Electronic Schrödinger Equation , 2012 .

[26]  Yonina C. Eldar,et al.  Introduction to Compressed Sensing , 2022 .

[27]  Henryk Wozniakowski,et al.  Approximation of infinitely differentiable multivariate functions is intractable , 2009, J. Complex..

[28]  Jianwei Ma,et al.  A Review of Curvelets and Recent Applications , 2009 .

[29]  G. Kutyniok,et al.  Construction of Compactly Supported Shearlet Frames , 2010, 1003.5481.

[30]  Armin Iske,et al.  Optimally Sparse Image Representation by the Easy Path Wavelet Transform , 2012, Int. J. Wavelets Multiresolution Inf. Process..

[31]  Wolfgang Dahmen,et al.  Classification algorithms using adaptive partitioning , 2014, 1411.0839.

[32]  Wang-Q Lim,et al.  Compactly Supported Shearlets , 2010, 1009.4359.

[33]  Denis Belomestny,et al.  Multilevel dual approach for pricing American style derivatives , 2012, Finance Stochastics.

[34]  K. Ritter,et al.  Variable Subspace Sampling and Multi-level Algorithms , 2009 .

[35]  Elisabeth Ullmann,et al.  Stochastic Galerkin Matrices , 2010, SIAM J. Matrix Anal. Appl..

[36]  Wang-Q Lim,et al.  Image Separation Using Shearlets , 2011 .

[37]  Wolfgang Dahmen,et al.  Convergence Rates for Greedy Algorithms in Reduced Basis Methods , 2010, SIAM J. Math. Anal..

[38]  J. Ballani,et al.  Black box approximation of tensors in hierarchical Tucker format , 2013 .

[39]  Winfried Sickel,et al.  Best m-term aproximation and tensor product of Sobolev and Besov spaces-the case of non-compact embeddings , 2010 .

[40]  W. Hackbusch,et al.  Black Box Low Tensor-Rank Approximation Using Fiber-Crosses , 2009 .

[41]  Wolfgang Dahmen,et al.  Adaptive Petrov-Galerkin methods for first order transport equations , 2011 .

[42]  E. Novak,et al.  Optimal Order of Convergence and (In)Tractability of Multivariate Approximation of Smooth Functions , 2009 .

[43]  Dirk A. Lorenz,et al.  Beyond convergence rates: exact recovery with the Tikhonov regularization with sparsity constraints , 2010, 1001.3276.

[44]  Steffen Dereich,et al.  Foundations of Computational Mathematics, Budapest 2011: On the Complexity of Computing Quadrature Formulas for SDEs , 2012 .

[45]  Howard C. Elman,et al.  Efficient Iterative Solvers for Stochastic Galerkin Discretizations of Log-Transformed Random Diffusion Problems , 2012, SIAM J. Sci. Comput..

[46]  Wolfgang Dahmen,et al.  Adaptivity and variational stabilization for convection-diffusion equations∗ , 2012 .

[47]  Christian Bender,et al.  Dual pricing of multi-exercise options under volume constraints , 2011, Finance Stochastics.

[48]  Harry Yserentant,et al.  The mixed regularity of electronic wave functions in fractional order and weighted Sobolev spaces , 2012, Numerische Mathematik.

[49]  K. Bredies,et al.  Regularization with non-convex separable constraints , 2009 .

[50]  Steffen Dereich,et al.  A multilevel Monte Carlo algorithm for Lévy-driven stochastic differential equations , 2011 .

[51]  G. Kutyniok,et al.  Irregular Shearlet Frames: Geometry and Approximation Properties , 2010, 1002.2657.

[52]  R. Schneider,et al.  The Alternating Linear Scheme for Tensor Optimisation in the TT Format , 2022 .

[53]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[54]  Michael Griebel,et al.  An Adaptive Sparse Grid Semi-Lagrangian Scheme for First Order Hamilton-Jacobi Bellman Equations , 2012, Journal of Scientific Computing.

[55]  Torsten Görner,et al.  Efficient and accurate computation of spherical mean values at scattered center points , 2012 .

[56]  V. N. Temli︠a︡kov Approximation of periodic functions , 1993 .

[57]  Reinhold Schneider,et al.  Dynamical Approximation by Hierarchical Tucker and Tensor-Train Tensors , 2013, SIAM J. Matrix Anal. Appl..

[58]  Christian Bender,et al.  Primal and Dual Pricing of Multiple Exercise Options in Continuous Time , 2011, SIAM J. Financial Math..

[59]  Harry Yserentant,et al.  The mixed regularity of electronic wave functions multiplied by explicit correlation factors , 2011 .

[60]  Reinhold Schneider,et al.  Error estimates for the Coupled Cluster method , 2013 .

[61]  E. Novak,et al.  On the power of function values for the approximation problem in various settings , 2010, 1011.3682.

[62]  E. Novak,et al.  Tractability of Multivariate Problems , 2008 .

[63]  D. Rudolf,et al.  Explicit error bounds for Markov chain Monte Carlo , 2011, 1108.3201.

[64]  Christian Bender,et al.  Error Criteria for Numerical Solutions of Backward SDEs , 2010 .

[65]  Lars Grasedyck,et al.  Polynomial Approximation in Hierarchical Tucker Format by Vector – Tensorization , 2010 .

[66]  Winfried Sickel,et al.  Tensor products of Sobolev-Besov spaces and applications to approximation from the hyperbolic cross , 2009, J. Approx. Theory.

[67]  Adrian Dumitrescu,et al.  On the Largest Empty Axis-Parallel Box Amidst n Points , 2009, Algorithmica.

[68]  S. Larsson,et al.  On Wavelet-Galerkin Methods for Semilinear Parabolic Equations with Additive Noise , 2012, 1208.0433.

[69]  Christian Bender,et al.  A Posteriori Estimates for Backward SDEs , 2013, SIAM/ASA J. Uncertain. Quantification.

[70]  Hermann G. Matthies,et al.  Efficient Analysis of High Dimensional Data in Tensor Formats , 2012 .

[71]  Michael Döhler,et al.  Nonequispaced Hyperbolic Cross Fast Fourier Transform , 2010, SIAM J. Numer. Anal..

[72]  Klaus Ritter,et al.  A Local Refinement Strategy for Constructive Quantization of Scalar SDEs , 2013, Found. Comput. Math..

[73]  J. Halton On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals , 1960 .

[74]  Henryk Wozniakowski,et al.  The curse of dimensionality for numerical integration of smooth functions , 2012, Math. Comput..

[75]  Markus Hansen,et al.  On tensor products of quasi-Banach spaces , 2010 .

[76]  Karsten Urban,et al.  On Wavelet-Galerkin Methods for Semilinear Parabolic Equations with Additive Noise , 2013 .

[77]  P. Maass,et al.  An analysis of electrical impedance tomography with applications to Tikhonov regularization , 2012 .

[78]  H. Harbrecht,et al.  On the low-rank approximation by the pivoted Cholesky decomposition , 2012 .

[79]  Thorsten Rohwedder,et al.  The continuous Coupled Cluster formulation for the electronic Schrödinger equation , 2013 .

[80]  Wolfgang Dahmen,et al.  Compressed Sensing and Electron Microscopy , 2010 .

[81]  A. Uschmajew,et al.  LOCAL CONVERGENCE OF ALTERNATING SCHEMES FOR OPTIMIZATION OF CONVEX PROBLEMS IN THE TT FORMAT , 2011 .

[82]  Wolfgang Dahmen,et al.  Fast high-dimensional approximation with sparse occupancy trees , 2011, J. Comput. Appl. Math..

[83]  Karsten Urban,et al.  Adaptive Wavelet Methods on Unbounded Domains , 2012, Journal of Scientific Computing.

[84]  Lutz Kämmerer Reconstructing hyperbolic cross trigonometric polynomials by sampling along generated sets , 2012 .

[85]  Steffen Dereich,et al.  Multilevel Monte Carlo algorithms for L\'{e}vy-driven SDEs with Gaussian correction , 2011, 1101.1369.

[86]  Erich Novak,et al.  Optimal approximation of elliptic problems by linear and nonlinear mappings IV: Errors in L2 and other norms , 2004, J. Complex..

[87]  Reinhold Schneider,et al.  Optimization problems in contracted tensor networks , 2011, Comput. Vis. Sci..

[88]  O. Ernst,et al.  ON THE CONVERGENCE OF GENERALIZED POLYNOMIAL CHAOS EXPANSIONS , 2011 .

[89]  Randolph E. Bank,et al.  On the $${H^1}$$H1-stability of the $${L_2}$$L2-projection onto finite element spaces , 2014, Numerische Mathematik.

[90]  Reinhold Schneider,et al.  Numerical analysis of augmented plane wave methods for full-potential electronic structure calculations , 2015 .

[91]  Erich Novak,et al.  Tractability of the Approximation of High-Dimensional Rank One Tensors , 2014, Constructive Approximation.

[92]  P. Maass,et al.  Function spaces and optimal currents in impedance tomography , 2011 .

[93]  G. Teschke,et al.  Accelerated projected steepest descent method for nonlinear inverse problems with sparsity constraints , 2010 .

[94]  Reinhold Schneider,et al.  On manifolds of tensors of fixed TT-rank , 2012, Numerische Mathematik.

[95]  Lutz Kämmerer,et al.  Interpolation lattices for hyperbolic cross trigonometric polynomials , 2012, J. Complex..

[96]  Stephan Dahlke,et al.  An adaptive wavelet method for parameter identification problems in parabolic partial differential equations , 2022 .

[97]  Lars Grasedyck,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig a Projection Method to Solve Linear Systems in Tensor Format a Projection Method to Solve Linear Systems in Tensor Format , 2022 .

[98]  Gabriele Steidl,et al.  Shearlet Coorbit Spaces: Compactly Supported Analyzing Shearlets, Traces and Embeddings , 2011 .

[99]  Reinhold Schneider,et al.  Numerical analysis of Gaussian approximations in quantum chemistry , 2012 .

[100]  Wang-Q Lim,et al.  Shearlets on Bounded Domains , 2010, 1007.3039.

[101]  G. Teschke,et al.  Inversion of the noisy Radon transform on SO(3) by Gabor frames and sparse recovery principles , 2011 .

[102]  Reinhold Schneider,et al.  An analysis for the DIIS acceleration method used in quantum chemistry calculations , 2011 .

[103]  Dirk A. Lorenz,et al.  Minimization of Non-smooth, Non-convex Functionals by Iterative Thresholding , 2015, J. Optim. Theory Appl..

[104]  Armin Iske,et al.  On Groupoid C∗-Algebras, Persistent Homology and Time-Frequency Analysis , 2011 .

[105]  Daniela Rosca,et al.  A New Hybrid Method for Image Approximation Using the Easy Path Wavelet Transform , 2011, IEEE Transactions on Image Processing.

[106]  Erich Novak,et al.  The Curse of Dimensionality for Monotone and Convex Functions of Many Variables , 2010, 1011.3680.

[107]  Gerlind Plonka-Hoch,et al.  Compressive Video Sampling With Approximate Message Passing Decoding , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[108]  Wang-Q Lim,et al.  Compactly supported shearlets are optimally sparse , 2010, J. Approx. Theory.

[109]  K. Ritter,et al.  Adaptive Wavelet Methods for Elliptic Stochastic Partial Differential Equations , 2022 .

[110]  Winfried Sickel,et al.  Best m-term approximation and Lizorkin-Triebel spaces , 2011, J. Approx. Theory.

[111]  D. Crisan,et al.  Robust filtering: Correlated noise and multidimensional observation , 2012, 1201.1858.

[112]  Daniel Rudolf,et al.  Error bounds for computing the expectation by Markov chain Monte Carlo , 2009, Monte Carlo Methods Appl..

[113]  G. Rote,et al.  Quasi-Monte-Carlo methods and the dispersion of point sequences , 1996 .

[114]  Tobias Jahnke,et al.  Error Bound for Piecewise Deterministic Processes Modeling Stochastic Reaction Systems , 2012, Multiscale Model. Simul..

[115]  Armin Iske,et al.  Adaptive ADER Methods Using Kernel-Based Polyharmonic Spline WENO Reconstruction , 2010, SIAM J. Sci. Comput..

[116]  M. Fornasier,et al.  Multilevel Preconditioning for Adaptive Sparse Optimization , 2009 .

[117]  Klaus Ritter,et al.  Spatial Besov Regularity for Stochastic Partial Differential Equations on Lipschitz Domains , 2010, 1011.1814.

[118]  Hans-Joachim Bungartz,et al.  Acta Numerica 2004: Sparse grids , 2004 .

[119]  Lutz Kämmerer,et al.  On the stability of the hyperbolic cross discrete Fourier transform , 2011, Numerische Mathematik.

[120]  Wolfgang Dahmen,et al.  DOUBLE GREEDY ALGORITHMS: REDUCED BASIS METHODS FOR TRANSPORT DOMINATED PROBLEMS ∗ , 2013, 1302.5072.

[121]  K. Ritter,et al.  On the convergence analysis of Rothe ’ s method , 2022 .

[122]  Fred J. Hickernell,et al.  Multi-level Monte Carlo algorithms for infinite-dimensional integration on RN , 2010, J. Complex..

[123]  Henryk Wozniakowski,et al.  Discontinuous information in the worst case and randomized settings , 2011, 1106.2945.

[124]  Konstantin Grella,et al.  Sparse tensor spherical harmonics approximation in radiative transfer , 2011, J. Comput. Phys..

[125]  Gerlind Plonka-Hoch,et al.  Curvelet-Wavelet Regularized Split Bregman Iteration for Compressed Sensing , 2011, Int. J. Wavelets Multiresolution Inf. Process..

[126]  G. Teschke,et al.  Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems , 2010 .

[127]  Sadegh Jokar,et al.  Sparse recovery and Kronecker products , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[128]  W. Hackbusch Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[129]  Henryk Wozniakowski,et al.  The curse of dimensionality for numerical integration of smooth functions II , 2012, J. Complex..

[130]  Stephan Dahlke,et al.  Spatial Besov regularity for semilinear stochastic partial differential equations on bounded Lipschitz domains , 2012, Int. J. Comput. Math..

[131]  Klaus Ritter,et al.  Derandomization of the Euler scheme for scalar stochastic differential equations , 2012, J. Complex..

[132]  Hermann G. Matthies,et al.  Efficient low-rank approximation of the stochastic Galerkin matrix in tensor formats , 2014, Comput. Math. Appl..

[133]  Tobias Jahnke,et al.  On Reduced Models for the Chemical Master Equation , 2011, Multiscale Model. Simul..

[134]  Michael Gnewuch,et al.  On weighted Hilbert spaces and integration of functions of infinitely many variables , 2014, J. Complex..

[135]  Stephan Dahlke,et al.  A note on quarkonial systems and multilevel partition of unity methods , 2013 .

[136]  Tobias Jahnke,et al.  Solving chemical master equations by adaptive wavelet compression , 2010, J. Comput. Phys..

[137]  S. Dereich,et al.  Constructive quantization: Approximation by empirical measures , 2011, 1108.5346.

[138]  Stephan Dahlke,et al.  Piecewise tensor product wavelet bases by extensions and approximation rates , 2013, Math. Comput..

[139]  D. Lorenz,et al.  Greedy solution of ill-posed problems: error bounds and exact inversion , 2009, 0904.0154.

[140]  Winfried Sickel,et al.  On Besov regularity of solutions to nonlinear elliptic partial differential equations , 2020, Nonlinear Analysis.

[141]  Fred J. Hickernell,et al.  Deterministic multi-level algorithms for infinite-dimensional integration on RN , 2011, J. Complex..

[142]  Gitta Kutyniok,et al.  Data Separation by Sparse Representations , 2011, Compressed Sensing.