Adaptive Near-Optimal Rank Tensor Approximation for High-Dimensional Operator Equations

We consider a framework for the construction of iterative schemes for operator equations that combine low-rank approximation in tensor formats and adaptive approximation in a basis. Under fairly general assumptions, we conduct a rigorous convergence analysis, where all parameters required for the execution of the methods depend only on the underlying infinite-dimensional problem, but not on a concrete discretization. Under certain assumptions on the rates for the involved low-rank approximations and basis expansions, we can also give bounds on the computational complexity of the iteration as a function of the prescribed target error. Our theoretical findings are illustrated and supported by computational experiments. These demonstrate that problems in very high dimensions can be treated with controlled solution accuracy.

[1]  W. Hackbusch Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[2]  Virginie Ehrlacher,et al.  Convergence of a greedy algorithm for high-dimensional convex nonlinear problems , 2010, 1004.0095.

[3]  Wolfgang Dahmen,et al.  Compressed Sensing and Electron Microscopy , 2010 .

[4]  Mark Coppejans,et al.  Breaking the Curse of Dimensionality , 2000 .

[5]  Klaus Ritter,et al.  A Local Refinement Strategy for Constructive Quantization of Scalar SDEs , 2013, Found. Comput. Math..

[6]  Markus Hansen,et al.  On tensor products of quasi-Banach spaces , 2010 .

[7]  P. Maass,et al.  An analysis of electrical impedance tomography with applications to Tikhonov regularization , 2012 .

[8]  Hermann G. Matthies,et al.  Efficient Analysis of High Dimensional Data in Tensor Formats , 2012 .

[9]  M. Hansen,et al.  n-term approximation rates and Besov regularity for elliptic PDEs on polyhedral domains , 2012 .

[10]  Martin J. Mohlenkamp,et al.  Numerical operator calculus in higher dimensions , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[11]  H. Yserentant,et al.  On the Complexity of the Electronic Schrödinger Equation , 2022 .

[12]  Michael Griebel,et al.  Approximation of bi-variate functions: singular value decomposition versus sparse grids , 2014 .

[13]  Stephan Dahlke,et al.  Piecewise tensor product wavelet bases by extensions and approximation rates , 2013, Math. Comput..

[14]  K AlpertBradley A class of bases in L2 for the sparse representations of integral operators , 1993 .

[15]  O. Ernst,et al.  ON THE CONVERGENCE OF GENERALIZED POLYNOMIAL CHAOS EXPANSIONS , 2011 .

[16]  R. DeVore,et al.  Approximation of Functions of Few Variables in High Dimensions , 2011 .

[17]  Reinhold Schneider,et al.  Dynamical Approximation by Hierarchical Tucker and Tensor-Train Tensors , 2013, SIAM J. Matrix Anal. Appl..

[18]  Stephan Dahlke,et al.  A note on quarkonial systems and multilevel partition of unity methods , 2013 .

[19]  Karsten Urban,et al.  Adaptive Wavelet Methods on Unbounded Domains , 2012, Journal of Scientific Computing.

[20]  G. Teschke,et al.  Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems , 2010 .

[21]  Christian Bender,et al.  Least-Squares Monte Carlo for Backward SDEs , 2012 .

[22]  Wolfgang Dahmen,et al.  DOUBLE GREEDY ALGORITHMS: REDUCED BASIS METHODS FOR TRANSPORT DOMINATED PROBLEMS ∗ , 2013, 1302.5072.

[23]  Armin Iske,et al.  On Groupoid C∗-Algebras, Persistent Homology and Time-Frequency Analysis , 2011 .

[24]  A. Cohen Numerical Analysis of Wavelet Methods , 2003 .

[25]  Stephan Dahlke,et al.  An adaptive wavelet method for parameter identification problems in parabolic partial differential equations , 2022 .

[26]  F. L. Hitchcock Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[27]  Winfried Sickel,et al.  On Besov regularity of solutions to nonlinear elliptic partial differential equations , 2020, Nonlinear Analysis.

[28]  W. Dahmen Wavelet and multiscale methods for operator equations , 1997, Acta Numerica.

[29]  K. Ritter,et al.  Adaptive Wavelet Methods for Elliptic Stochastic Partial Differential Equations , 2022 .

[30]  B. Alpert A class of bases in L 2 for the sparse representations of integral operators , 1993 .

[31]  Rob Stevenson,et al.  On the Compressibility of Operators in Wavelet Coordinates , 2004, SIAM J. Math. Anal..

[32]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[33]  Arend Aalberthus Roeland Metselaar Handling Wavelet Expansions in numerical Methods , 2002 .

[34]  André Uschmajew,et al.  Local Convergence of the Alternating Least Squares Algorithm for Canonical Tensor Approximation , 2012, SIAM J. Matrix Anal. Appl..

[35]  Wolfgang Dahmen,et al.  Convergence Rates for Greedy Algorithms in Reduced Basis Methods , 2010, SIAM J. Math. Anal..

[36]  Wang-Q Lim,et al.  Compactly Supported Shearlets , 2010, 1009.4359.

[37]  Thorsten Rohwedder,et al.  The continuous Coupled Cluster formulation for the electronic Schrödinger equation , 2013 .

[38]  Klaus Ritter,et al.  Spatial Besov Regularity for Stochastic Partial Differential Equations on Lipschitz Domains , 2010, 1011.1814.

[39]  Reinhold Schneider,et al.  An analysis for the DIIS acceleration method used in quantum chemistry calculations , 2011 .

[40]  Wolfgang Dahmen,et al.  Adaptive low rank wavelet methods and applications to two-electron Schrödinger equations , 2012 .

[41]  Hermann G. Matthies,et al.  Efficient low-rank approximation of the stochastic Galerkin matrix in tensor formats , 2014, Comput. Math. Appl..

[42]  Petru A. Cioica,et al.  On the $L_q(L_p)$-regularity and Besov smoothness of stochastic parabolic equations on bounded Lipschitz domains , 2013, 1301.1180.

[43]  Torsten Görner,et al.  Efficient and accurate computation of spherical mean values at scattered center points , 2012 .

[44]  G. Kutyniok,et al.  Construction of Compactly Supported Shearlet Frames , 2010, 1003.5481.

[45]  Hermann G. Matthies,et al.  Solving stochastic systems with low-rank tensor compression , 2012 .

[46]  Armin Iske,et al.  Adaptive ADER Methods Using Kernel-Based Polyharmonic Spline WENO Reconstruction , 2010, SIAM J. Sci. Comput..

[47]  Fred J. Hickernell,et al.  Deterministic multi-level algorithms for infinite-dimensional integration on RN , 2011, J. Complex..

[48]  André Uschmajew,et al.  Regularity of Tensor Product Approximations to Square Integrable Functions , 2011 .

[49]  Antonio Falcó,et al.  On Minimal Subspaces in Tensor Representations , 2012, Found. Comput. Math..

[50]  Andreas Zeiser,et al.  Wavelet Approximation in Weighted Sobolev Spaces of Mixed Order with Applications to the Electronic Schrödinger Equation , 2012 .

[51]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[52]  S. Dereich,et al.  Constructive quantization: Approximation by empirical measures , 2011, 1108.5346.

[53]  D. Rudolf,et al.  Explicit error bounds for Markov chain Monte Carlo , 2011, 1108.3201.

[54]  Wolfgang Dahmen,et al.  Adaptive Wavelet Methods II—Beyond the Elliptic Case , 2002, Found. Comput. Math..

[55]  Tobias Jahnke,et al.  Error Bound for Piecewise Deterministic Processes Modeling Stochastic Reaction Systems , 2012, Multiscale Model. Simul..

[56]  Boris N. Khoromskij,et al.  Tensor-Structured Galerkin Approximation of Parametric and Stochastic Elliptic PDEs , 2011, SIAM J. Sci. Comput..

[57]  Antonio Falcó,et al.  Proper generalized decomposition for nonlinear convex problems in tensor Banach spaces , 2011, Numerische Mathematik.

[58]  Winfried Sickel,et al.  Best m-Term Approximation and Sobolev–Besov Spaces of Dominating Mixed Smoothness—the Case of Compact Embeddings , 2012 .

[59]  Stephan Dahlke,et al.  Spatial Besov regularity for semilinear stochastic partial differential equations on bounded Lipschitz domains , 2012, Int. J. Comput. Math..

[60]  M. Griebel,et al.  Approximation of Two-Variate Functions: Singular Value Decomposition Versus Regular Sparse Grids , 2010 .

[61]  Wolfgang Hackbusch,et al.  An Introduction to Hierarchical (H-) Rank and TT-Rank of Tensors with Examples , 2011, Comput. Methods Appl. Math..

[62]  Reinhold Schneider,et al.  Error estimates for the Coupled Cluster method , 2013 .

[63]  Henryk Wozniakowski,et al.  The curse of dimensionality for numerical integration of smooth functions , 2012, Math. Comput..

[64]  Reinhold Schneider,et al.  Numerical analysis of augmented plane wave methods for full-potential electronic structure calculations , 2015 .

[65]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[66]  Reinhold Schneider,et al.  Numerical analysis of Gaussian approximations in quantum chemistry , 2012 .

[67]  H. Harbrecht,et al.  On the low-rank approximation by the pivoted Cholesky decomposition , 2012 .

[68]  J. Ballani,et al.  Black box approximation of tensors in hierarchical Tucker format , 2013 .

[69]  G. Kutyniok,et al.  Irregular Shearlet Frames: Geometry and Approximation Properties , 2010, 1002.2657.

[70]  Konstantin Grella,et al.  Sparse tensor spherical harmonics approximation in radiative transfer , 2011, J. Comput. Phys..

[71]  Wang-Q Lim,et al.  Compactly supported shearlets are optimally sparse , 2010, J. Approx. Theory.

[72]  Henryk Wozniakowski,et al.  Discontinuous information in the worst case and randomized settings , 2011, 1106.2945.

[73]  Eugene E. Tyrtyshnikov,et al.  Breaking the Curse of Dimensionality, Or How to Use SVD in Many Dimensions , 2009, SIAM J. Sci. Comput..

[74]  Henryk Wozniakowski,et al.  Approximation of infinitely differentiable multivariate functions is intractable , 2009, J. Complex..

[75]  Antonio Falcó,et al.  Geometric structures in tensor representations , 2013 .

[76]  Armin Iske,et al.  Curvature analysis of frequency modulated manifolds in dimensionality reduction , 2011 .

[77]  Christian Bender,et al.  Error Criteria for Numerical Solutions of Backward SDEs , 2010 .

[78]  D. Crisan,et al.  Robust filtering: Correlated noise and multidimensional observation , 2012, 1201.1858.

[79]  W. Hackbusch,et al.  A New Scheme for the Tensor Representation , 2009 .

[80]  Karsten Urban,et al.  On Wavelet-Galerkin Methods for Semilinear Parabolic Equations with Additive Noise , 2013 .

[81]  Armin Iske,et al.  Optimal representation of piecewise Hölder smooth bivariate functions by the Easy Path Wavelet Transform , 2013, J. Approx. Theory.

[82]  Klaus Ritter,et al.  Derandomization of the Euler scheme for scalar stochastic differential equations , 2012, J. Complex..

[83]  Harry Yserentant,et al.  The mixed regularity of electronic wave functions multiplied by explicit correlation factors , 2011 .

[84]  Reinhold Schneider,et al.  On manifolds of tensors of fixed TT-rank , 2012, Numerische Mathematik.

[85]  Lutz Kämmerer,et al.  Interpolation lattices for hyperbolic cross trigonometric polynomials , 2012, J. Complex..

[86]  Steffen Dereich,et al.  Foundations of Computational Mathematics, Budapest 2011: On the Complexity of Computing Quadrature Formulas for SDEs , 2012 .

[87]  Martin J. Mohlenkamp,et al.  Algorithms for Numerical Analysis in High Dimensions , 2005, SIAM J. Sci. Comput..

[88]  Michael Griebel,et al.  An Adaptive Sparse Grid Semi-Lagrangian Scheme for First Order Hamilton-Jacobi Bellman Equations , 2012, Journal of Scientific Computing.

[89]  Gitta Kutyniok,et al.  Data Separation by Sparse Representations , 2011, Compressed Sensing.

[90]  Christian Bender,et al.  A Posteriori Estimates for Backward SDEs , 2013, SIAM/ASA J. Uncertain. Quantification.

[91]  Markus Weimar Breaking the curse of dimensionality , 2015 .

[92]  Michael Gnewuch,et al.  On weighted Hilbert spaces and integration of functions of infinitely many variables , 2014, J. Complex..

[93]  Gabriele Steidl,et al.  Shearlet Coorbit Spaces: Compactly Supported Analyzing Shearlets, Traces and Embeddings , 2011 .

[94]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[95]  Wolfgang Dahmen,et al.  Classification algorithms using adaptive partitioning , 2014, 1411.0839.

[96]  Reinhold Schneider,et al.  Approximation rates for the hierarchical tensor format in periodic Sobolev spaces , 2014, J. Complex..

[97]  R. Schneider,et al.  The Alternating Linear Scheme for Tensor Optimisation in the TT Format , 2022 .

[98]  Boris N. Khoromskij,et al.  Approximate iterations for structured matrices , 2008, Numerische Mathematik.

[99]  Wang-Q Lim,et al.  Image Separation Using Shearlets , 2011 .

[100]  André Uschmajew,et al.  Well-posedness of convex maximization problems on Stiefel manifolds and orthogonal tensor product approximations , 2010, Numerische Mathematik.

[101]  Rob Stevenson,et al.  An Adaptive Wavelet Method for Solving High-Dimensional Elliptic PDEs , 2009 .

[102]  Howard C. Elman,et al.  Efficient Iterative Solvers for Stochastic Galerkin Discretizations of Log-Transformed Random Diffusion Problems , 2012, SIAM J. Sci. Comput..

[103]  Winfried Sickel,et al.  Best m-term aproximation and tensor product of Sobolev and Besov spaces-the case of non-compact embeddings , 2010 .

[104]  Simen Kvaal,et al.  Multiconfigurational time-dependent Hartree method to describe particle loss due to absorbing boundary conditions , 2011, 1102.3899.

[105]  Wolfgang Dahmen,et al.  Approximation of High-Dimensional Rank One Tensors , 2013, Constructive Approximation.

[106]  Ianwei,et al.  Compressive Video Sampling with Approximate Message Passing Decoding , 2011 .

[107]  Harry Yserentant,et al.  The mixed regularity of electronic wave functions in fractional order and weighted Sobolev spaces , 2012, Numerische Mathematik.

[108]  E. Novak,et al.  On the power of function values for the approximation problem in various settings , 2010, 1011.3682.

[109]  Wolfgang Hackbusch,et al.  Tensorisation of vectors and their efficient convolution , 2011, Numerische Mathematik.

[110]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[111]  Harold Gulliksen,et al.  Contributions to mathematical psychology , 1964 .

[112]  A. Uschmajew,et al.  LOCAL CONVERGENCE OF ALTERNATING SCHEMES FOR OPTIMIZATION OF CONVEX PROBLEMS IN THE TT FORMAT , 2011 .

[113]  Lars Grasedyck,et al.  Polynomial Approximation in Hierarchical Tucker Format by Vector – Tensorization , 2010 .

[114]  Wang-Q Lim,et al.  Shearlets on Bounded Domains , 2010, 1007.3039.

[115]  K. Ritter,et al.  On the convergence analysis of Rothe ’ s method , 2022 .

[116]  Wolfgang Dahmen,et al.  Fast high-dimensional approximation with sparse occupancy trees , 2011, J. Comput. Appl. Math..

[117]  Daniel Kressner,et al.  Preconditioned Low-Rank Methods for High-Dimensional Elliptic PDE Eigenvalue Problems , 2011, Comput. Methods Appl. Math..

[118]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[119]  Lutz Kämmerer Reconstructing hyperbolic cross trigonometric polynomials by sampling along generated sets , 2012 .

[120]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[121]  Wolfgang Dahmen,et al.  Adaptive Petrov-Galerkin Methods for First Order Transport Equations , 2011, SIAM J. Numer. Anal..

[122]  Sadegh Jokar,et al.  Sparse recovery and Kronecker products , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[123]  Erich Novak,et al.  The Curse of Dimensionality for Monotone and Convex Functions of Many Variables , 2010, 1011.3680.

[124]  Wolfgang Dahmen,et al.  Adaptivity and variational stabilization for convection-diffusion equations∗ , 2012 .

[125]  Reinhold Schneider,et al.  Optimization problems in contracted tensor networks , 2011, Comput. Vis. Sci..

[126]  Wolfgang Dahmen,et al.  Adaptive wavelet methods for elliptic operator equations: Convergence rates , 2001, Math. Comput..

[127]  Tobias Jahnke,et al.  On Reduced Models for the Chemical Master Equation , 2011, Multiscale Model. Simul..

[128]  Denis Belomestny,et al.  Multilevel dual approach for pricing American style derivatives , 2012, Finance Stochastics.

[129]  Lars Grasedyck,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig a Projection Method to Solve Linear Systems in Tensor Format a Projection Method to Solve Linear Systems in Tensor Format , 2022 .

[130]  Christian Bender,et al.  Primal and Dual Pricing of Multiple Exercise Options in Continuous Time , 2011, SIAM J. Financial Math..