On manifolds of tensors of fixed TT-rank

Recently, the format of TT tensors (Hackbusch and Kühn in J Fourier Anal Appl 15:706–722, 2009; Oseledets in SIAM J Sci Comput 2009, submitted; Oseledets and Tyrtyshnikov in SIAM J Sci Comput 31:5, 2009; Oseledets and Tyrtyshnikov in Linear Algebra Appl 2009, submitted) has turned out to be a promising new format for the approximation of solutions of high dimensional problems. In this paper, we prove some new results for the TT representation of a tensor $${U \in \mathbb{R}^{n_1\times \cdots\times n_d}}$$ and for the manifold of tensors of TT-rank $${\underline{r}}$$ . As a first result, we prove that the TT (or compression) ranks ri of a tensor U are unique and equal to the respective separation ranks of U if the components of the TT decomposition are required to fulfil a certain maximal rank condition. We then show that the set $${\mathbb{T}}$$ of TT tensors of fixed rank $${\underline{r}}$$ locally forms an embedded manifold in $${\mathbb{R}^{n_1\times\cdots\times n_d}}$$ , therefore preserving the essential theoretical properties of the Tucker format, but often showing an improved scaling behaviour. Extending a similar approach for matrices (Conte and Lubich in M2AN 44:759, 2010), we introduce certain gauge conditions to obtain a unique representation of the tangent space $${\mathcal{T}_U\mathbb{T}}$$ of $${\mathbb{T}}$$ and deduce a local parametrization of the TT manifold. The parametrisation of $${\mathcal{T}_{U}\mathbb{T}}$$ is often crucial for an algorithmic treatment of high-dimensional time-dependent PDEs and minimisation problems (Lubich in From quantum to classical molecular dynamics: reduced methods and numerical analysis, 2008). We conclude with remarks on those applications and present some numerical examples.

[1]  E. Hairer,et al.  Geometric Numerical Integration: Structure Preserving Algorithms for Ordinary Differential Equations , 2004 .

[2]  I. V. Oseledets,et al.  Tensors inside of matrices give logarithmic complexity , 2009 .

[3]  Armin Iske,et al.  Adaptive ADER Methods Using Kernel-Based Polyharmonic Spline WENO Reconstruction , 2010, SIAM J. Sci. Comput..

[4]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[5]  Reinhold Schneider,et al.  The Alternating Linear Scheme for Tensor Optimization in the Tensor Train Format , 2012, SIAM J. Sci. Comput..

[6]  Volker Mehrmann,et al.  Differential-Algebraic Equations: Analysis and Numerical Solution , 2006 .

[7]  Wang-Q Lim,et al.  Shearlets on Bounded Domains , 2010, 1007.3039.

[8]  Reinhold Schneider,et al.  Variational calculus with sums of elementary tensors of fixed rank , 2012, Numerische Mathematik.

[9]  Antonio Falcó,et al.  On Minimal Subspaces in Tensor Representations , 2012, Found. Comput. Math..

[10]  G. Teschke,et al.  Accelerated projected steepest descent method for nonlinear inverse problems with sparsity constraints , 2010 .

[11]  Lutz Kämmerer,et al.  Interpolation lattices for hyperbolic cross trigonometric polynomials , 2012, J. Complex..

[12]  M. Fannes,et al.  Finitely correlated states on quantum spin chains , 1992 .

[13]  R. Schneider,et al.  DIRECT MINIMIZATION FOR CALCULATING INVARIANT SUBSPACES IN DENSITY FUNCTIONAL COMPUTATIONS OF THE ELECTRONIC STRUCTURE , 2008, 0805.1190.

[14]  G. Teschke,et al.  A compressive Landweber iteration for solving ill-posed inverse problems , 2008 .

[15]  Winfried Sickel,et al.  Best m-term approximation and Lizorkin-Triebel spaces , 2011, J. Approx. Theory.

[16]  White,et al.  Density matrix formulation for quantum renormalization groups. , 1992, Physical review letters.

[17]  Karsten Urban,et al.  Adaptive Wavelet Methods on Unbounded Domains , 2012, Journal of Scientific Computing.

[18]  H. Neudecker,et al.  An approach ton-mode components analysis , 1986 .

[19]  L. Lathauwer Tensor decompositions and applications : a survey , 2009 .

[20]  Mark Coppejans,et al.  Breaking the Curse of Dimensionality , 2000 .

[21]  John M. Lee Manifolds and Differential Geometry , 2009 .

[22]  G. Teschke,et al.  Compressive sensing principles and iterative sparse recovery for inverse and ill-posed problems , 2010 .

[23]  Berkant Savas,et al.  A Newton-Grassmann Method for Computing the Best Multilinear Rank-(r1, r2, r3) Approximation of a Tensor , 2009, SIAM J. Matrix Anal. Appl..

[24]  Eugene E. Tyrtyshnikov,et al.  Breaking the Curse of Dimensionality, Or How to Use SVD in Many Dimensions , 2009, SIAM J. Sci. Comput..

[25]  Wolfgang Dahmen,et al.  Convergence Rates for Greedy Algorithms in Reduced Basis Methods , 2010, SIAM J. Math. Anal..

[26]  Daniela Rosca,et al.  A New Hybrid Method for Image Approximation Using the Easy Path Wavelet Transform , 2011, IEEE Transactions on Image Processing.

[27]  Lars Grasedyck,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig a Projection Method to Solve Linear Systems in Tensor Format a Projection Method to Solve Linear Systems in Tensor Format , 2022 .

[28]  Sadegh Jokar,et al.  Sparse recovery and Kronecker products , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[29]  K. Bredies,et al.  Regularization with non-convex separable constraints , 2009 .

[30]  Sanjoy Dasgupta,et al.  Adaptive Control Processes , 2010, Encyclopedia of Machine Learning and Data Mining.

[31]  Dajana Conte,et al.  Mathematical Modelling and Numerical Analysis an Error Analysis of the Multi-configuration Time-dependent Hartree Method of Quantum Dynamics , 2022 .

[32]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[33]  Daniel Rudolf,et al.  Error bounds for computing the expectation by Markov chain Monte Carlo , 2009, Monte Carlo Methods Appl..

[34]  Winfried Sickel,et al.  Best m-term aproximation and tensor product of Sobolev and Besov spaces-the case of non-compact embeddings , 2010 .

[35]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[36]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[37]  Boris N. Khoromskij,et al.  Multigrid Accelerated Tensor Approximation of Function Related Multidimensional Arrays , 2009, SIAM J. Sci. Comput..

[38]  Claude Klöckl,et al.  The density matrix renormalization group on tensor networks , 2013 .

[39]  S. Lang Fundamentals of differential geometry , 1998 .

[40]  J. Leeuw,et al.  Principal component analysis of three-mode data by means of alternating least squares algorithms , 1980 .

[41]  O. Ernst,et al.  ON THE CONVERGENCE OF GENERALIZED POLYNOMIAL CHAOS EXPANSIONS , 2011 .

[42]  P. Absil,et al.  Riemannian Geometry of Grassmann Manifolds with a View on Algorithmic Computation , 2004 .

[43]  Othmar Koch,et al.  Dynamical Low-Rank Approximation , 2007, SIAM J. Matrix Anal. Appl..

[44]  L. Lathauwer,et al.  On the Best Rank-1 and Rank-( , 2004 .

[45]  C. Lubich From Quantum to Classical Molecular Dynamics: Reduced Models and Numerical Analysis , 2008 .

[46]  J. Friedman Multivariate adaptive regression splines , 1990 .

[47]  Erich Novak,et al.  Optimal approximation of elliptic problems by linear and nonlinear mappings IV: Errors in L2 and other norms , 2004, J. Complex..

[48]  T. Schulte-Herbrüggen,et al.  Computations in quantum tensor networks , 2012, 1212.5005.

[49]  Othmar Koch,et al.  Dynamical Tensor Approximation , 2010, SIAM J. Matrix Anal. Appl..

[50]  Winfried Sickel,et al.  Best m-Term Approximation and Sobolev–Besov Spaces of Dominating Mixed Smoothness—the Case of Compact Embeddings , 2012 .

[51]  Erwan Faou,et al.  Computing Semiclassical Quantum Dynamics with Hagedorn Wavepackets , 2009, SIAM J. Sci. Comput..

[52]  T. Crawford,et al.  An Introduction to Coupled Cluster Theory for Computational Chemists , 2007 .

[53]  Andreas Zeiser,et al.  Wavelet Approximation in Weighted Sobolev Spaces of Mixed Order with Applications to the Electronic Schrödinger Equation , 2012 .

[54]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[55]  D. Lorenz,et al.  Greedy solution of ill-posed problems: error bounds and exact inversion , 2009, 0904.0154.

[56]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[57]  Wang-Q Lim,et al.  Compactly Supported Shearlets , 2010, 1009.4359.

[58]  W. Hackbusch,et al.  Black Box Low Tensor-Rank Approximation Using Fiber-Crosses , 2009 .

[59]  H. Yserentant,et al.  On the Complexity of the Electronic Schrödinger Equation , 2022 .

[60]  E. Novak,et al.  Optimal Order of Convergence and (In)Tractability of Multivariate Approximation of Smooth Functions , 2009 .

[61]  Andreas Schadschneider,et al.  Groundstate properties of a generalized VBS-model , 1992 .

[62]  J. Olsen,et al.  Molecular electronic-structure theory , 2000 .

[63]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[64]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[65]  Martin J. Mohlenkamp,et al.  Multivariate Regression and Machine Learning with Sums of Separable Functions , 2009, SIAM J. Sci. Comput..

[66]  Trygve Helgaker,et al.  Molecular Electronic-Structure Theory: Helgaker/Molecular Electronic-Structure Theory , 2000 .

[67]  Berkant Savas,et al.  Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors , 2009, SIAM J. Sci. Comput..

[68]  F. Verstraete,et al.  Complete-graph tensor network states: a new fermionic wave function ansatz for molecules , 2010, 1004.5303.

[69]  Wolfgang Dahmen,et al.  Fast high-dimensional approximation with sparse occupancy trees , 2011, J. Comput. Appl. Math..

[70]  G. Vidal Efficient classical simulation of slightly entangled quantum computations. , 2003, Physical review letters.

[71]  Christian Bender,et al.  Dual pricing of multi-exercise options under volume constraints , 2011, Finance Stochastics.

[72]  G. Plonka The Easy Path Wavelet Transform: A New Adaptive Wavelet Transform for Sparse Representation of Two-Dimensional Data , 2009 .

[73]  Armin Iske,et al.  Curvature analysis of frequency modulated manifolds in dimensionality reduction , 2011 .

[74]  W. Hackbusch,et al.  A New Scheme for the Tensor Representation , 2009 .

[75]  Ivan Oseledets,et al.  A new tensor decomposition , 2009 .

[76]  Harry Yserentant,et al.  The mixed regularity of electronic wave functions multiplied by explicit correlation factors , 2011 .

[77]  Stephan Dahlke,et al.  Adaptive wavelet methods and sparsity reconstruction for inverse heat conduction problems , 2010, Adv. Comput. Math..

[78]  Gerlind Plonka-Hoch,et al.  The Easy Path Wavelet Transform: A New Adaptive Wavelet Transform for Sparse Representation of Two-Dimensional Data , 2008, Multiscale Model. Simul..

[79]  Toshio Odanaka,et al.  ADAPTIVE CONTROL PROCESSES , 1990 .

[80]  M. Beck,et al.  The multiconfiguration time-dependent Hartree (MCTDH) method: A highly efficient algorithm for propa , 1999 .

[81]  Steffen Dereich,et al.  Multilevel Monte Carlo algorithms for L\'{e}vy-driven SDEs with Gaussian correction , 2011, 1101.1369.

[82]  Jerome H. Friedman Multivariate adaptive regression splines (with discussion) , 1991 .