Tensor Spaces and Hierarchical Tensor Representations

In the present report we provide a brief introduction into recently developed hierarchical tensor representations. The new formats extend the well-known Tucker format into a hierarchical framework, by combining its favourable characteristics with low-order scaling properties. We demonstrate the basic concept of subspace approximation and higher order SVD (HOSVD), and how to extend this in a hierarchical way. We highlight that the present tensor representations are constituting smooth manifolds, and give a perspective how these properties can be used to develop numerical solvers for tensor equations and tensor optimisation problems.

[1]  Wolfgang Hackbusch,et al.  Tensorisation of vectors and their efficient convolution , 2011, Numerische Mathematik.

[2]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[3]  W. Hackbusch.  Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[4]  Reinhold Schneider,et al.  Efficient time-stepping scheme for dynamics on TT-manifolds , 2012 .

[5]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[6]  U. Schollwoeck.  The density-matrix renormalization group in the age of matrix product states , 2010, 1008.3477.

[7]  Reinhold Schneider,et al.  The Alternating Linear Scheme for Tensor Optimization in the Tensor Train Format , 2012, SIAM J. Sci. Comput..

[8]  Reinhold Schneider,et al.  Optimization problems in contracted tensor networks , 2011, Comput. Vis. Sci..

[9]  E. Schmidt.  Zur Theorie der linearen und nichtlinearen Integralgleichungen , 1907 .

[10]  Wolfgang Hackbusch,et al.  An Introduction to Hierarchical (H-) Rank and TT-Rank of Tensors with Examples , 2011, Comput. Methods Appl. Math..

[11]  Reinhold Schneider,et al.  Dynamical Approximation by Hierarchical Tucker and Tensor-Train Tensors , 2013, SIAM J. Matrix Anal. Appl..

[12]  Lars Grasedyck,et al.  Tree Adaptive Approximation in the Hierarchical Tensor Format , 2014, SIAM J. Sci. Comput..

[13]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[14]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[15]  Reinhold Schneider,et al.  Convergence Results for Projected Line-Search Methods on Varieties of Low-Rank Matrices Via Łojasiewicz Inequality , 2014, SIAM J. Optim..

[16]  Ivan Oseledets,et al.  Quantics-TT Approximation of Elliptic Solution Operators in Higher Dimensions , 2009 .

[17]  Wolfgang Hackbusch,et al.  A regularized Newton method for the efficient approximation of tensors represented in the canonical tensor format , 2012, Numerische Mathematik.

[18]  Boris N. Khoromskij,et al.  Simultaneous state-time approximation of the chemical master equation using tensor product formats , 2015, Numer. Linear Algebra Appl..

[19]  Eugene E. Tyrtyshnikov,et al.  Algebraic Wavelet Transform via Quantics Tensor Train Decomposition , 2011, SIAM J. Sci. Comput..

[20]  Bart Vandereycken,et al.  The geometry of algorithms using hierarchical tensors , 2013, Linear Algebra and its Applications.

[21]  Eugene E. Tyrtyshnikov,et al.  Breaking the Curse of Dimensionality, Or How to Use SVD in Many Dimensions , 2009, SIAM J. Sci. Comput..

[22]  Daniel Kressner,et al.  Algorithm 941 , 2014 .

[23]  Reinhold Schneider,et al.  Tensor-Structured Factorized Calculation of Two-Electron Integrals in a General Basis , 2013, SIAM J. Sci. Comput..

[24]  Ivan Oseledets,et al.  QTT approximation of elliptic solution operators in higher dimensions , 2011 .

[25]  Antonio Falcó,et al.  Geometric structures in tensor representations , 2013 .

[26]  W. Hackbusch,et al.  A New Scheme for the Tensor Representation , 2009 .

[27]  F. Verstraete,et al.  Post-matrix product state methods: To tangent space and beyond , 2013, 1305.1894.

[28]  Wolfgang Hackbusch,et al.  Numerical tensor calculus* , 2014, Acta Numerica.

[29]  Reinhold Schneider,et al.  On manifolds of tensors of fixed TT-rank , 2012, Numerische Mathematik.

[30]  Ivan Oseledets,et al.  Fast solution of multi-dimensional parabolic problems in the TT/QTT-format with initial application to the Fokker-Planck equation , 2011 .

[31]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[32]  V. Khoromskaia,et al.  Møller-Plesset (MP2) energy correction using tensor factorization of the grid-based two-electron integrals , 2014, Comput. Phys. Commun..

[33]  Wolfgang Hackbusch.  $$L^{\infty }$$ estimation of tensor truncations , 2013, Numerische Mathematik.

[34]  Haobin Wang,et al.  Multilayer formulation of the multiconfiguration time-dependent Hartree theory , 2003 .

[35]  André Uschmajew,et al.  On Local Convergence of Alternating Schemes for Optimization of Convex Problems in the Tensor Train Format , 2013, SIAM J. Numer. Anal..

[36]  Peter Bro Miltersen,et al.  Strategy Iteration Is Strongly Polynomial for 2-Player Turn-Based Stochastic Games with a Constant Discount Factor , 2010, JACM.

[37]  Reinhold Schneider,et al.  Variational calculus with sums of elementary tensors of fixed rank , 2012, Numerische Mathematik.

[38]  Reinhold Schneider,et al.  Tensor Product Approximation (DMRG) and Coupled Cluster method in Quantum Chemistry , 2013, 1310.2736.

[39]  Martin J. Mohlenkamp,et al.  Algorithms for Numerical Analysis in High Dimensions , 2005, SIAM J. Sci. Comput..

[40]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[41]  Antonio Falcó,et al.  On Minimal Subspaces in Tensor Representations , 2012, Found. Comput. Math..

[42]  Reinhold Schneider,et al.  QTT Representation of the Hartree and Exchange Operators in Electronic Structure Calculations , 2011, Comput. Methods Appl. Math..

[43]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[44]  Othmar Koch,et al.  Dynamical Low-Rank Approximation , 2007, SIAM J. Matrix Anal. Appl..

[45]  C. Lubich.  From Quantum to Classical Molecular Dynamics: Reduced Models and Numerical Analysis , 2008 .

[46]  E. Schmidt.  Zur Theorie der linearen und nichtlinearen Integralgleichungen , 1989 .

[47]  Tobias Jahnke,et al.  On the approximation of high-dimensional differential equations in the hierarchical Tucker format , 2014 .

[48]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[49]  Reinhold Schneider,et al.  Approximation rates for the hierarchical tensor format in periodic Sobolev spaces , 2014, J. Complex..

[50]  Boris N. Khoromskij,et al.  Approximate iterations for structured matrices , 2008, Numerische Mathematik.

[51]  S. V. DOLGOV,et al.  Fast Solution of Parabolic Problems in the Tensor Train/Quantized Tensor Train Format with Initial Application to the Fokker-Planck Equation , 2012, SIAM J. Sci. Comput..

[52]  J. Landsberg.  Tensors: Geometry and Applications , 2011 .