Fast Multidimensional Convolution in Low-Rank Tensor Formats via Cross Approximation

We propose a new cross-conv algorithm for approximate computation of convolution in different low-rank tensor formats (tensor train, Tucker, hierarchical Tucker). It has better complexity with respect to the tensor rank than previous approaches. The new algorithm has a high potential impact in different applications. The key idea is based on applying cross approximation in the “frequency domain,” where convolution becomes a simple elementwise product. We illustrate efficiency of our algorithm by computing the three-dimensional Newton potential and by presenting preliminary results for solution of the Hartree--Fock equation on tensor-product grids.

[1]  Eugene E. Tyrtyshnikov,et al.  Optimal and Superoptimal Circulant Preconditioners , 1992, SIAM J. Matrix Anal. Appl..

[2]  J. Ballani,et al.  Black box approximation of tensors in hierarchical Tucker format , 2013 .

[3]  Wolfgang Hackbusch,et al.  An Introduction to Hierarchical (H-) Rank and TT-Rank of Tensors with Examples , 2011, Comput. Methods Appl. Math..

[4]  Boris N. Khoromskij,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig Fast and Accurate Tensor Approximation of Multivariate Convolution with Linear Scaling in Dimension Fast and Accurate Tensor Approximation of Multivariate Convolution with Linear Scaling in Dimension , 2022 .

[5]  Boris N. Khoromskij,et al.  Multigrid Accelerated Tensor Approximation of Function Related Multidimensional Arrays , 2009, SIAM J. Sci. Comput..

[6]  B. Khoromskij,et al.  Low rank Tucker-type tensor approximation to classical potentials , 2007 .

[7]  Boris N. Khoromskij,et al.  Tensor decomposition in electronic structure calculations on 3D Cartesian grids , 2009, J. Comput. Phys..

[8]  Russell D. Johnson,et al.  NIST Computational Chemistry Comparison and Benchmark Database , 2005 .

[9]  E. Tyrtyshnikov,et al.  TT-cross approximation for multidimensional arrays , 2010 .

[10]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[11]  S. Goreinov,et al.  The maximum-volume concept in approximation by low-rank matrices , 2001 .

[12]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[13]  Eugene E. Tyrtyshnikov,et al.  Linear algebra for tensor problems , 2009, Computing.

[14]  Ivan V. Oseledets,et al.  DMRG Approach to Fast Linear Algebra in the TT-Format , 2011, Comput. Methods Appl. Math..

[15]  Boris N. Khoromskij,et al.  Superfast Fourier Transform Using QTT Approximation , 2012 .

[16]  Ivan Oseledets,et al.  A computationally efficient technique for the solution of multi-dimensional PBMs of granulation via tensor decomposition , 2014, Comput. Chem. Eng..

[17]  Mario Bebendorf,et al.  Approximation of boundary element matrices , 2000, Numerische Mathematik.

[18]  Venera Khoromskaia.  Computation of the Hartree-Fock Exchange by the Tensor-Structured Methods , 2010, Comput. Methods Appl. Math..

[19]  Reinhold Schneider,et al.  Tensor-Structured Factorized Calculation of Two-Electron Integrals in a General Basis , 2013, SIAM J. Sci. Comput..

[20]  Martin J. Mohlenkamp,et al.  Algorithms for Numerical Analysis in High Dimensions , 2005, SIAM J. Sci. Comput..

[21]  Boris N. Khoromskij,et al.  Low-rank Kronecker-product Approximation to Multi-dimensional Nonlocal Operators. Part II. HKT Representation of Certain Operators , 2005, Computing.

[22]  Yinchu Zhu,et al.  Breaking the curse of dimensionality in regression , 2017, ArXiv.

[23]  W. Hackbusch,et al.  A New Scheme for the Tensor Representation , 2009 .

[24]  Venera Khoromskaia.  Black-Box Hartree–Fock Solver by Tensor Numerical Methods , 2014, Comput. Methods Appl. Math..

[25]  S. Goreinov,et al.  On cross approximation of multi-index arrays , 2008 .

[26]  Ivan V. Oseledets,et al.  Approximation of 2d˟2d Matrices Using Tensor Decomposition , 2010, SIAM J. Matrix Anal. Appl..

[27]  Jonathan A. D. Wattis,et al.  An introduction to mathematical models of coagulation–fragmentation processes: A discrete deterministic mean-field approach , 2006 .

[28]  Dmitry V. Savostyanov,et al.  Quasioptimality of maximum-volume cross interpolation of tensors , 2013, 1305.1818.

[29]  Gregory Beylkin,et al.  Multiresolution quantum chemistry: basic theory and initial applications. , 2004, The Journal of chemical physics.

[30]  Ivan V. Oseledets,et al.  Fast adaptive interpolation of multi-dimensional arrays in tensor train format , 2011, The 2011 International Workshop on Multidimensional (nD) Systems.

[31]  Eugene E. Tyrtyshnikov,et al.  Fast truncation of mode ranks for bilinear tensor operations , 2012, Numer. Linear Algebra Appl..

[32]  White,et al.  Density matrix formulation for quantum renormalization groups. , 1992, Physical review letters.

[33]  W. Hackbusch.  Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[34]  Eugene E. Tyrtyshnikov,et al.  Incomplete Cross Approximation in the Mosaic-Skeleton Method , 2000, Computing.

[35]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[36]  Ivan V. Oseledets,et al.  Wedderburn Rank Reduction and Krylov Subspace Method for Tensor Approximation. Part 1: Tucker Case , 2010, SIAM J. Sci. Comput..

[37]  Mario Bebendorf,et al.  Adaptive Cross Approximation of Multivariate Functions , 2011 .

[38]  U. Schollwoeck.  The density-matrix renormalization group in the age of matrix product states , 2010, 1008.3477.

[39]  B. Khoromskij.  O(dlog N)-Quantics Approximation of N-d Tensors in High-Dimensional Numerical Modeling , 2011 .

[40]  S. Goreinov,et al.  How to find a good submatrix , 2010 .

[41]  S. Loyalka,et al.  Direct simulation Monte Carlo aerosol dynamics: Collisional sampling algorithms , 2007 .

[42]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[43]  B. Khoromskij.  Structured Rank-(r1, . . . , rd) Decomposition of Function-related Tensors in R_D , 2006 .

[44]  John W. Woods.  Multidimensional Signal, Image, and Video Processing and Coding, Second Edition , 2011 .

[45]  Östlund,et al.  Thermodynamic limit of density matrix renormalization. , 1995, Physical review letters.

[46]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[47]  Eugene E. Tyrtyshnikov,et al.  Approximate multiplication of tensor matrices based on the individual filtering of factors , 2009 .

[48]  H. Y. Wong,et al.  Efficient Options Pricing Using the Fast Fourier Transform , 2010 .

[49]  Markus Weimar.  Breaking the curse of dimensionality , 2015 .

[50]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[51]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[52]  Boris N. Khoromskij,et al.  Numerical Solution of the Hartree - Fock Equation in Multilevel Tensor-Structured Format , 2011, SIAM J. Sci. Comput..

[53]  Vladimir A. Kazeev,et al.  Multilevel Toeplitz Matrices Generated by Tensor-Structured Vectors and Convolution with Logarithmic Complexity , 2013, SIAM J. Sci. Comput..

[54]  Wolfgang Hackbusch,et al.  Efficient convolution with the Newton potential in d dimensions , 2008, Numerische Mathematik.

[55]  A. Cichocki,et al.  Generalizing the column–row matrix decomposition to multi-way arrays , 2010 .

[56]  Eugene E. Tyrtyshnikov,et al.  Breaking the Curse of Dimensionality, Or How to Use SVD in Many Dimensions , 2009, SIAM J. Sci. Comput..

[57]  Wolfgang Hackbusch.  Fast and exact projected convolution for non-equidistant grids , 2007, Computing.

[58]  S. V. Dolgov,et al.  ALTERNATING MINIMAL ENERGY METHODS FOR LINEAR SYSTEMS IN HIGHER DIMENSIONS∗ , 2014 .

[59]  Gregory Beylkin,et al.  Multiresolution quantum chemistry in multiwavelet bases: Hartree-Fock exchange. , 2004, The Journal of chemical physics.

[60]  Reinhold Schneider,et al.  QTT Representation of the Hartree and Exchange Operators in Electronic Structure Calculations , 2011, Comput. Methods Appl. Math..

[61]  Boris N. Khoromskij,et al.  Low-rank Kronecker-product Approximation to Multi-dimensional Nonlocal Operators. Part I. Separable Approximation of Multi-variate Functions , 2005, Computing.

[62]  Eugene E. Tyrtyshnikov,et al.  Tucker Dimensionality Reduction of Three-Dimensional Arrays in Linear Time , 2008, SIAM J. Matrix Anal. Appl..

[63]  Martin J. Mohlenkamp,et al.  Numerical operator calculus in higher dimensions , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[64]  Venera Khoromskaia,et al.  Numerical solution of the Hartree-Fock equation by multilevel tensor-structured methods , 2011 .

[65]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[66]  Laurent Demanet,et al.  Sublinear Randomized Algorithms for Skeleton Decompositions , 2011, SIAM J. Matrix Anal. Appl..

[67]  B. Khoromskij.  Tensors-structured Numerical Methods in Scientific Computing: Survey on Recent Advances , 2012 .