A Tensor-Train accelerated solver for integral equations in complex geometries

We present a framework using the Quantized Tensor Train (QTT) decomposition to accurately and efficiently solve volume and boundary integral equations in three dimensions. We describe how the QTT decomposition can be used as a hierarchical compression and inversion scheme for matrices arising from the discretization of integral equations. For a broad range of problems, computational and storage costs of the inversion scheme are extremely modest $O(\log N)$ and once the inverse is computed, it can be applied in $O(N \log N)$. We analyze the QTT ranks for hierarchically low rank matrices and discuss its relationship to commonly used hierarchical compression techniques such as FMM and HSS. We prove that the QTT ranks are bounded for translation-invariant systems and argue that this behavior extends to non-translation invariant volume and boundary integrals. For volume integrals, the QTT decomposition provides an efficient direct solver requiring significantly less memory compared to other fast direct solvers. We present results demonstrating the remarkable performance of the QTT-based solver when applied to both translation and non-translation invariant volume integrals in 3D. For boundary integral equations, we demonstrate that using a QTT decomposition to construct preconditioners for a Krylov subspace method leads to an efficient and robust solver with a small memory footprint. We test the QTT preconditioners in the iterative solution of an exterior elliptic boundary value problem (Laplace) formulated as a boundary integral equation in complex, multiply connected geometries.

[1]  Stephen A. Vavasis,et al.  Preconditioning for Boundary Integral Equations , 1992, SIAM J. Matrix Anal. Appl..

[2]  VLADIMIR A. KAZEEV,et al.  Low-Rank Explicit QTT Representation of the Laplace Operator and Its Inverse , 2012, SIAM J. Matrix Anal. Appl..

[3]  A. Wathen Preconditioning and convergence in the right norm , 2007 .

[4]  Vipin Kumar,et al.  Parallel Hierarchical Solvers and Preconditioners for Boundary Element Methods , 1996, Proceedings of the 1996 ACM/IEEE Conference on Supercomputing.

[5]  Boris N. Khoromskij,et al.  Blended kernel approximation in the ℋ︁‐matrix techniques , 2002, Numer. Linear Algebra Appl..

[6]  L. Greengard,et al.  A Direct Adaptive Poisson Solver of Arbitrary Order Accuracy , 1996 .

[7]  Vladimir Rokhlin,et al.  High-Order Corrected Trapezoidal Quadrature Rules for Singular Functions , 1997 .

[8]  Wolfgang Hackbusch,et al.  Tensorisation of vectors and their efficient convolution , 2011, Numerische Mathematik.

[9]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[10]  Shivkumar Chandrasekaran,et al.  A Fast Solver for HSS Representations via Sparse Matrices , 2006, SIAM J. Matrix Anal. Appl..

[11]  Eugene E. Tyrtyshnikov,et al.  Algebraic Wavelet Transform via Quantics Tensor Train Decomposition , 2011, SIAM J. Sci. Comput..

[12]  Jianlin Xia,et al.  Fast algorithms for hierarchically semiseparable matrices , 2010, Numer. Linear Algebra Appl..

[13]  Eric Darve,et al.  The Inverse Fast Multipole Method: Using a Fast Approximate Direct Solver as a Preconditioner for Dense Linear Systems , 2015, SIAM J. Sci. Comput..

[14]  Leslie Greengard,et al.  On the Numerical Solution of Two-Point Boundary Value Problems , 1991 .

[15]  Marcus J. Grote,et al.  Parallel Preconditioning with Sparse Approximate Inverses , 1997, SIAM J. Sci. Comput..

[16]  Ivan V. Oseledets,et al.  Solution of Linear Systems and Matrix Inversion in the TT-Format , 2012, SIAM J. Sci. Comput..

[17]  Eugene E. Tyrtyshnikov,et al.  Tensor-Train Ranks for Matrices and Their Inverses , 2011, Comput. Methods Appl. Math..

[18]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[19]  T. Chan,et al.  Wavelet sparse approximate inverse preconditioners , 1997 .

[20]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[21]  J. C. Aguilar,et al.  High-order corrected trapezoidal quadrature rules for the coulomb potential in three dimensions , 2005 .

[22]  Vladimir Rokhlin,et al.  On the numerical solution of two-point boundary value problems II , 1994 .

[23]  Boris N. Khoromskij,et al.  Fast Quadrature Techniques for Retarded Potentials Based on TT/QTT Tensor Approximation , 2011, Comput. Methods Appl. Math..

[24]  Shivkumar Chandrasekaran,et al.  A Fast ULV Decomposition Solver for Hierarchically Semiseparable Representations , 2006, SIAM J. Matrix Anal. Appl..

[25]  Eric Darve,et al.  The Inverse Fast Multipole Method , 2014, ArXiv.

[26]  Eugene E. Tyrtyshnikov,et al.  Incomplete Cross Approximation in the Mosaic-Skeleton Method , 2000, Computing.

[27]  George Biros,et al.  On preconditioners for the Laplace double‐layer in 2D , 2013, Numer. Linear Algebra Appl..

[28]  Per-Gunnar Martinsson,et al.  On the Compression of Low Rank Matrices , 2005, SIAM J. Sci. Comput..

[29]  Andrew J. Wathen,et al.  A short survey on preconditioning techniques for large-scale dense complex linear systems in electromagnetics , 2007, Int. J. Comput. Math..

[30]  R. Coifman,et al.  Fast wavelet transforms and numerical algorithms I , 1991 .

[31]  E. Tyrtyshnikov,et al.  TT-cross approximation for multidimensional arrays , 2010 .

[32]  V. Rokhlin,et al.  A fast direct solver for boundary integral equations in two dimensions , 2003 .

[33]  Frank Thomson Leighton,et al.  Preconditioned, Adaptive, Multipole-Accelerated Iterative Methods for Three-Dimensional First-Kind Integral Equations of Potential Theory , 1994, SIAM J. Sci. Comput..

[34]  C. Loan,et al.  Approximation with Kronecker Products , 1992 .

[35]  Eugene E. Tyrtyshnikov,et al.  Tensor ranks for the inversion of tensor-product binomials , 2010, J. Comput. Appl. Math..

[36]  I. V. Oseledets,et al.  Tensors inside of matrices give logarithmic complexity , 2009 .

[37]  Ivan V. Oseledets,et al.  Approximation of 2d˟2d Matrices Using Tensor Decomposition , 2010, SIAM J. Matrix Anal. Appl..

[38]  S. Gratton,et al.  Incremental spectral preconditioners for sequences of linear systems , 2007 .

[39]  Mario Bebendorf,et al.  Hierarchical Matrices: A Means to Efficiently Solve Elliptic Boundary Value Problems , 2008 .

[40]  Leslie Greengard,et al.  A fast algorithm for particle simulations , 1987 .

[41]  Femke Olyslager,et al.  A rank‐revealing preconditioner for the fast integral‐equation‐based characterization of electromagnetic crystal devices , 2006 .

[42]  Ivan Oseledets,et al.  Superfast inversion of two-level Toeplitz matrices using Newton iteration and tensor-displacement structure , 2007 .

[43]  Bruno Carpentieri,et al.  Combining Fast Multipole Techniques and an Approximate Inverse Preconditioner for Large Electromagnetism Calculations , 2005, SIAM J. Sci. Comput..

[44]  R. Kanwal Linear Integral Equations , 1925, Nature.

[45]  Boris N. Khoromskij,et al.  Grid-based lattice summation of electrostatic potentials by assembled rank-structured tensor approximation , 2014, Comput. Phys. Commun..

[46]  Tamara G. Kolda,et al.  Parallel Tensor Compression for Large-Scale Scientific Data , 2015, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[47]  S. Börm Efficient Numerical Methods for Non-local Operators , 2010 .

[48]  Boris N. Khoromskij,et al.  Hierarchical Kronecker tensor-product approximations , 2005, J. Num. Math..

[49]  S. V. Dolgov,et al.  ALTERNATING MINIMAL ENERGY METHODS FOR LINEAR SYSTEMS IN HIGHER DIMENSIONS∗ , 2014 .

[50]  Samuel Williams,et al.  Roofline: an insightful visual performance model for multicore architectures , 2009, CACM.

[51]  E. Tyrtyshnikov,et al.  Tensor properties of multilevel Toeplitz and related matrices , 2006 .

[52]  Gene H. Golub,et al.  Matrix computations , 1983 .

[53]  Philipp Grohs,et al.  FFRT: A Fast Finite Ridgelet Transform for Radiative Transport , 2014, Multiscale Model. Simul..

[54]  Youssef M. Marzouk,et al.  Spectral Tensor-Train Decomposition , 2014, SIAM J. Sci. Comput..

[55]  Per-Gunnar Martinsson,et al.  An O(N) Direct Solver for Integral Equations on the Plane , 2013, 1303.5466.

[56]  Olof Runborg,et al.  Corrected trapezoidal rules for a class of singular functions , 2014 .

[57]  Boris N. Khoromskij,et al.  Approximate iterations for structured matrices , 2008, Numerische Mathematik.

[58]  Ran Duan,et al.  High-order quadratures for the solution of scattering problems in two dimensions , 2009, J. Comput. Phys..

[59]  Henk A. van der Vorst,et al.  Bi-CGSTAB: A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems , 1992, SIAM J. Sci. Comput..

[60]  M. Benzi Preconditioning techniques for large linear systems: a survey , 2002 .

[61]  Piet Hut,et al.  A hierarchical O(N log N) force-calculation algorithm , 1986, Nature.

[62]  Per-Gunnar Martinsson,et al.  On interpolation and integration in finite-dimensional spaces of bounded functions , 2005 .

[63]  V. Rokhlin Rapid solution of integral equations of classical potential theory , 1985 .

[64]  Ivano Benedetti,et al.  A fast 3D dual boundary element method based on hierarchical matrices , 2008 .

[65]  W. Hackbusch,et al.  On the fast matrix multiplication in the boundary element method by panel clustering , 1989 .

[66]  B. Khoromskij Tensor numerical methods for multidimensional PDES: theoretical analysis and initial applications , 2015 .

[67]  Lloyd N. Trefethen,et al.  How Fast are Nonsymmetric Matrix Iterations? , 1992, SIAM J. Matrix Anal. Appl..

[68]  Lexing Ying,et al.  Sparsifying Preconditioner for the Lippmann-Schwinger Equation , 2014, Multiscale Model. Simul..

[69]  Boris N. Khoromskij,et al.  Superfast Fourier Transform Using QTT Approximation , 2012 .

[70]  Per-Gunnar Martinsson,et al.  A direct solver with O(N) complexity for integral equations on one-dimensional domains , 2011, 1105.5372.

[71]  K. Abdella Numerical Solution of two-point boundary value problems using Sinc interpolation , 2012 .

[72]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[73]  B. Khoromskij O(dlog N)-Quantics Approximation of N-d Tensors in High-Dimensional Numerical Modeling , 2011 .

[74]  Bruno Carpentieri,et al.  A Class of Spectral Two-Level Preconditioners , 2003, SIAM J. Sci. Comput..

[75]  Mario Bebendorf,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig Hierarchical Lu Decomposition Based Preconditioners for Bem Hierarchical Lu Decomposition Based Preconditioners for Bem , 2022 .

[76]  Tim Beardsley,et al.  Strategic Defense Initiative: Academicians doubt efficacy , 1986, Nature.

[77]  L. Gürel,et al.  Iterative Near-Field Preconditioner for the Multilevel Fast Multipole Algorithm , 2010, SIAM J. Sci. Comput..

[78]  Leslie Greengard,et al.  A Fast Direct Solver for Structured Linear Systems by Recursive Skeletonization , 2012, SIAM J. Sci. Comput..

[79]  Christoph Schwab,et al.  Approximation of Singularities by Quantized-Tensor FEM ∗ , 2015 .

[80]  George Biros,et al.  Boundary integral method for the flow of vesicles with viscosity contrast in three dimensions , 2015, J. Comput. Phys..

[81]  Adrianna Gillman,et al.  Fast Direct Solvers for Elliptic Partial Differential Equations , 2011 .

[82]  Ian H. Sloan,et al.  Fully discrete spectral boundary integral methods for Helmholtz problems on smooth closed surfaces in ${\mathbb R}^3$ , 2002, Numerische Mathematik.

[83]  Jacob K. White,et al.  Second-kind integral formulations of the capacitance problem , 1996, Adv. Comput. Math..

[84]  Lexing Ying,et al.  Hierarchical Interpolative Factorization for Elliptic Operators: Integral Equations , 2013, 1307.2666.

[85]  L. Greengard,et al.  A new version of the Fast Multipole Method for the Laplace equation in three dimensions , 1997, Acta Numerica.