Step-truncation integrators for evolution equations on low-rank tensor manifolds

We develop a new class of algorithms, which we call step-truncation methods, to integrate in time an initial value problem for an ODE or a PDE on a low-rank tensor manifold. The new methods are based on performing a time step with a conventional time-stepping scheme followed by a truncation operation into a tensor manifold with prescribed rank. By considering such truncation operation as a nonlinear operator in the space of tensors, we prove various consistency results and errors estimates for a wide range of step-truncation algorithms. In particular, we establish consistency between the best step-truncation method and the best tangent space projection integrator via perturbation analysis. Numerical applications are presented and discussed for a Fokker-Planck equation on a torus of dimension two and four.

[1]  G. Karniadakis,et al.  A computable evolution equation for the joint response-excitation probability density function of stochastic dynamical systems , 2012, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[2]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[3]  Pierre F. J. Lermusiaux,et al.  Dynamically orthogonal field equations for continuous stochastic dynamical systems , 2009 .

[4]  Felix J. Herrmann,et al.  Optimization on the Hierarchical Tucker manifold – Applications to tensor completion , 2014, Linear Algebra and its Applications.

[5]  C. Lubich,et al.  A projector-splitting integrator for dynamical low-rank approximation , 2013, BIT Numerical Mathematics.

[6]  Valeriĭ Isaakovich Kli︠a︡t︠s︡kin.  Dynamics of stochastic systems , 2005 .

[7]  Lars Grasedyck,et al.  Distributed hierarchical SVD in the Hierarchical Tucker format , 2018, Numer. Linear Algebra Appl..

[8]  Reinhold Schneider,et al.  Approximation rates for the hierarchical tensor format in periodic Sobolev spaces , 2014, J. Complex..

[9]  Daniele Venturi,et al.  Stability analysis of hierarchical tensor methods for time-dependent PDEs , 2019, J. Comput. Phys..

[10]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[11]  Erich Novak,et al.  High dimensional polynomial interpolation on sparse grids , 2000, Adv. Comput. Math..

[12]  Stanley Osher,et al.  Unnormalized optimal transport , 2019, J. Comput. Phys..

[13]  Paris Perdikaris,et al.  Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data , 2019, J. Comput. Phys..

[14]  Othmar Koch,et al.  Dynamical Tensor Approximation , 2010, SIAM J. Matrix Anal. Appl..

[15]  B. Khoromskij.  Tensor numerical methods for multidimensional PDES: theoretical analysis and initial applications , 2015 .

[16]  C. Villani.  Optimal Transport: Old and New , 2008 .

[17]  Reinhold Schneider,et al.  Tensor Networks and Hierarchical Tensors for the Solution of High-Dimensional Partial Differential Equations , 2016, Foundations of Computational Mathematics.

[18]  Daniele Venturi,et al.  Dynamic tensor approximation of high-dimensional nonlinear PDEs , 2020, J. Comput. Phys..

[19]  Thomas Y. Hou,et al.  A dynamically bi-orthogonal method for time-dependent stochastic partial differential equations I: Derivation and algorithms , 2013, J. Comput. Phys..

[20]  Othmar Koch,et al.  Dynamical Low-Rank Approximation , 2007, SIAM J. Matrix Anal. Appl..

[21]  Albert Cohen,et al.  High-Dimensional Adaptive Sparse Polynomial Interpolation and Applications to Parametric PDEs , 2013, Foundations of Computational Mathematics.

[22]  Levon Nurbekyan,et al.  A machine learning framework for solving high-dimensional mean field game and mean field control problems , 2020, Proceedings of the National Academy of Sciences.

[23]  Jan S. Hesthaven,et al.  Spectral Methods for Time-Dependent Problems: Contents , 2007 .

[24]  André Uschmajew,et al.  On Local Convergence of Alternating Schemes for Optimization of Convex Problems in the Tensor Train Format , 2013, SIAM J. Numer. Anal..

[25]  Daniele Venturi,et al.  Spectral methods for nonlinear functionals and functional differential equations , 2020, Research in the Mathematical Sciences.

[26]  George E. Karniadakis,et al.  Multi-element probabilistic collocation method in high dimensions , 2010, J. Comput. Phys..

[27]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[28]  Daniele Venturi,et al.  Dynamically orthogonal tensor methods for high-dimensional nonlinear PDEs , 2019, J. Comput. Phys..

[29]  BachmayrMarkus,et al.  Tensor Networks and Hierarchical Tensors for the Solution of High-Dimensional Partial Differential Equations , 2016 .

[30]  George Em Karniadakis,et al.  PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs , 2019, ArXiv.

[31]  Colin B. Macdonald,et al.  Calculus on Surfaces with General Closest Point Functions , 2012, SIAM J. Numer. Anal..

[32]  G. Stewart.  Perturbation theory for the singular value decomposition , 1990 .

[33]  J. Strikwerda.  Finite Difference Schemes and Partial Differential Equations , 1989 .

[34]  Reinhold Schneider,et al.  On manifolds of tensors of fixed TT-rank , 2012, Numerische Mathematik.

[35]  D Venturi,et al.  Convolutionless Nakajima–Zwanzig equations for stochastic analysis in nonlinear dynamical systems , 2014, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[36]  Reinhold Schneider,et al.  Dynamical Approximation by Hierarchical Tucker and Tensor-Train Tensors , 2013, SIAM J. Matrix Anal. Appl..

[37]  D. Venturi.  The Numerical Approximation of Nonlinear Functionals and Functional Differential Equations , 2016, 1604.05250.

[38]  Fabio Nobile,et al.  Error Analysis of the Dynamically Orthogonal Approximation of Time Dependent Random PDEs , 2015, SIAM J. Sci. Comput..

[39]  H. Bungartz,et al.  Sparse grids , 2004, Acta Numerica.

[40]  Ed Anderson,et al.  LAPACK Users' Guide , 1995 .

[41]  Daniel Kressner,et al.  Algorithm 941 , 2014 .

[42]  Lars Karlsson,et al.  Parallel algorithms for tensor completion in the CP format , 2016, Parallel Comput..

[43]  Akil C. Narayan,et al.  Adaptive Leja Sparse Grid Constructions for Stochastic Collocation and High-Dimensional Approximation , 2014, SIAM J. Sci. Comput..

[44]  E Weinan,et al.  A mean-field optimal control formulation of deep learning , 2018, Research in the Mathematical Sciences.

[45]  Bart Vandereycken,et al.  The geometry of algorithms using hierarchical tensors , 2013, Linear Algebra and its Applications.

[46]  Daniel M. Tartakovsky,et al.  Parallel tensor methods for high-dimensional linear PDEs , 2018, J. Comput. Phys..

[47]  W. Hackbusch.  Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[48]  George E. Karniadakis,et al.  Hidden physics models: Machine learning of nonlinear partial differential equations , 2017, J. Comput. Phys..

[49]  Xiaoli Ma,et al.  First-Order Perturbation Analysis of Singular Vectors in Singular Value Decomposition , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.