Variational calculus with sums of elementary tensors of fixed rank

In this article we introduce a calculus of variations for sums of elementary tensors and apply it to functionals of practical interest. The survey provides all necessary ingredients for applying minimization methods in a general setting. The important cases of target functionals which are linear and quadratic with respect to the tensor product are discussed, and combinations of these functionals are presented in detail. As an example, we consider the solution of a linear system in structured tensor format. Moreover, we discuss the solution of an eigenvalue problem with sums of elementary tensors. This example can be viewed as a prototype of a constrained minimization problem. For the numerical treatment, we suggest a method which has the same order of complexity as the popular alternating least square algorithm and demonstrate the rate of convergence in numerical tests.

[1]  Othmar Koch,et al.  Dynamical Tensor Approximation , 2010, SIAM J. Matrix Anal. Appl..

[2]  Berkant Savas,et al.  Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors , 2009, SIAM J. Sci. Comput..

[3]  W. Hackbusch,et al.  Black Box Low Tensor-Rank Approximation Using Fiber-Crosses , 2009 .

[4]  Dietrich Braess,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig Approximation of 1/x by Exponential Sums in [1, ∞) , 2022 .

[5]  J. Leeuw,et al.  Principal component analysis of three-mode data by means of alternating least squares algorithms , 1980 .

[6]  Martin J. Mohlenkamp,et al.  Preliminary results on approximating a wavefunction as an unconstrained sum of Slater determinants , 2007 .

[7]  Sabine Van Huffel,et al.  Best Low Multilinear Rank Approximation of Higher-Order Tensors, Based on the Riemannian Trust-Region Scheme , 2011, SIAM J. Matrix Anal. Appl..

[8]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[9]  Lars Grasedyck,et al.  Existence and Computation of Low Kronecker-Rank Approximations for Large Linear Systems of Tensor Product Structure , 2004, Computing.

[10]  W. Greub.  Linear Algebra , 1981 .

[11]  Berkant Savas,et al.  A Newton-Grassmann Method for Computing the Best Multilinear Rank-(r1, r2, r3) Approximation of a Tensor , 2009, SIAM J. Matrix Anal. Appl..

[12]  Boris N. Khoromskij,et al.  Approximate iterations for structured matrices , 2008, Numerische Mathematik.

[13]  Wolfgang Hackbusch,et al.  A regularized Newton method for the efficient approximation of tensors represented in the canonical tensor format , 2012, Numerische Mathematik.

[14]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[15]  Antonio Falcó,et al.  On Minimal Subspaces in Tensor Representations , 2012, Found. Comput. Math..

[16]  H. Neudecker,et al.  An approach ton-mode components analysis , 1986 .

[17]  Martin J. Mohlenkamp,et al.  Numerical operator calculus in higher dimensions , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Ivan P. Gavrilyuk,et al.  Hierarchical Tensor-Product Approximation to the Inverse and Related Operators for High-Dimensional Elliptic Problems , 2004, Computing.

[19]  Martin J. Mohlenkamp,et al.  Algorithms for Numerical Analysis in High Dimensions , 2005, SIAM J. Sci. Comput..

[20]  Boris N. Khoromskij,et al.  Low-rank Kronecker-product Approximation to Multi-dimensional Nonlocal Operators. Part II. HKT Representation of Certain Operators , 2005, Computing.

[21]  André Uschmajew,et al.  Well-posedness of convex maximization problems on Stiefel manifolds and orthogonal tensor product approximations , 2010, Numerische Mathematik.

[22]  Reinhold Schneider,et al.  The Alternating Linear Scheme for Tensor Optimization in the Tensor Train Format , 2012, SIAM J. Sci. Comput..

[23]  Othmar Koch,et al.  Dynamical Low-Rank Approximation , 2007, SIAM J. Matrix Anal. Appl..

[24]  T. Yokonuma.  Tensor Spaces and Exterior Algebra , 1992 .

[25]  Boris N. Khoromskij,et al.  Low-rank Kronecker-product Approximation to Multi-dimensional Nonlocal Operators. Part I. Separable Approximation of Multi-variate Functions , 2005, Computing.

[26]  W. Hackbusch,et al.  A New Scheme for the Tensor Representation , 2009 .

[27]  Ivan Oseledets,et al.  A new tensor decomposition , 2009 .

[28]  Reinhold Schneider,et al.  On manifolds of tensors of fixed TT-rank , 2012, Numerische Mathematik.

[29]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.