A regularized Newton method for the efficient approximation of tensors represented in the canonical tensor format

In the present survey, we consider a rank approximation algorithm for tensors represented in the canonical format in arbitrary pre-Hilbert tensor product spaces. It is shown that the original approximation problem is equivalent to a finite dimensional ℓ2 minimization problem. The ℓ2 minimization problem is solved by a regularized Newton method which requires the computation and evaluation of the first and second derivative of the objective function. A systematic choice of the initial guess for the iterative scheme is introduced. The effectiveness of the approach is demonstrated in numerical experiments.

[1]  Peter Kosmol,et al.  Methoden zur numerischen Behandlung nichtlinearer Gleichungen und Optimierungsaufgaben , 1989 .

[2]  H. Schwetlick.  Numerische Lösung nichtlinearer Gleichungen , 1978 .

[3]  Wolfgang Hackbusch,et al.  Canonical Tensor Products as a Generalization of Gaussian-type Orbitals , 2010 .

[4]  Dietrich Braess,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig Approximation of 1/x by Exponential Sums in [1, ∞) , 2022 .

[5]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[6]  Martin J. Mohlenkamp,et al.  Numerical operator calculus in higher dimensions , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[7]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[8]  S. D. Cutkosky,et al.  Multilinear Algebra , 2019, Differential Forms.

[9]  Martin J. Mohlenkamp,et al.  Algorithms for Numerical Analysis in High Dimensions , 2005, SIAM J. Sci. Comput..

[10]  T. Gasser,et al.  Nonparametric Density Estimation under Unimodality and Monotonicity Constraints , 1999 .

[11]  I. V. Oseledets,et al.  Minimization methods for approximating tensors and their comparison , 2006 .

[12]  Wolfgang Hackbusch,et al.  Tensor product approximation with optimal rank in quantum chemistry. , 2007, The Journal of chemical physics.

[13]  Boris N. Khoromskij,et al.  Approximate iterations for structured matrices , 2008, Numerische Mathematik.

[14]  W. Rheinboldt.  Schwetlick H., Numerische Lösung nichtlinearer Gleichungen. Berlin VEB Deutscher Verlag der Wissenschaften 1979. 346 S., 21 Abb., M 68,— , 1980 .

[15]  P. Paatero.  A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[16]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[17]  P. Paatero.  The Multilinear Engine—A Table-Driven, Least Squares Program for Solving Multilinear Problems, Including the n-Way Parallel Factor Analysis Model , 1999 .