On Local Convergence of Alternating Schemes for Optimization of Convex Problems in the Tensor Train Format

Alternating linear schemes (ALS), with the alternating least squares algorithm a notable special case, provide one of the simplest and most popular choices for the treatment of optimization tasks by tensor methods. An according adaptation of ALS for the recent TT (= tensor train) format [I. V. Oseledets, SIAM J. Sci. Comput., 33 (2011), pp. 2295--2317], known in quantum computations as matrix product states, has recently been investigated in [S. Holtz, T. Rohwedder, and R. Schneider, SIAM J. Sci. Comput., 34 (2012), pp. A683--A713]. With the present work, the positive practical experience with TT-ALS is backed up with an according local linear convergence theory for the optimization of convex functionals $J$. The main assumption entering the proof is that the redundancy introduced by the TT parametrization $\tau$ matches the null space of the Hessian of the induced functional $j = J \circ \tau$, and we give conditions under which this assumption can be expected to hold. In particular, this is the case if ...

[1]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[2]  Ivan V. Oseledets,et al.  Solution of Linear Systems and Matrix Inversion in the TT-Format , 2012, SIAM J. Sci. Comput..

[3]  J. Ballani,et al.  Black box approximation of tensors in hierarchical Tucker format , 2013 .

[4]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[5]  S. White.  Density matrix renormalization group algorithms with a single center site , 2005, cond-mat/0508709.

[6]  Antonio Falcó,et al.  On Minimal Subspaces in Tensor Representations , 2012, Found. Comput. Math..

[7]  G. Vidal.  Efficient classical simulation of slightly entangled quantum computations. , 2003, Physical review letters.

[8]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[9]  Claude Klöckl,et al.  The density matrix renormalization group on tensor networks , 2013 .

[10]  Johannes Weissinger,et al.  Verallgemeinerungen des Seidelschen Iterationsverfahrens. Herrn R. v. Mises zum 70. Geburtstag gewidmet , 1953 .

[11]  T. Schulte-Herbrüggen,et al.  Exploiting matrix symmetries and physical symmetries in matrix product states and tensor trains , 2013, 1301.0746.

[12]  U. Schollwoeck.  The density-matrix renormalization group in the age of matrix product states , 2010, 1008.3477.

[13]  J I Cirac,et al.  Renormalization-group transformations on quantum states. , 2004, Physical review letters.

[14]  Reinhold Schneider,et al.  Dynamical Approximation by Hierarchical Tucker and Tensor-Train Tensors , 2013, SIAM J. Matrix Anal. Appl..

[15]  White,et al.  Density matrix formulation for quantum renormalization groups. , 1992, Physical review letters.

[16]  Reinhold Schneider,et al.  Optimization problems in contracted tensor networks , 2011, Comput. Vis. Sci..

[17]  W. Hackbusch,et al.  A New Scheme for the Tensor Representation , 2009 .

[18]  Reinhold Schneider,et al.  On manifolds of tensors of fixed TT-rank , 2012, Numerische Mathematik.

[19]  Berkant Savas,et al.  Perturbation Theory and Optimality Conditions for the Best Multilinear Rank Approximation of a Tensor , 2011, SIAM J. Matrix Anal. Appl..

[20]  André Uschmajew,et al.  Local Convergence of the Alternating Least Squares Algorithm for Canonical Tensor Approximation , 2012, SIAM J. Matrix Anal. Appl..

[21]  J. Ortega,et al.  Nonlinear Difference Equations and Gauss-Seidel Type Iterative Methods , 1966 .

[22]  Antonio Falcó,et al.  Proper generalized decomposition for nonlinear convex problems in tensor Banach spaces , 2011, Numerische Mathematik.

[23]  W. Hackbusch.  Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[24]  Dajana Conte,et al.  Mathematical Modelling and Numerical Analysis an Error Analysis of the Multi-configuration Time-dependent Hartree Method of Quantum Dynamics , 2022 .

[25]  E. Tyrtyshnikov,et al.  TT-cross approximation for multidimensional arrays , 2010 .

[26]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[27]  James C. Bezdek,et al.  Convergence of Alternating Optimization , 2003, Neural Parallel Sci. Comput..

[28]  Lars Grasedyck,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig a Projection Method to Solve Linear Systems in Tensor Format a Projection Method to Solve Linear Systems in Tensor Format , 2022 .

[29]  Bart Vandereycken,et al.  The geometry of algorithms using hierarchical tensors , 2013, Linear Algebra and its Applications.

[30]  Daniel Kressner,et al.  Preconditioned Low-Rank Methods for High-Dimensional Elliptic PDE Eigenvalue Problems , 2011, Comput. Methods Appl. Math..

[31]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[32]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[33]  Young-Ju,et al.  CONVERGENCE ANALYSIS ON ITERATIVE METHODS FOR SEMIDEFINITE SYSTEMS , 2008 .

[34]  S. Schechter.  ITERATION METHODS FOR NONLINEAR PROBLEMS , 1962 .

[35]  H. Keller.  On the Solution of Singular and Semidefinite Linear Systems by Iteration , 1965 .

[36]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[37]  Reinhold Schneider,et al.  Variational calculus with sums of elementary tensors of fixed rank , 2012, Numerische Mathematik.

[38]  Jinchao Xu,et al.  On the Convergence of Iterative Methods for Semidefinite Linear Systems , 2006, SIAM J. Matrix Anal. Appl..

[39]  E. Hairer,et al.  Geometric Numerical Integration: Structure Preserving Algorithms for Ordinary Differential Equations , 2004 .

[40]  B. Khoromskij,et al.  DMRG+QTT approach to computation of the ground state for the molecular Schrödinger operator , 2010 .

[41]  Reinhold Schneider,et al.  The Alternating Linear Scheme for Tensor Optimization in the Tensor Train Format , 2012, SIAM J. Sci. Comput..

[42]  J. Dieudonne.  Treatise on Analysis , 1969 .

[43]  Wolfgang Hackbusch,et al.  An Introduction to Hierarchical (H-) Rank and TT-Rank of Tensors with Examples , 2011, Comput. Methods Appl. Math..

[44]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[45]  Othmar Koch,et al.  Dynamical Low-Rank Approximation , 2007, SIAM J. Matrix Anal. Appl..

[46]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.