A Krylov-Schur-like method for computing the best rank-(r1,r2,r3) approximation of large and sparse tensors

The paper is concerned with methods for computing the best low multilinear rank approximation of large and sparse tensors. Krylov-type methods have been used for this problem; here block versions are introduced. For the computation of partial eigenvalue and singular value decompositions of matrices the Krylov-Schur (restarted Arnoldi) method is used. We describe a generalization of this method to tensors, for computing the best low multilinear rank approximation of large and sparse tensors. In analogy to the matrix case, the large tensor is only accessed in multiplications between the tensor and blocks of vectors, thus avoiding excessive memory usage. It is proved that, if the starting approximation is good enough, then the tensor Krylov-Schur method is convergent. Numerical examples are given for synthetic tensors and sparse tensors from applications, which demonstrate that for most large problems the Krylov-Schur method converges faster and more robustly than higher order orthogonal iteration.

[1]  Yangyang Xu,et al.  On the convergence of higher-order orthogonal iteration , 2015 .

[2]  Eugene E. Tyrtyshnikov,et al.  Tucker Dimensionality Reduction of Three-Dimensional Arrays in Linear Time , 2008, SIAM J. Matrix Anal. Appl..

[3]  Sabine Van Huffel,et al.  Best Low Multilinear Rank Approximation of Higher-Order Tensors, Based on the Riemannian Trust-Region Scheme , 2011, SIAM J. Matrix Anal. Appl..

[4]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[5]  Tamara G. Kolda,et al.  Categories and Subject Descriptors: G.4 [Mathematics of Computing]: Mathematical Software— , 2022 .

[6]  Berkant Savas,et al.  A Newton-Grassmann Method for Computing the Best Multilinear Rank-(r1, r2, r3) Approximation of a Tensor , 2009, SIAM J. Matrix Anal. Appl..

[7]  B. Khoromskij,et al.  Low rank Tucker-type tensor approximation to classical potentials , 2007 .

[8]  Bart De Moor,et al.  Multiview Partitioning via Tensor Methods , 2013, IEEE Transactions on Knowledge and Data Engineering.

[9]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[10]  Tamara G. Kolda,et al.  Higher-order Web link analysis using multilinear algebra , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[11]  Berkant Savas,et al.  Handwritten digit classification using higher order singular value decomposition , 2007, Pattern Recognit..

[12]  Mason A. Porter,et al.  Comparing Community Structure to Characteristics in Online Collegiate Social Networks , 2008, SIAM Rev..

[13]  Boris N. Khoromskij,et al.  Multigrid Accelerated Tensor Approximation of Function Related Multidimensional Arrays , 2009, SIAM J. Sci. Comput..

[14]  L. Lathauwer,et al.  On the best low multilinear rank approximation of higher-order tensors , 2010 .

[15]  M. J. D. Powell,et al.  On search directions for minimization algorithms , 1973, Math. Program..

[16]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[17]  Sabine Van Huffel,et al.  Differential-geometric Newton method for the best rank-(R1, R2, R3) approximation of tensors , 2008, Numerical Algorithms.

[18]  Berkant Savas,et al.  Krylov-Type Methods for Tensor Computations , 2010, 1005.0683.

[19]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[20]  Rasmus Bro,et al.  Improving the speed of multi-way algorithms:: Part I. Tucker3 , 1998 .

[21]  Martin Rosvall,et al.  Maps of sparse Markov chains efficiently reveal community structure in network flows with memory , 2016, ArXiv.

[22]  Tamara G. Kolda,et al.  Scalable Tensor Decompositions for Multi-aspect Data Mining , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[23]  VandewalleJoos,et al.  On the Best Rank-1 and Rank-(R1,R2,. . .,RN) Approximation of Higher-Order Tensors , 2000 .

[24]  Tamara G. Kolda,et al.  Efficient MATLAB Computations with Sparse and Factored Tensors , 2007, SIAM J. Sci. Comput..

[25]  Bora Uçar,et al.  High Performance Parallel Algorithms for the Tucker Decomposition of Sparse Tensors , 2016, 2016 45th International Conference on Parallel Processing (ICPP).

[26]  Christos Faloutsos,et al.  Mining billion-scale tensors: algorithms and discoveries , 2016, The VLDB Journal.

[27]  Chao Yang,et al.  ARPACK users' guide - solution of large-scale eigenvalue problems with implicitly restarted Arnoldi methods , 1998, Software, environments, tools.

[28]  Pierre Comon,et al.  Decomposition of quantics in sums of powers of linear forms , 1996, Signal Process..

[29]  Berkant Savas,et al.  Perturbation Theory and Optimality Conditions for the Best Multilinear Rank Approximation of a Tensor , 2011, SIAM J. Matrix Anal. Appl..

[30]  Berkant Savas,et al.  Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors , 2009, SIAM J. Sci. Comput..

[31]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[32]  Danny C. Sorensen,et al.  Implicit Application of Polynomial Filters in a k-Step Arnoldi Method , 1992, SIAM J. Matrix Anal. Appl..

[33]  Maryam Dehghan,et al.  Analyzing Large and Sparse Tensor Data using Spectral Low-Rank Approximation , 2020, ArXiv.

[34]  G. Stewart.  Matrix Algorithms, Volume II: Eigensystems , 2001 .

[35]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.

[36]  G. W. Stewart,et al.  A Krylov-Schur Algorithm for Large Eigenproblems , 2001, SIAM J. Matrix Anal. Appl..

[37]  Vladimir Batagelj,et al.  Density based approaches to network analysis Analysis of Reuters terror news network , 2003 .

[38]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[39]  Daniela Calvetti,et al.  Matrix methods in data mining and pattern recognition , 2009, Math. Comput..

[40]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[41]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[42]  Lars Eldén,et al.  Spectral partitioning of large and sparse 3‐tensors using low‐rank tensor approximation , 2020, Numer. Linear Algebra Appl..

[43]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[44]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[45]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[46]  Ivan V. Oseledets,et al.  Wedderburn Rank Reduction and Krylov Subspace Method for Tensor Approximation. Part 1: Tucker Case , 2010, SIAM J. Sci. Comput..

[47]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[48]  Axel Ruhe,et al.  Algorithms for separable nonlinear least squares problems , 1980 .

[49]  Yousef Saad,et al.  Block Krylov–Schur method for large symmetric eigenvalue problems , 2008, Numerical Algorithms.