An Adaptive Algebraic Multigrid Algorithm for Low-Rank Canonical Tensor Decomposition

A new algorithm based on algebraic multigrid is presented for computing the rank-$R$ canonical decomposition of a tensor for small $R$. Standard alternating least squares (ALS) is used as the relaxation method. Transfer operators and coarse-level tensors are constructed in an adaptive setup phase that combines multiplicative correction and bootstrap algebraic multigrid. An accurate solution is computed by an additive solve phase based on the full approximation scheme. Numerical tests show that for certain test problems our multilevel method significantly outperforms standalone ALS when a high level of accuracy is required.

[1]  Achi Brandt,et al.  Bootstrap AMG , 2011, SIAM J. Sci. Comput..

[2]  Ralf Hiptmair,et al.  Analysis of tensor product multigrid , 2001, Numerical Algorithms.

[3]  Boris N. Khoromskij,et al.  Multigrid Accelerated Tensor Approximation of Function Related Multidimensional Arrays , 2009, SIAM J. Sci. Comput..

[4]  H. Keller.  On the Solution of Singular and Semidefinite Linear Systems by Iteration , 1965 .

[5]  Achi Brandt,et al.  A Bootstrap Algebraic Multilevel Method for Markov Chains , 2010, SIAM J. Sci. Comput..

[6]  Boris N. Khoromskij.  On tensor approximation of Green iterations for Kohn-Sham equations , 2008 .

[7]  Achi Brandt,et al.  Efficient Multilevel Eigensolvers with Applications to Data Analysis Tasks , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[9]  Irad Yavneh,et al.  On-the-Fly Adaptive Smoothed Aggregation Multigrid for Markov Chains , 2011, SIAM J. Sci. Comput..

[10]  T. Manteuffel,et al.  Adaptive Smoothed Aggregation ( α SA ) Multigrid ∗ , 2005 .

[11]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[12]  A. Borzì,et al.  Algebraic multigrid methods for solving generalized eigenvalue problems , 2006 .

[13]  G. Styan.  Hadamard products and multivariate statistical analysis , 1973 .

[14]  D. Brandt,et al.  Multi-level adaptive solutions to boundary-value problems math comptr , 1977 .

[15]  Boris N. Khoromskij,et al.  Tensor-Structured Preconditioners and Approximate Inverse of Elliptic Operators in ℝd , 2009 .

[16]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[17]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[18]  Eugene E. Tyrtyshnikov,et al.  Tucker Dimensionality Reduction of Three-Dimensional Arrays in Linear Time , 2008, SIAM J. Matrix Anal. Appl..

[19]  Hans De Sterck,et al.  A Nonlinear GMRES Optimization Algorithm for Canonical Tensor Decomposition , 2011, SIAM J. Sci. Comput..

[20]  Mariya Ishteva.  Numerical Methods for the Best Low Multilinear Rank Approximation of Higher-Order Tensors (Numerieke methoden voor de beste lage multilineaire rang benadering van hogere-orde tensoren) , 2009 .

[21]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[22]  Thomas A. Manteuffel,et al.  Adaptive Algebraic Multigrid , 2005, SIAM J. Sci. Comput..

[23]  Joos Vandewalle,et al.  Computation of the Canonical Decomposition by Means of a Simultaneous Generalized Schur Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[24]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[25]  Hans De Sterck,et al.  A Self-Learning Algebraic Multigrid Method for Extremal Singular Triplets and Eigenpairs , 2011, SIAM J. Sci. Comput..

[26]  T. Kolda.  Multilinear operators for higher-order decompositions , 2006 .

[27]  Matemática,et al.  Society for Industrial and Applied Mathematics , 2010 .

[28]  I. V. Oseledets,et al.  Minimization methods for approximating tensors and their comparison , 2006 .

[29]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[30]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part III: Alternating Least Squares Algorithms , 2008, SIAM J. Matrix Anal. Appl..

[31]  Lars Grasedyck,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig a Projection Method to Solve Linear Systems in Tensor Format a Projection Method to Solve Linear Systems in Tensor Format , 2022 .

[32]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[33]  Irad Yavneh,et al.  Fast multilevel methods for Markov chains , 2011, Numer. Linear Algebra Appl..

[34]  Thomas A. Manteuffel,et al.  Adaptive Smoothed Aggregation (AlphaSA) Multigrid , 2005, SIAM Rev..

[35]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[36]  Daniel M. Dunlavy,et al.  A scalable optimization approach for fitting canonical tensor decompositions , 2011 .