Randomized algorithms for the low multilinear rank approximations of tensors

In this paper, we focus on developing randomized algorithms for the computation of low multilinear rank approximations of tensors based on the random projection and the singular value decomposition. Following the theory of the singular values of sub-Gaussian matrices, we make a probabilistic analysis for the error bounds for the randomized algorithm. We demonstrate the effectiveness of proposed algorithms via several numerical examples.

[1]  Gene H. Golub,et al.  Numerical methods for solving linear least squares problems , 1965, Milestones in Matrix Computation.

[2]  Eugene E. Tyrtyshnikov,et al.  Cross approximation in tensor electron density computations , 2010, Numer. Linear Algebra Appl..

[3]  Petros Drineas,et al.  Tensor-CUR Decompositions for Tensor-Based Data , 2008, SIAM J. Matrix Anal. Appl..

[4]  Raf Vandebril,et al.  A New Truncation Strategy for the Higher-Order Singular Value Decomposition , 2012, SIAM J. Sci. Comput..

[5]  Gregory Beylkin,et al.  Randomized interpolative decomposition of separated representations , 2013, J. Comput. Phys..

[6]  Yimin Wei,et al.  Randomized algorithms for the approximations of Tucker and the tensor train decompositions , 2018, Advances in Computational Mathematics.

[7]  P WoodruffDavid.  Sketching as a Tool for Numerical Linear Algebra , 2014 .

[8]  Christos Boutsidis,et al.  An improved approximation algorithm for the column subset selection problem , 2008, SODA.

[9]  Michael W. Mahoney,et al.  A randomized algorithm for a tensor-based generalization of the singular value decomposition , 2007 .

[10]  Charalampos E. Tsourakakis.  MACH: Fast Randomized Tensor Decompositions , 2009, SDM.

[11]  C. Pan.  On the existence and computation of rank-revealing LU factorizations , 2000 .

[12]  Arvind K. Saibaba,et al.  HOID: Higher Order Interpolatory Decomposition for Tensors Based on Tucker Representation , 2015, SIAM J. Matrix Anal. Appl..

[13]  VandewalleJoos,et al.  On the Best Rank-1 and Rank-(R1,R2,. . .,RN) Approximation of Higher-Order Tensors , 2000 .

[14]  C. Chui,et al.  Article in Press Applied and Computational Harmonic Analysis a Randomized Algorithm for the Decomposition of Matrices , 2022 .

[15]  Yimin Wei,et al.  Theory and Computation of Complex Tensors and its Applications , 2020 .

[16]  Amir Averbuch,et al.  Randomized LU Decomposition , 2013, ArXiv.

[17]  Berkant Savas,et al.  Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors , 2009, SIAM J. Sci. Comput..

[18]  C. Pan,et al.  Rank-Revealing QR Factorizations and the Singular Value Decomposition , 1992 .

[19]  T. Chan.  Rank revealing QR factorizations , 1987 .

[20]  V. Rokhlin,et al.  A randomized algorithm for the approximation of matrices , 2006 .

[21]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[22]  Yang Guo,et al.  Low-Rank Tucker Approximation of a Tensor From Streaming Data , 2019, SIAM J. Math. Data Sci..

[23]  Xingyu Wang,et al.  Fast nonnegative tensor factorization based on accelerated proximal gradient and low-rank approximation , 2016, Neurocomputing.

[24]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[25]  Andrzej Cichocki,et al.  Fast Nonnegative Matrix/Tensor Factorization Based on Low-Rank Approximation , 2012, IEEE Transactions on Signal Processing.

[26]  Arvind K. Saibaba,et al.  Randomized algorithms for low-rank tensor decompositions in the Tucker format , 2019, SIAM J. Math. Data Sci..

[27]  Lieven De Lathauwer,et al.  Low multilinear rank tensor approximation via semidefinite programming , 2009, 2009 17th European Signal Processing Conference.

[28]  L. Lathauwer,et al.  On the best low multilinear rank approximation of higher-order tensors , 2010 .

[29]  Andrzej Cichocki,et al.  Decomposition of Big Tensors With Low Multilinear Rank , 2014, ArXiv.

[30]  Nico Vervliet,et al.  Breaking the Curse of Dimensionality Using Decompositions of Incomplete Tensors: Tensor-based scientific computing in big data analysis , 2014, IEEE Signal Processing Magazine.

[31]  P. Tang,et al.  Bounds on Singular Values Revealed by QR Factorizations , 1999 .

[32]  Andrzej Cichocki,et al.  Efficient Nonnegative Tucker Decompositions: Algorithms and Uniqueness , 2014, IEEE Transactions on Image Processing.

[33]  Berkant Savas,et al.  Handwritten digit classification using higher order singular value decomposition , 2007, Pattern Recognit..

[34]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[35]  Andrzej Cichocki,et al.  Randomized Algorithms for Computation of Tucker Decomposition and Higher Order SVD (HOSVD) , 2021, IEEE Access.

[36]  Ming Gu,et al.  Efficient Algorithms for Computing a Strong Rank-Revealing QR Factorization , 1996, SIAM J. Sci. Comput..

[37]  Berkant Savas,et al.  A Newton-Grassmann Method for Computing the Best Multilinear Rank-(r1, r2, r3) Approximation of a Tensor , 2009, SIAM J. Matrix Anal. Appl..

[38]  Petros Drineas,et al.  FAST MONTE CARLO ALGORITHMS FOR MATRICES II: COMPUTING A LOW-RANK APPROXIMATION TO A MATRIX∗ , 2004 .

[39]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[40]  Eugene E. Tyrtyshnikov,et al.  Tucker Dimensionality Reduction of Three-Dimensional Arrays in Linear Time , 2008, SIAM J. Matrix Anal. Appl..

[41]  Sabine Van Huffel,et al.  Best Low Multilinear Rank Approximation of Higher-Order Tensors, Based on the Riemannian Trust-Region Scheme , 2011, SIAM J. Matrix Anal. Appl..

[42]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[43]  Pierre Comon,et al.  Tensor Decompositions, State of the Art and Applications , 2002 .

[44]  M. Rudelson,et al.  The smallest singular value of a random rectangular matrix , 2008, 0802.3956.

[45]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[46]  Nico Vervliet,et al.  A Randomized Block Sampling Approach to Canonical Polyadic Decomposition of Large-Scale Tensors , 2016, IEEE Journal of Selected Topics in Signal Processing.

[47]  V. Rokhlin,et al.  A fast randomized algorithm for the approximation of matrices ✩ , 2007 .

[48]  S. Goreinov,et al.  The maximum-volume concept in approximation by low-rank matrices , 2001 .

[49]  Santosh S. Vempala,et al.  Matrix approximation and projective clustering via volume sampling , 2006, SODA '06.

[50]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[51]  Sjsu ScholarWorks,et al.  Rank revealing QR factorizations , 2014 .

[52]  Trac D. Tran,et al.  Tensor sparsification via a bound on the spectral norm of random tensors , 2010, ArXiv.

[53]  Michael W. Mahoney.  Randomized Algorithms for Matrices and Data , 2011, Found. Trends Mach. Learn..

[54]  Michael P. Friedlander,et al.  Computing non-negative tensor factorizations , 2008, Optim. Methods Softw..

[55]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[56]  Danny C. Sorensen,et al.  A DEIM Induced CUR Factorization , 2014, SIAM J. Sci. Comput..

[57]  A. Cichocki,et al.  Generalizing the column–row matrix decomposition to multi-way arrays , 2010 .

[58]  Omar Rivasplata,et al.  Smallest singular value of sparse random matrices , 2011, 1106.0938.

[59]  Daniel Kressner,et al.  Recompression of Hadamard Products of Tensors in Tucker Format , 2017, SIAM J. Sci. Comput..

[60]  Berkant Savas,et al.  Krylov-Type Methods for Tensor Computations , 2010, 1005.0683.

[61]  P. Alam.  ‘A’ , 2021, Composites Engineering: An A–Z Guide.

[62]  M. Rudelson,et al.  Smallest singular value of random matrices and geometry of random polytopes , 2005 .

[63]  Xuezhong Wang,et al.  Tensor neural network models for tensor singular value decompositions , 2020, Computational Optimization and Applications.

[64]  Tamara G. Kolda,et al.  A Practical Randomized CP Tensor Decomposition , 2017, SIAM J. Matrix Anal. Appl..

[65]  Ivan V. Oseledets,et al.  Wedderburn Rank Reduction and Krylov Subspace Method for Tensor Approximation. Part 1: Tucker Case , 2010, SIAM J. Sci. Comput..

[66]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[67]  G. W. Stewart,et al.  Four algorithms for the the efficient computation of truncated pivoted QR approximations to a sparse matrix , 1999, Numerische Mathematik.