Low-Rank Tucker Approximation of a Tensor From Streaming Data

This paper describes a new algorithm for computing a low-Tucker-rank approximation of a tensor. The method applies a randomized linear map to the tensor to obtain a sketch that captures the important directions within each mode, as well as the interactions among the modes. The sketch can be extracted from streaming or distributed data or with a single pass over the tensor, and it uses storage proportional to the degrees of freedom in the output Tucker approximation. The algorithm does not require a second pass over the tensor, although it can exploit another view to compute a superior approximation. The paper provides a rigorous theoretical guarantee on the approximation error. Extensive numerical experiments show that that the algorithm produces useful results that improve on the state of the art for streaming Tucker decomposition.

[1]  David P. Woodruff,et al.  Turnstile streaming algorithms might as well be linear sketches , 2014, STOC.

[2]  Bora Uçar,et al.  High Performance Parallel Algorithms for the Tucker Decomposition of Sparse Tensors , 2016, 2016 45th International Conference on Parallel Processing (ICPP).

[3]  David P. Woodruff Sketching as a Tool for Numerical Linear Algebra , 2014, Found. Trends Theor. Comput. Sci..

[4]  Charalampos E. Tsourakakis MACH: Fast Randomized Tensor Decompositions , 2009, SDM.

[5]  Madeleine Udell,et al.  Tensor Random Projection for Low Memory Dimension Reduction , 2021, ArXiv.

[6]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[7]  M. Rudelson Row products of random matrices , 2011, 1102.1947.

[8]  Christos Boutsidis,et al.  Improved Matrix Algorithms via the Subsampled Randomized Hadamard Transform , 2012, SIAM J. Matrix Anal. Appl..

[9]  Guillaume Blanquart,et al.  Differential diffusion effects, distributed burning, and local extinctions in high Karlovitz premixed flames , 2015 .

[10]  Andrzej Cichocki,et al.  Decomposition of Big Tensors With Low Multilinear Rank , 2014, ArXiv.

[11]  Bernard Chazelle,et al.  The Fast Johnson--Lindenstrauss Transform and Approximate Nearest Neighbors , 2009, SIAM J. Comput..

[12]  Stephen Becker,et al.  Low-Rank Tucker Decomposition of Large Tensors Using TensorSketch , 2018, NeurIPS.

[13]  Moses Charikar,et al.  Finding frequent items in data streams , 2004, Theor. Comput. Sci..

[14]  S. Muthukrishnan,et al.  Data streams: algorithms and applications , 2005, SODA '03.

[15]  Tamara G. Kolda,et al.  Scalable Tensor Decompositions for Multi-aspect Data Mining , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[16]  Volkan Cevher,et al.  Streaming Low-Rank Matrix Approximation with an Application to Scientific Simulation , 2019, SIAM J. Sci. Comput..

[17]  David P. Woodruff,et al.  Low rank approximation and regression in input sparsity time , 2013, STOC '13.

[18]  Tamara G. Kolda,et al.  A Practical Randomized CP Tensor Decomposition , 2017, SIAM J. Matrix Anal. Appl..

[19]  Peter Lindstrom,et al.  TTHRESH: Tensor Compression for Multidimensional Visual Data , 2018, IEEE Transactions on Visualization and Computer Graphics.

[20]  Demetri Terzopoulos,et al.  Multilinear Analysis of Image Ensembles: TensorFaces , 2002, ECCV.

[21]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[22]  Dimitris Achlioptas,et al.  Database-friendly random projections: Johnson-Lindenstrauss with binary coins , 2003, J. Comput. Syst. Sci..

[23]  V. Rokhlin,et al.  A randomized algorithm for the approximation of matrices , 2006 .

[24]  Joel A. Tropp,et al.  Universality laws for randomized dimension reduction, with applications , 2015, ArXiv.

[25]  Tamara G. Kolda,et al.  Parallel Tensor Compression for Large-Scale Scientific Data , 2015, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[26]  Andrzej Cichocki,et al.  Tensor Decompositions: A New Concept in Brain Data Analysis? , 2013, ArXiv.

[27]  Benoît Meister,et al.  Efficient and scalable computations with sparse tensors , 2012, 2012 IEEE Conference on High Performance Extreme Computing.

[28]  Raf Vandebril,et al.  A New Truncation Strategy for the Higher-Order Singular Value Decomposition , 2012, SIAM J. Sci. Comput..

[29]  David P. Woodruff,et al.  D ec 2 01 7 Sketching for Kronecker Product Regression and P-splines Huaian , 2018 .

[30]  Yangyang Xu,et al.  On the convergence of higher-order orthogonal iteration , 2015 .

[31]  Alexander J. Smola,et al.  Fast and Guaranteed Tensor Decomposition via Sketching , 2015, NIPS.

[32]  Joel A. Tropp,et al.  Improved Analysis of the subsampled Randomized Hadamard Transform , 2010, Adv. Data Sci. Adapt. Anal..

[33]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[34]  Sanjeev Arora,et al.  Computational Complexity: A Modern Approach , 2009 .

[35]  Volkan Cevher,et al.  Practical Sketching Algorithms for Low-Rank Matrix Approximation , 2016, SIAM J. Matrix Anal. Appl..

[36]  Kenneth Ward Church,et al.  Very sparse random projections , 2006, KDD '06.

[37]  Jimeng Sun,et al.  An input-adaptive and in-place approach to dense tensor-times-matrix multiply , 2015, SC15: International Conference for High Performance Computing, Networking, Storage and Analysis.

[38]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[39]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[40]  V. Rokhlin,et al.  A fast randomized algorithm for the approximation of matrices ✩ , 2007 .

[41]  Philip S. Yu,et al.  Incremental tensor analysis: Theory and applications , 2008, TKDD.