Long Random Matrices and Tensor Unfolding

In this paper, we consider the singular values and singular vectors of low rank perturbations of large rectangular random matrices, in the regime the matrix is “long”: we allow the number of rows (columns) to grow polynomially in the number of columns (rows). We prove there exists a critical signal-to-noise ratio (depending on the dimensions of the matrix), and the extreme singular values and singular vectors exhibit a BBP type phase transition. As a main application, we investigate the tensor unfolding algorithm for the asymmetric rank-one spiked tensor model, and obtain an exact threshold, which is independent of the procedure of tensor unfolding. If the signal-to-noise ratio is above the threshold, tensor unfolding detects the signals; otherwise, it fails to capture the signals.

[1]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[2]  Tselil Schramm,et al.  Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors , 2015, STOC.

[3]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[4]  Olivier Ledoit,et al.  Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices , 2011, 1207.5322.

[5]  H. Yau,et al.  Isotropic local laws for sample covariance and generalized Wigner matrices , 2013, 1308.5729.

[6]  Ankur Moitra,et al.  Optimality and Sub-optimality of PCA for Spiked Random Matrices and Synchronization , 2016, ArXiv.

[7]  Anru R. Zhang,et al.  Tensor SVD: Statistical and Computational Limits , 2017, IEEE Transactions on Information Theory.

[8]  Jiaoyang Huang.  Mesoscopic Perturbations of Large Random Matrices , 2014, 1412.4193.

[9]  Dong Wang,et al.  Eigenvector distribution in the critical regime of BBP transition , 2021, Probability Theory and Related Fields.

[10]  G. Biroli,et al.  Complex Energy Landscapes in Spiked-Tensor and Simple Glassy Models: Ruggedness, Arrangements of Local Minima, and Phase Transitions , 2018, Physical Review X.

[11]  T. Cai,et al.  Sparse PCA: Optimal rates and adaptive estimation , 2012, 1211.1309.

[12]  G. B. Arous,et al.  The Landscape of the Spiked Tensor Model , 2017, Communications on Pure and Applied Mathematics.

[13]  Anru R. Zhang,et al.  An Optimal Statistical and Computational Framework for Generalized Tensor Estimation , 2020, The Annals of Statistics.

[14]  Ivan V. Oseledets,et al.  Tensor methods and recommender systems , 2016, Wiley Interdiscip. Rev. Data Min. Knowl. Discov..

[15]  Florent Krzakala,et al.  Statistical and computational phase transitions in spiked tensor estimation , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[16]  Florent Krzakala,et al.  Who is Afraid of Big Bad Minima? Analysis of Gradient-Flow in a Spiked Matrix-Tensor Model , 2019, NeurIPS.

[17]  D. Paul.  ASYMPTOTICS OF SAMPLE EIGENSTRUCTURE FOR A LARGE DIMENSIONAL SPIKED COVARIANCE MODEL , 2007 .

[18]  VandewalleJoos,et al.  On the Best Rank-1 and Rank-(R1,R2,. . .,RN) Approximation of Higher-Order Tensors , 2000 .

[19]  Debashis Paul,et al.  PCA in High Dimensions: An Orientation , 2018, Proceedings of the IEEE.

[20]  Gilad Lerman,et al.  Phase transition in random tensors with multiple spikes , 2018, ArXiv.

[21]  S. Péché,et al.  Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices , 2004, math/0403022.

[22]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[23]  Lars Schmidt-Thieme,et al.  Pairwise interaction tensor factorization for personalized tag recommendation , 2010, WSDM '10.

[24]  Noureddine El Karoui.  Spectrum estimation for large dimensional covariance matrices using random matrix theory , 2006, math/0609418.

[25]  Jonathan Shi,et al.  Tensor principal component analysis via sum-of-square proofs , 2015, COLT.

[26]  Yuetian Luo,et al.  Tensor Clustering with Planted Structures: Statistical Optimality and Computational Limits , 2020, The Annals of Statistics.

[27]  Nuria Oliver,et al.  Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering , 2010, RecSys '10.

[28]  Afonso S. Bandeira,et al.  Statistical limits of spiked tensor models , 2016, Annales de l'Institut Henri Poincaré, Probabilités et Statistiques.

[29]  I. Johnstone,et al.  On Consistency and Sparsity for Principal Components Analysis in High Dimensions , 2009, Journal of the American Statistical Association.

[30]  Federico Ricci-Tersenghi,et al.  How to iron out rough landscapes and get optimal performances: averaged gradient descent and its application to tensor PCA , 2019, Journal of Physics A: Mathematical and Theoretical.

[31]  A. Guionnet,et al.  Fluctuations of the Extreme Eigenvalues of Finite Rank Deformations of Random Matrices , 2010, 1009.0145.

[32]  I. Johnstone.  On the distribution of the largest eigenvalue in principal components analysis , 2001 .

[33]  Andrea Montanari,et al.  A statistical model for tensor PCA , 2014, NIPS.

[34]  J. W. Silverstein,et al.  Eigenvalues of large sample covariance matrices of spiked population models , 2004, math/0408165.

[35]  Raj Rao Nadakuditi,et al.  The singular values and vectors of low rank perturbations of large rectangular random matrices , 2011, J. Multivar. Anal..

[36]  Jianfeng Yao,et al.  On sample eigenvalues in a generalized spiked population model , 2008, J. Multivar. Anal..

[37]  Yuetian Luo,et al.  Open Problem: Average-Case Hardness of Hypergraphic Planted Clique Detection , 2020, COLT 2020.

[38]  Hongtu Zhu,et al.  Tensor Regression with Applications in Neuroimaging Data Analysis , 2012, Journal of the American Statistical Association.

[39]  B. Nadler,et al.  MINIMAX BOUNDS FOR SPARSE PCA WITH NOISY HIGH-DIMENSIONAL DATA. , 2012, Annals of statistics.

[40]  I. Johnstone,et al.  Optimal Shrinkage of Eigenvalues in the Spiked Covariance Model. , 2013, Annals of statistics.

[41]  Afonso S. Bandeira,et al.  Notes on computational-to-statistical gaps: predictions using statistical physics , 2018, Portugaliae Mathematica.

[42]  Daniel Z. Huang,et al.  Power Iteration for Tensor PCA , 2020, J. Mach. Learn. Res..

[43]  W. Hackbusch.  Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[44]  Raj Rao Nadakuditi,et al.  The eigenvalues and eigenvectors of finite, low rank perturbations of large random matrices , 2009, 0910.2120.

[45]  Andrea Montanari,et al.  On the Limitation of Spectral Methods: From the Gaussian Hidden Clique Problem to Rank One Perturbations of Gaussian Tensors , 2014, IEEE Transactions on Information Theory.

[46]  G. B. Arous,et al.  Algorithmic thresholds for tensor PCA , 2018, The Annals of Probability.

[47]  T. Cai,et al.  Optimal estimation and rank detection for sparse spiked covariance matrices , 2013, Probability theory and related fields.

[48]  Pierre Comon,et al.  Tensors : A brief introduction , 2014, IEEE Signal Processing Magazine.

[49]  Aukosh Jagannath,et al.  Statistical thresholds for tensor PCA , 2018, The Annals of Applied Probability.

[50]  Janice Chen,et al.  Dynamic reconfiguration of the default mode network during narrative comprehension , 2016, Nature Communications.

[51]  Wei-Kuo Chen,et al.  Phase transition in the spiked random tensor with Rademacher prior , 2017, The Annals of Statistics.

[52]  Michael J. Feldman,et al.  Spiked singular values and vectors under extreme aspect ratios , 2021, Journal of Multivariate Analysis.

[53]  Michel X. Goemans,et al.  Community detection in hypergraphs, spiked tensor models, and Sum-of-Squares , 2017, 2017 International Conference on Sampling Theory and Applications (SampTA).

[54]  Anru R. Zhang,et al.  A Sharp Blockwise Tensor Perturbation Bound for Orthogonal Iteration , 2020, J. Mach. Learn. Res..

[55]  Zongming Ma.  Sparse Principal Component Analysis and Iterative Thresholding , 2011, 1112.2432.

[56]  Vincent Q. Vu,et al.  MINIMAX SPARSE PRINCIPAL SUBSPACE ESTIMATION IN HIGH DIMENSIONS , 2012, 1211.0373.

[57]  Marcelo J. Moreira,et al.  Asymptotic power of sphericity tests for high-dimensional data , 2013, 1306.4867.