Algorithmic thresholds for tensor PCA

We study the algorithmic thresholds for principal component analysis of Gaussian $k$-tensors with a planted rank-one spike, via Langevin dynamics and gradient descent. In order to efficiently recover the spike from natural initializations, the signal to noise ratio must diverge in the dimension. Our proof shows that the mechanism for the success/failure of recovery is the strength of the "curvature" of the spike on the maximum entropy region of the initial data. To demonstrate this, we study the dynamics on a generalized family of high-dimensional landscapes with planted signals, containing the spiked tensor models as specific instances. We identify thresholds of signal-to-noise ratios above which order 1 time recovery succeeds; in the case of the spiked tensor model these match the thresholds conjectured for algorithms such as Approximate Message Passing. Below these thresholds, where the curvature of the signal on the maximal entropy region is weak, we show that recovery from certain natural initializations takes at least stretched exponential time. Our approach combines global regularity estimates for spin glasses with point-wise estimates, to study the recovery problem by a perturbative approach.

[1]  I. J. Schoenberg.  Positive definite functions on spheres , 1942 .

[2]  D. W. Stroock,et al.  Multidimensional Diffusion Processes , 1979 .

[3]  p>2 spin glasses with first order ferromagnetic transitions , 1999, cond-mat/9912201.

[4]  I. Johnstone.  On the distribution of the largest principal component , 2000 .

[5]  M. Ledoux.  The concentration of measure phenomenon , 2001 .

[6]  Elton P. Hsu.  Stochastic analysis on manifolds , 2002 .

[7]  J. Wellner,et al.  High Dimensional Probability III , 2003 .

[8]  S. Péché,et al.  Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices , 2004, math/0403022.

[9]  Mylène Maïda,et al.  Large deviations for the largest eigenvalue of rank one deformations of Gaussian ensembles , 2006 .

[10]  V. Koltchinskii,et al.  High Dimensional Probability , 2006, math/0612726.

[11]  S. Péché.  The largest eigenvalue of small rank perturbations of Hermitian random matrices , 2006 .

[12]  C. Donati-Martin,et al.  The largest eigenvalues of finite rank deformation of large Wigner matrices: Convergence and nonuniversality of the fluctuations. , 2007, 0706.0136.

[13]  A. Guionnet,et al.  Large deviations of the extreme eigenvalues of random deformations of matrices , 2010, Probability Theory and Related Fields.

[14]  Antonio Auffinger,et al.  Random Matrices and Complexity of Spin Glasses , 2010, 1003.1129.

[15]  Y. Gliklikh.  Stochastic Analysis on Manifolds , 2011 .

[16]  P. Rigollet,et al.  Optimal detection of sparse principal components in high dimension , 2012, 1202.5070.

[17]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[18]  Antonio Auffinger,et al.  Complexity of random smooth functions on the high-dimensional sphere , 2011, 1110.5872.

[19]  Andrea Montanari,et al.  A statistical model for tensor PCA , 2014, NIPS.

[20]  Jonathan Shi,et al.  Tensor principal component analysis via sum-of-square proofs , 2015, COLT.

[21]  Florent Krzakala,et al.  Statistical physics of inference: thresholds and algorithms , 2015, ArXiv.

[22]  Eliran Subag,et al.  The complexity of spherical p-spin models - a second moment approach , 2015, 1504.02251.

[23]  Tselil Schramm,et al.  Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors , 2015, STOC.

[24]  Ankur Moitra,et al.  Noisy tensor completion via the sum-of-squares hierarchy , 2015, Mathematical Programming.

[25]  Pravesh Kothari,et al.  A Nearly Tight Sum-of-Squares Lower Bound for the Planted Clique Problem , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[26]  Afonso S. Bandeira,et al.  Statistical limits of spiked tensor models , 2016, Annales de l'Institut Henri Poincaré, Probabilités et Statistiques.

[27]  Ankur Moitra,et al.  Optimality and Sub-optimality of PCA for Spiked Random Matrices and Synchronization , 2016, ArXiv.

[28]  Florent Krzakala,et al.  Statistical and computational phase transitions in spiked tensor estimation , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[29]  Andrea Montanari,et al.  On the Limitation of Spectral Methods: From the Gaussian Hidden Clique Problem to Rank One Perturbations of Gaussian Tensors , 2014, IEEE Transactions on Information Theory.

[30]  Michel X. Goemans,et al.  Community detection in hypergraphs, spiked tensor models, and Sum-of-Squares , 2017, 2017 International Conference on Sampling Theory and Applications (SampTA).

[31]  G. Ben Arous,et al.  Spectral Gap Estimates in Mean Field Spin Glasses , 2017, 1705.04243.

[32]  David Gamarnik,et al.  High Dimensional Regression with Binary Coefficients. Estimating Squared Error and a Phase Transtition , 2017, COLT.

[33]  Tengyu Ma,et al.  On the optimization landscape of tensor decompositions , 2017, Mathematical Programming.

[34]  Emmanuel Abbe,et al.  Proof of the Achievability Conjectures for the General Stochastic Block Model , 2018 .

[35]  Reza Gheissari,et al.  On the spectral gap of spherical spin glass dynamics , 2016, Annales de l'Institut Henri Poincaré, Probabilités et Statistiques.

[36]  Reza Gheissari,et al.  Bounding Flows for Spherical Spin Glass Dynamics , 2018, Communications in Mathematical Physics.

[37]  Wei-Kuo Chen,et al.  Phase transition in the spiked random tensor with Rademacher prior , 2017, The Annals of Statistics.

[38]  G. B. Arous,et al.  The Landscape of the Spiked Tensor Model , 2017, Communications on Pure and Applied Mathematics.

[39]  G. Biroli,et al.  Complex Energy Landscapes in Spiked-Tensor and Simple Glassy Models: Ruggedness, Arrangements of Local Minima, and Phase Transitions , 2018, Physical Review X.