Spectral methods from tensor networks

A tensor network is a diagram that specifies a way to ``multiply'' a collection of tensors together to produce another tensor (or matrix). Many existing algorithms for tensor problems (such as tensor decomposition and tensor PCA), although they are not presented this way, can be viewed as spectral methods on matrices built from simple tensor networks. In this work we leverage the full power of this abstraction to design new algorithms for certain continuous tensor decomposition problems. An important and challenging family of tensor problems comes from orbit recovery, a class of inference problems involving group actions (inspired by applications such as cryo-electron microscopy). Orbit recovery problems over finite groups can often be solved via standard tensor methods. However, for infinite groups, no general algorithms are known. We give a new spectral algorithm based on tensor networks for one such problem: continuous multi-reference alignment over the infinite group SO(2). Our algorithm extends to the more general heterogeneous case.

[1]  Jonathan Shi,et al.  Tensor principal component analysis via sum-of-square proofs , 2015, COLT.

[2]  Anima Anandkumar,et al.  A Spectral Algorithm for Latent Dirichlet Allocation , 2012, Algorithmica.

[3]  Afonso S. Bandeira,et al.  Estimation under group actions: recovering orbits from invariants , 2017, Applied and Computational Harmonic Analysis.

[4]  Yutong Chen,et al.  NON-UNIQUE GAMES OVER COMPACT GROUPS AND ORIENTATION ESTIMATION IN CRYO-EM , 2015, Inverse problems.

[5]  J. Biamonte,et al.  Tensor Networks in a Nutshell , 2017, 1708.00006.

[6]  A. Bandeira Convex Relaxations for Certain Inverse Problems on Graphs , 2015 .

[7]  Ankur Moitra,et al.  Algorithmic Aspects of Machine Learning , 2018 .

[8]  Amit Singer,et al.  Orthogonal matrix retrieval in cryo-electron microscopy , 2014, 2015 IEEE 12th International Symposium on Biomedical Imaging (ISBI).

[9]  Yoel Shkolnisky,et al.  Three-Dimensional Structure Determination from Common Lines in Cryo-EM by Eigenvectors and Semidefinite Programming , 2011, SIAM J. Imaging Sci..

[10]  R. Diamond On the multiple simultaneous superposition of molecular structures by rigid body transformations , 1992, Protein science : a publication of the Protein Society.

[11]  Amit Singer,et al.  Multireference alignment using semidefinite programming , 2013, ITCS.

[12]  Zhizhen Zhao,et al.  Bispectrum Inversion With Application to Multireference Alignment , 2017, IEEE Transactions on Signal Processing.

[13]  Afonso S. Bandeira,et al.  Optimal rates of estimation for multi-reference alignment , 2017, Mathematical Statistics and Learning.

[14]  Tselil Schramm,et al.  Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors , 2015, STOC.

[15]  Elchanan Mossel,et al.  Learning nonsingular phylogenies and hidden Markov models , 2005, STOC '05.

[16]  A. Singer Angular Synchronization by Eigenvectors and Semidefinite Programming. , 2009, Applied and computational harmonic analysis.

[17]  Sham M. Kakade,et al.  Learning mixtures of spherical gaussians: moment methods and spectral decompositions , 2012, ITCS '13.

[18]  Tengyu Ma,et al.  Decomposing Overcomplete 3rd Order Tensors using Sum-of-Squares Algorithms , 2015, APPROX-RANDOM.

[19]  Jianqing Fan,et al.  High-Dimensional Statistics , 2014 .

[20]  Manuel Rosa-Zurera,et al.  Using Multilayer Perceptrons to Align High Range Resolution Radar Signals , 2005, ICANN.

[21]  Amit Singer,et al.  Sample complexity of the boolean multireference alignment problem , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[22]  F. Groen,et al.  Fast Translation Invariant Classification of (HRR) Range Profiles in a Zero Phase Representation , 2003 .

[23]  Zhizhen Zhao,et al.  A Spectral Method for Stable Bispectrum Inversion With Application to Multireference Alignment , 2018, IEEE Signal Processing Letters.

[24]  David Steurer,et al.  Exact tensor completion with sum-of-squares , 2017, COLT.

[25]  Maxim Sviridenko,et al.  Concentration and moment inequalities for polynomials of independent random variables , 2012, SODA.

[26]  Hossein Mobahi,et al.  Homotopy Analysis for Tensor PCA , 2016, COLT.

[27]  Roy R. Lederman,et al.  Heterogeneous multireference alignment: A single pass approach , 2017, 2018 52nd Annual Conference on Information Sciences and Systems (CISS).

[28]  Prateek Jain,et al.  Learning Mixtures of Discrete Product Distributions using Spectral Decompositions , 2013, COLT.

[29]  Pravesh Kothari,et al.  A Nearly Tight Sum-of-Squares Lower Bound for the Planted Clique Problem , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[30]  Tengyu Ma,et al.  Polynomial-Time Tensor Decompositions with Sum-of-Squares , 2016, 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS).

[31]  Douglas L. Theobald,et al.  Optimal simultaneous superpositioning of multiple structures with missing data , 2012, Bioinform..

[32]  J. Dubochet,et al.  Cryo-electron microscopy of viruses , 1984, Nature.

[33]  Amit Singer,et al.  Estimation in the Group Action Channel , 2018, 2018 IEEE International Symposium on Information Theory (ISIT).

[34]  Ankur Moitra,et al.  Message‐Passing Algorithms for Synchronization Problems over Compact Groups , 2016, ArXiv.

[35]  Andrea Montanari,et al.  A statistical model for tensor PCA , 2014, NIPS.

[36]  Amit Singer,et al.  The sample complexity of multi-reference alignment , 2017, SIAM J. Math. Data Sci..

[37]  Alexander S. Wein Statistical estimation in the presence of group actions , 2018 .

[38]  Anima Anandkumar,et al.  Analyzing Tensor Power Method Dynamics: Applications to Learning Overcomplete Latent Variable Models , 2014, ArXiv.

[39]  Anima Anandkumar,et al.  A tensor approach to learning mixed membership community models , 2013, J. Mach. Learn. Res..

[40]  E. Nogales The development of cryo-EM into a mainstream structural biology technique , 2015, Nature Methods.

[41]  Anima Anandkumar,et al.  Analyzing Tensor Power Method Dynamics in Overcomplete Regime , 2014, J. Mach. Learn. Res..