Most Tensor Problems Are NP-Hard

We prove that multilinear (tensor) analogues of many efficiently computable problems in numerical linear algebra are NP-hard. Our list includes: determining the feasibility of a system of bilinear equations, deciding whether a 3-tensor possesses a given eigenvalue, singular value, or spectral norm; approximating an eigenvalue, eigenvector, singular vector, or the spectral norm; and determining the rank or best rank-1 approximation of a 3-tensor. Furthermore, we show that restricting these problems to symmetric tensors does not alleviate their NP-hardness. We also explain how deciding nonnegative definiteness of a symmetric 4-tensor is NP-hard and how computing the combinatorial hyperdeterminant is NP-, #P-, and VNP-hard.

[1]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[2]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[3]  A. Church.  Review: A. M. Turing, On Computable Numbers, with an Application to the Entscheidungsproblem , 1937 .

[4]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[5]  S. Banach.  Über homogene Polynome in ($L^{2}$) , 1938 .

[6]  Fred Gruenberger.  A terminology proposal , 1960, CACM.

[7]  H. Putnam,et al.  The Decision Problem for Exponential Diophantine Equations , 1961 .

[8]  T. Motzkin,et al.  Maxima for Graphs and a New Proof of a Theorem of Turán , 1965, Canadian Journal of Mathematics.

[9]  J. Edmonds.  Systems of distinct representatives and linear algebra , 1967 .

[10]  W. Greub.  Theory of a linear transformation , 1967 .

[11]  Donald Ervin Knuth,et al.  The Art of Computer Programming , 1968 .

[12]  Donald Ervin Knuth,et al.  The Art of Computer Programming, Volume II: Seminumerical Algorithms , 1970 .

[13]  V. Strassen.  Gaussian elimination is not optimal , 1969 .

[14]  J. Wrench.  Table errata: The art of computer programming, Vol. 2: Seminumerical algorithms (Addison-Wesley, Reading, Mass., 1969) by Donald E. Knuth , 1970 .

[15]  B. Buchberger.  Ein algorithmisches Kriterium für die Lösbarkeit eines algebraischen Gleichungssystems , 1970 .

[16]  Stephen A. Cook,et al.  The complexity of theorem-proving procedures , 1971, STOC.

[17]  C. Siegel.  Zur Theorie der quadratischen Formen , 1972 .

[18]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[19]  D. E. Knuth,et al.  A terminological proposal , 1974, SIGA.

[20]  D. E. Knuth,et al.  Postscript about NP-hard problems , 1974, SIGA.

[21]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[22]  Leslie G. Valiant,et al.  The Complexity of Computing the Permanent , 1979, Theor. Comput. Sci..

[23]  Leslie G. Valiant,et al.  Completeness classes in algebra , 1979, STOC.

[24]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[25]  Donald E. Knuth,et al.  The art of computer programming. Vol.2: Seminumerical algorithms , 1981 .

[26]  D. Bayer.  The division algorithm and the hilbert scheme , 1982 .

[27]  James P. Jones.  Universal Diophantine Equation , 1982, J. Symb. Log..

[28]  James P. Jones,et al.  Register Machine Proof of the Theorem on Exponential Diophantine Representation of Enumerable Sets , 1984, J. Symb. Log..

[29]  D. Deutsch.  Quantum theory, the Church–Turing principle and the universal quantum computer , 1985, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[30]  T. Coleman,et al.  The null space problem I. complexity , 1986 .

[31]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[32]  S. Smale,et al.  On a theory of computation and complexity over the real numbers; np-completeness , 1989 .

[33]  Johan Håstad,et al.  Tensor Rank is NP-Complete , 1989, ICALP.

[34]  Volker Strassen,et al.  Algebraic Complexity Theory , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[35]  J. George Shanthikumar,et al.  Convex separable optimization is not much harder than linear optimization , 1990, JACM.

[36]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[37]  L. Blum.  A Theory of Computation and Complexity over the real numbers , 1991 .

[38]  S. Vavasis.  Nonlinear optimization: complexity issues , 1991 .

[39]  Joel Friedman,et al.  The Spectra of Infinite Hypertrees , 1991, SIAM J. Comput..

[40]  Alexander I. Barvinok,et al.  Feasibility testing for systems of real quadratic equations , 1992, STOC '92.

[41]  Marie-Françoise Roy,et al.  Real algebraic geometry , 1992 .

[42]  Michael Greenacre,et al.  Multiway data analysis , 1992 .

[43]  B. Reznick.  Sums of Even Powers of Real Linear Forms , 1992 .

[44]  David A. Cox,et al.  Ideals, Varieties, and Algorithms: An Introduction to Computational Algebraic Geometry and Commutative Algebra, 3/e (Undergraduate Texts in Mathematics) , 2007 .

[45]  Yuri Matiyasevich,et al.  Hilbert’s tenth problem , 2019, 100 Years of Math Milestones.

[46]  Alexander I. Barvinok.  Feasibility testing for systems of real quadratic equations , 1993, Discret. Comput. Geom..

[47]  I. M. Gelʹfand,et al.  Discriminants, Resultants, and Multidimensional Determinants , 1994 .

[48]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[49]  László Lovász,et al.  Stable sets and polynomials , 1994, Discret. Math..

[50]  Alexander I. Barvinok,et al.  New algorithms for lineark-matroid intersection and matroidk-parity problems , 1995, Math. Program..

[51]  M. Aigner.  Turán’s graph theorem , 1995 .

[52]  D. Loera,et al.  Gröbner bases and graph colorings. , 1995 .

[53]  Ming Gu.  Finding Well-Conditioned Similarities to Block-Diagonalize Nonsymmetric Matrices Is NP-Hard , 1995, J. Complex..

[54]  Avi Wigderson,et al.  On the second eigenvalue of hypergraphs , 1995, Comb..

[55]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[56]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[57]  Michael Sipser,et al.  Introduction to the Theory of Computation , 1996, SIGA.

[58]  William Kahan,et al.  Lecture Notes on the Status of IEEE Standard 754 for Binary Floating-Point Arithmetic , 1996 .

[59]  Dorit S. Hochbaum,et al.  Approximation Algorithms for NP-Hard Problems , 1996 .

[60]  W. Wootters,et al.  Entanglement of a Pair of Quantum Bits , 1997, quant-ph/9703041.

[61]  C. Tomasi,et al.  Systems of Bilinear Equations , 1997 .

[62]  Michael Clausen,et al.  Algebraic complexity theory , 1997, Grundlehren der mathematischen Wissenschaften.

[63]  Dorit S. Hochba,et al.  Approximation Algorithms for NP-Hard Problems , 1997, SIGA.

[64]  Victor Y. Pan,et al.  Solving a Polynomial Equation: Some History and Recent Progress , 1997, SIAM Rev..

[65]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[66]  David A. Cox,et al.  Ideals, Varieties, and Algorithms , 1997 .

[67]  Umesh V. Vazirani,et al.  Quantum Complexity Theory , 1997, SIAM J. Comput..

[68]  S. Smale.  Mathematical problems for the next century , 1998 .

[69]  Lenore Blum,et al.  Complexity and Real Computation , 1997, Springer New York.

[70]  J. Håstad.  Clique is hard to approximate withinn1−ε , 1999 .

[71]  Kyriakos Kalorkoti.  ALGEBRAIC COMPLEXITY THEORY (Grundlehren der Mathematischen Wissenschaften 315) , 1999 .

[72]  Chee-Keng Yap,et al.  Fundamental problems of algorithmic algebra , 1999 .

[73]  Lance Fortnow,et al.  Complexity limitations on quantum computation , 1999, J. Comput. Syst. Sci..

[74]  D. Hilbert.  Mathematical Problems , 2019, Mathematics: People · Problems · Results.

[75]  D. Haskell,et al.  Model theory, algebra, and geometry , 2000 .

[76]  Klaus Weihrauch,et al.  Computable Analysis: An Introduction , 2014, Texts in Theoretical Computer Science. An EATCS Series.

[77]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[78]  Jitendra Malik,et al.  Normalized Cuts and Image Segmentation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[79]  Peter Bürgisser,et al.  Completeness and Reduction in Algebraic Complexity Theory , 2000, Algorithms and computation in mathematics.

[80]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[81]  Timothy H. McNicholl.  Review of "Complexity and real computation" by Blum, Cucker, Shub, and Smale. Springer-Verlag. , 2001, SIGA.

[82]  Michael L. Overton,et al.  Numerical Computing with IEEE Floating Point Arithmetic , 2001 .

[83]  Demetri Terzopoulos,et al.  Multilinear image analysis for facial recognition , 2002, Object recognition supported by user interaction for service robots.

[84]  M. Alex O. Vasilescu.  Human motion signatures: analysis, synthesis, recognition , 2002, Object recognition supported by user interaction for service robots.

[85]  Etienne de Klerk,et al.  Approximation of the Stability Number of a Graph via Copositive Programming , 2002, SIAM J. Optim..

[86]  Martin J. Mohlenkamp,et al.  Numerical operator calculus in higher dimensions , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[87]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[88]  Akimasa Miyake,et al.  Multipartite entanglement and hyperdeterminants , 2002, Quantum Inf. Comput..

[89]  Willi Meier,et al.  Solving Underdefined Systems of Multivariate Quadratic Equations , 2002, Public Key Cryptography.

[90]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[91]  B. Poonen.  Hilbert's Tenth Problem and Mazur's Conjecture for large subrings of $\mathbb{Q}$ , 2003, math/0306277.

[92]  Leonid Gurvits.  Classical deterministic complexity of Edmonds' Problem and quantum entanglement , 2003, STOC '03.

[93]  Y. Nesterov.  Random walk in a simplex and quadratic optimization over convex polytopes , 2003 .

[94]  P. Goldbart,et al.  Geometric measure of entanglement and applications to bipartite and multipartite quantum states , 2003, quant-ph/0307219.

[95]  Guy Kindler,et al.  Optimal inapproximability results for MAX-CUT and other 2-variable CSPs? , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[96]  Demetri Terzopoulos,et al.  TensorTextures: multilinear image-based rendering , 2004, ACM Trans. Graph..

[97]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[98]  F. Grunewald,et al.  On the integer solutions of quadratic equations , 2004 .

[99]  Pierre Comon,et al.  Blind identification and source separation in 2×3 under-determined mixtures , 2004, IEEE Trans. Signal Process..

[100]  Santosh S. Vempala,et al.  Tensor decomposition and approximation schemes for constraint satisfaction problems , 2005, STOC '05.

[101]  Tamir Hazan,et al.  Non-negative tensor factorization with applications to statistics and computer vision , 2005, ICML.

[102]  Lek-Heng Lim,et al.  Singular values and eigenvalues of tensors: a variational approach , 2005, 1st IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, 2005..

[103]  Leonid Gurvits,et al.  On the Complexity of Mixed Discriminants and Related Problems , 2005, MFCS.

[104]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[105]  Liqun Qi,et al.  Eigenvalues of a real supersymmetric tensor , 2005, J. Symb. Comput..

[106]  C. Bachoc,et al.  New upper bounds for kissing numbers from semidefinite programming , 2006, math/0608426.

[107]  Noga Alon,et al.  Approximating the Cut-Norm via Grothendieck's Inequality , 2006, SIAM J. Comput..

[108]  Dan Suciu,et al.  Journal of the ACM , 2006 .

[109]  Branimir Lambov.  RealLib: An efficient implementation of exact real arithmetic , 2007, Math. Struct. Comput. Sci..

[110]  Martin Fürer.  Faster integer multiplication , 2007, STOC '07.

[111]  L. Qi,et al.  The degree of the E-characteristic polynomial of an even order tensor , 2007 .

[112]  Ryan O'Donnell,et al.  Optimal Inapproximability Results for MAX-CUT and Other 2-Variable CSPs? , 2007, SIAM J. Comput..

[113]  David Zuckerman.  Linear Degree Extractors and the Inapproximability of Max Clique and Chromatic Number , 2007, Theory Comput..

[114]  Tero Harju,et al.  Undecidability Bounds for Integer Matrices Using Claus Instances , 2007, Int. J. Found. Comput. Sci..

[115]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[116]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.

[117]  A. Tonge,et al.  Norm attaining polynomials , 2007 .

[118]  Daureen Steinberg.  COMPUTATION OF MATRIX NORMS WITH APPLICATIONS TO ROBUST OPTIMIZATION , 2007 .

[119]  L. Qi.  Eigenvalues and invariants of tensors , 2007 .

[120]  Anindya De,et al.  Fast integer multiplication using modular arithmetic , 2008, STOC.

[121]  Fernando Mário Oliveira Filho,et al.  Lower Bounds for Measurable Chromatic Numbers , 2008, 0801.1059.

[122]  Nicholas J. Higham,et al.  Functions of matrices - theory and computation , 2008 .

[123]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[124]  Jesús A. De Loera,et al.  Hilbert's nullstellensatz and an algorithm for proving combinatorial infeasibility , 2008, ISSAC '08.

[125]  Hans-Peter Seidel,et al.  Estimating Crossing Fibers: A Tensor Decomposition Approach , 2008, IEEE Transactions on Visualization and Computer Graphics.

[126]  Bernd Sturmfels,et al.  The hyperdeterminant and triangulations of the 4-cube , 2006, Math. Comput..

[127]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[128]  N. Higham.  Functions of Matrices: Theory and Computation (Other Titles in Applied Mathematics) , 2008 .

[129]  Christopher J. Hillar,et al.  Algebraic characterization of uniquely vertex colorable graphs , 2008, J. Comb. Theory, Ser. B.

[130]  Elizabeth S. Allman,et al.  Phylogenetic ideals and varieties for the general Markov model , 2004, Adv. Appl. Math..

[131]  Etienne de Klerk,et al.  The complexity of optimizing over a simplex, hypercube or sphere: a short survey , 2008, Central Eur. J. Oper. Res..

[132]  Santosh S. Vempala,et al.  Random Tensors and Planted Cliques , 2009, APPROX-RANDOM.

[133]  Arthur Cayley,et al.  The Collected Mathematical Papers: On the Theory of Linear Transformations , 2009 .

[134]  Lance Fortnow,et al.  The status of the P versus NP problem , 2009, CACM.

[135]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[136]  Alexander Olshevsky,et al.  Matrix P-norms are NP-hard to approximate if p ≠1,2,∞ , 2009 .

[137]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[138]  Bernd Sturmfels,et al.  Reconstructing spatiotemporal gene expression data from partial observations , 2009, Bioinform..

[139]  L. M. Pardo,et al.  Efficient polynomial system-solving by numerical methods , 2009 .

[140]  Frank Vallentin,et al.  The Positive Semidefinite Grothendieck Problem with Rank Constraint , 2009, ICALP.

[141]  Julien M. Hendrickx,et al.  Matrix p-Norms Are NP-Hard to Approximate If p!=q1, 2, INFINITY , 2010, SIAM J. Matrix Anal. Appl..

[142]  Bruno Grenet,et al.  The Multivariate Resultant Is NP-hard in Any Characteristic , 2010, MFCS.

[143]  Shuzhong Zhang,et al.  Approximation algorithms for homogeneous polynomial optimization with quadratic constraints , 2010, Math. Program..

[144]  S. Friedland,et al.  UPPER BOUNDS ON THE MAGNITUDE OF SOLUTIONS OF CERTAIN LINEAR SYSTEMS WITH INTEGER COEFFICIENTS , 2011, 1108.4078.

[145]  J. Landsberg.  Tensors: Geometry and Applications , 2011 .

[146]  S. Friedland.  Best rank one approximation of real symmetric tensors can be chosen symmetric , 2011, 1110.5689.

[147]  Erik Massop.  Hilbert's tenth problem , 2012 .

[148]  B. Poonen.  UNDECIDABLE PROBLEMS: A SAMPLER , 2012, 1204.0299.

[149]  Sanjeev Arora,et al.  Computing a nonnegative matrix factorization -- provably , 2011, STOC '12.

[150]  Lek-Heng Lim.  Tensors and Hypermatrices , 2013 .

[151]  B. Reznick.  ON THE LENGTH OF BINARY FORMS , 2010, 1007.5485.

[152]  B. Sturmfels,et al.  The number of eigenvalues of a tensor , 2010, 1004.4953.

[153]  John N. Tsitsiklis,et al.  NP-hardness of deciding convexity of quartic polynomials and related problems , 2010, Math. Program..

[154]  Frank Vallentin,et al.  Grothendieck Inequalities for Semidefinite Programs with Rank Constraint , 2010, Theory Comput..

[155]  Peter J. C. Dickinson,et al.  On the computational complexity of membership problems for the completely positive cone and its dual , 2014, Comput. Optim. Appl..

[156]  D. Kressner.  Bivariate Matrix Functions , 2014 .