On Generic Nonexistence of the Schmidt-Eckart-Young Decomposition for Complex Tensors

The Schmidt--Eckart--Young theorem for matrices states that the optimal rank-$r$ approximation of a matrix is obtained by retaining the first $r$ terms from the singular value decomposition of that matrix. This paper considers a generalization of this optimal truncation property to the rank decomposition (Candecomp/Parafac) of tensors and establishes a necessary orthogonality condition. We prove that this condition is not satisfied at least by an open set of positive Lebesgue measure in complex tensor spaces. It is proved, moreover, that for complex tensors of small rank this condition can be satisfied only by a set of tensors of Lebesgue measure zero. Finally, we demonstrate that generic tensors in cubic tensor spaces are not optimally truncatable.

[1]  Raf Vandebril,et al.  A New Truncation Strategy for the Higher-Order Singular Value Decomposition , 2012, SIAM J. Sci. Comput..

[2]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[3]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[4]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[5]  A. Nouy.  Proper Generalized Decompositions and Separated Representations for the Numerical Solution of High Dimensional Stochastic Problems , 2010 .

[6]  Alwin Stegeman,et al.  Candecomp/Parafac: From Diverging Components to a Decomposition in Block Terms , 2012, SIAM J. Matrix Anal. Appl..

[7]  Anima Anandkumar,et al.  Tensor decompositions for learning latent variable models , 2012, J. Mach. Learn. Res..

[8]  F. Chinesta,et al.  On the Convergence of a Greedy Rank-One Update Algorithm for a Class of Linear Systems , 2010 .

[9]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[10]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[11]  Alessandro Gimigliano,et al.  Secant varieties of ℙ¹×\cdots×ℙ¹ (-times) are NOT defective for ≥5 , 2011 .

[12]  G. W. STEWARTt.  ON THE EARLY HISTORY OF THE SINGULAR VALUE DECOMPOSITION * , 2022 .

[13]  E. Davidson,et al.  Strategies for analyzing data from video fluorometric monitoring of liquid chromatographic effluents , 1981 .

[14]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[15]  Grazia Lotti,et al.  Approximate Solutions for the Bilinear Form Computational Problem , 1980, SIAM J. Comput..

[16]  C. Bocci,et al.  Refined methods for the identifiability of tensors , 2013, 1303.6915.

[17]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[18]  Chris Peterson,et al.  Induction for secant varieties of Segre varieties , 2006, math/0607191.

[19]  Adrien Leygue,et al.  An overview of the proper generalized decomposition with applications in computational rheology , 2011 .

[20]  P. Paatero.  Construction and analysis of degenerate PARAFAC models , 2000 .

[21]  A. Stegeman.  Degeneracy in Candecomp/Parafac and Indscal Explained For Several Three-Sliced Arrays With A Two-Valued Typical Rank , 2007, Psychometrika.

[22]  E. Schmidt.  Zur Theorie der linearen und nichtlinearen Integralgleichungen , 1907 .

[23]  J. Huisman.  Real abelian varieties with complex multiplication , 1992 .

[24]  Tamara G. Kolda,et al.  Orthogonal Tensor Decompositions , 2000, SIAM J. Matrix Anal. Appl..

[25]  V. Strassen.  Rank and optimal computation of generic tensors , 1983 .

[26]  K. Meerbergen,et al.  A probabilistic numerical algorithm based on Terracini's lemma for proving nondefectivity of Segre varieties , 2012 .

[27]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[28]  Alwin Stegeman,et al.  Low-Rank Approximation of Generic p˟q˟2 Arrays and Diverging Components in the Candecomp/Parafac Model , 2008, SIAM J. Matrix Anal. Appl..

[29]  W. Hackbusch.  Tensor Spaces and Numerical Tensor Calculus , 2012, Springer Series in Computational Mathematics.

[30]  T. Willmore.  Algebraic Geometry , 1973, Nature.

[31]  Yousef Saad,et al.  On the Tensor SVD and the Optimal Low Rank Orthogonal Approximation of Tensors , 2008, SIAM J. Matrix Anal. Appl..

[32]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[33]  Ju. A. Drozd,et al.  Tame and wild matrix problems , 1980 .

[34]  Alwin Stegeman,et al.  A Three-Way Jordan Canonical Form as Limit of Low-Rank Tensor Approximations , 2013, SIAM J. Matrix Anal. Appl..

[35]  Tamara G. Kolda,et al.  A Counterexample to the Possibility of an Extension of the Eckart-Young Low-Rank Approximation Theorem for the Orthogonal Rank Tensor Decomposition , 2002, SIAM J. Matrix Anal. Appl..

[36]  A. Geramita,et al.  Ranks of tensors, secant varieties of Segre varieties and fat points , 2002 .

[37]  A. Stegeman.  Degeneracy in Candecomp/Parafac explained for p × p × 2 arrays of rank p + 1 or higher , 2006 .

[38]  Alessandro Terracini,et al.  Sulle vk per cui la varietÀ degli sh (h + 1) seganti ha dimensione minore delĽordinario , 1911 .

[39]  Martin J. Mohlenkamp,et al.  Algorithms for Numerical Analysis in High Dimensions , 2005, SIAM J. Sci. Comput..

[40]  Athina P. Petropulu,et al.  Joint singular value decomposition - a new tool for separable representation of images , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[41]  Alessandro Gimigliano,et al.  Secant Varieties of (P ^1) X .... X (P ^1) (n-times) are NOT Defective for n \geq 5 , 2008 .

[42]  Giorgio Ottaviani,et al.  On Generic Identifiability of 3-Tensors of Small Rank , 2011, SIAM J. Matrix Anal. Appl..

[43]  Morten Mørup,et al.  Applications of tensor (multiway array) factorizations and decompositions in data mining , 2011, WIREs Data Mining Knowl. Discov..

[44]  Michael Clausen,et al.  Typical Tensorial Rank , 1997 .

[45]  L. Nazarova.  REPRESENTATIONS OF QUIVERS OF INFINITE TYPE , 1973 .

[46]  Fulvio Gesmundo.  An asymptotic bound for secant varieties of Segre varieties , 2012, 1209.1732.

[47]  J. M. Landsberg,et al.  On the third secant variety , 2011, 1111.7005.

[48]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[49]  W. Hackbusch,et al.  A New Scheme for the Tensor Representation , 2009 .

[50]  Lars Grasedyck,et al.  F ¨ Ur Mathematik in Den Naturwissenschaften Leipzig a Projection Method to Solve Linear Systems in Tensor Format a Projection Method to Solve Linear Systems in Tensor Format , 2022 .

[51]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[52]  G. Ottaviani,et al.  On the Alexander–Hirschowitz theorem , 2007, math/0701409.

[53]  M. SIAMJ..  A COUNTEREXAMPLE TO THE POSSIBILITY OF AN EXTENSION OF THE ECKART – YOUNG LOW-RANK APPROXIMATION THEOREM FOR THE ORTHOGONAL RANK TENSOR DECOMPOSITION , 2003 .

[54]  J. Landsberg.  Tensors: Geometry and Applications , 2011 .

[55]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[56]  Dario Bini,et al.  Border Rank of a pxqx2 Tensor and the Optimal Approximation od a Pair of Bilinear Forms , 1980, ICALP.

[57]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[58]  F. R. Gantmakher.  The Theory of Matrices , 1984 .

[59]  Lieven De Lathauwer,et al.  A Method to Avoid Diverging Components in the Candecomp/Parafac Model for Generic I˟J˟2 Arrays , 2008, SIAM J. Matrix Anal. Appl..

[60]  D. Simson,et al.  Elements of the Representation Theory of Associative Algebras , 2007 .

[61]  Jorge N. Tendeiro,et al.  Some New Results on Orthogonally Constrained Candecomp , 2011, J. Classif..

[62]  Raf Vandebril,et al.  A randomized algorithm for testing nonsingularity of structured matrices with an application to asserting nondefectivity of Segre varieties , 2015 .

[63]  Dario Bini.  Relations between exact and approximate bilinear algorithms. Applications , 1980 .

[64]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[65]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[66]  J. Jaja.  An Addendum to Kronecker’s Theory of Pencils , 1979 .

[67]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[68]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[69]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[70]  Daniel Kressner,et al.  Krylov Subspace Methods for Linear Systems with Tensor Product Structure , 2010, SIAM J. Matrix Anal. Appl..

[71]  Claus Michael Ringel,et al.  Representations of finite dimensional algebras and related topics in Lie theory and geometry , 2004 .

[72]  E. Schmidt.  Zur Theorie der linearen und nichtlinearen Integralgleichungen , 1989 .