Subtracting a best rank-1 approximation does not necessarily decrease tensor rank

It has been shown that a best rank-R approximation of an order-k tensor may not exist when R ≥ 2 and k ≥ 3. This poses a serious problem to data analysts using tensor decompositions. It has been observed numerically that, generally, this issue cannot be solved by consecutively computing and subtracting best rank-1 approximations. The reason for this is that subtracting a best rank-1 approximation generally does not decrease tensor rank. In this paper, we provide a mathematical treatment of this property for real-valued 2 × 2 × 2 tensors, with symmetric tensors as a special case. Regardless of the symmetry, we show that for generic 2×2×2 tensors (which have rank 2 or 3), subtracting a best rank-1 approximation results in a tensor that has rank 3 and lies on the boundary between the rank-2 and rank-3 sets. Hence, for a typical tensor of rank 2, subtracting a best rank-1 approximation increases the tensor rank.

[1]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[2]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[3]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[4]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[5]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[6]  Joseph JáJá.  Optimal Evaluation of Pairs of Bilinear Forms , 1978, STOC.

[7]  V. Strassen.  Rank and optimal computation of generic tensors , 1983 .

[8]  G. Rota,et al.  The invariant theory of binary forms , 1984 .

[9]  J. Leeuw,et al.  Explicit candecomp/parafac solutions for a contrived 2 × 2 × 2 array of rank three , 1988 .

[10]  J. Kruskal.  Rank, decomposition, and uniqueness for 3-way and n -way arrays , 1989 .

[11]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[12]  Pierre Comon,et al.  Decomposition of quantics in sums of powers of linear forms , 1996, Signal Process..

[13]  Michael Clausen,et al.  Algebraic complexity theory , 1997, Grundlehren der mathematischen Wissenschaften.

[14]  Pierre Comon,et al.  Blind channel identification and extraction of more sources than sensors , 1998, Optics & Photonics.

[15]  J. Berge,et al.  Simplicity of core arrays in three-way principal component analysis and the typical rank of p×q×2 arrays , 1999 .

[16]  J. Vandewalle,et al.  An introduction to independent component analysis , 2000 .

[17]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[18]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..

[19]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[20]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[21]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[22]  L. Lathauwer,et al.  On the Best Rank-1 and Rank-( , 2004 .

[23]  J. Berge,et al.  Typical rank and indscal dimensionality for symmetric three-way arrays of order I×2×2 or I×3×3 , 2004 .

[24]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[25]  Liqun Qi,et al.  Rank and eigenvalues of a supersymmetric tensor, the multivariate homogeneous polynomial and the algebraic hypersurface it defines , 2006, J. Symb. Comput..

[26]  A. Stegeman.  Degeneracy in Candecomp/Parafac explained for p × p × 2 arrays of rank p + 1 or higher , 2006 .

[27]  A. Stegeman.  Degeneracy in Candecomp/Parafac and Indscal Explained For Several Three-Sliced Arrays With A Two-Valued Typical Rank , 2007, Psychometrika.

[28]  Lieven De Lathauwer,et al.  Tensor-based techniques for the blind separation of DS-CDMA signals , 2007, Signal Process..

[29]  Yiju Wang,et al.  On the best rank-1 approximation to higher-order symmetric tensors , 2007, Math. Comput. Model..

[30]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[31]  André Lima Férrer de Almeida,et al.  PARAFAC-based unified tensor modeling for wireless communication systems with application to blind multiuser equalization , 2007, Signal Process..

[32]  Alwin Stegeman,et al.  Low-Rank Approximation of Generic p˟q˟2 Arrays and Diverging Components in the Candecomp/Parafac Model , 2008, SIAM J. Matrix Anal. Appl..

[33]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[34]  A. Stegeman,et al.  On the Non-Existence of Optimal Solutions and the Occurrence of “Degeneracy” in the CANDECOMP/PARAFAC Model , 2008, Psychometrika.

[35]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[36]  P. Kroonenberg.  Wiley Series in Probability and Statistics , 2008 .

[37]  Lieven De Lathauwer,et al.  A Method to Avoid Diverging Components in the Candecomp/Parafac Model for Generic I˟J˟2 Arrays , 2008, SIAM J. Matrix Anal. Appl..

[38]  P. Kroonenberg.  Applied Multiway Data Analysis , 2008 .

[39]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..