Low-Rank Approximation of Generic p˟q˟2 Arrays and Diverging Components in the Candecomp/Parafac Model

We consider the low-rank approximation over the real field of generic $p \times q \times 2$ arrays. For all possible combinations of $p$, $q$, and $R$, we present conjectures on the existence of a best rank-$R$ approximation. Our conjectures are motivated by a detailed analysis of the boundary of the set of arrays with at most rank $R$. We link these results to the Candecomp/Parafac (CP) model for three-way component analysis. Essentially, CP tries to find a best rank-$R$ approximation to a given three-way array. In the case of $p \times q \times 2$ arrays, we show (under some regularity condition) that if a best rank-$R$ approximation does not exist, then any sequence of CP updates will exhibit diverging CP components, which implies that several components are highly correlated in all three modes and their component weights become arbitrarily large. This extends Stegeman [Psychometrika, 71 (2006), pp. 483-501], who considers $p \times p \times 2$ arrays of rank $p+1$ or higher. We illustrate our results by means of simulations.

[1]  J. Kruskal.  Rank, decomposition, and uniqueness for 3-way and n -way arrays , 1989 .

[2]  Michael Clausen,et al.  Algebraic complexity theory , 1997, Grundlehren der mathematischen Wissenschaften.

[3]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[4]  J. Berge,et al.  Kruskal's polynomial for 2×2×2 arrays and a generalization to 2×n×n arrays , 1991 .

[5]  David P. Dobkin,et al.  On the optimal evaluation of a set of bilinear forms , 1973, SWAT.

[6]  P. Paatero.  Construction and analysis of degenerate PARAFAC models , 2000 .

[7]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[8]  David P. Dobkin,et al.  On the optimal evaluation of a set of bilinear forms , 1978 .

[9]  A. Stegeman.  Degeneracy in Candecomp/Parafac explained for p × p × 2 arrays of rank p + 1 or higher , 2006 .

[10]  Peter Lancaster,et al.  The theory of matrices , 1969 .

[11]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[12]  J. Kruskal,et al.  How 3-MFA data can cause degenerate parafac solutions, among other relationships , 1989 .

[13]  W. Rayens,et al.  Two-factor degeneracies and a stabilization of PARAFAC , 1997 .

[14]  Grazia Lotti,et al.  O(n2.7799) Complexity for n*n Approximate Matrix Multiplication , 1979, Inf. Process. Lett..

[15]  J. Leeuw,et al.  Explicit candecomp/parafac solutions for a contrived 2 × 2 × 2 array of rank three , 1988 .

[16]  P. Paatero,et al.  THREE-WAY (PARAFAC) FACTOR ANALYSIS : EXAMINATION AND COMPARISON OF ALTERNATIVE COMPUTATIONAL METHODS AS APPLIED TO ILL-CONDITIONED DATA , 1998 .

[17]  Volker Strassen,et al.  Algebraic Complexity Theory , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[18]  N.D. Sidiropoulos,et al.  Low-rank decomposition of multi-way arrays: a signal processing perspective , 2004, Processing Workshop Proceedings, 2004 Sensor Array and Multichannel Signal.

[19]  Dario Bini.  Relations between exact and approximate bilinear algorithms. Applications , 1980 .

[20]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[21]  Ben C. Mitchell,et al.  Slowly converging parafac sequences: Swamps and two‐factor degeneracies , 1994 .

[22]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[23]  Dario Bini.  Border rank of m×n×(mn−q) tensors , 1986 .

[24]  A. Stegeman.  Degeneracy in Candecomp/Parafac and Indscal Explained For Several Three-Sliced Arrays With A Two-Valued Typical Rank , 2007, Psychometrika.

[25]  Henk A. L. Kiers,et al.  Degenerate solutions obtained from several variants of factor analysis , 2002 .

[26]  J. Landsberg.  The border rank of the multiplication of 2×2 matrices is seven , 2005 .

[27]  R. A. Harshman,et al.  Data preprocessing and the extended PARAFAC model , 1984 .

[28]  K. Ramachandra,et al.  Vermeidung von Divisionen. , 1973 .

[29]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[30]  Joseph JáJá.  Optimal Evaluation of Pairs of Bilinear Forms , 1979, SIAM J. Comput..

[31]  J. Jaja.  An Addendum to Kronecker’s Theory of Pencils , 1979 .

[32]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[33]  Dario Bini,et al.  Border Rank of a pxqx2 Tensor and the Optimal Approximation od a Pair of Bilinear Forms , 1980, ICALP.

[34]  P. Paatero.  The Multilinear Engine—A Table-Driven, Least Squares Program for Solving Multilinear Problems, Including the n-Way Parallel Factor Analysis Model , 1999 .

[35]  Y. Z. Cao,et al.  A PARAFAC algorithm using penalty diagonalization error (PDE) for three-way data array resolution. , 2000, The Analyst.

[36]  B. Everitt,et al.  Three-Mode Principal Component Analysis. , 1986 .

[37]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[38]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[39]  P. Comon,et al.  Optimal solutions to non-negative PARAFAC/multilinear NMF always exist , 2005 .

[40]  F. R. Gantmakher.  The Theory of Matrices , 1984 .

[41]  Grazia Lotti,et al.  Approximate Solutions for the Bilinear Form Computational Problem , 1980, SIAM J. Comput..

[42]  J. Berge,et al.  Simplicity of core arrays in three-way principal component analysis and the typical rank of p×q×2 arrays , 1999 .

[43]  J. Berge,et al.  Partial uniqueness in CANDECOMP/PARAFAC , 2004 .