A Method to Avoid Diverging Components in the Candecomp/Parafac Model for Generic I˟J˟2 Arrays

Computing the Candecomp/Parafac (CP) solution of $R$ components (i.e., the best rank-$R$ approximation) for a generic $I\times J\times 2$ array may result in diverging components, also known as “degeneracy.” In such a case, several components are highly correlated in all three modes, and their component weights become arbitrarily large. Evidence exists that this is caused by the nonexistence of an optimal CP solution. Instead of using CP, we propose to compute the best approximation by means of a generalized Schur decomposition (GSD), which always exists. The obtained GSD solution is the limit point of the sequence of CP updates (whether it features diverging components or not) and can be separated into a nondiverging CP part and a sparse Tucker3 part or into a nondiverging CP part and a smaller GSD part. We show how to obtain both representations and illustrate our results with numerical experiments.

[1]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[2]  J. Kruskal,et al.  How 3-MFA data can cause degenerate parafac solutions, among other relationships , 1989 .

[3]  Joos Vandewalle,et al.  Computation of the Canonical Decomposition by Means of a Simultaneous Generalized Schur Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[4]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[5]  J. Leeuw,et al.  Explicit candecomp/parafac solutions for a contrived 2 × 2 × 2 array of rank three , 1988 .

[6]  W. Rayens,et al.  Two-factor degeneracies and a stabilization of PARAFAC , 1997 .

[7]  P. Paatero,et al.  THREE-WAY (PARAFAC) FACTOR ANALYSIS : EXAMINATION AND COMPARISON OF ALTERNATIVE COMPUTATIONAL METHODS AS APPLIED TO ILL-CONDITIONED DATA , 1998 .

[8]  Joseph JáJá.  Optimal Evaluation of Pairs of Bilinear Forms , 1979, SIAM J. Comput..

[9]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[10]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[11]  S. Leurgans,et al.  A Decomposition for Three-Way Arrays , 1993, SIAM J. Matrix Anal. Appl..

[12]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[13]  P. Comon,et al.  Optimal solutions to non-negative PARAFAC/multilinear NMF always exist , 2005 .

[14]  A. Stegeman,et al.  On the Non-Existence of Optimal Solutions and the Occurrence of “Degeneracy” in the CANDECOMP/PARAFAC Model , 2008, Psychometrika.

[15]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[16]  Wim P. Krijnen,et al.  Convergence of the sequence of parameters generated by alternating least squares algorithms , 2006, Comput. Stat. Data Anal..

[17]  P. Paatero.  Construction and analysis of degenerate PARAFAC models , 2000 .

[18]  P. Comon.  Independent Component Analysis , 1992 .

[19]  R. A. Harshman,et al.  Data preprocessing and the extended PARAFAC model , 1984 .

[20]  J. Vandewalle,et al.  An introduction to independent component analysis , 2000 .

[21]  Nikos D. Sidiropoulos,et al.  Kruskal's permutation lemma and the identification of CANDECOMP/PARAFAC and bilinear models with constant modulus constraints , 2004, IEEE Transactions on Signal Processing.

[22]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[23]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[24]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[25]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[26]  J. Berge,et al.  Simplicity of core arrays in three-way principal component analysis and the typical rank of p×q×2 arrays , 1999 .

[27]  P. Kroonenberg.  Applied Multiway Data Analysis , 2008 .

[28]  Gene H. Golub,et al.  Ill-conditioned eigensystems and the computation of the Jordan canonical form , 1975, Milestones in Matrix Computation.

[29]  Arogyaswami Paulraj,et al.  An analytical constant modulus algorithm , 1996, IEEE Trans. Signal Process..

[30]  Gene H. Golub,et al.  Matrix computations , 1983 .

[31]  L. Lathauwer,et al.  Sufficient conditions for uniqueness in Candecomp/Parafac and Indscal with random component matrices , 2006 .

[32]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[33]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[34]  P. Paatero.  The Multilinear Engine—A Table-Driven, Least Squares Program for Solving Multilinear Problems, Including the n-Way Parallel Factor Analysis Model , 1999 .

[35]  Alwin Stegeman,et al.  Low-Rank Approximation of Generic p˟q˟2 Arrays and Diverging Components in the Candecomp/Parafac Model , 2008, SIAM J. Matrix Anal. Appl..

[36]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..

[37]  A. Stegeman.  Degeneracy in Candecomp/Parafac and Indscal Explained For Several Three-Sliced Arrays With A Two-Valued Typical Rank , 2007, Psychometrika.

[38]  A. Stegeman.  Degeneracy in Candecomp/Parafac explained for p × p × 2 arrays of rank p + 1 or higher , 2006 .

[39]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[40]  Lieven De Lathauwer,et al.  Tensor-based techniques for the blind separation of DS-CDMA signals , 2007, Signal Process..

[41]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[42]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[43]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[44]  Y. Z. Cao,et al.  A PARAFAC algorithm using penalty diagonalization error (PDE) for three-way data array resolution. , 2000, The Analyst.

[45]  Ben C. Mitchell,et al.  Slowly converging parafac sequences: Swamps and two‐factor degeneracies , 1994 .

[46]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .