Low-rank decomposition of multi-way arrays: a signal processing perspective

In many signal processing applications of linear algebra tools, the signal part of a postulated model lies in a so-called signal sub-space, while the parameters of interest are in one-to-one correspondence with a certain basis of this subspace. The signal sub-space can often be reliably estimated from measured data, but the particular basis of interest cannot be identified without additional problem-specific structure. This is a manifestation of rotational indeterminacy, i.e., non-uniqueness of low-rank matrix decomposition. The situation is very different for three-or higher-way arrays, i.e., arrays indexed by three or more independent variables, for which low-rank decomposition is unique under mild conditions. This has fundamental implications for DSP problems which deal with such data. This paper provides a brief lour of the basic elements of this theory, along with many examples of application in problems of current interest in the signal processing community.

[1]  Nikos D. Sidiropoulos,et al.  Blind high resolution localization and tracking of multiple frequency hopped signals , 2001, Conference Record of Thirty-Fifth Asilomar Conference on Signals, Systems and Computers (Cat.No.01CH37256).

[2]  Nikos D. Sidiropoulos,et al.  Blind Identification of Out-of-Cell Users in DS-CDMA , 2004, EURASIP J. Adv. Signal Process..

[3]  N. Sidiropoulos,et al.  Maximum likelihood fitting using ordinary least squares algorithms , 2002 .

[4]  Nikos D. Sidiropoulos,et al.  Identifiability results for blind beamforming in incoherent multipath with small delay spread , 2001, IEEE Trans. Signal Process..

[5]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[6]  Nikos D. Sidiropoulos,et al.  Khatri-Rao space-time codes , 2002, IEEE Trans. Signal Process..

[7]  Rasmus Bro,et al.  On the difference between low-rank and subspace approximation: improved model for multi-linear PLS regression , 2001 .

[8]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[9]  Richard A. Harshman,et al.  Determination and Proof of Minimum Uniqueness Conditions for PARAFAC1 , 1972 .

[10]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[11]  J. Berge,et al.  On uniqueness in candecomp/parafac , 2002 .

[12]  Nikos D. Sidiropoulos,et al.  Generalizing Carathéodory's uniqueness of harmonic parameterization to N dimensions , 2001, IEEE Trans. Inf. Theory.

[13]  Michael Clausen,et al.  Algebraic complexity theory , 1997, Grundlehren der mathematischen Wissenschaften.

[14]  Bruce R. Kowalski,et al.  Generalized rank annihilation factor analysis , 1986 .

[15]  Joos Vandewalle,et al.  Independent component analysis and (simultaneous) third-order tensor diagonalization , 2001, IEEE Trans. Signal Process..

[16]  B. Kowalski,et al.  Review of Chemometrics Applied to Spectroscopy: 1985-95, Part 3 - Multi-way Analysis , 1997 .

[17]  Georgios B. Giannakis,et al.  PARAFAC STAP for the UESA Radar , 2000 .

[18]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[19]  Graham K. Rand,et al.  Quantitative Applications in the Social Sciences , 1983 .

[20]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[21]  Nicholas D. Sidiropoulos,et al.  Parafac techniques for signal separation , 2000 .

[22]  Nikos D. Sidiropoulos,et al.  On communication diversity for blind identifiability and the uniqueness of low-rank decomposition of N-way arrays , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[23]  Nikos D. Sidiropoulos,et al.  Almost sure identifiability of constant modulus multidimensional harmonic retrieval , 2002, IEEE Trans. Signal Process..

[24]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..

[25]  Nikos D. Sidiropoulos,et al.  Blind identification of out-of-cell users in DS-CDMA: An algebraic approach , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[26]  L. Lathauwer,et al.  On the Best Rank-1 and Rank-( , 2004 .

[27]  VandewalleJoos,et al.  A Multilinear Singular Value Decomposition , 2000 .

[28]  Roberto Rocci,et al.  A simplification of a result by zellini on the maximal rank of symmetric three-way arrays , 1994 .

[29]  Sergiy A. Vorobyov,et al.  Blind spatial signature estimation using time-varying user power loading and parallel factor analysis , 2003, 2003 IEEE 58th Vehicular Technology Conference. VTC 2003-Fall (IEEE Cat. No.03CH37484).

[30]  Nikos D. Sidiropoulos,et al.  Kruskal's permutation lemma and the identification of CANDECOMP/PARAFAC and bilinear models with constant modulus constraints , 2004, IEEE Transactions on Signal Processing.

[31]  R. Bro.  PARAFAC. Tutorial and applications , 1997 .

[32]  N.D. Sidiropoulos,et al.  Blind multiuser detection in W-CDMA systems with large delay spread , 2001, IEEE Signal Processing Letters.

[33]  Nikos D. Sidiropoulos,et al.  Almost-sure identifiability of multidimensional harmonic retrieval , 2001, IEEE Trans. Signal Process..

[34]  Henk A. L. Kiers,et al.  Some clarifications of the CANDECOMP algorithm applied to INDSCAL , 1991 .

[35]  P. Schönemann,et al.  An algebraic solution for a class of subjective metrics models , 1972 .

[36]  J. Kruskal.  Rank, decomposition, and uniqueness for 3-way and n -way arrays , 1989 .

[37]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[38]  J. Berge,et al.  Typical rank and indscal dimensionality for symmetric three-way arrays of order I×2×2 or I×3×3 , 2004 .

[39]  J. Berge,et al.  The typical rank of tall three-way arrays , 2000 .

[40]  J. Berge,et al.  Some uniqueness results for PARAFAC2 , 1996 .

[41]  Kyriakos Kalorkoti.  ALGEBRAIC COMPLEXITY THEORY (Grundlehren der Mathematischen Wissenschaften 315) , 1999 .

[42]  Nikos D. Sidiropoulos,et al.  Blind spatial signature estimation via time-varying user power loading and parallel factor analysis , 2005, IEEE Transactions on Signal Processing.

[43]  Rasmus Bro,et al.  User separation in DS-CDMA systems with unknown long PN spreading codes , 1999, 1999 2nd IEEE Workshop on Signal Processing Advances in Wireless Communications (Cat. No.99EX304).

[44]  J. Berge,et al.  Simplicity of core arrays in three-way principal component analysis and the typical rank of p×q×2 arrays , 1999 .

[45]  H. Law.  Research methods for multimode data analysis , 1984 .

[46]  N. Sidiropoulos,et al.  Least squares algorithms under unimodality and non‐negativity constraints , 1998 .

[47]  Joe Brewer,et al.  Kronecker products and matrix calculus in system theory , 1978 .

[48]  Nikos D. Sidiropoulos,et al.  Cramer-Rao lower bounds for low-rank decomposition of multidimensional arrays , 2001, IEEE Trans. Signal Process..

[49]  R. Cattell.  “Parallel proportional profiles” and other principles for determining the choice of factors by rotation , 1944 .

[50]  N. Sidiropoulos,et al.  On the uniqueness of multilinear decomposition of N‐way arrays , 2000 .

[51]  Nikos D. Sidiropoulos,et al.  Robust iterative fitting of multilinear models , 2005, IEEE Transactions on Signal Processing.

[52]  Nikos D. Sidiropoulos,et al.  Almost sure identifiability of multidimensional harmonic retrieval , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).