Bilinear factorizations subject to monomial equality constraints via tensor decompositions

Abstract The Canonical Polyadic Decomposition (CPD), which decomposes a tensor into a sum of rank one terms, plays an important role in signal processing and machine learning. In this paper we extend the CPD framework to the more general case of bilinear factorizations subject to monomial equality constraints. This includes extensions of multilinear algebraic uniqueness conditions originally developed for the CPD. We obtain a deterministic uniqueness condition that admits a constructive interpretation. Computationally, we reduce the bilinear factorization problem into a CPD problem, which can be solved via a matrix EigenValue Decomposition (EVD). Under the given conditions, the discussed EVD-based algorithms are guaranteed to return the exact bilinear factorization. Finally, we make a connection between bilinear factorizations subject to monomial equality constraints and the coupled block term decomposition, which allows us to translate monomial structures into low-rank structures.

[1]  Arogyaswami Paulraj,et al.  An analytical constant modulus algorithm , 1996, IEEE Trans. Signal Process..

[2]  Lieven De Lathauwer,et al.  Canonical Polyadic Decomposition of Third-Order Tensors: Reduction to Generalized Eigenvalue Decomposition , 2013, SIAM J. Matrix Anal. Appl..

[3]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[4]  Pierre Comon,et al.  Nonnegative approximations of nonnegative tensors , 2009, ArXiv.

[5]  Thomas Kailath,et al.  ESPRIT-estimation of signal parameters via rotational invariance techniques , 1989, IEEE Trans. Acoust. Speech Signal Process..

[6]  Lieven De Lathauwer,et al.  Generic Uniqueness of a Structured Matrix Factorization and Applications in Blind Source Separation , 2016, IEEE Journal of Selected Topics in Signal Processing.

[7]  Lieven De Lathauwer,et al.  Canonical Polyadic Decomposition of a Tensor That Has Missing Fibers: A Monomial Factorization Approach , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[9]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[10]  Yuval Kluger,et al.  Learning Binary Latent Variable Models: A Tensor Eigenpair Approach , 2018, ICML.

[11]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[12]  Lieven De Lathauwer,et al.  On the Uniqueness of the Canonical Polyadic Decomposition of Third-Order Tensors - Part I: Basic Results and Uniqueness of One Factor Matrix , 2013, SIAM J. Matrix Anal. Appl..

[13]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[14]  C. Matias,et al.  Identifiability of parameters in latent structure models with many observed variables , 2008, 0809.5032.

[15]  Tamir Hazan,et al.  Non-negative tensor factorization with applications to statistics and computer vision , 2005, ICML.

[16]  Lieven De Lathauwer,et al.  Multidimensional Harmonic Retrieval via Coupled Canonical Polyadic Decomposition—Part I: Model and Identifiability , 2017, IEEE Transactions on Signal Processing.

[17]  M. Lagunas-Hernandez,et al.  ARMA model maximum entropy power spectral estimation , 1984 .

[18]  S. Leurgans,et al.  A Decomposition for Three-Way Arrays , 1993, SIAM J. Matrix Anal. Appl..

[19]  Pierre Comon,et al.  Analytical blind discrete source separation , 2000, 2000 10th European Signal Processing Conference.

[20]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[21]  L. Lathauwer,et al.  Canonical polyadic decomposition of third-order tensors: relaxed uniqueness conditions and algebraic algorithm , 2015, 1501.07251.

[22]  Lieven De Lathauwer,et al.  Blind Multichannel Deconvolution and Convolutive Extensions of Canonical Polyadic and Block Term Decompositions , 2017, IEEE Transactions on Signal Processing.

[23]  M. Marcus,et al.  On the relation between the determinant and the permanent , 1961 .

[24]  Lieven De Lathauwer,et al.  On the Uniqueness of the Canonical Polyadic Decomposition of Third-Order Tensors - Part II: Uniqueness of the Overall Decomposition , 2013, SIAM J. Matrix Anal. Appl..

[25]  Nikos D. Sidiropoulos,et al.  From K-Means to Higher-Way Co-Clustering: Multilinear Decomposition With Sparse Latent Factors , 2013, IEEE Transactions on Signal Processing.

[26]  Alle-Jan van der Veen,et al.  Analytical method for blind binary signal separation , 1997, Proceedings of 13th International Conference on Digital Signal Processing.

[27]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[28]  Nikos D. Sidiropoulos,et al.  Kruskal's permutation lemma and the identification of CANDECOMP/PARAFAC and bilinear models with constant modulus constraints , 2004, IEEE Transactions on Signal Processing.

[29]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[30]  Lieven De Lathauwer,et al.  Multidimensional Harmonic Retrieval via Coupled Canonical Polyadic Decomposition—Part II: Algorithm and Multirate Sampling , 2017, IEEE Transactions on Signal Processing.

[31]  Ravindra B. Bapat,et al.  Mixed discriminants of positive semidefinite matrices , 1989 .

[32]  Lieven De Lathauwer,et al.  Coupled Canonical Polyadic Decompositions and (Coupled) Decompositions in Multilinear Rank- (Lr, n, Lr, n, 1) Terms - Part II: Algorithms , 2015, SIAM J. Matrix Anal. Appl..

[33]  Lieven De Lathauwer,et al.  Coupled Canonical Polyadic Decompositions and Multiple Shift Invariance in Array Processing , 2018, IEEE Transactions on Signal Processing.

[34]  Tamara G. Kolda,et al.  Temporal Analysis of Semantic Graphs Using ASALSAN , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[35]  C. Bocci,et al.  Refined methods for the identifiability of tensors , 2013, 1303.6915.

[36]  Matthias Hein,et al.  Matrix factorization with binary components , 2013, NIPS.

[37]  Xiao Fu,et al.  Tensors, Learning, and “Kolmogorov Extension” for Finite-Alphabet Random Vectors , 2017, IEEE Transactions on Signal Processing.

[38]  Pierre Comon,et al.  Identifiability of an X-Rank Decomposition of Polynomial Maps , 2016, SIAM J. Appl. Algebra Geom..

[39]  Thomas L. Griffiths,et al.  Infinite latent feature models and the Indian buffet process , 2005, NIPS.

[40]  Lieven De Lathauwer,et al.  Coupled Canonical Polyadic Decompositions and (Coupled) Decompositions in Multilinear Rank-(Lr, n, Lr, n, 1) Terms - Part I: Uniqueness , 2015, SIAM J. Matrix Anal. Appl..

[41]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[42]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.