A survey of tensor methods

Matrix decompositions have always been at the heart of signal, circuit and system theory. In particular, the Singular Value Decomposition (SVD) has been an important tool. There is currently a shift of paradigm in the algebraic foundations of these fields. Quite recently, Nonnegative Matrix Factorization (NMF) has been shown to outperform SVD at a number of tasks. Increasing research efforts are spent on the study and application of decompositions of higher-order tensors or multi-way arrays. This paper is a partial survey on tensor generalizations of the SVD and their applications. We also touch on Nonnegative Tensor Factorizations.

[1]  Nikos D. Sidiropoulos,et al.  Robust iterative fitting of multilinear models , 2005, IEEE Transactions on Signal Processing.

[2]  Max Welling,et al.  Positive tensor factorization , 2001, Pattern Recognit. Lett..

[3]  I. Jolliffe.  Principal Component Analysis , 2002 .

[4]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..

[5]  Sabine Van Huffel,et al.  Best multilinear rank approximation of tensors : a trust-region algorithm and local mimina (Best poster award) , 2009 .

[6]  Lieven De Lathauwer,et al.  A Method to Avoid Diverging Components in the Candecomp/Parafac Model for Generic I˟J˟2 Arrays , 2008, SIAM J. Matrix Anal. Appl..

[7]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[8]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[9]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[10]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[11]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[12]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[13]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[14]  R. Bro,et al.  A new efficient method for determining the number of components in PARAFAC models , 2003 .

[15]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[16]  L. De Lathauwer,et al.  A tensor-based blind ds-cdma receiver using simultaneous matrix diagonalization , 2007, 2007 IEEE 8th Workshop on Signal Processing Advances in Wireless Communications.

[17]  Sabine Van Huffel,et al.  Exponential data fitting using multilinear algebra: the single‐channel and multi‐channel case , 2005, Numer. Linear Algebra Appl..

[18]  I. Mechelen,et al.  Three-way component analysis: principles and illustrative application. , 2001, Psychological methods.

[19]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[20]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[21]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part I: Lemmas for Partitioned Matrices , 2008, SIAM J. Matrix Anal. Appl..

[22]  Bülent Yener,et al.  Unsupervised Multiway Data Analysis: A Literature Survey , 2009, IEEE Transactions on Knowledge and Data Engineering.

[23]  Lieven De Lathauwer,et al.  An enhanced line search scheme for complex-valued tensor decompositions. Application in DS-CDMA , 2008, Signal Process..

[24]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[25]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[26]  Joos Vandewalle,et al.  Independent component analysis and (simultaneous) third-order tensor diagonalization , 2001, IEEE Trans. Signal Process..

[27]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[28]  Demetri Terzopoulos,et al.  Multilinear subspace analysis of image ensembles , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Tamara G. Kolda,et al.  Multilinear Algebra for Analyzing Data with Multiple Linkages , 2006, Graph Algorithms in the Language of Linear Algebra.

[30]  L. Lathauwer,et al.  An enhanced plane search scheme for complex-valued tensor decompositions , 2010 .

[31]  Bülent Yener,et al.  Collective Sampling and Analysis of High Order Tensors for Chatroom Communications , 2006, ISI.

[32]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[33]  Donghui Chen,et al.  Nonnegativity constraints in numerical analysis , 2009, The Birth of Numerical Analysis.

[34]  Lieven De Lathauwer,et al.  Blind Deconvolution of DS-CDMA Signals by Means of Decomposition in Rank-$(1,L,L)$ Terms , 2008, IEEE Transactions on Signal Processing.

[35]  P. Paatero.  The Multilinear Engine—A Table-Driven, Least Squares Program for Solving Multilinear Problems, Including the n-Way Parallel Factor Analysis Model , 1999 .

[36]  Lieven De Lathauwer,et al.  Swamp reducing technique for tensor decomposition , 2008, 2008 16th European Signal Processing Conference.

[37]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[38]  L. Lathauwer,et al.  Dimensionality reduction for higher-order tensors: algorithms and applications , 2008 .

[39]  Berkant Savas,et al.  A Newton-Grassmann Method for Computing the Best Multilinear Rank-(r1, r2, r3) Approximation of a Tensor , 2009, SIAM J. Matrix Anal. Appl..

[40]  J. Chisholm.  Approximation by Sequences of Padé Approximants in Regions of Meromorphy , 1966 .

[41]  Carla D. Moravitz Martin,et al.  A Jacobi-Type Method for Computing Orthogonal Tensor Decompositions , 2008, SIAM J. Matrix Anal. Appl..

[42]  Alwin Stegeman,et al.  Low-Rank Approximation of Generic p˟q˟2 Arrays and Diverging Components in the Candecomp/Parafac Model , 2008, SIAM J. Matrix Anal. Appl..

[43]  Tamara G. Kolda,et al.  Orthogonal Tensor Decompositions , 2000, SIAM J. Matrix Anal. Appl..

[44]  Joos Vandewalle,et al.  Computation of the Canonical Decomposition by Means of a Simultaneous Generalized Schur Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[45]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[46]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[47]  Yeung Yam,et al.  Trade-off between approximation accuracy and complexity: TS controller design via HOSVD based complexity minimization , 2003 .

[48]  Berkant Savas,et al.  Best multilinear rank approximation of tensors with quasi-Newton methods on Grassmannians , 2008 .

[49]  N. Sidiropoulos,et al.  On the uniqueness of multilinear decomposition of N‐way arrays , 2000 .

[50]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[51]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[52]  L. De Lathauwer,et al.  Exponential data fitting using multilinear algebra: the decimative case , 2009 .

[53]  Pierre Comon,et al.  Decomposition of quantics in sums of powers of linear forms , 1996, Signal Process..

[54]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part III: Alternating Least Squares Algorithms , 2008, SIAM J. Matrix Anal. Appl..

[55]  Patrick Dupont,et al.  Canonical decomposition of ictal scalp EEG reliably detects the seizure onset zone , 2007, NeuroImage.

[56]  André Lima Férrer de Almeida,et al.  PARAFAC-based unified tensor modeling for wireless communication systems with application to blind multiuser equalization , 2007, Signal Process..

[57]  M. Alex O. Vasilescu,et al.  TensorTextures: multilinear image-based rendering , 2004, SIGGRAPH 2004.

[58]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[59]  P. Kroonenberg.  Applied Multiway Data Analysis , 2008 .

[60]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[61]  Tamara G. Kolda,et al.  Higher-order Web link analysis using multilinear algebra , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[62]  Berkant Savas,et al.  Handwritten digit classification using higher order singular value decomposition , 2007, Pattern Recognit..

[63]  S. Amari,et al.  Nonnegative Matrix and Tensor Factorization [Lecture Notes] , 2008, IEEE Signal Processing Magazine.

[64]  L. Lathauwer,et al.  Sufficient conditions for uniqueness in Candecomp/Parafac and Indscal with random component matrices , 2006 .

[65]  Rasmus Bro,et al.  Multiway analysis of epilepsy tensors , 2007, ISMB/ECCB.

[66]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[67]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[68]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[69]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[70]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[71]  L. Lathauwer,et al.  Dimensionality reduction in higher-order signal processing and rank-(R1,R2,…,RN) reduction in multilinear algebra , 2004 .

[72]  J. Vandewalle,et al.  An introduction to independent component analysis , 2000 .

[73]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[74]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[75]  A. Cichocki,et al.  Flexible HALS algorithms for sparse non-negative matrix/tensor factorization , 2008, 2008 IEEE Workshop on Machine Learning for Signal Processing.

[76]  I. Dhillon,et al.  Fast Projection-Based Methods for the Least Squares Nonnegative Matrix Approximation Problem , 2008 .

[77]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[78]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[79]  Lars Kai Hansen,et al.  Algorithms for Sparse Nonnegative Tucker Decompositions , 2008, Neural Computation.

[80]  Lieven De Lathauwer,et al.  A Block Component Model-Based Blind DS-CDMA Receiver , 2008, IEEE Transactions on Signal Processing.

[81]  Andrzej Cichocki,et al.  Fast and Efficient Algorithms for Nonnegative Tucker Decomposition , 2008, ISNN.

[82]  William S Rayens,et al.  Structure-seeking multilinear methods for the analysis of fMRI data , 2004, NeuroImage.