Partitioned Hierarchical alternating least squares algorithm for CP tensor decomposition

Canonical polyadic decomposition (CPD), also known as PARAFAC, is a representation of a given tensor as a sum of rank-one tensors. Traditional method for accomplishing CPD is the alternating least squares (ALS) algorithm. This algorithm is easy to implement with very low computational complexity per iteration. A disadvantage is that in difficult scenarios, where factor matrices in the decomposition contain nearly collinear columns, the number of iterations needed to achieve convergence might be very large. In this paper, we propose a modification of the algorithm which has similar complexity per iteration as ALS, but in difficult scenarios it needs a significantly lower number of iterations.

[1]  Martin Haardt,et al.  Extension of the semi-algebraic framework for approximate CP decompositions via non-symmetric simultaneous matrix diagonalization , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  André Uschmajew,et al.  Local Convergence of the Alternating Least Squares Algorithm for Canonical Tensor Approximation , 2012, SIAM J. Matrix Anal. Appl..

[3]  Andrzej Cichocki,et al.  Partitioned Alternating Least Squares Technique for Canonical Polyadic Tensor Decomposition , 2016, IEEE Signal Processing Letters.

[4]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[5]  Lieven De Lathauwer,et al.  Structured Data Fusion , 2015, IEEE Journal of Selected Topics in Signal Processing.

[6]  P. Comon.  Tensor Diagonalization, A useful Tool in Signal Processing , 1994 .

[7]  P. Paatero.  A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[8]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[9]  Andrzej Cichocki,et al.  A further improvement of a fast damped Gauss-Newton algorithm for candecomp-parafac tensor decomposition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[10]  Andrzej Cichocki,et al.  Tensor diagonalization - a new tool for PARAFAC and block-term decomposition , 2014, ArXiv.

[11]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[12]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[13]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[14]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[15]  B. Kowalski,et al.  Tensorial resolution: A direct trilinear decomposition , 1990 .

[16]  Andrzej Cichocki,et al.  Numerical CP decomposition of some difficult tensors , 2016, J. Comput. Appl. Math..

[17]  Andrzej Cichocki,et al.  Tensor Deflation for CANDECOMP/PARAFAC— Part II: Initialization and Error Analysis , 2015, IEEE Transactions on Signal Processing.

[18]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[19]  Andrzej Cichocki,et al.  Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC , 2012, SIAM J. Matrix Anal. Appl..

[20]  Florian Roemer,et al.  A semi-algebraic framework for approximate CP decompositions via simultaneous matrix diagonalizations (SECSI) , 2013, Signal Process..