Canonical Polyadic Tensor Decomposition With Low-Rank Factor Matrices

This paper proposes a constrained canonical polyadic (CP) tensor decomposition method with low-rank factor matrices. In this way, we allow the CP decomposition with high rank while keeping the number of the model parameters small. First, we propose an algorithm to decompose the tensors into factor matrices of given ranks. Second, we propose an algorithm which can determine the ranks of the factor matrices automatically, such that the fitting error is bounded by a user- selected constant. The algorithms are verified on the decomposition of a tensor of the MNIST hand-written image dataset.

[1]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[2]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[3]  Jian Sun,et al.  Accelerating Very Deep Convolutional Networks for Classification and Detection , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[5]  Ivan V. Oseledets,et al.  Speeding-up Convolutional Neural Networks Using Fine-tuned CP-Decomposition , 2014, ICLR.

[6]  Andrzej Cichocki,et al.  Sensitivity in Tensor Decomposition , 2019, IEEE Signal Processing Letters.

[7]  Eunhyeok Park,et al.  Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications , 2015, ICLR.

[8]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[9]  P. Paatero.  A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[10]  Rasmus Bro,et al.  Improving the speed of multiway algorithms: Part II: Compression , 1998 .

[11]  Andrzej Cichocki,et al.  Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC , 2012, SIAM J. Matrix Anal. Appl..

[12]  Andrzej Cichocki,et al.  Weighted Krylov-Levenberg-Marquardt Method for Canonical Polyadic Tensor Decomposition , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[14]  Andrzej Cichocki,et al.  Partitioned Alternating Least Squares Technique for Canonical Polyadic Tensor Decomposition , 2016, IEEE Signal Processing Letters.

[15]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[16]  R. Harshman,et al.  Modeling multi‐way data with linearly dependent loadings , 2009 .

[17]  Andrzej Cichocki,et al.  Fast Alternating LS Algorithms for High Order CANDECOMP/PARAFAC Tensor Factorizations , 2013, IEEE Transactions on Signal Processing.

[18]  Pierre Comon,et al.  CONFAC Decomposition Approach to Blind Identification of Underdetermined Mixtures Based on Generating Function Derivatives , 2012, IEEE Transactions on Signal Processing.

[19]  J. Kruskal,et al.  Candelinc: A general approach to multidimensional analysis of many-way arrays with linear constraints on parameters , 1980 .