Numerical CP decomposition of some difficult tensors

In this paper, a numerical method is proposed for canonical polyadic (CP) decomposition of small size tensors. The focus is primarily on decomposition of tensors that correspond to small matrix multiplications. Here, rank of the tensors is equal to the smallest number of scalar multiplications that are necessary to accomplish the matrix multiplication. The proposed method is based on a constrained Levenberg-Marquardt optimization. Numerical results indicate the rank and border ranks of tensors that correspond to multiplication of matrices of the size 2x3 and 3x2, 3x3 and 3x2, 3x3 and 3x3, and 3x4 and 4x3. The ranks are 11, 15, 23 and 29, respectively. In particular, a novel algorithm for multiplying the matrices of the sizes 3x3 and 3x2 with 15 multiplications is presented.

[1]  Éric Schost,et al.  Optimization techniques for small matrix multiplication , 2011, Theor. Comput. Sci..

[2]  John E. Hopcroft,et al.  Duality applied to the complexity of matrix multiplications and other bilinear forms , 1973, STOC '73.

[3]  G. Schachtel,et al.  A Noncommutative Algorithm for Multiplying 5*5 Matrices Using 103 Multiplications , 1978, Inf. Process. Lett..

[4]  O. M. Makarov.  A non-commutative algorithm for multiplying 5×5 matrices using one hundred multiplications , 1988 .

[5]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[6]  John E. Hopcroft,et al.  Duality Applied to the Complexity of Matrix Multiplication and Other Bilinear Forms , 1973, SIAM J. Comput..

[7]  Shmuel Winograd,et al.  On multiplication of 2 × 2 matrices , 1971 .

[8]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[9]  Andrzej Cichocki,et al.  Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC , 2012, SIAM J. Matrix Anal. Appl..

[10]  V. Strassen.  Gaussian elimination is not optimal , 1969 .

[11]  Éric Schost,et al.  Optimization techniques for small matrix multiplication , 2011, ACCA.

[12]  Julian D. Laderman,et al.  A noncommutative algorithm for multiplying $3 \times 3$ matrices using 23 multiplications , 1976 .

[13]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[14]  R. Brent.  Algorithms for matrix multiplication , 1970 .

[15]  Virginia Vassilevska Williams,et al.  Multiplying matrices faster than coppersmith-winograd , 2012, STOC '12.

[16]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[17]  Lieven De Lathauwer,et al.  Structured Data Fusion , 2015, IEEE Journal of Selected Topics in Signal Processing.

[18]  Bijan Afsari,et al.  Sensitivity Analysis for the Problem of Matrix Joint Diagonalization , 2008, SIAM J. Matrix Anal. Appl..

[19]  J. Landsberg.  Tensors: Geometry and Applications , 2011 .

[20]  D. D. Morrison.  Methods for nonlinear least squares problems and convergence proofs , 1960 .