Jacobi Algorithm for the Best Low Multilinear Rank Approximation of Symmetric Tensors

The problem discussed in this paper is the symmetric best low multilinear rank approximation of third-order symmetric tensors. We propose an algorithm based on Jacobi rotations, for which symmetry is preserved at each iteration. Two numerical examples are provided indicating the need for such algorithms. An important part of the paper consists of proving that our algorithm converges to stationary points of the objective function. This can be considered an advantage of the proposed algorithm over existing symmetry-preserving algorithms in the literature.

[1]  Lieven De Lathauwer,et al.  A survey of tensor methods , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[2]  Joos Vandewalle,et al.  Computation of the Canonical Decomposition by Means of a Simultaneous Generalized Schur Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[3]  Rasmus Bro,et al.  Improving the speed of multi-way algorithms:: Part I. Tucker3 , 1998 .

[4]  Berkant Savas,et al.  Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors , 2009, SIAM J. Sci. Comput..

[5]  Pierre-Antoine Absil,et al.  A Modified Particle Swarm Optimization Algorithm for the Best Low Multilinear Rank Approximation of Higher-Order Tensors , 2010, ANTS Conference.

[6]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[7]  Berkant Savas,et al.  Krylov Subspace Methods for Tensor Computations , 2009 .

[8]  J. Leeuw,et al.  Some additional results on principal components analysis of three-mode data by means of alternating least squares algorithms , 1987 .

[9]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[10]  Antoine Souloumiac,et al.  Jacobi Angles for Simultaneous Diagonalization , 1996, SIAM J. Matrix Anal. Appl..

[11]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[12]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[13]  F. L. Hitchcock The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[14]  F. L. Hitchcock Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[15]  Phillip A. Regalia,et al.  Monotonic convergence of fixed-point algorithms for ICA , 2003, IEEE Trans. Neural Networks.

[16]  Elijah Polak,et al.  Computational methods in optimization , 1971 .

[17]  L. Lathauwer,et al.  Dimensionality reduction in higher-order signal processing and rank-(R1,R2,…,RN) reduction in multilinear algebra , 2004 .

[18]  J. Leeuw,et al.  Principal component analysis of three-mode data by means of alternating least squares algorithms , 1980 .

[19]  P. McCullagh Tensor Methods in Statistics , 1987 .

[20]  Carla D. Moravitz Martin,et al.  A Jacobi-Type Method for Computing Orthogonal Tensor Decompositions , 2008, SIAM J. Matrix Anal. Appl..

[21]  C. L. Nikias,et al.  Signal processing with higher-order spectra , 1993, IEEE Signal Processing Magazine.

[22]  Berkant Savas,et al.  A Newton-Grassmann Method for Computing the Best Multilinear Rank-(r1, r2, r3) Approximation of a Tensor , 2009, SIAM J. Matrix Anal. Appl..

[23]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[24]  L. Lathauwer,et al.  On the best low multilinear rank approximation of higher-order tensors , 2010 .

[25]  Sabine Van Huffel,et al.  Differential-geometric Newton method for the best rank-(R1, R2, R3) approximation of tensors , 2008, Numerical Algorithms.

[26]  C. L. Nikias,et al.  Higher-order spectra analysis : a nonlinear signal processing framework , 1993 .

[27]  Ganesh R. Naik,et al.  Introduction: Independent Component Analysis , 2012 .

[28]  Yurii Nesterov,et al.  Generalized Power Method for Sparse Principal Component Analysis , 2008, J. Mach. Learn. Res..

[29]  L. Lathauwer First-order perturbation analysis of the best rank-(R1, R2, R3) approximation in multilinear algebra , 2004 .

[30]  P. Kroonenberg Applied Multiway Data Analysis , 2008 .

[31]  Berkant Savas,et al.  Krylov-Type Methods for Tensor Computations , 2010, 1005.0683.

[32]  Sabine Van Huffel,et al.  Best Low Multilinear Rank Approximation of Higher-Order Tensors, Based on the Riemannian Trust-Region Scheme , 2011, SIAM J. Matrix Anal. Appl..

[33]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[34]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[35]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[36]  Phillip A. Regalia,et al.  The higher-order power method revisited: convergence proofs and effective initialization , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[37]  Pierre-Antoine Absil,et al.  Accelerated Line-search and Trust-region Methods , 2009, SIAM J. Numer. Anal..

[38]  Mohammed Bennani Dosse,et al.  The Assumption of Proportional Components when Candecomp is Applied to Symmetric Matrices in the Context of Indscal , 2008, Psychometrika.

[39]  P. Regalia Monotonically convergent algorithms for symmetric tensor approximation , 2013 .

[40]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[41]  E. Polak,et al.  Computational methods in optimization : a unified approach , 1972 .

[42]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[43]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[44]  Sabine Van Huffel,et al.  Tucker compression and local optima , 2011 .

[45]  Tamara G. Kolda,et al.  Shifted Power Method for Computing Tensor Eigenpairs , 2010, SIAM J. Matrix Anal. Appl..

[46]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[47]  Joos Vandewalle,et al.  Independent component analysis and (simultaneous) third-order tensor diagonalization , 2001, IEEE Trans. Signal Process..

[48]  S. Friedland Best rank one approximation of real symmetric tensors can be chosen symmetric , 2011, 1110.5689.

[49]  Lieven De Lathauwer,et al.  Tensor-based techniques for the blind separation of DS-CDMA signals , 2007, Signal Process..

[50]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..