Semidefinite Relaxations for Best Rank-1 Tensor Approximations

This paper studies the problem of finding best rank-1 approximations for both symmetric and nonsymmetric tensors. For symmetric tensors, this is equivalent to optimizing homogeneous polynomials over unit spheres; for nonsymmetric tensors, this is equivalent to optimizing multiquadratic forms over multispheres. We propose semidefinite relaxations, based on sum of squares representations, to solve these polynomial optimization problems. Their special properties and structures are studied. In applications, the resulting semidefinite programs are often large scale. The recent Newton-CG augmented Lagrangian method by Zhao, Sun, and Toh [SIAM J. Optim., 20 (2010), pp. 1737--1765] is suitable for solving these semidefinite relaxations. Extensive numerical experiments are presented to show that this approach is efficient in getting best rank-1 approximations.

[1]  O. Taussky.  Sums of Squares , 1970 .

[2]  Man-Duen Choi.  Positive semidefinite biquadratic forms , 1975 .

[3]  Olga Taussky-Todd.  SOME CONCRETE ASPECTS OF HILBERT'S 17TH PROBLEM , 1996 .

[4]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[5]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[6]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[7]  Pablo A. Parrilo,et al.  Minimizing Polynomial Functions , 2001, Algorithmic and Quantitative Aspects of Real Algebraic Geometry in Mathematics and Computer Science.

[8]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[9]  Pierre Comon,et al.  Tensor Decompositions, State of the Art and Applications , 2002 .

[10]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[11]  Renato D. C. Monteiro,et al.  A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization , 2003, Math. Program..

[12]  Renato D. C. Monteiro,et al.  Digital Object Identifier (DOI) 10.1007/s10107-004-0564-1 , 2004 .

[13]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[14]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[15]  Pierre Comon,et al.  Symmetric tensor decomposition , 2009, 2009 17th European Signal Processing Conference.

[16]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[17]  J. Lasserre.  Moments, Positive Polynomials And Their Applications , 2009 .

[18]  M. Laurent.  Sums of Squares, Moment Matrices and Optimization Over Polynomials , 2009 .

[19]  Chen Ling,et al.  Biquadratic Optimization Over Unit Spheres and Semidefinite Programming Relaxations , 2009, SIAM J. Optim..

[20]  Kim-Chuan Toh,et al.  A Newton-CG Augmented Lagrangian Method for Semidefinite Programming , 2010, SIAM J. Optim..

[21]  Berkant Savas,et al.  Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors , 2009, SIAM J. Sci. Comput..

[22]  Liqun Qi,et al.  The Best Rank-One Approximation Ratio of a Tensor Space , 2011, SIAM J. Matrix Anal. Appl..

[23]  J. Landsberg.  Tensors: Geometry and Applications , 2011 .

[24]  Tamara G. Kolda,et al.  Shifted Power Method for Computing Tensor Eigenpairs , 2010, SIAM J. Matrix Anal. Appl..

[25]  Jiawang Nie.  Sum of squares methods for minimizing polynomial forms over spheres and hypersurfaces , 2012 .

[26]  Li Wang,et al.  Regularization Methods for SDP Relaxations in Large-Scale Polynomial Optimization , 2009, SIAM J. Optim..

[27]  Chen Ling,et al.  The Best Rank-1 Approximation of a Symmetric Tensor and Related Spherical Optimization Problems , 2012, SIAM J. Matrix Anal. Appl..

[28]  Pierre Comon,et al.  General tensor decomposition, moment matrices and applications , 2013, J. Symb. Comput..

[29]  Zheng-Hai Huang,et al.  Finding the extreme Z‐eigenvalues of tensors via a sequential semidefinite programming method , 2013, Numer. Linear Algebra Appl..

[30]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[31]  Luke Oeding,et al.  Eigenvectors of tensors and algorithms for Waring decomposition , 2011, J. Symb. Comput..

[32]  Paul Van Dooren,et al.  Jacobi Algorithm for the Best Low Multilinear Rank Approximation of Symmetric Tensors , 2013, SIAM J. Matrix Anal. Appl..

[33]  Monique Laurent,et al.  Semidefinite optimization , 2019, Graphs and Geometry.

[34]  Jiawang Nie,et al.  Optimality conditions and finite convergence of Lasserre’s hierarchy , 2012, Math. Program..

[35]  A. Ivic.  Sums of squares , 2020, An Introduction to 𝑞-analysis.