Greedy Approaches to Symmetric Orthogonal Tensor Decomposition

Finding the symmetric and orthogonal decomposition of a tensor is a recurring problem in signal processing, machine learning, and statistics. In this paper, we review, establish, and compare the perturbation bounds for two natural types of incremental rank-one approximation approaches. Numerical experiments and open questions are also presented and discussed.

[1]  H. Weyl.  Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung) , 1912 .

[2]  W. Kahan,et al.  The Rotation of Eigenvectors by a Perturbation. III , 1970 .

[3]  N. Z. Shor.  An approach to obtaining global extremums in polynomial mathematical programming problems , 1987 .

[4]  T. Rao,et al.  Tensor Methods in Statistics , 1989 .

[5]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[6]  Yurii Nesterov,et al.  Squared Functional Systems and Optimization Problems , 2000 .

[7]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[8]  P. Parrilo.  Structured semidefinite programs and semialgebraic geometry methods in robustness and optimization , 2000 .

[9]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[10]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[11]  Tamara G. Kolda,et al.  Orthogonal Tensor Decompositions , 2000, SIAM J. Matrix Anal. Appl..

[12]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[13]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[14]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[15]  Pablo A. Parrilo,et al.  Semidefinite programming relaxations for semialgebraic problems , 2003, Math. Program..

[16]  Lek-Heng Lim,et al.  Singular values and eigenvalues of tensors: a variational approach , 2005, 1st IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, 2005..

[17]  Liqun Qi,et al.  On the successive supersymmetric rank‐1 decomposition of higher‐order supersymmetric tensors , 2007, Numer. Linear Algebra Appl..

[18]  Didier Henrion,et al.  GloptiPoly 3: moments, optimization and semidefinite programming , 2007, Optim. Methods Softw..

[19]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[20]  Tamara G. Kolda,et al.  Shifted Power Method for Computing Tensor Eigenpairs , 2010, SIAM J. Matrix Anal. Appl..

[21]  Huan Wang,et al.  Exact Recovery of Sparsely-Used Dictionaries , 2012, COLT.

[22]  Lixing Han.  An unconstrained optimization approach for finding real eigenvalues of even order symmetric tensors , 2012, 1203.5150.

[23]  Shuzhong Zhang,et al.  Maximum Block Improvement and Polynomial Optimization , 2012, SIAM J. Optim..

[24]  Chen Ling,et al.  The Best Rank-1 Approximation of a Symmetric Tensor and Related Spherical Optimization Problems , 2012, SIAM J. Matrix Anal. Appl..

[25]  Qingzhi Yang,et al.  Properties and methods for finding the best rank-one approximation to higher-order tensors , 2014, Comput. Optim. Appl..

[26]  Li Wang,et al.  Semidefinite Relaxations for Best Rank-1 Tensor Approximations , 2013, SIAM J. Matrix Anal. Appl..

[27]  Anima Anandkumar,et al.  Tensor decompositions for learning latent variable models , 2012, J. Mach. Learn. Res..

[28]  Donald Goldfarb,et al.  Successive Rank-One Approximations for Nearly Orthogonally Decomposable Symmetric Tensors , 2015, SIAM J. Matrix Anal. Appl..

[29]  John Wright,et al.  Complete dictionary recovery over the sphere , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[30]  Yu-Hong Dai,et al.  A sequential subspace projection method for extreme Z-eigenvalues of supersymmetric tensors , 2015, Numer. Linear Algebra Appl..

[31]  Shiqian Ma,et al.  Tensor principal component analysis via convex optimization , 2012, Math. Program..

[32]  Pierre Comon,et al.  A Finite Algorithm to Compute Rank-1 Tensor Approximations , 2016, IEEE Signal Processing Letters.

[33]  Z. Wen,et al.  A note on semidefinite programming relaxations for polynomial optimization over a single sphere , 2016 .

[34]  Elina Robeva,et al.  Orthogonal Decomposition of Symmetric Tensors , 2014, SIAM J. Matrix Anal. Appl..

[35]  Yun S. Song,et al.  Orthogonal Tensor Decompositions via Two-Mode Higher-Order SVD (HOSVD) , 2016, ArXiv.