Overcomplete tensor decomposition via convex optimization

This work develops theories and computational methods for overcomplete, non-orthogonal tensor decomposition using convex optimization. Under an incoherence condition of the rank-one factors, we show that one can retrieve tensor decomposition by solving a convex, infinite-dimensional analog of ℓ1 minimization on the space of measures. The optimal value of this optimization defines the tensor nuclear norm. Two computational schemes are proposed to solve the infinite-dimensional optimization: semidefinite programs based on sum-of-squares relaxations and nonlinear programs that are an exact reformulation of the tensor nuclear norm. The latter exhibits superior performance compared with the state-of-the-art tensor decomposition methods.

[1]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[2]  Gongguo Tang,et al.  Near minimax line spectral estimation , 2013, 2013 47th Annual Conference on Information Sciences and Systems (CISS).

[3]  John Wright,et al.  Provable Low-Rank Tensor Recovery , 2014 .

[4]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[5]  Rekha R. Thomas,et al.  Semidefinite Optimization and Convex Algebraic Geometry , 2012 .

[6]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[7]  Renato D. C. Monteiro,et al.  A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization , 2003, Math. Program..

[8]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[9]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[10]  G. Watson.  Characterization of the subdifferential of some matrix norms , 1992 .

[11]  Reinhard Heckel,et al.  Super-Resolution Radar , 2014, ArXiv.

[12]  Anima Anandkumar,et al.  Analyzing Tensor Power Method Dynamics: Applications to Learning Overcomplete Latent Variable Models , 2014, ArXiv.

[13]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[14]  Bo Huang,et al.  Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery , 2013, ICML.

[15]  Alexander Barvinok,et al.  A course in convexity , 2002, Graduate studies in mathematics.

[16]  Parikshit Shah,et al.  Optimal Low-Rank Tensor Recovery from Separable Measurements: Four Contractions Suffice , 2015, ArXiv.

[17]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[18]  Pierre Comon,et al.  Tensor Decompositions, State of the Art and Applications , 2002 .

[19]  Pierre Comon,et al.  Blind Multilinear Identification , 2012, IEEE Transactions on Information Theory.

[20]  Gabriel Peyré,et al.  Exact Support Recovery for Sparse Spikes Deconvolution , 2013, Foundations of Computational Mathematics.

[21]  Benjamin Recht,et al.  Atomic norm denoising with applications to line spectral estimation , 2011, Allerton.

[22]  Parikshit Shah,et al.  Guaranteed Tensor Decomposition: A Moment Approach , 2015, ICML.

[23]  Shai Dekel,et al.  Super-Resolution on the Sphere Using Convex Optimization , 2014, IEEE Transactions on Signal Processing.

[24]  WonkaPeter,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013 .

[25]  Gongguo Tang.  Atomic Decomposition of Mixtures of Translation-Invariant Signals , 2013 .

[26]  Gongguo Tang,et al.  Resolution limits for atomic decompositions via Markov-Bernstein type inequalities , 2015, 2015 International Conference on Sampling Theory and Applications (SampTA).

[27]  Parikshit Shah,et al.  Compressed Sensing Off the Grid , 2012, IEEE Transactions on Information Theory.

[28]  Shai Dekel,et al.  Robust Recovery of Stream of Pulses using Convex Optimization , 2014, ArXiv.

[29]  Prateek Jain,et al.  Provable Tensor Factorization with Missing Data , 2014, NIPS.

[30]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[31]  Carlos Fernandez-Granda.  Support detection in super-resolution , 2013, ArXiv.

[32]  Santosh S. Vempala,et al.  Fourier PCA and robust tensor decomposition , 2013, STOC.

[33]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[34]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  J. Lasserre.  Moments, Positive Polynomials And Their Applications , 2009 .

[36]  Emmanuel J. Cand.  Towards a Mathematical Theory of Super-Resolution , 2012 .

[37]  Anima Anandkumar,et al.  Guaranteed Non-Orthogonal Tensor Decomposition via Alternating Rank-1 Updates , 2014, ArXiv.

[38]  Sham M. Kakade,et al.  Learning mixtures of spherical gaussians: moment methods and spectral decompositions , 2012, ITCS '13.

[39]  Pierre Comon,et al.  Multiarray Signal Processing: Tensor decomposition meets compressed sensing , 2010, ArXiv.

[40]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[41]  Thomas B. Schön,et al.  2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2015 , 2016 .

[42]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[43]  David Steurer,et al.  Dictionary Learning and Tensor Decomposition via the Sum-of-Squares Method , 2014, STOC.

[44]  Annie Chen,et al.  Context-Aware Collaborative Filtering System: Predicting the User's Preference in the Ubiquitous Computing Environment , 2005, LoCA.

[45]  Jean-Francois Cardoso,et al.  Source separation using higher order moments , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[46]  Parikshit Shah,et al.  Robust line spectral estimation , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[47]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.