Low-Rank Tensor Recovery using Sequentially Optimal Modal Projections in Iterative Hard Thresholding (SeMPIHT)

Iterative hard thresholding (IHT) is a simple and effective approach to parsimonious data recovery. Its multilinear rank (mrank)-based application to low-rank tensor recovery (LRTR) is especially valuable given the difficulties involved in this problem. In this paper, we propose a novel IHT algorithm for LRTR, choosing sequential per-mode SVD truncation as its thresholding operator. This operator is less costly than those used in existing IHT algorithms for LRTR, and often leads to superior performance. Furthermore, by exploiting the sequential optimality of the employed modal projections, we derive recovery guarantees relying on restricted isometry constants. Though these guarantees are suboptimal, our numerical studies indicate that a quasi-optimal number of Gaussian measurements suffices for perfect data reconstruction. We also investigate a continuation technique which yields a sequence of progressively more complex estimated models until attaining a target mrank. When recovering real-world data, this...

[1]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[2]  Bo Huang,et al.  Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery , 2013, ICML.

[3]  Mike E. Davies,et al.  Normalized Iterative Hard Thresholding: Guaranteed Stability and Performance , 2010, IEEE Journal of Selected Topics in Signal Processing.

[4]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[5]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[6]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[7]  Ryota Tomioka,et al.  On the extension of trace norm to tensors , 2010 .

[8]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[9]  Ming Yuan,et al.  On Tensor Completion via Nuclear Norm Minimization , 2014, Foundations of Computational Mathematics.

[10]  Bart Vandereycken,et al.  Low-rank tensor completion by Riemannian optimization , 2014 .

[11]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[12]  Gérard Favier,et al.  An iterative hard thresholding algorithm with improved convergence for low-rank tensor recovery , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[13]  Johan A. K. Suykens,et al.  Tensor Versus Matrix Completion: A Comparison With Application to Spectral Data , 2011, IEEE Signal Processing Letters.

[14]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[15]  Lars Grasedyck,et al.  Variants of Alternating Least Squares Tensor Completion in the Tensor Train Format , 2015, SIAM J. Sci. Comput..

[16]  Hisashi Kashima,et al.  Statistical Performance of Convex Tensor Decomposition , 2011, NIPS.

[17]  Volkan Cevher,et al.  Matrix Recipes for Hard Thresholding Methods , 2012, Journal of Mathematical Imaging and Vision.

[18]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[19]  N. Kreimer,et al.  Tensor completion based on nuclear norm minimization for 5D seismic data reconstruction , 2013 .

[20]  Ryota Tomioka,et al.  Estimation of low-rank tensors via convex optimization , 2010, 1010.0789.

[21]  Johan A. K. Suykens,et al.  A Rank-One Tensor Updating Algorithm for Tensor Completion , 2015, IEEE Signal Processing Letters.

[22]  Lars Grasedyck,et al.  Hierarchical Singular Value Decomposition of Tensors , 2010, SIAM J. Matrix Anal. Appl..

[23]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[24]  T. Blumensath,et al.  Iterative Thresholding for Sparse Approximations , 2008 .

[25]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[26]  Jared Tanner,et al.  Normalized Iterative Hard Thresholding for Matrix Completion , 2013, SIAM J. Sci. Comput..

[27]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[28]  Raf Vandebril,et al.  A New Truncation Strategy for the Higher-Order Singular Value Decomposition , 2012, SIAM J. Sci. Comput..

[29]  Inderjit S. Dhillon,et al.  Guaranteed Rank Minimization via Singular Value Projection , 2009, NIPS.

[30]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[31]  Reinhold Schneider,et al.  Low rank tensor recovery via iterative hard thresholding , 2016, ArXiv.

[32]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[33]  Pablo A. Parrilo,et al.  The Convex Geometry of Linear Inverse Problems , 2010, Foundations of Computational Mathematics.

[34]  Wotao Yin,et al.  Parallel matrix factorization for low-rank tensor completion , 2013, ArXiv.

[35]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[37]  Shmuel Friedland,et al.  Computational Complexity of Tensor Nuclear Norm , 2014, ArXiv.

[38]  Lei Yang,et al.  Minimum n-rank approximation via iterative hard thresholding , 2013, Appl. Math. Comput..

[39]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[40]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[41]  Felix J. Herrmann,et al.  Optimization on the Hierarchical Tucker manifold – Applications to tensor completion , 2014, Linear Algebra and its Applications.

[42]  Hiroyuki Kasai,et al.  Riemannian preconditioning for tensor completion , 2015, ArXiv.