Rank minimization on tensor ring: an efficient approach for tensor decomposition and completion

In recent studies, tensor ring decomposition (TRD) has become a promising model for tensor completion. However, TRD suffers from the rank selection problem due to the undetermined multilinear rank. For tensor decomposition with missing entries, the sub-optimal rank selection of traditional methods leads to the overfitting/underfitting problem. In this paper, we first explore the latent space of the TRD and theoretically prove the relationship between the TR-rank and the rank of the tensor unfoldings. Then, we propose two tensor completion models by imposing the different low-rank regularizations on the TR-factors, by which the TR-rank of the underlying tensor is minimized and the low-rank structures of the underlying tensor are exploited. By employing the alternating direction method of multipliers scheme, our algorithms obtain the TR factors and the underlying tensor simultaneously. In experiments of tensor completion tasks, our algorithms show robustness to rank selection and high computation efficiency, in comparison to traditional low-rank approximation algorithms.

[1]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[2]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[3]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Naotaka Fujii,et al.  Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Qibin Zhao,et al.  Tensor-ring Nuclear Norm Minimization and Application for Visual : Data Completion , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  James Caverlee,et al.  Tensor Completion Algorithms in Big Data Analytics , 2017, ACM Trans. Knowl. Discov. Data.

[7]  Masashi Sugiyama,et al.  Learning Efficient Tensor Representations with Ring-structured Networks , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Liqing Zhang,et al.  Tensor Ring Decomposition , 2016, ArXiv.

[9]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[10]  Alexander Novikov,et al.  Tensorizing Neural Networks , 2015, NIPS.

[11]  Liangpei Zhang,et al.  Total-Variation-Regularized Low-Rank Matrix Factorization for Hyperspectral Image Restoration , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Ce Zhu,et al.  Low Rank Tensor Completion for Multiway Visual Data , 2018, 1805.03967.

[13]  Chao Li,et al.  Rank Minimization on Tensor Ring: A New Paradigm in Scalable Tensor Decomposition and Completion , 2018, ArXiv.

[14]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[15]  Bo Du,et al.  PLTD: Patch-Based Low-Rank Tensor Decomposition for Hyperspectral Images , 2017, IEEE Transactions on Multimedia.

[16]  Lexing Ying,et al.  Efficient Construction of Tensor Ring Representations from Sampling , 2017, Multiscale Model. Simul..

[17]  Larry S. Davis,et al.  Collaborative Fashion Recommendation: A Functional Tensor Factorization Approach , 2015, ACM Multimedia.

[18]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[19]  Jianting Cao,et al.  High-order tensor completion via gradient-based optimization under tensor train format , 2019, Signal Process. Image Commun..

[20]  Andrzej Cichocki,et al.  Smooth PARAFAC Decomposition for Tensor Completion , 2015, IEEE Transactions on Signal Processing.

[21]  Liqing Zhang,et al.  Bayesian Robust Tensor Factorization for Incomplete Multiway Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Ming Yuan,et al.  On Tensor Completion via Nuclear Norm Minimization , 2014, Foundations of Computational Mathematics.

[23]  Hong Cheng,et al.  Trace Norm Regularized CANDECOMP/PARAFAC Decomposition With Missing Data , 2015, IEEE Transactions on Cybernetics.

[24]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[25]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Minh N. Do,et al.  Efficient Tensor Completion for Color Image and Video Recovery: Low-Rank Tensor Train , 2016, IEEE Transactions on Image Processing.

[27]  Taiji Suzuki,et al.  Convex Tensor Decomposition via Structured Schatten Norm Regularization , 2013, NIPS.

[28]  Jianting Cao,et al.  Higher-dimension Tensor Completion via Low-rank Tensor Ring Decomposition , 2018, 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[29]  Hidekata Hontani,et al.  Missing Slice Recovery for Tensors Using a Low-Rank Model in Embedded Space , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  V. Aggarwal,et al.  Efficient Low Rank Tensor Ring Completion , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[31]  Shuicheng Yan,et al.  Sharing Residual Units Through Collective Tensor Factorization To Improve Deep Neural Networks , 2018, IJCAI.

[32]  Hong Cheng,et al.  Generalized Higher Order Orthogonal Iteration for Tensor Learning and Decomposition , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Misha Elena Kilmer,et al.  Novel Methods for Multilinear Data Completion and De-noising Based on Tensor-SVD , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Liangpei Zhang,et al.  Hyperspectral Image Restoration Using Low-Rank Matrix Recovery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Ce Zhu,et al.  Image Completion Using Low Tensor Tree Rank and Total Variation Minimization , 2019, IEEE Transactions on Multimedia.