A Fused CP Factorization Method for Incomplete Tensors

Low-rank tensor completion methods have been advanced recently for modeling sparsely observed data with a multimode structure. However, low-rank priors may fail to interpret the model factors of general tensor objects. The most common method to address this drawback is to use regularizations together with the low-rank priors. However, due to the complex nature and diverse characteristics of real-world multiway data, the use of a single or a few regularizations remains far from efficient, and there are limited systematic experimental reports on the advantages of these regularizations for tensor completion. To fill these gaps, we propose a modified CP tensor factorization framework that fuses the <inline-formula> <tex-math notation="LaTeX">$l_{2}$ </tex-math></inline-formula> norm constraint, sparseness (<inline-formula> <tex-math notation="LaTeX">$l_{1}$ </tex-math></inline-formula> norm), manifold, and smooth information simultaneously. The factorization problem is addressed through a combination of Nesterov’s optimal gradient descent method and block coordinate descent. Here, we construct a smooth approximation to the <inline-formula> <tex-math notation="LaTeX">$l_{1}$ </tex-math></inline-formula> norm and TV norm regularizations, and then, the tensor factor is updated using the projected gradient method, where the step size is determined by the Lipschitz constant. Extensive experiments on simulation data, visual data completion, intelligent transportation systems, and GPS data of user involvement are conducted, and the efficiency of our method is confirmed by the results. Moreover, the obtained results reveal the characteristics of these commonly used regularizations for tensor completion in a certain sense and give experimental guidance concerning how to use them.

[1]  Bin Ran,et al.  Short‐term highway traffic flow prediction based on a hybrid strategy considering temporal–spatial information , 2016 .

[2]  Chun-Hou Zheng,et al.  Comparison of Non-negative Matrix Factorization Methods for Clustering Genomic Data , 2016, ICIC.

[3]  Haiying Li,et al.  Capacity-oriented passenger flow control under uncertain demand: Algorithm development and real-world case study , 2016 .

[4]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Chris H. Q. Ding,et al.  Collaborative Filtering: Weighted Nonnegative Matrix Factorization Incorporating User and Item Graphs , 2010, SDM.

[6]  Yunming Ye,et al.  MR-NTD: Manifold Regularization Nonnegative Tucker Decomposition for Tensor Data Dimension Reduction and Representation , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[8]  Bin Ran,et al.  Short-Term Traffic Prediction Based on Dynamic Tensor Completion , 2016, IEEE Transactions on Intelligent Transportation Systems.

[9]  Dacheng Tao,et al.  Subspaces Indexing Model on Grassmann Manifold for Image Search , 2011, IEEE Transactions on Image Processing.

[10]  Bin Ran,et al.  A hybrid deep learning based traffic flow prediction method and its understanding , 2018 .

[11]  Johan A. K. Suykens,et al.  Parallelized Tensor Train Learning of Polynomial Classifiers , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Zongben Xu,et al.  Folded-concave penalization approaches to tensor completion , 2015, Neurocomputing.

[13]  Wensheng Zhang,et al.  The Twist Tensor Nuclear Norm for Video Completion , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Qi Dai,et al.  The Physics of Compressive Sensing and the Gradient-Based Recovery Algorithms , 2009, ArXiv.

[15]  Christian Jutten,et al.  Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects , 2015, Proceedings of the IEEE.

[16]  Liqing Zhang,et al.  Bayesian Robust Tensor Factorization for Incomplete Multiway Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[17]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[18]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[19]  Xing Xie,et al.  Collaborative Filtering Meets Mobile Recommendation: A User-Centered Approach , 2010, AAAI.

[20]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[21]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[22]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Xi Chen,et al.  Smoothing proximal gradient method for general structured sparse regression , 2010, The Annals of Applied Statistics.

[24]  Bin Ran,et al.  Tensor based missing traffic data completion with spatial–temporal correlation , 2016 .

[25]  Marko Filipovic,et al.  Tucker factorization with missing data with application to low-$$n$$n-rank tensor completion , 2015, Multidimens. Syst. Signal Process..

[26]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[27]  Bin Ran,et al.  Tensor completion via a multi-linear low-n-rank factorization model , 2014, Neurocomputing.

[28]  Yue Gao,et al.  Camera Constraint-Free View-Based 3-D Object Retrieval , 2012, IEEE Transactions on Image Processing.

[29]  Li Li,et al.  Efficient missing data imputing for traffic flow by considering temporal and spatial dependence , 2013 .

[30]  Johan A. K. Suykens,et al.  Tensor Versus Matrix Completion: A Comparison With Application to Spectral Data , 2011, IEEE Signal Processing Letters.

[31]  Wotao Yin,et al.  A Block Coordinate Descent Method for Regularized Multiconvex Optimization with Applications to Nonnegative Tensor Factorization and Completion , 2013, SIAM J. Imaging Sci..

[32]  Huachun Tan,et al.  Robust tensor decomposition based on Cauchy distribution and its applications , 2017, Neurocomputing.

[33]  Tamara G. Kolda,et al.  Scalable Tensor Factorizations for Incomplete Data , 2010, ArXiv.

[34]  Minh N. Do,et al.  Efficient tensor completion: Low-rank tensor train , 2016, ArXiv.

[35]  Johan A. K. Suykens,et al.  Learning with tensors: a framework based on convex optimization and spectral regularization , 2014, Machine Learning.

[36]  Y. Nesterov A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[37]  F. L. Hitchcock The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[38]  Andrzej Cichocki,et al.  Smooth PARAFAC Decomposition for Tensor Completion , 2015, IEEE Transactions on Signal Processing.

[39]  Guillermo Sapiro,et al.  Kernelized Probabilistic Matrix Factorization: Exploiting Graphs and Side Information , 2012, SDM.

[40]  Hong Cheng,et al.  Generalized Higher Order Orthogonal Iteration for Tensor Learning and Decomposition , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[41]  Fabien Moutarde,et al.  Analysis of network-level traffic states using locality preservative non-negative matrix factorization , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[42]  Jian Zhang,et al.  Using Tensor Completion Method to Achieving Better Coverage of Traffic State Estimation from Sparse Floating Car Data , 2016, PloS one.

[43]  Ivan V. Oseledets,et al.  Tensor methods and recommender systems , 2016, Wiley Interdiscip. Rev. Data Min. Knowl. Discov..

[44]  Zenglin Xu,et al.  Bayesian Nonparametric Models for Multiway Data Analysis , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Guangdong Feng,et al.  A Tensor Based Method for Missing Traffic Data Completion , 2013 .

[46]  Lieven De Lathauwer,et al.  Structured Data Fusion , 2015, IEEE Journal of Selected Topics in Signal Processing.

[47]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[48]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Jen-Tzung Chien,et al.  Tensor-Factorized Neural Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[50]  Gonzalo Mateos,et al.  Rank Regularization and Bayesian Inference for Tensor Completion and Extrapolation , 2013, IEEE Transactions on Signal Processing.

[51]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[52]  Xuan Song,et al.  CitySpectrum: a non-negative tensor factorization approach , 2014, UbiComp.

[53]  Bo Huang,et al.  Square Deal: Lower Bounds and Improved Relaxations for Tensor Recovery , 2013, ICML.

[54]  Xinjian Chen,et al.  Fast algorithms for multidimensional DCT-to-DCT computation between a block and its associated subblocks , 2005, IEEE Trans. Signal Process..

[55]  Wenjiang J. Fu Penalized Regressions: The Bridge versus the Lasso , 1998 .

[56]  Ali Taylan Cemgil,et al.  Link prediction in heterogeneous data via generalized coupled tensor factorization , 2013, Data Mining and Knowledge Discovery.

[57]  Hisashi Kashima,et al.  Tensor factorization using auxiliary information , 2011, Data Mining and Knowledge Discovery.

[58]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[59]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..