Robust tensor decomposition based on Cauchy distribution and its applications

Abstract Tensor analysis has reached a celebrity status in the areas of machine learning, computer vision, and artificial intelligence. Completing and recovering tensor is an important problem for tensor analysis. It involves recovering a tensor from either a subset of its entries or the whole entries contaminated by noise. Classical tensor completion/recovery methods are based mainly on l2-norm (Gaussian distribution noise) models, and are sensitive to noise of large magnitude. While l1-norm (Laplacian distribution noise) based methods are robust to noise of large magnitude, they do not deal with dense noise effectively. In this paper, we present a novel Cauchy tensor decomposition method for simultaneously recovering and completing a low rank tensor with both missing data and complex noise. We utilize the Cauchy distribution to model noise and derive the objective function of Cauchy tensor decomposition under the maximum likelihood estimation (MLE) framework. Then we developed a robust tensor decomposition framework that used first-order optimization approaches to optimize the objective function. Extensive numerical experiments are conducted and show that our method is able to successfully recover and complete tensors with large/dense noise and missing data. We further demonstrate the usefulness of Cauchy tensor decomposition on three real-world applications, image inpainting, traffic data process and foreground/background separation. The experimental results show that the method is applicable to a wide range of problems.

[1]  Zongben Xu,et al.  Folded-concave penalization approaches to tensor completion , 2015, Neurocomputing.

[2]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[3]  Nojun Kwak,et al.  Principal Component Analysis Based on L1-Norm Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  F. Hampel.  The Influence Curve and Its Role in Robust Estimation , 1974 .

[5]  Liangpei Zhang,et al.  Hyperspectral Image Restoration Using Low-Rank Matrix Recovery , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[6]  Tamara G. Kolda,et al.  Scalable Tensor Factorizations for Incomplete Data , 2010, ArXiv.

[7]  L. Lathauwer,et al.  On the Best Rank-1 and Rank-( , 2004 .

[8]  F. Xavier Roca,et al.  Exploiting multiple cues in motion segmentation based on background subtraction , 2013, Neurocomputing.

[9]  Bin Ran,et al.  Tensor completion via a multi-linear low-n-rank factorization model , 2014, Neurocomputing.

[10]  Johan A. K. Suykens,et al.  Learning with tensors: a framework based on convex optimization and spectral regularization , 2014, Machine Learning.

[11]  Gonzalo R. Arce,et al.  Statistically-Efficient Filtering in Impulsive Environments: Weighted Myriad Filters , 2002, EURASIP J. Adv. Signal Process..

[12]  Der-Horng Lee,et al.  Probe Vehicle Population and Sample Size for Arterial Speed Estimation , 2002 .

[13]  Xiao Zhang,et al.  Tensor completion via multi-shared-modes canonical correlation analysis , 2016, Neurocomputing.

[14]  Wuhong Wang,et al.  Weighted Tensor Completion-Based Traffic State Estimation Model , 2015 .

[15]  B. Recht,et al.  Tensor completion and low-n-rank tensor recovery via convex optimization , 2011 .

[16]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[17]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[18]  Guangdong Feng,et al.  Low-n-rank tensor recovery based on multi-linear augmented Lagrange multiplier method , 2013, Neurocomputing.

[19]  Chris H. Q. Ding,et al.  Robust tensor factorization using R1 norm , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Yin Li,et al.  Optimum Subspace Learning and Error Correction for Tensors , 2010, ECCV.

[21]  Jian Yang,et al.  Sparse Tensor Discriminant Color Space for Face Verification , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[22]  Weisheng Dong,et al.  3D magnetic resonance image denoising using low-rank tensor approximation , 2016, Neurocomputing.

[23]  Gonzalo R. Arce,et al.  Optimality of the myriad filter in practical impulsive-noise environments , 2001, IEEE Trans. Signal Process..

[24]  Donald Goldfarb,et al.  Robust Low-Rank Tensor Recovery: Models and Algorithms , 2013, SIAM J. Matrix Anal. Appl..

[25]  Xiaoqin Zhang,et al.  Incremental Tensor Subspace Learning and Its Applications to Foreground Segmentation and Tracking , 2011, International Journal of Computer Vision.

[26]  Xiaohu You,et al.  Energy-efficient resource allocation for OFDMA relay systems with imperfect CSIT , 2015, Science China Information Sciences.

[27]  Lei Zhang,et al.  Robust low-rank tensor factorization by cyclic weighted median , 2014, Science China Information Sciences.

[28]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2013, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Demetri Terzopoulos,et al.  Multilinear Analysis of Image Ensembles: TensorFaces , 2002, ECCV.

[30]  Bernardete Ribeiro,et al.  Analysis of trends in seasonal electrical energy consumption via non-negative tensor factorization , 2015, Neurocomputing.

[31]  Mehran Yazdi,et al.  Compression of Hyperspectral Images Using Discerete Wavelet Transform and Tucker Decomposition , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[32]  Eric C. Chi,et al.  Making Tensor Factorizations Robust to Non-Gaussian Noise , 2010, 1010.3043.

[33]  Xingyu Wang,et al.  Fast nonnegative tensor factorization based on accelerated proximal gradient and low-rank approximation , 2016, Neurocomputing.

[34]  Johan A. K. Suykens,et al.  Robust Low-Rank Tensor Recovery With Regularized Redescending M-Estimator , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Zenglin Xu,et al.  Bayesian Nonparametric Models for Multiway Data Analysis , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  A. Ben Hamza,et al.  Image denoising: a nonlinear robust statistical approach , 2001, IEEE Trans. Signal Process..

[37]  Guangdong Feng,et al.  A Tensor Based Method for Missing Traffic Data Completion , 2013 .

[38]  Stan Sclaroff,et al.  Segmenting foreground objects from a dynamic textured background via a robust Kalman filter , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[39]  Marko Filipovic,et al.  Tucker factorization with missing data with application to low-$$n$$n-rank tensor completion , 2015, Multidimens. Syst. Signal Process..

[40]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[41]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[42]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[43]  Bin Ran,et al.  Robust Missing Traffic Flow Imputation Considering Nonnegativity and Road Capacity , 2014 .

[44]  Huachun Tan,et al.  Low Multilinear Rank Approximation of Tensors and Application in Missing Traffic Data , 2014 .

[45]  Liqing Zhang,et al.  Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Kenneth E. Barner,et al.  Robust Sampling and Reconstruction Methods for Sparse Signals in the Presence of Impulsive Noise , 2010, IEEE Journal of Selected Topics in Signal Processing.

[47]  Lei Zhang,et al.  An Adaptive Longitudinal Driving Assistance System Based on Driver Characteristics , 2013, IEEE Transactions on Intelligent Transportation Systems.

[48]  Jian Yang,et al.  Modified Principal Component Analysis: An Integration of Multiple Similarity Subspace Models , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[49]  John Law,et al.  Robust Statistics—The Approach Based on Influence Functions , 1986 .