A Fast and Accurate Matrix Completion Method Based on QR Decomposition and $L_{2,1}$ -Norm Minimization

Low-rank matrix completion aims to recover matrices with missing entries and has attracted considerable attention from machine learning researchers. Most of the existing methods, such as weighted nuclear-norm-minimization-based methods and Qatar Riyal (QR)-decomposition-based methods, cannot provide both convergence accuracy and convergence speed. To investigate a fast and accurate completion method, an iterative QR-decomposition-based method is proposed for computing an approximate singular value decomposition. This method can compute the largest <inline-formula> <tex-math notation="LaTeX">$r (r>0)$ </tex-math></inline-formula> singular values of a matrix by iterative QR decomposition. Then, under the framework of matrix trifactorization, a method for computing an approximate SVD based on QR decomposition (CSVD-QR)-based <inline-formula> <tex-math notation="LaTeX">$L_{2,1}$ </tex-math></inline-formula>-norm minimization method (LNM-QR) is proposed for fast matrix completion. Theoretical analysis shows that this QR-decomposition-based method can obtain the same optimal solution as a nuclear norm minimization method, i.e., the <inline-formula> <tex-math notation="LaTeX">$L_{2,1}$ </tex-math></inline-formula>-norm of a submatrix can converge to its nuclear norm. Consequently, an LNM-QR-based iteratively reweighted <inline-formula> <tex-math notation="LaTeX">$L_{2,1}$ </tex-math></inline-formula>-norm minimization method (IRLNM-QR) is proposed to improve the accuracy of LNM-QR. Theoretical analysis shows that IRLNM-QR is as accurate as an iteratively reweighted nuclear norm minimization method, which is much more accurate than the traditional QR-decomposition-based matrix completion methods. Experimental results <italic>obtained</italic> on both synthetic and real-world visual data sets show that our methods are much faster and more accurate than the state-of-the-art methods.

[1]  Zhao Kang,et al.  Robust Subspace Clustering via Tighter Rank Approximation , 2015, CIKM.

[2]  Shuicheng Yan,et al.  Nonconvex Nonsmooth Low Rank Minimization via Iteratively Reweighted Nuclear Norm , 2015, IEEE Transactions on Image Processing.

[3]  Xuelong Li,et al.  Fast and Accurate Matrix Completion via Truncated Nuclear Norm Regularization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Justin P. Haldar,et al.  Rank-Constrained Solutions to Linear Matrix Equations Using PowerFactorization , 2009, IEEE Signal Processing Letters.

[5]  Jie Zhang,et al.  Structure-Constrained Low-Rank Representation , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Lin Wu,et al.  Multiview Spectral Clustering via Structured Low-Rank Matrix Factorization , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[7]  Xuelong Li,et al.  Robust Semi-Supervised Subspace Clustering via Non-Negative Low-Rank Representation , 2016, IEEE Transactions on Cybernetics.

[8]  Bart De Schutter,et al.  The QR Decomposition and the Singular Value Decomposition in the Symmetrized Max-Plus Algebra , 1998 .

[9]  Samuel Cheng,et al.  Decomposition Approach for Low-Rank Matrix Completion and Its Applications , 2014, IEEE Transactions on Signal Processing.

[10]  Dong Xu,et al.  Robust Kernel Low-Rank Representation , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[11]  Matthieu Puigt,et al.  Fast Nonnegative Matrix Factorization and Completion Using Nesterov Iterations , 2017, LVA/ICA.

[12]  Shuicheng Yan,et al.  Generalized Singular Value Thresholding , 2014, AAAI.

[13]  Yiu-Ming Cheung,et al.  Rank-One Matrix Completion With Automatic Rank Estimation via L1-Norm Regularization , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[15]  Tommi S. Jaakkola,et al.  Weighted Low-Rank Approximations , 2003, ICML.

[16]  Xuelong Li,et al.  Joint Embedding Learning and Sparse Regression: A Framework for Unsupervised Feature Selection , 2014, IEEE Transactions on Cybernetics.

[17]  Feiping Nie,et al.  Efficient and Robust Feature Selection via Joint ℓ2, 1-Norms Minimization , 2010, NIPS.

[18]  Feiping Nie,et al.  Optimal Mean Robust Principal Component Analysis , 2014, ICML.

[19]  Chi Fang,et al.  Robust Image Restoration via Adaptive Low-Rank Approximation and Joint Kernel Regression , 2014, IEEE Transactions on Image Processing.

[20]  Yangyang Kang,et al.  Robust and scalable matrix completion , 2016, 2016 International Conference on Big Data and Smart Computing (BigComp).

[21]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[22]  James T. Kwok,et al.  Large-Scale Nyström Kernel Matrix Approximation Using Randomized SVD , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Fillia Makedon,et al.  Learning from Incomplete Ratings Using Non-negative Matrix Factorization , 2006, SDM.

[24]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[25]  Wen Hong,et al.  Matrix-Completion-Based Airborne Tomographic SAR Inversion Under Missing Data , 2015, IEEE Geoscience and Remote Sensing Letters.

[26]  Jong Chul Ye,et al.  Annihilating Filter-Based Low-Rank Hankel Matrix Approach for Image Inpainting , 2015, IEEE Transactions on Image Processing.

[27]  Qing Liu,et al.  A Truncated Nuclear Norm Regularization Method Based on Weighted Residual Error for Matrix Completion , 2016, IEEE Transactions on Image Processing.

[28]  Alexandre Bernardino,et al.  Matrix Completion for Weakly-Supervised Multi-Label Image Classification , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[31]  Zhao Kang,et al.  Top-N Recommender System via Matrix Completion , 2016, AAAI.

[32]  Zhao Kang,et al.  Robust PCA Via Nonconvex Rank Approximation , 2015, 2015 IEEE International Conference on Data Mining.

[33]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[34]  Zhengming Ding,et al.  Robust Multiview Data Analysis Through Collective Low-Rank Subspace , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[35]  Feiping Nie,et al.  Low-Rank Matrix Recovery via Efficient Schatten p-Norm Minimization , 2012, AAAI.

[36]  Alexandre Bernardino,et al.  Unifying Nuclear Norm and Bilinear Factorization Approaches for Low-Rank Matrix Decomposition , 2013, 2013 IEEE International Conference on Computer Vision.

[37]  James Demmel,et al.  Accurate Singular Values of Bidiagonal Matrices , 1990, SIAM J. Sci. Comput..

[38]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.

[39]  Yinghuan Shi,et al.  Incomplete-Data Oriented Multiview Dimension Reduction via Sparse Low-Rank Representation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Athanasios Mouchtaris,et al.  Joint low-rank representation and matrix completion under a singular value thresholding framework , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[41]  Zhao Kang,et al.  Kernel-driven similarity learning , 2017, Neurocomputing.

[42]  Feiping Nie,et al.  Robust Discrete Matrix Completion , 2013, AAAI.

[43]  Licheng Jiao,et al.  A fast tri-factorization method for low-rank matrix recovery and completion , 2013, Pattern Recognit..

[44]  Yudong Chen,et al.  Incoherence-Optimal Matrix Completion , 2013, IEEE Transactions on Information Theory.

[45]  Yong Luo,et al.  Multiview Matrix Completion for Multilabel Image Classification , 2015, IEEE Transactions on Image Processing.

[46]  Feiping Nie,et al.  Robust Matrix Completion via Joint Schatten p-Norm and lp-Norm Minimization , 2012, 2012 IEEE 12th International Conference on Data Mining.

[47]  Peder A. Olsen,et al.  Nuclear Norm Minimization via Active Subspace Selection , 2014, ICML.

[48]  Junbin Gao,et al.  Tensor LRR and Sparse Coding-Based Subspace Clustering , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[49]  James Demmel,et al.  Jacobi's Method is More Accurate than QR , 1989, SIAM J. Matrix Anal. Appl..

[50]  Hanghang Tong,et al.  Robust bilinear factorization with missing and grossly corrupted observations , 2015, Inf. Sci..