A nonconvex approach to low-rank matrix completion using convex optimization

Summary This paper deals with the problem of recovering an unknown low-rank matrix from a sampling of its entries. For its solution, we consider a nonconvex approach based on the minimization of a nonconvex functional that is the sum of a convex fidelity term and a nonconvex, nonsmooth relaxation of the rank function. We show that by a suitable choice of this nonconvex penalty, it is possible, under mild assumptions, to use also in this matrix setting the iterative forward–backward splitting method. Specifically, we propose the use of certain parameter dependent nonconvex penalties that with a good choice of the parameter value allow us to solve in the backward step a convex minimization problem, and we exploit this result to prove the convergence of the iterative forward–backward splitting algorithm. Based on the theoretical results, we develop for the solution of the matrix completion problem the efficient iterative improved matrix completion forward–backward algorithm, which exhibits lower computing times and improved recovery performance when compared with the best state-of-the-art algorithms for matrix completion. Copyright © 2016 John Wiley & Sons, Ltd.

[1]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[2]  Damiana Lazzaro,et al.  A fast algorithm for nonconvex approaches to sparse recovery problems , 2013, Signal Process..

[3]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[4]  Massimo Fornasier,et al.  Low-rank Matrix Recovery via Iteratively Reweighted Least Squares Minimization , 2010, SIAM J. Optim..

[5]  Reinhold Schneider,et al.  Convergence Results for Projected Line-Search Methods on Varieties of Low-Rank Matrices Via Łojasiewicz Inequality , 2014, SIAM J. Optim..

[6]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[7]  A. Kruger On Fréchet Subdifferentials , 2003 .

[8]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[9]  Ivan W. Selesnick,et al.  Convex 1-D Total Variation Denoising with Non-convex Regularization , 2015, IEEE Signal Processing Letters.

[10]  Bing Zeng,et al.  Directional Discrete Cosine Transforms—A New Framework for Image Coding , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[12]  Shuicheng Yan,et al.  Generalized Nonconvex Nonsmooth Low-Rank Minimization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Wotao Yin,et al.  Improved Iteratively Reweighted Least Squares for Unconstrained Smoothed 퓁q Minimization , 2013, SIAM J. Numer. Anal..

[14]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[15]  Maryam Fazel,et al.  Iterative reweighted algorithms for matrix rank minimization , 2012, J. Mach. Learn. Res..

[16]  David Suter,et al.  Recovering the missing components in a large noisy low-rank matrix: application to SFM , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  L. Armytage History and context , 2012 .

[18]  Damiana Lazzaro,et al.  Fast Sparse Image Reconstruction Using Adaptive Nonlinear Filtering , 2011, IEEE Transactions on Image Processing.

[19]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[20]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[21]  A. Lewis The Convex Analysis of Unitarily Invariant Matrix Functions , 1995 .

[22]  Patrick L. Combettes,et al.  Proximal Thresholding Algorithm for Minimization over Orthonormal Bases , 2007, SIAM J. Optim..

[23]  Petros Drineas,et al.  Fast Monte Carlo Algorithms for Matrices I: Approximating Matrix Multiplication , 2006, SIAM J. Comput..

[24]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[25]  Ivan W. Selesnick,et al.  Sparse Signal Estimation by Maximally Sparse Convex Optimization , 2013, IEEE Transactions on Signal Processing.

[26]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[27]  Zuowei Shen,et al.  Robust video denoising using low rank matrix completion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Dima Grigoriev,et al.  Complexity of Quantifier Elimination in the Theory of Algebraically Closed Fields , 1984, MFCS.

[29]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[30]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[31]  Ivan W. Selesnick,et al.  Convex Denoising using Non-Convex Tight Frame Regularization , 2015, IEEE Signal Processing Letters.

[32]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[33]  Shimon Ullman,et al.  Uncovering shared structures in multiclass classification , 2007, ICML '07.

[34]  D. Goldfarb,et al.  Solving low-rank matrix completion problems efficiently , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[35]  Toshihiro Furukawa,et al.  Iterative partial matrix shrinkage algorithm for matrix rank minimization , 2014, Signal Process..

[36]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[37]  Massimiliano Pontil,et al.  Multi-Task Feature Learning , 2006, NIPS.

[38]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[39]  L. Mirsky A trace inequality of John von Neumann , 1975 .

[40]  Kim-Chuan Toh,et al.  Solving semidefinite-quadratic-linear programs using SDPT3 , 2003, Math. Program..

[41]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[42]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, ISIT.

[43]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[44]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[45]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).