Enhanced Low-Rank Matrix Approximation

This letter proposes to estimate low-rank matrices by formulating a convex optimization problem with nonconvex regularization. We employ parameterized nonconvex penalty functions to estimate the nonzero singular values more accurately than the nuclear norm. A closed-form solution for the global optimum of the proposed objective function (sum of data fidelity and the nonconvex regularizer) is also derived. The solution reduces to singular value thresholding method as a special case. The proposed method is demonstrated for image denoising.

[1]  Rick Chartrand,et al.  Nonconvex Splitting for Regularized Low-Rank + Sparse Decomposition , 2012, IEEE Transactions on Signal Processing.

[2]  Zhang Liu,et al.  Subspace system identification via weighted nuclear norm optimization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[3]  Nicolas Vayatis,et al.  Estimation of Simultaneously Sparse and Low Rank Matrices , 2012, ICML.

[4]  M. Nikolova,et al.  Estimation of binary images by minimizing convex criteria , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[5]  Jack Xin,et al.  Minimization of Transformed L1 Penalty: Closed Form Representation and Iterative Thresholding Algorithms , 2014, ArXiv.

[6]  Paul Tseng,et al.  Hankel Matrix Rank Minimization with Applications to System Identification and Realization , 2013, SIAM J. Matrix Anal. Appl..

[7]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[8]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[9]  Sanjeev Arora,et al.  Learning mixtures of arbitrary gaussians , 2001, STOC '01.

[10]  John Wright,et al.  Provable Models for Robust Low-Rank Tensor Completion , 2015 .

[11]  Arjan Kuijper,et al.  Scale Space and Variational Methods in Computer Vision , 2013, Lecture Notes in Computer Science.

[12]  Santosh S. Vempala,et al.  A spectral algorithm for learning mixtures of distributions , 2002, The 43rd Annual IEEE Symposium on Foundations of Computer Science, 2002. Proceedings..

[13]  Shuicheng Yan,et al.  Generalized Nonconvex Nonsmooth Low-Rank Minimization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Guangming Shi,et al.  Nonlocal Image Restoration With Bilateral Variance Estimation: A Low-Rank Approach , 2013, IEEE Transactions on Image Processing.

[15]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[16]  Shuicheng Yan,et al.  Nonconvex Nonsmooth Low Rank Minimization via Iteratively Reweighted Nuclear Norm , 2015, IEEE Transactions on Image Processing.

[17]  Raj Rao Nadakuditi,et al.  OptShrink: An Algorithm for Improved Low-Rank Signal Matrix Denoising by Optimal, Data-Driven Singular Value Shrinkage , 2013, IEEE Transactions on Information Theory.

[18]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[19]  Ivan Markovsky,et al.  Structured low-rank approximation and its applications , 2008, Autom..

[20]  Minh N. Do,et al.  Denoising MR Spectroscopic Imaging Data With Low-Rank Approximations , 2013, IEEE Transactions on Biomedical Engineering.

[21]  Rob Fergus,et al.  Fast Image Deconvolution using Hyper-Laplacian Priors , 2009, NIPS.

[22]  Ivan W. Selesnick,et al.  Group-Sparse Signal Denoising: Non-Convex Regularization, Convex Optimization , 2013, IEEE Transactions on Signal Processing.

[23]  Santosh S. Vempala,et al.  Spectral Algorithms , 2009, Found. Trends Theor. Comput. Sci..

[24]  Rick Chartrand,et al.  Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[25]  Ivan W. Selesnick,et al.  Convex Denoising using Non-Convex Tight Frame Regularization , 2015, IEEE Signal Processing Letters.

[26]  Donald Geman,et al.  Constrained Restoration and the Recovery of Discontinuities , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  A. Bruce,et al.  WAVESHRINK WITH FIRM SHRINKAGE , 1997 .

[28]  Armando Manduca,et al.  Highly Undersampled Magnetic Resonance Image Reconstruction via Homotopic $\ell_{0}$ -Minimization , 2009, IEEE Transactions on Medical Imaging.

[29]  Feiping Nie,et al.  Robust Matrix Completion via Joint Schatten p-Norm and lp-Norm Minimization , 2012, 2012 IEEE 12th International Conference on Data Mining.

[30]  Zhihua Zhang,et al.  Nonconvex Relaxation Approaches to Robust Matrix Recovery , 2013, IJCAI.

[31]  Serena Morigi,et al.  Convex Image Denoising via Non-Convex Regularization , 2015, SSVM.

[32]  Mathews Jacob,et al.  Nonlocal Regularization of Inverse Problems: A Unified Variational Framework , 2013, IEEE Transactions on Image Processing.

[33]  Maryam Fazel,et al.  Iterative reweighted algorithms for matrix rank minimization , 2012, J. Mach. Learn. Res..

[34]  Ivan W. Selesnick,et al.  Sparse Signal Estimation by Maximally Sparse Convex Optimization , 2013, IEEE Transactions on Signal Processing.

[35]  James A. Cadzow,et al.  Signal enhancement-a composite property mapping algorithm , 1988, IEEE Trans. Acoust. Speech Signal Process..

[36]  Ivan W. Selesnick,et al.  Artifact-Free Wavelet Denoising: Non-convex Sparse Regularization, Convex Optimization , 2015, IEEE Signal Processing Letters.

[37]  Patrick L. Combettes,et al.  Proximal Thresholding Algorithm for Minimization over Orthonormal Bases , 2007, SIAM J. Optim..

[38]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[39]  Yuanyuan Liu,et al.  Scalable Algorithms for Tractable Schatten Quasi-Norm Minimization , 2016, AAAI.

[40]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[41]  Lei Zhang,et al.  Centralized sparse representation for image restoration , 2011, 2011 International Conference on Computer Vision.

[42]  Dimitris Achlioptas,et al.  Fast computation of low-rank matrix approximations , 2007, JACM.

[43]  Sanjoy Dasgupta,et al.  Learning mixtures of Gaussians , 1999, 40th Annual Symposium on Foundations of Computer Science (Cat. No.99CB37039).

[44]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[45]  Donald W. Tufts,et al.  Estimation of a signal waveform from noisy data using low-rank approximation to a data matrix , 1993, IEEE Trans. Signal Process..

[46]  Shuicheng Yan,et al.  Generalized Singular Value Thresholding , 2014, AAAI.

[47]  C. N. hyndhavi,et al.  Group-sparse signal denoising : Non-convex Regularization , Convex Optimization , 2018 .

[48]  Mila Nikolova,et al.  Markovian reconstruction using a GNC approach , 1999, IEEE Trans. Image Process..

[49]  S. Chatterjee,et al.  Matrix estimation by Universal Singular Value Thresholding , 2012, 1212.1247.

[50]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[51]  Nathan Srebro,et al.  Concentration-Based Guarantees for Low-Rank Matrix Reconstruction , 2011, COLT.

[52]  Ilker Bayram,et al.  On the Convergence of the Iterative Shrinkage/Thresholding Algorithm With a Weakly Convex Penalty , 2015, IEEE Transactions on Signal Processing.

[53]  Jack Xin,et al.  Transformed Schatten-1 Iterative Thresholding Algorithms for Matrix Rank Minimization and Applications , 2015, ArXiv.

[54]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[55]  Petros Drineas,et al.  FAST MONTE CARLO ALGORITHMS FOR MATRICES II: COMPUTING A LOW-RANK APPROXIMATION TO A MATRIX∗ , 2004 .

[56]  I. Selesnick,et al.  Convex fused lasso denoising with non-convex regularization and its use for pulse detection , 2015, 2015 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[57]  Jacques Froment,et al.  Patch-Based Low-Rank Minimization for Image Denoising , 2015, ArXiv.

[58]  Brendt Wohlberg,et al.  A nonconvex ADMM algorithm for group sparsity with sparse groups , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[59]  Kyogu Lee,et al.  Vocal separation using extended robust principal component analysis with Schatten p/lp-norm and scale compression , 2014, 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[60]  Stefano Lucidi,et al.  A Derivative-Free Algorithm for Inequality Constrained Nonlinear Programming via Smoothing of an linfty Penalty Function , 2009, SIAM J. Optim..

[61]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Ruslan Salakhutdinov,et al.  Matrix reconstruction with the local max norm , 2012, NIPS.

[63]  Zuowei Shen,et al.  Robust video denoising using low rank matrix completion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[64]  Yanyang Zi,et al.  Sparsity-based Algorithm for Detecting Faults in Rotating Machines , 2015, ArXiv.

[65]  A. Lewis.  The Convex Analysis of Unitarily Invariant Matrix Functions , 1995 .

[66]  José V. Manjón,et al.  MRI denoising using Non-Local Means , 2008, Medical Image Anal..

[67]  Ivan W. Selesnick,et al.  Convex 1-D Total Variation Denoising with Non-convex Regularization , 2015, IEEE Signal Processing Letters.

[68]  Cun-Hui Zhang.  Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[69]  Ilker Bayram.  On the Convergence of the Iterative Shrinkage/Thresholding Algorithm With a Weakly Convex Penalty , 2016, IEEE Trans. Signal Process..

[70]  Louis L. Scharf,et al.  The SVD and reduced rank signal processing , 1991, Signal Process..