Elastic-Net Regularization for Low-Rank Matrix Recovery

This paper considers the problem of recovering a low-rank matrix from a small number of measurements consisting of linear combinations of the matrix entries. We extend the elastic-net regularization in compressive sensing to a more general setting, the matrix recovery setting, and consider the elastic-net regularization scheme for matrix recovery. To investigate on the statistical properties of this scheme and in particular on its convergence properties, we set up a suitable mathematic framework. We characterize some properties of the estimator and construct a natural iterative procedure to compute it. The convergence analysis shows that the sequence of iterates converges, which then underlies successful applications of the matrix elastic-net regularization algorithm. In addition, the error bounds of the proposed algorithm for low-rank matrix and even for full-rank matrix are presented in this paper.

[1]  Yuan Yan Tang,et al.  Wavelet Orthonormal Decompositions for Extracting Features in Pattern Recognition , 1999, Int. J. Pattern Recognit. Artif. Intell..

[2]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[3]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[4]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[5]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[6]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[7]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[8]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[9]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[10]  Yuanyan Tang,et al.  Status of pattern recognition with wavelet analysis , 2008, Frontiers of Computer Science in China.

[11]  Yuanqing Li,et al.  Equivalence Probability and Sparsity of Two Sparse Solutions in Sparse Representation , 2008, IEEE Transactions on Neural Networks.

[12]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..

[13]  Lorenzo Rosasco,et al.  Elastic-net regularization in learning theory , 2008, J. Complex..

[14]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[15]  Andrea Montanari,et al.  Matrix completion from a few entries , 2009, ISIT.

[16]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[17]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[18]  Zhihua Zhang,et al.  On an Efficient Sparse Representation of Objects of General Shape via Continuous Extension and Wavelet Approximation , 2010, Int. J. Wavelets Multiresolution Inf. Process..

[19]  Xin Yao,et al.  Sparse Approximation Through Boosting for Learning Large Scale Kernel Machines , 2010, IEEE Transactions on Neural Networks.

[20]  Yuan Yan Tang,et al.  Local Learning Estimates by Integral Operators , 2010, Int. J. Wavelets Multiresolution Inf. Process..

[21]  Yuan Yan Tang,et al.  Locality Preserving Nonnegative Matrix Factorization with Application to Face Recognition , 2010, Int. J. Wavelets Multiresolution Inf. Process..

[22]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[23]  Emmanuel J. Candès,et al.  Matrix Completion With Noise , 2009, Proceedings of the IEEE.

[24]  Zongben Xu,et al.  Generalization Bounds of Regularization Algorithms Derived Simultaneously through Hypothesis Space Complexity, Algorithmic Stability and Data Quality , 2011, Int. J. Wavelets Multiresolution Inf. Process..

[25]  Emmanuel J. Candès,et al.  Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements , 2011, IEEE Transactions on Information Theory.

[26]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..

[27]  Gerlind Plonka-Hoch,et al.  Curvelet-Wavelet Regularized Split Bregman Iteration for Compressed Sensing , 2011, Int. J. Wavelets Multiresolution Inf. Process..

[28]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[29]  Luoqing Li,et al.  Least Square Regression with coefficient Regularization by Gradient Descent , 2012, Int. J. Wavelets Multiresolution Inf. Process..