Approximation of rank function and its application to the nearest low-rank correlation matrix

The rank function rank(.) is neither continuous nor convex which brings much difficulty to the solution of rank minimization problems. In this paper, we provide a unified framework to construct the approximation functions of rank(.), and study their favorable properties. Particularly, with two families of approximation functions, we propose a convex relaxation method for the rank minimization problems with positive semidefinite cone constraints, and illustrate its application by computing the nearest low-rank correlation matrix. Numerical results indicate that this convex relaxation method is comparable with the sequential semismooth Newton method (Li and Qi in SIAM J Optim 21:1641–1666, 2011) and the majorized penalty approach (Gao and Sun, 2010) in terms of the quality of solutions.

[1]  Yong-Jin Liu,et al.  An implementable proximal point algorithmic framework for nuclear norm minimization , 2012, Math. Program..

[2]  Stephen P. Boyd,et al.  Rank minimization and applications in system theory , 2004, Proceedings of the 2004 American Control Conference.

[3]  Emmanuel J. Candès,et al.  A Singular Value Thresholding Algorithm for Matrix Completion , 2008, SIAM J. Optim..

[4]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[5]  Stephen P. Boyd,et al.  Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices , 2003, Proceedings of the 2003 American Control Conference, 2003..

[6]  Kim-Chuan Toh,et al.  An introduction to a class of matrix cone programming , 2012, Mathematical Programming.

[7]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[8]  Lars Elden,et al.  Matrix methods in data mining and pattern recognition , 2007, Fundamentals of algorithms.

[9]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2008, Found. Comput. Math..

[10]  Houduo Qi,et al.  A Sequential Semismooth Newton Method for the Nearest Low-rank Correlation Matrix Problem , 2011, SIAM J. Optim..

[11]  R. Rebonato.  Review Paper. Interest–rate term–structure pricing models: a review , 2004, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[12]  Jérôme Malick,et al.  A Dual Approach to Semidefinite Least-Squares Problems , 2004, SIAM J. Matrix Anal. Appl..

[13]  Yin Zhang,et al.  Maximum stable set formulations and heuristics based on continuous optimization , 2002, Math. Program..

[14]  Defeng Sun,et al.  The Strong Second-Order Sufficient Condition and Constraint Nondegeneracy in Nonlinear Semidefinite Programming and Their Implications , 2006, Math. Oper. Res..

[15]  D. Brigo,et al.  A Note on Correlation and Rank Reduction , 2002 .

[16]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[17]  Olvi L. Mangasarian,et al.  A class of smoothing functions for nonlinear and mixed complementarity problems , 1996, Comput. Optim. Appl..

[18]  Stephen P. Boyd,et al.  Least-Squares Covariance Matrix Adjustment , 2005, SIAM J. Matrix Anal. Appl..

[19]  Yun-Bin Zhao,et al.  Approximation Theory of Matrix Rank Minimization and Its Application to Quadratic Equations , 2010, 1010.0851.

[20]  Defeng Sun,et al.  Semismooth Homeomorphisms and Strong Stability of Semidefinite and Lorentz Complementarity Problems , 2003, Math. Oper. Res..

[21]  Zhenyue Zhang,et al.  Optimal low-rank approximation to a correlation matrix , 2003 .

[22]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[23]  David P. Williamson,et al.  .879-approximation algorithms for MAX CUT and MAX 2SAT , 1994, STOC '94.

[24]  Defeng Sun,et al.  Semismooth Matrix-Valued Functions , 2002, Math. Oper. Res..

[25]  Weiyu Xu,et al.  Null space conditions and thresholds for rank minimization , 2011, Math. Program..

[26]  Miriam Hodge.  Fast at-the-money calibration of the Libor market model using Lagrange multipliers , 2003 .

[27]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[28]  M. Fielden,et al.  The liver pharmacological and xenobiotic gene response repertoire , 2008, Molecular systems biology.

[29]  Lixin Wu.  Fast at-the-money calibration of the LIBOR market model through Lagrange multipliers , 2002 .

[30]  S. Yun,et al.  An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems , 2009 .

[31]  Defeng Sun,et al.  Strong Semismoothness of Eigenvalues of Symmetric Matrices and Its Application to Inverse Eigenvalue Problems , 2002, SIAM J. Numer. Anal..

[32]  Defeng Sun,et al.  A Majorized Penalty Approach for Calibrating Rank Constrained Correlation Matrix Problems , 2010 .

[33]  Igor Grubisic,et al.  Efficient Rank Reduction of Correlation Matrices , 2004, cond-mat/0403477.

[34]  N. Higham.  Computing the nearest correlation matrix—a problem from finance , 2002 .

[35]  Defeng Sun,et al.  A Quadratically Convergent Newton Method for Computing the Nearest Correlation Matrix , 2006, SIAM J. Matrix Anal. Appl..

[36]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[37]  P. Groenen,et al.  Rank Reduction of Correlation Matrices by Majorization , 2004 .

[38]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[39]  Yin Zhang,et al.  Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm , 2012, Mathematical Programming Computation.

[40]  Kim-Chuan Toh,et al.  A Newton-CG Augmented Lagrangian Method for Semidefinite Programming , 2010, SIAM J. Optim..

[41]  Takeo Kanade,et al.  A sequential factorization method for recovering shape and motion from image streams , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  A. Lewis.  The Convex Analysis of Unitarily Invariant Matrix Functions , 1995 .

[43]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[44]  Lieven Vandenberghe,et al.  Interior-Point Method for Nuclear Norm Approximation with Application to System Identification , 2009, SIAM J. Matrix Anal. Appl..

[45]  Kim-Chuan Toh,et al.  An inexact primal–dual path following algorithm for convex quadratic SDP , 2007, Math. Program..

[46]  M. Mesbahi.  On the rank minimization problem and its control applications , 1998 .

[47]  A. S. Lewis,et al.  Derivatives of Spectral Functions , 1996, Math. Oper. Res..

[48]  Franz Rendl,et al.  Regularization Methods for Semidefinite Programming , 2009, SIAM J. Optim..

[49]  Shiqian Ma,et al.  Fixed point and Bregman iterative methods for matrix rank minimization , 2009, Math. Program..