Necessary and sufficient conditions for success of the nuclear norm heuristic for rank minimization

Minimizing the rank of a matrix subject to constraints is a challenging is a challenging problem that arises in many control applications including controller design, realization theory and model reduction. This class of optimization problems, known as rank minimization, is NP-HARD, and for most practical problems there are no efficient algorithms that yield exact solutions. A popular heuristic algorithm replaces the rank function with the nuclear norm-equal to the sum of the singular values-of the decision variable. In this paper, we provide a necessary and sufficient condition that quantifies when this heuristic successfully finds the minimum rank solution of a linear constraint set. We further show that most of the problems of interest in control can be formulated as rank minimization subject to such linear constraints. We additionally provide a probability distribution over instances of the affine rank minimization problem such that instances sampled from this distribution satisfy our conditions for success with overwhelming probability provided the number of constraints is appropriately large. Finally, we give empirical evidence that these probabilistic bounds provide accurate predictions of the heuristic¿s performance in non-asymptotic scenarios.

[1]  D. Slepian.  The one-sided barrier problem for Gaussian noise , 1962 .

[2]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[3]  S. Szarek.  The finite dimensional basis problem with an appendix on nets of Grassmann manifolds , 1983 .

[4]  Y. Gordon.  Some inequalities for Gaussian processes and applications , 1985 .

[5]  Y. Gordon.  Gaussian Processes and Almost Spherical Sections of Convex Bodies , 1988 .

[6]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[7]  M. Talagrand,et al.  Probability in Banach spaces , 1991 .

[8]  Nathan Linial,et al.  The geometry of graphs and some of its algorithmic applications , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[9]  S. Szarek.  Metric Entropy of Homogeneous Spaces , 1997, math/9701213.

[10]  G. Papavassilopoulos,et al.  On the rank minimization problem over a positive semidefinite linear matrix inequality , 1997, IEEE Trans. Autom. Control..

[11]  C. Beck,et al.  Computational study and comparisons of LFT reducibility methods , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[12]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[13]  Pablo A. Parrilo,et al.  On cone-invariant linear matrix inequalities , 2000, IEEE Trans. Autom. Control..

[14]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[15]  Kilian Q. Weinberger,et al.  Unsupervised Learning of Image Manifolds by Semidefinite Programming , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[16]  Nathan Srebro,et al.  Fast maximum margin matrix factorization for collaborative prediction , 2005, ICML.

[17]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[18]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[19]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[21]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[22]  Laurent El Ghaoui,et al.  Rank Minimization under LMI constraints: A Framework for Output Feedback Problems , 2007 .

[23]  Inderjit S. Dhillon,et al.  Rank minimization via online learning , 2008, ICML '08.

[24]  Z. Bai,et al.  METHODOLOGIES IN SPECTRAL ANALYSIS OF LARGE DIMENSIONAL RANDOM MATRICES, A REVIEW , 2008 .

[25]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[26]  Weiyu Xu,et al.  Compressed sensing - probabilistic analysis of a null-space characterization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[27]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..