Low-Rank Semidefinite Programming: Theory and Applications

Finding low-rank solutions of semidefinite programs is important in many applications. For example, semidefinite programs that arise as relaxations of polynomial optimization problems are exact relaxations when the semidefinite program has a rank-1 solution. Unfortunately, computing a minimum-rank solution of a semidefinite program is an NP-hard problem. In this paper we review the theory of low-rank semidefinite programming, presenting theorems that guarantee the existence of a low-rank solution, heuristics for computing low-rank solutions, and algorithms for finding low-rank approximate solutions. Then we present applications of the theory to trust-region problems and signal processing.

[1]  M. Hestenes.  Multiplier and gradient methods , 1969 .

[2]  Yinyu Ye,et al.  On affine scaling algorithms for nonconvex quadratic programming , 1992, Math. Program..

[3]  Tamás Terlaky,et al.  A Survey of the S-Lemma , 2007, SIAM Rev..

[4]  Samuel Burer,et al.  Second-Order-Cone Constraints for Extended Trust-Region Subproblems , 2013, SIAM J. Optim..

[5]  Y. Nesterov.  Quality of semidefinite relaxation for nonconvex quadratic optimization , 1997 .

[6]  K. Mani Chandy,et al.  Quadratically Constrained Quadratic Programs on Acyclic Graphs With Application to Power Flow , 2012, IEEE Transactions on Control of Network Systems.

[7]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[8]  Jorge J. Moré,et al.  The Levenberg-Marquardt algo-rithm: Implementation and theory , 1977 .

[9]  Yin Zhang,et al.  Computing a Celis-Dennis-Tapia trust-region step for equality constrained optimization , 1992, Math. Program..

[10]  G. Strang.  Introduction to Linear Algebra , 1993 .

[11]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[12]  Franz Rendl,et al.  A Spectral Bundle Method for Semidefinite Programming , 1999, SIAM J. Optim..

[13]  Renato D. C. Monteiro,et al.  First- and second-order methods for semidefinite programming , 2003, Math. Program..

[14]  D. Goldfarb.  A family of variable-metric methods derived by variational means , 1970 .

[15]  Raymond E. Miller,et al.  Complexity of Computer Computations , 1972 .

[16]  Henry Wolkowicz,et al.  Indefinite Trust Region Subproblems and Nonsymmetric Eigenvalue Perturbations , 1995, SIAM J. Optim..

[17]  Yong Xia,et al.  S-lemma with equality and its applications , 2014, Math. Program..

[18]  Qiang Li,et al.  Optimal and Robust Transmit Designs for MISO Channel Secrecy by Semidefinite Programming , 2010, IEEE Transactions on Signal Processing.

[19]  Ya-Xiang Yuan,et al.  Optimality Conditions for the Minimization of a Quadratic with Two Quadratic Constraints , 1997, SIAM J. Optim..

[20]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[21]  Luca Trevisan,et al.  Gadgets, approximation, and linear programming , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[22]  Daniel Pérez Palomar,et al.  Rank-Constrained Separable Semidefinite Programming With Applications to Optimal Beamforming , 2010, IEEE Transactions on Signal Processing.

[23]  Shuzhong Zhang,et al.  Complex Matrix Decomposition and Quadratic Programming , 2007, Math. Oper. Res..

[24]  D. Sorensen.  Newton's method with a model trust region modification , 1982 .

[25]  S. Goldfeld,et al.  Maximization by Quadratic Hill-Climbing , 1966 .

[26]  Renato D. C. Monteiro,et al.  Digital Object Identifier (DOI) 10.1007/s10107-004-0564-1 , 2004 .

[27]  Johan Håstad,et al.  Some optimal inapproximability results , 2001, JACM.

[28]  S. Gusev,et al.  Kalman-Popov-Yakubovich lemma and the S-procedure: A historical essay , 2006 .

[29]  Javad Lavaei.  Zero duality gap for classical opf problem convexifies fundamental nonlinear power problems , 2011, Proceedings of the 2011 American Control Conference.

[30]  Carl D. Meyer,et al.  Matrix Analysis and Applied Linear Algebra , 2000 .

[31]  Alexander I. Barvinok,et al.  A Remark on the Rank of Positive Semidefinite Matrices Subject to Affine Constraints , 2001, Discret. Comput. Geom..

[32]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[33]  J. Dennis,et al.  A robust trust region algorithm for nonlinear programming , 1990 .

[34]  Samuel Burer,et al.  The trust region subproblem with non-intersecting linear constraints , 2015, Math. Program..

[35]  Wing-Kin Ma,et al.  Low-complexity robust MISO downlink precoder optimization for the limited feedback case , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[36]  Chong-Yung Chi,et al.  Worst-case robust multiuser transmit beamforming using semidefinite relaxation: duality and implications , 2011, 2011 Conference Record of the Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR).

[37]  S. Sojoudi,et al.  Optimal charging of plug-in hybrid electric vehicles in smart grids , 2011, 2011 IEEE Power and Energy Society General Meeting.

[38]  E.J. Candes.  Compressive Sampling , 2022 .

[39]  Shuzhong Zhang,et al.  On Cones of Nonnegative Quadratic Functions , 2003, Math. Oper. Res..

[40]  M. J. D. Powell,et al.  A method for nonlinear constraints in minimization problems , 1969 .

[41]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[42]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[43]  Stephen P. Boyd,et al.  Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices , 2003, Proceedings of the 2003 American Control Conference, 2003..

[44]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[45]  Gérard Cornuéjols,et al.  Valid inequalities for mixed integer linear programs , 2007, Math. Program..

[46]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[47]  Francis R. Bach,et al.  Low-Rank Optimization on the Cone of Positive Semidefinite Matrices , 2008, SIAM J. Optim..

[48]  Ralph E. Gomory,et al.  An algorithm for integer solutions to linear programs , 1958 .

[49]  José Mario Martínez,et al.  A trust-region strategy for minimization on arbitrary domains , 1995, Math. Program..

[50]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[51]  R. Monteiro,et al.  A projected gradient algorithm for solving the maxcut SDP relaxation , 2001 .

[52]  Yonina C. Eldar,et al.  Strong Duality in Nonconvex Quadratic Optimization with Two Quadratic Constraints , 2006, SIAM J. Optim..

[53]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[54]  Laurent El Ghaoui,et al.  Robust Optimization , 2021, ICORES.

[55]  Alexander I. Barvinok,et al.  Problems of distance geometry and convex properties of quadratic maps , 1995, Discret. Comput. Geom..

[56]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[57]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[58]  R. Gomory.  AN ALGORITHM FOR THE MIXED INTEGER PROBLEM , 1960 .

[59]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[60]  J. Lavaei,et al.  Physics of power networks makes hard optimization problems easy to solve , 2012, 2012 IEEE Power and Energy Society General Meeting.

[61]  Renato D. C. Monteiro,et al.  A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization , 2003, Math. Program..

[62]  José Mario Martínez,et al.  Local Minimizers of Quadratic Functions on Euclidean Balls and Spheres , 1994, SIAM J. Optim..

[63]  Daniel Bienstock,et al.  Polynomial Solvability of Variants of the Trust-Region Subproblem , 2014, SODA.

[64]  Gábor Pataki,et al.  On the Rank of Extreme Matrices in Semidefinite Programs and the Multiplicity of Optimal Eigenvalues , 1998, Math. Oper. Res..

[65]  Shuzhong Zhang,et al.  On the Low Rank Solutions for Linear Matrix Inequalities , 2008, Math. Oper. Res..

[66]  Franz Rendl,et al.  A semidefinite framework for trust region subproblems with applications to large scale minimization , 1997, Math. Program..

[67]  Yin Zhang,et al.  Interior-Point Algorithms for Semidefinite Programming Based on a Nonlinear Formulation , 2002, Comput. Optim. Appl..

[68]  Ruizhi Shi,et al.  Tightness of Semidefinite Programming Relaxation to Robust Transmit Beamforming with SINR Constraints , 2013 .

[69]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[70]  David M. author-Gay.  Computing Optimal Locally Constrained Steps , 1981 .

[71]  Mehran Mesbahi,et al.  On the semi-definite programming solution of the least order dynamic output feedback synthesis , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[72]  Javad Lavaei,et al.  Geometry of power flows in tree networks , 2012, 2012 IEEE Power and Energy Society General Meeting.

[73]  Holger Boche,et al.  Accepted for Publication in Ieee Transactions on Signal Processing 1 Robust Qos-constrained Optimization of Downlink Multiuser Miso Systems , 2022 .

[74]  Yinyu Ye,et al.  Convergence behavior of interior-point algorithms , 1993, Math. Program..

[75]  Nikos D. Sidiropoulos,et al.  Convex Optimization-Based Beamforming , 2010, IEEE Signal Processing Magazine.

[76]  Mats Bengtsson,et al.  Convex Optimization-based Beamforming : From Receive to Transmit and Network Designs , 2010 .

[77]  Anthony Man-Cho So,et al.  A Unified Theorem on Sdp Rank Reduction , 2008, Math. Oper. Res..

[78]  Shuzhong Zhang,et al.  New Results on Quadratic Minimization , 2003, SIAM J. Optim..

[79]  Javad Lavaei,et al.  Efficient convex relaxation for stochastic optimal distributed control problem , 2014, 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[80]  Masakazu Kojima,et al.  Exact Solutions of Some Nonconvex Quadratic Optimization Problems via SDP and SOCP Relaxations , 2003, Comput. Optim. Appl..

[81]  Gaston H. Gonnet,et al.  On the LambertW function , 1996, Adv. Comput. Math..

[82]  Yin Zhang,et al.  A computational study of a gradient-based log-barrier algorithm for a class of large-scale SDPs , 2003, Math. Program..

[83]  Christopher I. Byrnes,et al.  Analysis and control of nonlinear systems , 1988 .

[84]  Javad Lavaei,et al.  On the exactness of semidefinite relaxation for nonlinear optimization over graphs: Part I , 2013, 52nd IEEE Conference on Decision and Control.

[85]  Zhi-Quan Luo,et al.  Robust SINR-constrained MISO downlink beamforming: when is semidefinite programming relaxation tight? , 2012, EURASIP Journal on Wireless Communications and Networking.

[86]  S. Low,et al.  Zero Duality Gap in Optimal Power Flow Problem , 2012, IEEE Transactions on Power Systems.

[87]  Subhash Khot.  On the power of unique 2-prover 1-round games , 2002, STOC '02.

[88]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 2. The New Algorithm , 1970 .

[89]  Hoang Duong Tuan,et al.  Generalized S-Lemma and strong duality in nonconvex quadratic programming , 2013, J. Glob. Optim..

[90]  E. Candès,et al.  Sparsity and incoherence in compressive sampling , 2006, math/0611957.

[91]  Timothy N. Davidson,et al.  Convex Conic Formulations of Robust Downlink Precoder Designs With Quality of Service Constraints , 2007, IEEE Journal of Selected Topics in Signal Processing.

[92]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[93]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[94]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[95]  Ryan O'Donnell,et al.  Optimal Inapproximability Results for MAX-CUT and Other 2-Variable CSPs? , 2007, SIAM J. Comput..

[96]  Jean Levine,et al.  Analysis and Control of Nonlinear Systems , 2009 .

[97]  C. Carathéodory.  Über den Variabilitätsbereich der Koeffizienten von Potenzreihen, die gegebene Werte nicht annehmen , 1907 .

[98]  David Tse,et al.  Geometry of injection regions of power networks , 2011, IEEE Transactions on Power Systems.

[99]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[100]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[101]  Vaithilingam Jeyakumar,et al.  Alternative Theorems for Quadratic Inequality Systems and Global Quadratic Optimization , 2009, SIAM J. Optim..

[102]  Mauricio G. C. Resende,et al.  An interior point algorithm to solve computationally difficult set covering problems , 1991, Math. Program..

[103]  Gérard Cornuéjols,et al.  Revival of the Gomory cuts in the 1990’s , 2007, Ann. Oper. Res..

[104]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.