Rank-constrained optimization and its applications

Abstract This paper investigates an iterative approach to solve the general rank-constrained optimization problems (RCOPs) defined to optimize a convex objective function subject to a set of convex constraints and rank constraints on unknown rectangular matrices. In addition, rank minimization problems (RMPs) are introduced and equivalently transformed into RCOPs by introducing a quadratic matrix equality constraint. The rank function is discontinuous and nonconvex, thus the general RCOPs are classified as NP-hard in most of the cases. An iterative rank minimization (IRM) method, with convex formulation at each iteration, is proposed to gradually approach the constrained rank. The proposed IRM method aims at solving RCOPs with rank inequalities constrained by upper or lower bounds, as well as rank equality constraints. Proof of the convergence to a local minimizer with at least a sublinear convergence rate is provided. Four representative applications of RCOPs and RMPs, including system identification, output feedback stabilization, and structured H 2 controller design problems, are presented with comparative simulation results to verify the feasibility and improved performance of the proposed IRM method.

[1]  John B. Moore,et al.  A Newton-like method for solving rank constrained linear matrix inequalities , 2006, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[2]  J. Berge,et al.  A numerical approach to the approximate and the exact minimum rank of a covariance matrix , 1991 .

[3]  Stephen P. Boyd,et al.  Rank minimization and applications in system theory , 2004, Proceedings of the 2004 American Control Conference.

[4]  Ran Dai,et al.  Spacecraft Attitude Control under Constrained Zones via Quadratically Constrained Quadratic Programming , 2015 .

[5]  Ramón A. Delgado,et al.  A novel representation of rank constraints for real matrices , 2016 .

[6]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[7]  Ran Dai,et al.  An iterative approach to Rank Minimization Problems , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[8]  Young-Hyun Moon,et al.  Technical communique: Structurally constrained H2 and H∞ control: A rank-constrained LMI approach , 2006 .

[9]  Graham C. Goodwin,et al.  A Rank-Constrained Optimization approach: Application to Factor Analysis , 2014 .

[10]  Ivan Markovsky,et al.  Software for weighted structured low-rank approximation , 2014, J. Comput. Appl. Math..

[11]  J. Willems.  Paradigms and puzzles in the theory of dynamical systems , 1991 .

[12]  Inderjit S. Dhillon,et al.  Rank minimization via online learning , 2008, ICML '08.

[13]  M. Mesbahi.  On the rank minimization problem and its control applications , 1998 .

[14]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[15]  Juan C. Agüero,et al.  Low-order control design using a novel rank-constrained optimization approach , 2016, 2016 Australian Control Conference (AuCC).

[16]  Ivan Markovsky,et al.  Low Rank Approximation - Algorithms, Implementation, Applications , 2018, Communications and Control Engineering.

[17]  Stephen P. Boyd,et al.  A path-following method for solving BMI problems in control , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[18]  Ohad Shamir,et al.  Large-Scale Convex Minimization with a Low-Rank Constraint , 2011, ICML.

[19]  L. Ghaoui,et al.  A cone complementarity linearization algorithm for static output-feedback and related problems , 1997, IEEE Trans. Autom. Control..

[20]  Stephen P. Boyd,et al.  Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices , 2003, Proceedings of the 2003 American Control Conference, 2003..

[21]  Jon C. Dattorro,et al.  Convex Optimization & Euclidean Distance Geometry , 2004 .

[22]  R. Dai,et al.  Path Planning of Spatial Rigid Motion with Constrained Attitude , 2015 .

[23]  Yoram Bresler,et al.  ADMiRA: Atomic Decomposition for Minimum Rank Approximation , 2009, IEEE Transactions on Information Theory.

[24]  I. Markovsky,et al.  Left vs right representations for solving weighted low-rank approximation problems , 2007 .

[25]  Maryam Fazel,et al.  Iterative reweighted algorithms for matrix rank minimization , 2012, J. Mach. Learn. Res..

[26]  I. Markovsky,et al.  High-performance numerical algorithms and software for structured total least squares , 2005 .

[27]  Laurent El Ghaoui,et al.  Rank Minimization under LMI constraints: A Framework for Output Feedback Problems , 2007 .

[28]  Gilles Meyer.  Geometric optimization algorithms for linear regression on fixed-rank matrices , 2011 .

[29]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[30]  Benjamin Recht,et al.  A Simpler Approach to Matrix Completion , 2009, J. Mach. Learn. Res..

[31]  Yun-Bin Zhao,et al.  Approximation Theory of Matrix Rank Minimization and Its Application to Quadratic Equations , 2010, 1010.0851.

[32]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[33]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[34]  Laurent El Ghaoui,et al.  Graph weight design for Laplacian eigenvalue constraints with multi-agent systems applications , 2011, IEEE Conference on Decision and Control and European Control Conference.

[35]  Bart De Moor,et al.  Algorithms for deterministic balanced subspace identification , 2005, Autom..

[36]  G. Stewart,et al.  A generalization of the Eckart-Young-Mirsky matrix approximation theorem , 1987 .

[37]  Ivan Markovsky,et al.  Structured low-rank approximation and its applications , 2008, Autom..

[38]  B. Fares,et al.  An augmented Lagrangian method for a class of LMI-constrained problems in robust control theory , 2001 .

[39]  Karolos M. Grigoriadis,et al.  Low-order control design for LMI problems using alternating projection methods , 1996, Autom..

[40]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[41]  G. Papavassilopoulos,et al.  On the rank minimization problem over a positive semidefinite linear matrix inequality , 1997, IEEE Trans. Autom. Control..

[42]  Emmanuel J. Candès,et al.  Exact Matrix Completion via Convex Optimization , 2009, Found. Comput. Math..