A Rank-Constrained Optimization approach: Application to Factor Analysis

Abstract In this paper, we present a general method for rank-constrained optimization. We use an iterative convex optimization procedure where it is possible to include any extra convex constraints. The proposed approach has potential application in several areas. We focus on the problem of Factor Analysis. In this case, our approach provides sufficient flexibility to handle correlated errors. The benefits of the method is demonstrated via a simulation study.

[1]  John B. Moore,et al.  A Newton-like method for solving rank constrained linear matrix inequalities , 2006, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[2]  J. Berge,et al.  A numerical approach to the approximate and the exact minimum rank of a covariance matrix , 1991 .

[3]  Arvind Ganesh,et al.  Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix , 2009 .

[4]  Young-Hyun Moon,et al.  Technical communique: Structurally constrained H2 and H∞ control: A rank-constrained LMI approach , 2006 .

[5]  Ivan Markovsky,et al.  Structured low-rank approximation and its applications , 2008, Autom..

[6]  J. Bai,et al.  Inferential Theory for Factor Models of Large Dimensions , 2003 .

[7]  D. Hunter,et al.  Optimization Transfer Using Surrogate Objective Functions , 2000 .

[8]  A. d'Aspremont,et al.  A semidefinite representation for some minimum cardinality problems , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[9]  Catherine Doz,et al.  A Quasi–Maximum Likelihood Approach for Large, Approximate Dynamic Factor Models , 2006, Review of Economics and Statistics.

[10]  Bart Vandereycken.  Riemannian and Multilevel Optimization for Rank-Constrained Matrix Problems (with Applications to Lyapunov Equations) (Riemannse en meerschalige optimalisatie voor matrixproblemen met rangbeperkingen) , 2010 .

[11]  Ivan Markovsky,et al.  Low Rank Approximation - Algorithms, Implementation, Applications , 2018, Communications and Control Engineering.

[12]  Stephen P. Boyd,et al.  A rank minimization heuristic with application to minimum order system approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[13]  Michael L. Overton,et al.  Optimality conditions and duality theory for minimizing sums of the largest eigenvalues of symmetric matrices , 2015, Math. Program..

[14]  Alexander Shapiro,et al.  Statistical inference of minimum rank factor analysis , 2002 .

[15]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[16]  Kathrin Klamroth,et al.  Biconvex sets and optimization with biconvex functions: a survey and extensions , 2007, Math. Methods Oper. Res..

[17]  Sam T. Roweis,et al.  EM Algorithms for PCA and SPCA , 1997, NIPS.

[18]  Serena Ng,et al.  Are more data always better for factor analysis , 2006 .

[19]  Graham C. Goodwin,et al.  EM-Based Channel Estimation in OFDM Systems with Phase Noise , 2011, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[20]  Gilles Meyer.  Geometric optimization algorithms for linear regression on fixed-rank matrices , 2011 .

[21]  Yoram Bresler,et al.  ADMiRA: Atomic Decomposition for Minimum Rank Approximation , 2009, IEEE Transactions on Information Theory.

[22]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[23]  Ivan Markovsky,et al.  Recent progress on variable projection methods for structured low-rank approximation , 2014, Signal Process..

[24]  Manfred Morari,et al.  System identification with missing data via nuclear norm regularization , 2009, 2009 European Control Conference (ECC).

[25]  M. Rothschild,et al.  Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets , 1982 .

[26]  Graham C. Goodwin,et al.  A novel approach to model error modelling using the expectation-maximization algorithm , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[27]  Ohad Shamir,et al.  Large-Scale Convex Minimization with a Low-Rank Constraint , 2011, ICML.

[28]  Jon C. Dattorro,et al.  Convex Optimization & Euclidean Distance Geometry , 2004 .

[29]  Graham C. Goodwin,et al.  Dual time-frequency domain system identification , 2012, Autom..

[30]  J. Stock,et al.  Forecasting Using Principal Components From a Large Number of Predictors , 2002 .