Cone-Constrained Principal Component Analysis

Estimating a vector from noisy quadratic observations is a task that arises naturally in many contexts, from dimensionality reduction, to synchronization and phase retrieval problems. It is often the case that additional information is available about the unknown vector (for instance, sparsity, sign or magnitude of its entries). Many authors propose non-convex quadratic optimization problems that aim at exploiting optimally this information. However, solving these problems is typically NP-hard. We consider a simple model for noisy quadratic observation of an unknown vector v0. The unknown vector is constrained to belong to a cone C ∋ v0. While optimal estimation appears to be intractable for the general problems in this class, we provide evidence that it is tractable when C is a convex cone with an efficient projection. This is surprising, since the corresponding optimization problem is non-convex and -from a worst case perspective- often NP hard. We characterize the resulting minimax risk in terms of the statistical dimension of the cone δ(C). This quantity is already known to control the risk of estimation from gaussian observations and random linear measurements. It is rather surprising that the same quantity plays a role in the estimation risk from quadratic measurements.

[1]  Martin J. Wainwright,et al.  High-dimensional analysis of semidefinite relaxations for sparse principal components , 2008, ISIT.

[2]  P. Paatero Least squares formulation of robust non-negative factor analysis , 1997 .

[3]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[4]  Emmanuel J. Candès,et al.  PhaseLift: Exact and Stable Signal Recovery from Magnitude Measurements via Convex Programming , 2011, ArXiv.

[5]  Xiaodong Li,et al.  Sparse Signal Recovery from Quadratic Measurements via Convex Programming , 2012, SIAM J. Math. Anal..

[6]  B. Nadler,et al.  MINIMAX BOUNDS FOR SPARSE PCA WITH NOISY HIGH-DIMENSIONAL DATA. , 2012, Annals of statistics.

[7]  Bernd Grtner,et al.  Approximation Algorithms and Semidefinite Programming , 2012 .

[8]  J R Fienup,et al.  Phase retrieval algorithms: a comparison. , 1982, Applied optics.

[9]  Yinyu Ye,et al.  Semidefinite programming for ad hoc wireless sensor network localization , 2004, Third International Symposium on Information Processing in Sensor Networks, 2004. IPSN 2004.

[10]  Babak Hassibi,et al.  Recovery of sparse 1-D signals from the magnitudes of their Fourier transform , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[11]  I. Johnstone,et al.  Sparse Principal Components Analysis , 2009, 0901.4392.

[12]  Joel A. Tropp,et al.  Living on the edge: A geometric theory of phase transitions in convex optimization , 2013, ArXiv.

[13]  Marc Teboulle,et al.  Conditional Gradient Algorithmsfor Rank-One Matrix Approximations with a Sparsity Constraint , 2011, SIAM Rev..

[14]  B. Nadler,et al.  Do Semidefinite Relaxations Really Solve Sparse PCA , 2013 .

[15]  Yihong Wu,et al.  Computational Barriers in Minimax Submatrix Detection , 2013, ArXiv.

[16]  D. Féral,et al.  The Largest Eigenvalue of Rank One Deformation of Large Wigner Matrices , 2006, math/0605624.

[17]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[18]  Andrea Montanari,et al.  Sparse PCA via Covariance Thresholding , 2013, J. Mach. Learn. Res..

[19]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[20]  R. Tibshirani,et al.  Sparse Principal Component Analysis , 2006 .

[21]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, 2010 IEEE International Symposium on Information Theory.

[22]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[23]  Amit Singer,et al.  A remark on global positioning from local distances , 2008, Proceedings of the National Academy of Sciences.

[24]  Andrea Montanari,et al.  Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising , 2011, IEEE Transactions on Information Theory.

[25]  Andrea Montanari,et al.  Non-Negative Principal Component Analysis: Message Passing Algorithms and Sharp Asymptotics , 2014, IEEE Transactions on Information Theory.