Covariance estimation with nonnegative partial correlations

We study the problem of high-dimensional covariance estimation under the constraint that the partial correlations are nonnegative. The sign constraints dramatically simplify estimation: the Gaussian maximum likelihood estimator is well defined with only two observations regardless of the number of variables. We analyze its performance in the setting where the dimension may be much larger than the sample size. We establish that the estimator is both high-dimensionally consistent and minimax optimal in the symmetrized Stein loss. We also prove a negative result which shows that the sign-constraints can introduce substantial bias for estimating the top eigenvalue of the covariance matrix.

[1]  Olivier Ledoit,et al.  Optimal Estimation of a Large-Dimensional Covariance Matrix Under Stein's Loss , 2017, Bernoulli.

[2]  Inderjit S. Dhillon,et al.  Matrix Nearness Problems with Bregman Divergences , 2007, SIAM J. Matrix Anal. Appl..

[3]  S. Geer,et al.  Confidence intervals for high-dimensional inverse covariance estimation , 2014, 1403.6752.

[4]  Antonio Ortega,et al.  Graph Learning From Data Under Laplacian and Structural Constraints , 2016, IEEE Journal of Selected Topics in Signal Processing.

[5]  Geurt Jongbloed,et al.  Nonparametric Estimation under Shape Constraints , 2014 .

[6]  Adityanand Guntuboyina,et al.  Nonparametric Shape-Restricted Regression , 2017, Statistical Science.

[7]  Caroline Uhler,et al.  Covariance Matrix Estimation under Total Positivity for Portfolio Selection* , 2019, 1909.04222.

[8]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[9]  S. Lauritzen,et al.  Maximum likelihood estimation in Gaussian models under total positivity , 2017, The Annals of Statistics.

[10]  Collin Carbno,et al.  Actuarial Theory for Dependent Risks: Measures, Orders, and Models , 2007, Technometrics.

[11]  H. Jeffreys.  An invariant form for the prior probability in estimation problems , 1946, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences.

[12]  Lones Smith,et al.  Student Portfolios and the College Admissions Problem , 2013 .

[13]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[14]  Clifford S. Stein.  Estimation of a covariance matrix , 1975 .

[15]  Moshe Shaked,et al.  Some notions of multivariate positive dependence , 2005 .

[16]  S. Karlin,et al.  M-Matrices as covariance matrices of multinormal distributions☆ , 1983 .

[17]  Matthias Hein,et al.  Estimation of positive definite M-matrices and structure learning for attractive Gaussian Markov Random fields , 2014, 1404.6640.

[18]  D. Dey,et al.  Estimation of a covariance matrix under Stein's loss , 1985 .

[19]  Trevor J. Hastie,et al.  The Graphical Lasso: New Insights and Alternatives , 2011, Electronic journal of statistics.

[20]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[21]  Harrison H. Zhou,et al.  Estimating structured high-dimensional covariance and precision matrices: Optimal rates and adaptive estimation , 2016 .

[22]  Joshua B. Tenenbaum,et al.  Discovering Structure by Learning Sparse Graphs , 2010 .

[23]  Harrison H. Zhou,et al.  Estimating Sparse Precision Matrix: Optimal Rates of Convergence and Adaptive Estimation , 2012, 1212.2882.

[24]  C. Stein.  Lectures on the theory of estimation of many parameters , 1986 .

[25]  Matthias Hein,et al.  Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization , 2012, 1205.0953.

[26]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Max Likelihood Estimation Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data , 2022 .

[27]  Paul R. Milgrom,et al.  A theory of auctions and competitive bidding , 1982 .

[28]  T. Cai,et al.  A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation , 2011, 1102.2233.

[29]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[30]  Ojas D. Parekh,et al.  On Factor Width and Symmetric H-matrices , 2005 .

[31]  Antonio Ortega,et al.  Generalized Laplacian precision matrix estimation for graph signal processing , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[32]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[33]  Cun-Hui Zhang,et al.  Sparse matrix inversion with scaled Lasso , 2012, J. Mach. Learn. Res..

[34]  Martin J. Wainwright,et al.  High-Dimensional Statistics , 2019 .

[35]  Arlene K. H. Kim.  Obtaining minimax lower bounds: a review , 2020, Journal of the Korean Statistical Society.

[36]  Caroline Uhler,et al.  Learning High-Dimensional Gaussian Graphical Models under Total Positivity without Tuning Parameters , 2019 .

[37]  Eduardo Pavez,et al.  Learning Graphs With Monotone Topology Properties and Multiple Connected Components , 2017, IEEE Transactions on Signal Processing.

[38]  Yuhao Wang,et al.  Learning High-dimensional Gaussian Graphical Models under Total Positivity without Adjustment of Tuning Parameters , 2020, AISTATS.

[39]  I. Johnstone,et al.  Optimal Shrinkage of Eigenvalues in the Spiked Covariance Model. , 2013, Annals of statistics.

[40]  Maher Moakher,et al.  Symmetric Positive-Definite Matrices: From Geometry to Applications and Visualization , 2006, Visualization and Processing of Tensor Fields.

[41]  Harrison H. Zhou,et al.  OPTIMAL RATES OF CONVERGENCE FOR SPARSE COVARIANCE MATRIX ESTIMATION , 2012, 1302.3030.

[42]  Ming Yuan,et al.  High Dimensional Inverse Covariance Matrix Estimation via Linear Programming , 2010, J. Mach. Learn. Res..

[43]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[44]  R. Plemmons.  M-matrix characterizations.I—nonsingular M-matrices , 1977 .

[45]  S. Geman.  A Limit Theorem for the Norm of Random Matrices , 1980 .