Estimation When Both Covariance and Precision Matrices are Sparse

We offer a method to estimate a covariance matrix in the special case that both the covariance matrix and the precision matrix are sparse — a constraint we call double sparsity. The estimation method is maximum likelihood, subject to the double sparsity constraint. In our method, only a particular class of sparsity pattern is allowed: both the matrix and its inverse must be subordinate to the same chordal graph. Compared to a naive enforcement of double sparsity, our chordal graph 1 ar X iv :2 10 8. 06 63 8v 1 [ st at .M E ] 1 5 A ug 2 02 1 approach exploits a special algebraic local inverse formula. This local inverse property makes computations that would usually involve an inverse (of either precision matrix or covariance matrix) much faster. In the context of estimation of covariance matrices, our proposal appears to be the first to find such special pairs of covariance and precision matrices.

[1]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[2]  Tri Nguyen,et al.  The Interplay of Ranks of Submatrices , 2004, SIAM Rev..

[3]  Charles R. Johnson,et al.  Local inversion of matrices with sparse inverses , 1998 .

[4]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[5]  G. Peters,et al.  Explicit solutions to correlation matrix completion problems, with an application to risk management and insurance , 2017, Royal Society Open Science.

[6]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[7]  Charles R. Johnson,et al.  Positive definite completions of partial Hermitian matrices , 1984 .

[8]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.

[9]  Martin S. Andersen,et al.  Chordal Graphs and Semidefinite Optimization , 2015, Found. Trends Optim..

[10]  T. Speed,et al.  Gaussian Markov Distributions over Finite Graphs , 1986 .

[11]  H. Dym,et al.  Extensions of band matrices with band inverses , 1981 .

[12]  Mohsen Pourahmadi,et al.  High-Dimensional Covariance Estimation , 2013 .

[13]  G. Strang,et al.  A Local Inverse Formula and a Factorization , 2016, 1610.01230.

[14]  Barry L. Nelson,et al.  Gaussian Markov Random Fields for Discrete Optimization via Simulation: Framework and Algorithms , 2019, Oper. Res..

[15]  Gilbert Strang.  Fast transforms: Banded matrices with banded inverses , 2010, Proceedings of the National Academy of Sciences.

[16]  R. Vandebril,et al.  Matrix Computations and Semiseparable Matrices , 2007 .

[17]  G. Golub,et al.  Separable nonlinear least squares: the variable projection method and its applications , 2003 .

[18]  Olivier Ledoit,et al.  Honey, I Shrunk the Sample Covariance Matrix , 2003 .

[19]  Michael I. Jordan.  Graphical Models , 2003 .

[20]  Adam J. Rothman,et al.  A new approach to Cholesky-based covariance regularization in high dimensions , 2009, 0903.0645.

[21]  B. Peyton,et al.  An Introduction to Chordal Graphs and Clique Trees , 1993 .