Precision Matrix Estimation With ROPE

ABSTRACT It is known that the accuracy of the maximum likelihood-based covariance and precision matrix estimates can be improved by penalized log-likelihood estimation. In this article, we propose a ridge-type operator for the precision matrix estimation, ROPE for short, to maximize a penalized likelihood function where the Frobenius norm is used as the penalty function. We show that there is an explicit closed form representation of a shrinkage estimator for the precision matrix when using a penalized log-likelihood, which is analogous to ridge regression in a regression context. The performance of the proposed method is illustrated by a simulation study and real data applications. Computer code used in the example analyses as well as other supplementary materials for this article are available online.

[1]  C. R. Henderson,et al.  Best linear unbiased estimation and prediction under a selection model. , 1975, Biometrics.

[2]  Olivier Ledoit,et al.  Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices , 2011, 1207.5322.

[3]  H. D. Patterson,et al.  Recovery of inter-block information when block sizes are unequal , 1971 .

[4]  Ting Yuan,et al.  A coordinate descent algorithm for sparse positive definite matrix estimation , 2013, Stat. Anal. Data Min..

[5]  P. VanRaden,et al.  Efficient methods to compute genomic predictions. , 2008, Journal of dairy science.

[6]  T. Cai,et al.  A Constrained ℓ1 Minimization Approach to Sparse Precision Matrix Estimation , 2011, 1102.2233.

[7]  E. Stacy.  A Generalization of the Gamma Distribution , 1962 .

[8]  J. Friedman,et al.  New Insights and Faster Computations for the Graphical Lasso , 2011 .

[9]  Xinwei Deng,et al.  Penalized Covariance Matrix Estimation Using a Matrix-Logarithm Transformation , 2013 .

[10]  A. U.S.,et al.  Sparse Estimation of a Covariance Matrix , 2010 .

[11]  Olivier Ledoit,et al.  Honey, I Shrunk the Sample Covariance Matrix , 2003 .

[12]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[13]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[14]  D. Reich,et al.  Population Structure and Eigenanalysis , 2006, PLoS genetics.

[15]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[16]  Zitong Li,et al.  Overview of LASSO-related penalized regression methods for quantitative trait mapping and genomic selection , 2012, Theoretical and Applied Genetics.

[17]  Seung-Jean Kim,et al.  Maximum Likelihood Covariance Estimation with a Condition Number Constraint , 2006, 2006 Fortieth Asilomar Conference on Signals, Systems and Computers.

[18]  Peter Bühlmann,et al.  High-Dimensional Statistics with a View Toward Applications in Biology , 2014 .

[19]  Michael Wolf,et al.  Spectrum Estimation: A Unified Framework for Covariance Matrix Estimation and PCA in Large Dimensions , 2013, J. Multivar. Anal..

[20]  Mohsen Pourahmadi,et al.  High-Dimensional Covariance Estimation: Pourahmadi/High-Dimensional , 2013 .

[21]  N. Higham,et al.  Numerical analysis of a quadratic matrix equation , 2000 .

[22]  Jeffrey B. Endelman,et al.  Ridge Regression and Other Kernels for Genomic Selection with R Package rrBLUP , 2011 .

[23]  Mohsen Pourahmadi,et al.  High-Dimensional Covariance Estimation , 2013 .

[24]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[25]  W. Wonham.  On a Matrix Riccati Equation of Stochastic Control , 1968 .

[26]  Wessel N. van Wieringen,et al.  Ridge estimation of inverse covariance matrices from high-dimensional data , 2014, Comput. Stat. Data Anal..

[27]  Weidong Liu,et al.  Fast and adaptive sparse precision matrix estimation in high dimensions , 2012, J. Multivar. Anal..

[28]  Mikko J Sillanpää,et al.  Use of Wishart Prior and Simple Extensions for Sparse Precision Matrix Estimation , 2016, PloS one.

[29]  A. Laub.  A schur method for solving algebraic Riccati equations , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[30]  Harry Dym,et al.  Linear Algebra in Action , 2006, Graduate Studies in Mathematics.

[31]  David I. Warton,et al.  Penalized Normal Likelihood and Ridge Regularization of Correlation and Covariance Matrices , 2008 .

[32]  Jianhua Z. Huang,et al.  Covariance matrix selection and estimation via penalised normal likelihood , 2006 .

[33]  Robert J. Vanderbei,et al.  The fastclime package for linear programming and large-scale precision matrix estimation in R , 2014, J. Mach. Learn. Res..

[34]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[35]  Albrecht E. Melchinger,et al.  Shrinkage estimation of the genomic relationship matrix can improve genomic estimated breeding values in the training set , 2015, Theoretical and Applied Genetics.

[36]  José Crossa,et al.  Genomic‐Enabled Prediction Based on Molecular Markers and Pedigree Using the Bayesian Linear Regression Package in R , 2010, The plant genome.

[37]  Jianqing Fan,et al.  NETWORK EXPLORATION VIA THE ADAPTIVE LASSO AND SCAD PENALTIES. , 2009, The annals of applied statistics.

[38]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[39]  José Crossa,et al.  Prediction of Genetic Values of Quantitative Traits in Plant Breeding Using Pedigree and Molecular Markers , 2010, Genetics.

[40]  Jianhua Z. Huang.  Covariance selection and estimation via penalised normal likelihood , 2005 .