The Bayesian Covariance Lasso.

Estimation of sparse covariance matrices and their inverse subject to positive definiteness constraints has drawn a lot of attention in recent years. The abundance of high-dimensional data, where the sample size (n) is less than the dimension (d), requires shrinkage estimation methods since the maximum likelihood estimator is not positive definite in this case. Furthermore, when n is larger than d but not sufficiently larger, shrinkage estimation is more stable than maximum likelihood as it reduces the condition number of the precision matrix. Frequentist methods have utilized penalized likelihood methods, whereas Bayesian approaches rely on matrix decompositions or Wishart priors for shrinkage. In this paper we propose a new method, called the Bayesian Covariance Lasso (BCLASSO), for the shrinkage estimation of a precision (covariance) matrix. We consider a class of priors for the precision matrix that leads to the popular frequentist penalties as special cases, develop a Bayes estimator for the precision matrix, and propose an efficient sampling scheme that does not precalculate boundaries for positive definiteness. The proposed method is permutation invariant and performs shrinkage and estimation simultaneously for non-full rank data. Simulations show that the proposed BCLASSO performs similarly as frequentist methods for non-full rank data.

[1]  Dinggang Shen,et al.  Evidence on the emergence of the brain's default network from 2-week-old to 2-year-old healthy pediatric subjects , 2009, Proceedings of the National Academy of Sciences.

[2]  M. West,et al.  Sparse graphical models for exploring gene expression data , 2004 .

[3]  Xiao-Li Meng,et al.  Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage , 2000 .

[4]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[5]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[6]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[7]  D. Dunson,et al.  Random Effects Selection in Linear Mixed Models , 2003, Biometrics.

[8]  Jun S. Liu,et al.  The Multiple-Try Method and Local Optimization in Metropolis Sampling , 2000 .

[9]  M. Yuan,et al.  Model selection and estimation in the Gaussian graphical model , 2007 .

[10]  D. Paul.  ASYMPTOTICS OF SAMPLE EIGENSTRUCTURE FOR A LARGE DIMENSIONAL SPIKED COVARIANCE MODEL , 2007 .

[11]  Sylvia Frühwirth-Schnatter,et al.  Bayesian parsimonious covariance estimation for hierarchical linear mixed models , 2008, Stat. Comput..

[12]  James G. Scott,et al.  Feature-Inclusion Stochastic Search for Gaussian Graphical Models , 2008 .

[13]  James G. Scott,et al.  Objective Bayesian model selection in Gaussian graphical models , 2009 .

[14]  M. Pourahmadi.  Maximum likelihood estimation of generalised linear models for multivariate normal covariance matrix , 2000 .

[15]  J. Berger,et al.  Estimation of a Covariance Matrix Using the Reference Prior , 1994 .

[16]  E. Davidson,et al.  Gene regulatory networks. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[17]  Jianhua Z. Huang,et al.  Covariance matrix selection and estimation via penalised normal likelihood , 2006 .

[18]  R. Christensen.  Introduction to Graphical Modeling , 2001 .

[19]  K. Sachs,et al.  Causal Protein-Signaling Networks Derived from Multiparameter Single-Cell Data , 2005, Science.

[20]  H. Massam,et al.  FLEXIBLE COVARIANCE ESTIMATION IN GRAPHICAL , 2008 .

[21]  R. Kohn,et al.  Efficient estimation of covariance selection models , 2003 .

[22]  L. R. Haff.  Minimax estimators for a multinormal precision matrix , 1977 .

[23]  Y. Escoufier.  LE TRAITEMENT DES VARIABLES VECTORIELLES , 1973 .

[24]  Alexandre d'Aspremont,et al.  Model Selection Through Sparse Maximum Likelihood Estimation , 2007, ArXiv.

[25]  R. Kohn,et al.  Parsimonious Covariance Matrix Estimation for Longitudinal Data , 2002 .

[26]  M. Drton,et al.  Model selection for Gaussian concentration graphs , 2004 .

[27]  Jianqing Fan,et al.  NETWORK EXPLORATION VIA THE ADAPTIVE LASSO AND SCAD PENALTIES. , 2009, The annals of applied statistics.

[28]  K. Strimmer,et al.  Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics , 2011 .