Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices

Many statistical applications require an estimate of a covariance matrix and/or its inverse. When the matrix dimension is large compared to the sample size, which happens frequently, the sample covariance matrix is known to perform poorly and may suffer from ill-conditioning. There already exists an extensive literature concerning improved estimators in such situations. In the absence of further knowledge about the structure of the true covariance matrix, the most successful approach so far, arguably, has been shrinkage estimation. Shrinking the sample covariance matrix to a multiple of the identity, by taking a weighted average of the two, turns out to be equivalent to linearly shrinking the sample eigenvalues to their grand mean, while retaining the sample eigenvectors. Our paper extends this approach by considering nonlinear transformations of the sample eigenvalues. We show how to construct an estimator that is asymptotically equivalent to an oracle estimator suggested in previous work. As demonstrated in extensive Monte Carlo simulations, the resulting bona fide estimator can result in sizeable improvements over the sample covariance matrix and also over linear shrinkage.

[1]  H. D. Brunk,et al.  AN EMPIRICAL DISTRIBUTION FUNCTION FOR SAMPLING WITH INCOMPLETE INFORMATION , 1955 .

[2]  E. Wigner.  Characteristic Vectors of Bordered Matrices with Infinite Dimensions I , 1955 .

[3]  J. Lamperti.  ON CONVERGENCE OF STOCHASTIC PROCESSES , 1962 .

[4]  V. Marčenko,et al.  DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES , 1967 .

[5]  J. Capon.  High-resolution frequency-wavenumber spectrum analysis , 1969 .

[6]  Clifford S. Stein.  Estimation of a covariance matrix , 1975 .

[7]  L. R. Haff.  Empirical Bayes Estimation of the Multivariate Normal Covariance Matrix , 1980 .

[8]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[9]  C. Stein.  Lectures on the theory of estimation of many parameters , 1986 .

[10]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[11]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[12]  J. W. Silverstein,et al.  Analysis of the limiting spectral distribution of large dimensional random matrices , 1995 .

[13]  T. Stieltjes.  Recherches sur les fractions continues , 1995 .

[14]  J. W. Silverstein,et al.  On the empirical distribution of eigenvalues of a class of large dimensional random matrices , 1995 .

[15]  J. W. Silverstein.  Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices , 1995 .

[16]  J. W. Silverstein,et al.  No eigenvalues outside the support of the limiting spectral distribution of large-dimensional sample covariance matrices , 1998 .

[17]  J. W. Silverstein,et al.  EXACT SEPARATION OF EIGENVALUES OF LARGE DIMENSIONAL SAMPLE COVARIANCE MATRICES , 1999 .

[18]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[19]  Theodore P. Hill,et al.  Necessary and sufficient condition that the limit of Stieltjes transforms is a Stieltjes transform , 2003, J. Approx. Theory.

[20]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[21]  J. Neyman,et al.  INADMISSIBILITY OF THE USUAL ESTIMATOR FOR THE MEAN OF A MULTIVARIATE NORMAL DISTRIBUTION , 2005 .

[22]  Sean R. Collins,et al.  A strategy for extracting and analyzing large-scale quantitative epistatic interaction data , 2006, Genome Biology.

[23]  Riccardo Colacito,et al.  Testing and Valuing Dynamic Correlations for Asset Allocation , 2005 .

[24]  Seung-Jean Kim,et al.  Maximum Likelihood Covariance Estimation with a Condition Number Constraint , 2006, 2006 Fortieth Asilomar Conference on Signals, Systems and Computers.

[25]  Xavier Mestre,et al.  Finite sample size effect on minimum variance beamformers: optimum diagonal loading factor for large arrays , 2006, IEEE Transactions on Signal Processing.

[26]  Noureddine El Karoui.  Spectrum estimation for large dimensional covariance matrices using random matrix theory , 2006, math/0609418.

[27]  Jianqing Fan,et al.  High dimensional covariance matrix estimation using a factor model , 2007, math/0701124.

[28]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.

[29]  Bin Yu,et al.  High-dimensional covariance estimation by minimizing ℓ1-penalized log-determinant divergence , 2008, 0811.3628.

[30]  Xavier Mestre,et al.  On the Asymptotic Behavior of the Sample Estimates of Eigenvalues and Eigenvectors of Covariance Matrices , 2008, IEEE Transactions on Signal Processing.

[31]  Carlos M. Carvalho,et al.  FLEXIBLE COVARIANCE ESTIMATION IN GRAPHICAL GAUSSIAN MODELS , 2008, 0901.3267.

[32]  S. Péché,et al.  Eigenvectors of some large sample covariance matrix ensembles , 2009 .

[33]  S. Krantz.  Explorations in Harmonic Analysis , 2009 .

[34]  A. Tsybakov,et al.  Estimation of high-dimensional low-rank matrices , 2009, 0912.5338.

[35]  Bing-Yi Jing,et al.  Nonparametric estimate of spectral density functions of sample covariance matrices: A first step , 2010, 1211.3230.

[36]  Guangming Pan,et al.  Strong convergence of the empirical distribution of eigenvalues of sample covariance matrices with a perturbation matrix , 2010, J. Multivar. Anal..

[37]  Michael Wolf,et al.  Nonlinear Shrinkage Estimation of Large-Dimensional Covariance Matrices , 2011 .

[38]  Kshitij Khare,et al.  Wishart distributions for decomposable covariance graph models , 2011, 1103.1768.

[39]  Harrison H. Zhou,et al.  MINIMAX ESTIMATION OF LARGE COVARIANCE MATRICES UNDER ℓ1-NORM , 2012 .

[40]  Walter Distaso,et al.  Design-Free Estimation of Variance Matrices , 2011 .