Geometric sensitivity of random matrix results: consequences for shrinkage estimators of covariance and related statistical methods

Shrinkage estimators of covariance are an important tool in modern applied and theoretical statistics. They play a key role in regularized estimation problems, such as ridge regression (aka Tykhonov regularization), regularized discriminant analysis and a variety of optimization problems. In this paper, we bring to bear the tools of random matrix theory to understand their behavior, and in particular, that of quadratic forms involving inverses of those estimators, which are important in practice. We use very mild assumptions compared to the usual assumptions made in random matrix theory, requiring only mild conditions on the moments of linear and quadratic forms in our random vectors. In particular, we show that our results apply for instance to log-normal data, which are of interest in financial applications. Our study highlights the relative sensitivity of random matrix results (and their practical consequences) to geometric assumptions which are often implicitly made by random matrix theorists and may not be relevant in data analytic practice.

[1]  J. Lindeberg.  Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung , 1922 .

[2]  D. Burkholder.  Distribution Function Inequalities for Martingales , 1973 .

[3]  L. R. Haff.  Empirical Bayes Estimation of the Multivariate Normal Covariance Matrix , 1980 .

[4]  B. Efron,et al.  The Jackknife Estimate of Variance , 1981 .

[5]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[6]  G. Pisier.  Probabilistic methods in the geometry of Banach spaces , 1986 .

[7]  David E. Tyler.  A Distribution-Free $M$-Estimator of Multivariate Scatter , 1987 .

[8]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[9]  Vi︠a︡cheslav Leonidovich Girko,et al.  Theory of random determinants , 1990 .

[10]  Charles R. Johnson,et al.  Topics in Matrix Analysis , 1991 .

[11]  J. W. Silverstein,et al.  On the empirical distribution of eigenvalues of a class of large dimensional random matrices , 1995 .

[12]  D. Stroock,et al.  Probability Theory: An Analytic View. , 1995 .

[13]  J. W. Silverstein.  Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices , 1995 .

[14]  G. Lugosi,et al.  On Concentration-of-Measure Inequalities , 1998 .

[15]  Z. Bai.  METHODOLOGIES IN SPECTRAL ANALYSIS OF LARGE DIMENSIONAL RANDOM MATRICES , A REVIEW , 1999 .

[16]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[17]  Y. Chikuse.  Statistics on special manifolds , 2003 .

[18]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[19]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[20]  P. Bickel,et al.  Some theory for Fisher''s linear discriminant function , 2004 .

[21]  A. Meucci.  Risk and asset allocation , 2005 .

[22]  I. Johnstone.  High Dimensional Statistical Inference and Random Matrices , 2006, math/0611589.

[23]  J. W. Silverstein,et al.  Spectral Analysis of Large Dimensional Random Matrices , 2009 .

[24]  Noureddine El Karoui,et al.  Concentration of measure and spectra of random matrices: Applications to correlation matrices, elliptical distributions and beyond , 2009, 0912.1950.

[25]  Noureddine El Karoui,et al.  High-dimensionality effects in the Markowitz problem and other quadratic programs with linear equality constraints: risk underestimation , 2009 .

[26]  Noureddine El Karoui,et al.  High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation , 2010, 1211.2917.

[27]  Noureddine El Karoui,et al.  On the Realized Risk of High-Dimensional Markowitz Portfolios , 2013, SIAM J. Financial Math..