Improved multivariate normal mean estimation with unknown covariance when p is greater than n

We consider the problem of estimating the mean vector of a p-variate normal $(\theta,\Sigma)$ distribution under invariant quadratic loss, $(\delta-\theta)'\Sigma^{-1}(\delta-\theta)$, when the covariance is unknown. We propose a new class of estimators that dominate the usual estimator $\delta^0(X)=X$. The proposed estimators of $\theta$ depend upon X and an independent Wishart matrix S with n degrees of freedom, however, S is singular almost surely when p>n. The proof of domination involves the development of some new unbiased estimators of risk for the p>n setting. We also find some relationships between the amount of domination and the magnitudes of n and p.

[1]  L. Gleser Minimax Estimation of a Normal Mean Vector When the Covariance Matrix is Unknown , 1979 .

[2]  Lawrence D. Brown,et al.  Estimation with Incompletely Specified Loss Functions (the Case of Several Location Parameters) , 1975 .

[3]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[4]  Tatsuya Kubokawa,et al.  Estimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data , 2008 .

[5]  Tatsuya Kubokawa,et al.  Robust Improvement in Estimation of a Mean Matrix in an Elliptically Contoured Distribution , 2001 .

[6]  L. Brown,et al.  Ensemble Minimax Estimation for Multivariate Normal Means , 2011 .

[7]  Jan R. Magnus,et al.  On certain moments relating to ratios of quadratic forms in normal variables : Further results , 1990 .

[8]  R. Muirhead Aspects of Multivariate Statistical Theory , 1982, Wiley Series in Probability and Statistics.

[9]  George Casella,et al.  Minimax Ridge Regression Estimation , 1980 .

[10]  C. Stein Estimation of the Mean of a Multivariate Normal Distribution , 1981 .

[11]  Alexandre d'Aspremont,et al.  First-Order Methods for Sparse Covariance Selection , 2006, SIAM J. Matrix Anal. Appl..

[12]  L. R. Haff,et al.  A CLASS OF MINIMAX ESTIMATORS OF A NORMAL MEAN VECTOR FOR ARBITRARY QUADRATIC LOSS AND UNKNOWN COVARIANCE MATRIX , 1983 .

[13]  Adam J. Rothman,et al.  Sparse estimation of large covariance matrices via a nested Lasso penalty , 2008, 0803.3872.

[14]  L. R. Haff An identity for the Wishart distribution with applications , 1979 .

[15]  Muni S. Srivastava,et al.  Stein estimation under elliptical distributions , 1989 .

[16]  M. Srivastava Multivariate Theory for Analyzing High Dimensional Data , 2007 .

[17]  P. Bickel,et al.  Regularized estimation of large covariance matrices , 2008, 0803.1909.

[18]  Yoshihiko Konno,et al.  Shrinkage estimators for large covariance matrices in multivariate real and complex normal distributions under an invariant quadratic loss , 2009, J. Multivar. Anal..

[19]  James O. Berger,et al.  Combining Independent Normal Mean Estimation Problems with Unknown Variances , 1976 .

[20]  Martin Lysy,et al.  Shrinkage Estimation in Multilevel Normal Models , 2012, 1203.5610.

[21]  Martin T. Wells,et al.  Robust shrinkage estimation for elliptically symmetric distributions with unknown covariance matrix , 2003 .

[22]  L. Brown,et al.  Minimax Estimation of a Normal Mean Vector for Arbitrary Quadratic Loss and Unknown Covariance Matrix , 1977 .

[23]  S. Kotz,et al.  Symmetric Multivariate and Related Distributions , 1989 .

[24]  Dominique Fourdrinier,et al.  On Bayes and unbiased estimators of loss , 2003 .

[25]  Gene H. Golub,et al.  The differentiation of pseudo-inverses and non-linear least squares problems whose variables separate , 1972, Milestones in Matrix Computation.

[26]  A. Baranchik,et al.  A Family of Minimax Estimators of the Mean of a Multivariate Normal Distribution , 1970 .

[27]  Muni S. Srivastava,et al.  Singular Wishart and multivariate beta distributions , 2003 .

[28]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[29]  C. Stein Lectures on the theory of estimation of many parameters , 1986 .

[30]  Yasunori Fujikoshi,et al.  Multivariate analysis of variance with fewer observations than the dimension , 2006 .

[31]  Adam J. Rothman,et al.  Sparse permutation invariant covariance estimation , 2008, 0801.4837.

[32]  D L Streiner,et al.  An Introduction to Multivariate Statistics , 1993, Canadian journal of psychiatry. Revue canadienne de psychiatrie.