General maximum likelihood empirical Bayes estimation of normal means

We propose a general maximum likelihood empirical Bayes (GMLEB) method for the estimation of a mean vector based on observations with i.i.d. normal errors. We prove that under mild moment conditions on the unknown means, the average mean squared error (MSE) of the GMLEB is within an infinitesimal fraction of the minimum average MSE among all separable estimators which use a single deterministic estimating function on individual observations, provided that the risk is of greater order than (log n) 5 /n. We also prove that the GMLEB is uniformly approximately minimax in regular and weak l p balls when the order of the length-normalized norm of the unknown means is between (log n) k1 /n 1/(p^2) and n/(log n) k2 . Simulation experiments demonstrate that the GMLEB outperforms the James―Stein and several state-of-the-art threshold estimators in a wide range of settings without much down side.

[1]  A. V. D. Vaart,et al.  Posterior convergence rates of Dirichlet mixtures at smooth densities , 2007, 0708.1885.

[2]  I. Johnstone.  Minimax Bayes, Asymptotic Minimax and Sparse Wavelet Priors , 1994 .

[3]  L. Brown.  Admissible Estimators, Recurrent Diffusions, and Insoluble Boundary Value Problems , 1971 .

[4]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[5]  H. Robbins.  Asymptotically Subminimax Solutions of Compound Statistical Decision Problems , 1985 .

[6]  Cun-Hui Zhang,et al.  Compound decision theory and empirical bayes methods , 2003 .

[7]  Lawrence D. Brown,et al.  NONPARAMETRIC EMPIRICAL BAYES AND COMPOUND DECISION APPROACHES TO ESTIMATION OF A HIGH-DIMENSIONAL VECTOR OF NORMAL MEANS , 2009, 0908.1712.

[8]  Thomas M. Cover,et al.  An algorithm for maximizing expected log investment return , 1984, IEEE Trans. Inf. Theory.

[9]  Yimin Xiao,et al.  On Block Thresholding in Wavelet Regression with Long Memory Correlated Noise , 2007 .

[10]  J. Kiefer,et al.  CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN THE PRESENCE OF INFINITELY MANY INCIDENTAL PARAMETERS , 1956 .

[11]  I. Johnstone,et al.  Adapting to unknown sparsity by controlling the false discovery rate , 2005, math/0505374.

[12]  Cun-Hui Zhang.  General empirical Bayes wavelet methods and exactly adaptive minimax estimation , 2005, math/0504501.

[13]  B. Efron,et al.  Empirical Bayes on vector observations: An extension of Stein's method , 1972 .

[14]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[15]  Y. Vardi,et al.  From image deblurring to optimal investments : maximum likelihood solutions for positive linear inverse problems , 1993 .

[16]  H. Robbins.  An Empirical Bayes Approach to Statistics , 1956 .

[17]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[18]  Cun-Hui Zhang,et al.  GENERALIZED MAXIMUM LIKELIHOOD ESTIMATION OF NORMAL MIXTURE DENSITIES , 2009 .

[19]  C. Carathéodory.  Über den variabilitätsbereich der fourier’schen konstanten von positiven harmonischen funktionen , 1911 .

[20]  B. Efron,et al.  Stein's Estimation Rule and Its Competitors- An Empirical Bayes Approach , 1973 .

[21]  P. Massart,et al.  Gaussian model selection , 2001 .

[22]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[23]  A. V. D. Vaart,et al.  Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities , 2001 .

[24]  E. George.  Minimax Multiple Shrinkage Estimation , 1986 .

[25]  Cun-Hui Zhang,et al.  EMPIRICAL BAYES AND COMPOUND ESTIMATION OF NORMAL MEANS , 1997 .

[26]  I. Johnstone,et al.  Minimax Risk over l p-Balls for l q-error , 1994 .

[27]  J. Neyman,et al.  INADMISSIBILITY OF THE USUAL ESTIMATOR FOR THE MEAN OF A MULTIVARIATE NORMAL DISTRIBUTION , 2005 .

[28]  B. Efron.  Robbins, Empirical Bayes, And Microarrays , 2001 .

[29]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[30]  I. Johnstone,et al.  Minimax risk overlp-balls forlp-error , 1994 .

[31]  C. Morris.  Parametric Empirical Bayes Inference: Theory and Applications , 1983 .

[32]  B. Silverman,et al.  Incorporating Information on Neighboring Coefficients Into Wavelet Estimation , 2001 .

[33]  I. Johnstone,et al.  Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences , 2004, math/0410088.

[34]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[35]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[36]  D. Donoho,et al.  Minimax risk over / p-balls for / q-error , 2022 .

[37]  H. Robbins.  The Empirical Bayes Approach to Statistical Decision Problems , 1964 .

[38]  H. Robbins.  Some Thoughts on Empirical Bayes Estimation , 1983 .

[39]  Dean P. Foster,et al.  The risk inflation criterion for multiple regression , 1994 .

[40]  Cun-Hui Zhang,et al.  Empirical Bayes methods for controlling the false discovery rate with dependent data , 2007, 0708.0978.

[41]  C. Borell.  The Brunn-Minkowski inequality in Gauss space , 1975 .