A non-degenerate estimator for hierarchical variance parameters via penalized likelihood estimation

Variance parameters in mixed or multilevel models can be difficult to estimate, especially when the number of groups is small. We propose a maximum penalized likelihood approach which is equivalent to estimating variance parameters by their marginal posterior mode, given a weakly informative prior distribution. By choosing the prior from the gamma family with at least 1 degree of freedom, we ensure that the prior density is zero at the boundary and thus the marginal posterior mode of the group-level variance will be positive. The use of a weakly informative prior allows us to stabilize our estimates while remaining faithful to the data.

[1]  J. Neyman,et al.  Consistent Estimates Based on Partially Consistent Observations , 1948 .

[2]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[3]  P. J. Huber.  The behavior of maximum likelihood estimates under nonstandard conditions , 1967 .

[4]  H. D. Patterson,et al.  Recovery of inter-block information when block sizes are unequal , 1971 .

[5]  D. Harville.  Bayesian inference for variance components using only error contrasts , 1974 .

[6]  L. Gleser,et al.  Classical asymptotic properties of a certain estimator related to the maximum likelihood estimator , 1975 .

[7]  A. O'Hagan,et al.  On posterior joint and marginal modes , 1976 .

[8]  D. Harville.  Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems , 1977 .

[9]  J. Miller,et al.  Asymptotic Properties of Maximum Likelihood Estimates in the Mixed Model of the Analysis of Variance , 1977 .

[10]  THE EFFECTS OF SPECIAL PREPARATION ON SAT‐VERBAL SCORES , 1979 .

[11]  H. White.  A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity , 1980 .

[12]  D. Rubin.  Estimation in Parallel Randomized Experiments , 1981 .

[13]  J. Ware,et al.  Random-effects models for longitudinal data. , 1982, Biometrics.

[14]  John F. Monahan,et al.  Monte Carlo Comparison of ANOVA, MIVQUE, REML, and ML Estimators of Variance Components , 1984 .

[15]  S. Raudenbush,et al.  Empirical Bayes Meta-Analysis , 1985 .

[16]  H. Swaminathan,et al.  Bayesian estimation in the two-parameter logistic model , 1985 .

[17]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[18]  Robert J. Mislevy,et al.  Bayes modal estimation in item response models , 1986 .

[19]  Hsin Ying Lin,et al.  Bayesian estimation of item response curves , 1986 .

[20]  Andrea Rotnitzky,et al.  Regression Models for Discrete Longitudinal Responses , 1993 .

[21]  Roel Bosker,et al.  Standard Errors and Sample Sizes for Two-Level Research , 1993 .

[22]  Peter McCullagh,et al.  [Regression Models for Discrete Longitudinal Responses]: Comment , 1993 .

[23]  T. Mathew,et al.  Improved Nonnegative Estimation of Variance Components in Balanced Multivariate Mixed Models , 1994 .

[24]  R. J. Kelly,et al.  Improved Nonnegative Estimation of Variance Components in Some Mixed Models With Unbalanced Data , 1994 .

[25]  Walter R. Gilks,et al.  Model checking and model improvement , 1995 .

[26]  David Draper,et al.  Assessment and Propagation of Model Uncertainty , 2011 .

[27]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[28]  M. Kenward,et al.  Small sample inference for fixed effects from restricted maximum likelihood. , 1997, Biometrics.

[29]  S. Thompson,et al.  Detecting and describing heterogeneity in meta-analysis. , 1998, Statistics in medicine.

[30]  M. Srivastava,et al.  Improved nonnegative estimation of multivariate components of variance , 1999 .

[31]  E. Maris.  Estimating multiple classification latent class models , 1999 .

[32]  Nicholas T. Longford.  On Estimating Standard Errors in Multilevel Analysis , 2000 .

[33]  S. Greenland.  When Should Epidemiologic Regressions Use Random Coefficients? , 2000, Biometrics.

[34]  Sarah E. Brockwell,et al.  A comparison of statistical methods for meta‐analysis , 2001, Statistics in medicine.

[35]  Ana Ivelisse Avilés,et al.  Linear Mixed Models for Longitudinal Data , 2001, Technometrics.

[36]  D. Ruppert,et al.  Likelihood ratio tests in linear mixed models with one variance component , 2003 .

[37]  D. Ruppert,et al.  Some properties of Likelihood Ratio Tests in Linear Mixed Models , 2003 .

[38]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[39]  Jeroen K. Vermunt,et al.  Bayesian Posterior Estimation of Logit Parameters with Small Samples , 2004 .

[40]  T. Kubokawa,et al.  Estimation of Covariance Matrices in Fixed and Mixed Effects Linear Models (Subsequently published in "Journal of Multivariate Analysis", 97, 2242-2261, 2006. ) , 2006 .

[41]  A. Gelman,et al.  Rich State, Poor State, Red State, Blue State: What's the Matter with Connecticut? , 2005 .

[42]  S. Rabe-Hesketh,et al.  Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects , 2005 .

[43]  Sophia Rabe-Hesketh,et al.  Multilevel and Longitudinal Modeling Using Stata , 2005 .

[44]  Jeroen K. Vermunt,et al.  AVOIDING BOUNDARY ESTIMATES IN LATENT CLASS ANALYSIS BY BAYESIAN POSTERIOR MODE ESTIMATION , 2006 .

[45]  Elizabeth L. Scott,et al.  Consistent Estimates Based on Partially Consistent Observations Author ( s ) : , 2007 .

[46]  Sophia Rabe-Hesketh,et al.  Multilevel and Longitudinal Modeling Using Stata, Second Edition , 2008 .

[47]  P. Verde,et al.  Comment on: Efficacy and safety of tigecycline: a systematic review and meta-analysis. , 2011, The Journal of antimicrobial chemotherapy.

[48]  Nicholas T. Longford,et al.  Random Coefficient Models , 1994, International Encyclopedia of Statistical Science.