Optimal Model Averaging Estimation for Generalized Linear Models and Generalized Linear Mixed-Effects Models

ABSTRACT Considering model averaging estimation in generalized linear models, we propose a weight choice criterion based on the Kullback–Leibler (KL) loss with a penalty term. This criterion is different from that for continuous observations in principle, but reduces to the Mallows criterion in the situation. We prove that the corresponding model averaging estimator is asymptotically optimal under certain assumptions. We further extend our concern to the generalized linear mixed-effects model framework and establish associated theory. Numerical experiments illustrate that the proposed method is promising.

[1]  Geert Molenberghs,et al.  A Family of Generalized Linear Models for Repeated Measures With Normal and Conjugate Random Effects (pp.188) , 2010, 1101.0990.

[2]  Jiming Jiang,et al.  MAXIMUM POSTERIOR ESTIMATION OF RANDOM EFFECTS IN GENERALIZED LINEAR MIXED MODELS , 2001 .

[3]  Linda S. L. Tan,et al.  Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations , 2012, 1205.3906.

[4]  M. Wand,et al.  General design Bayesian generalized linear mixed models , 2006, math/0606491.

[5]  P. Whittle,et al.  Bounds for the Moments of Linear and Quadratic Forms in Independent Variables , 1960 .

[6]  Hollis Ashbaugh-Skaife,et al.  The Effects of Corporate Governance on Firms' Credit Ratings , 2004 .

[7]  K. Burnham,et al.  Model selection: An integral part of inference , 1997 .

[8]  Marno Verbeek,et al.  A Guide to Modern Econometrics , 2000 .

[9]  Adrian E. Raftery,et al.  Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data , 2005, Bioinform..

[10]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[11]  Jianqing Fan,et al.  Sure independence screening in generalized linear models with NP-dimensionality , 2009, The Annals of Statistics.

[12]  Ker-Chau Li,et al.  A Model-Averaging Approach for High-Dimensional Regression , 2014 .

[13]  Jun S. Liu,et al.  Model selection principles in misspecified models , 2010, 1005.5483.

[14]  Hua Liang,et al.  Focused information criterion and model averaging for generalized additive partial linear models , 2011, 1103.1480.

[15]  N. Hjort,et al.  Frequentist Model Average Estimators , 2003 .

[16]  Yuhong Yang,et al.  Combining Linear Regression Models , 2005, Journal of the American Statistical Association.

[17]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[18]  Ryo Okui,et al.  Heteroscedasticity�?Robust C Model Averaging , 2012 .

[19]  Jeffrey S. Racine,et al.  Jackknife model averaging , 2012 .

[20]  C. Conflitti.  Opinion Surveys on the Euro: A Multilevel Multinomial Logistic Analysis , 2011 .

[21]  Lei Nie,et al.  Convergence rate of MLE in generalized linear and nonlinear mixed-effects models: Theory and applications , 2007 .

[22]  Nicholas T. Longford,et al.  Editorial: Model selection and efficiency—is ‘Which model …?’ the right question? , 2005 .

[23]  H. White.  Maximum Likelihood Estimation of Misspecified Models , 1982 .

[24]  F. Vaida,et al.  Conditional Akaike information under generalized linear and proportional hazards mixed models. , 2011, Biometrika.

[25]  Yuhong Yang.  Can the Strengths of AIC and BIC Be Shared , 2005 .

[26]  B. Hansen.  Least Squares Model Averaging , 2007 .

[27]  Subhash R. Lele,et al.  Estimability and Likelihood Inference for Generalized Linear Mixed Models Using Data Cloning , 2010 .

[28]  G. Kitagawa,et al.  Information Criteria and Statistical Modeling , 2007 .

[29]  Jiming Jiang,et al.  Conditional inference about generalized linear mixed models , 1999 .

[30]  Jianqing Fan,et al.  Sure Independence Screening , 2018 .

[31]  Xinyu Zhang,et al.  Adaptively combined forecasting for discrete response time series , 2013 .

[32]  Guohua Zou,et al.  Least squares model averaging by Mallows criterion , 2010 .

[33]  Cheryl J. Flynn,et al.  Efficiency for Regularization Parameter Selection in Penalized Likelihood Estimation of Misspecified Models , 2013, 1302.2068.

[34]  Hua Liang,et al.  Model averaging and weight choice in linear mixed-effects models , 2014 .

[35]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[36]  Peng Wang,et al.  Conditional Inference Functions for Mixed-Effects Models With Unspecified Random-Effects Distribution , 2012 .

[37]  Alan T. K. Wan,et al.  Optimal Weight Choice for Frequentist Model Average Estimators , 2011 .

[38]  Adrian E. Raftery,et al.  Bayesian Model Averaging: A Tutorial , 2016 .

[39]  Bruce E. Hansen,et al.  Model averaging, asymptotic risk, and regressor groups , 2014 .

[40]  Christophe Croux,et al.  Variable Selection for Logistic Regression Using a Prediction‐Focused Information Criterion , 2006, Biometrics.