Efficient Shrinkage for Generalized Linear Mixed Models Under Linear Restrictions

In this paper, we consider the pretest, shrinkage, and penalty estimation procedures for generalized linear mixed models when it is conjectured that some of the regression parameters are restricted to a linear subspace. We develop the statistical properties of the pretest and shrinkage estimation methods, which include asymptotic distributional biases and risks. We show that the pretest and shrinkage estimators have a significantly higher relative efficiency than the classical estimator. Furthermore, we consider the penalty estimator LASSO (Least Absolute Shrinkage and Selection Operator), and numerically compare its relative performance with that of the other estimators. A series of Monte Carlo simulation experiments are conducted with different combinations of inactive predictors, and the performance of each estimator is evaluated in terms of the simulated mean squared error. The study shows that the shrinkage and pretest estimators are comparable to the LASSO estimator when the number of inactive predictors in the model is relatively large. The estimators under consideration are applied to a real data set to illustrate the usefulness of the procedures in practice.

[1]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[2]  Gerhard Tutz,et al.  Variable selection for generalized linear mixed models by L1-penalized estimation , 2012, Statistics and Computing.

[3]  Noah Simon,et al.  A Sparse-Group Lasso , 2013 .

[4]  S. Hossain,et al.  Application of shrinkage estimation in linear regression models with autoregressive errors , 2015 .

[5]  S. Ejaz Ahmed,et al.  Shrinkage, pretest, and penalty estimators in generalized linear models , 2015 .

[6]  J. Goeman.  L1 Penalized Estimation in the Cox Proportional Hazards Model , 2009, Biometrical journal. Biometrische Zeitschrift.

[7]  T. Louis.  Finding the Observed Information Matrix When Using the EM Algorithm , 1982 .

[8]  K. Doksum,et al.  L 1 penalty and shrinkage estimation in partially linear models with random coefficient autoregressive errors , 2012 .

[9]  J. Shao,et al.  Regularizing lasso: A consistent variable selection method , 2015 .

[10]  S. R. Searle,et al.  Generalized, Linear, and Mixed Models , 2005 .

[11]  Liangjun Su,et al.  Shrinkage Estimation of Dynamic Panel Data Models with Interactive Fixed Effects , 2015 .

[12]  Pranab Kumar Sen,et al.  Risk comparison of some shrinkage M-estimators in linear models , 2006 .

[13]  A. Sommer,et al.  Increased risk of respiratory disease and diarrhea in children with preexisting mild vitamin A deficiency. , 1984, The American journal of clinical nutrition.

[14]  P. McCullagh,et al.  Generalized Linear Models , 1972, Predictive Analytics.

[15]  Ryan J. Tibshirani,et al.  Efficient Implementations of the Generalized Lasso Dual Path Algorithm , 2014, ArXiv.

[16]  Jianqing Fan,et al.  A Selective Overview of Variable Selection in High Dimensional Feature Space. , 2009, Statistica Sinica.

[17]  Jürg Schelldorfer,et al.  GLMMLasso: An Algorithm for High-Dimensional Generalized Linear Mixed Models Using ℓ1-Penalization , 2011, 1109.4003.

[18]  S. Sinha,et al.  Testing for generalized linear mixed models with cluster correlated data under linear inequality constraints , 2012 .

[19]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[20]  S. Ahmed,et al.  Shrinkage estimation and selection for a logistic regression model , 2014 .

[21]  Kjell A. Doksum,et al.  LASSO and shrinkage estimation in Weibull censored regression models , 2012 .

[22]  H. Zou,et al.  Sparse precision matrix estimation via lasso penalized D-trace loss , 2014 .

[23]  M. Ghahramani,et al.  Efficient estimation for time series following generalized linear models , 2016 .

[24]  Mortaza Jamshidian,et al.  On Algorithms for Restricted Maximum Likelihood Estimation , 2002, Comput. Stat. Data Anal..

[25]  Ludwig Fahrmeir,et al.  Bayesian smoothing and regression for longitudinal, spatial and event history data / Ludwig Fahrmeir , 2011 .

[26]  P. Diggle,et al.  Analysis of Longitudinal Data , 2003 .

[27]  S. Nkurunziza,et al.  A class of Stein‐rules in multivariate regression model with structural changes , 2016 .

[28]  Heng Lian,et al.  Shrinkage estimation for identification of linear components in additive models , 2012 .

[29]  Chin-Tsang Chiang,et al.  KERNEL SMOOTHING ON VARYING COEFFICIENT MODELS WITH LONGITUDINAL DEPENDENT VARIABLE , 2000 .