Penalty, Shrinkage and Pretest Strategies: Variable Selection and Estimation

The objective of this book is to compare the statistical properties of penalty and non-penalty estimation strategies for some popular models. Specifically, it considers the full model, submodel, penalty, pretest and shrinkage estimation techniques for three regression models before presenting the asymptotic properties of the non-penalty estimators and their asymptotic distributional efficiency comparisons. Further, the risk properties of the non-penalty estimators and penalty estimators are explored through a Monte Carlo simulation study. Showcasing examples based on real datasets, the book will be useful for students and applied researchers in a host of applied fields.The books level of presentation and style make it accessible to a broad audience. It offers clear, succinct expositions of each estimation strategy. More importantly, it clearly describes how to use each estimation strategy for the problem at hand. The book is largely self-contained, as are the individual chapters, so that anyone interested in a particular topic or area of application may read only that specific chapter. The book is specially designed for graduate students who want to understand the foundations and concepts underlying penalty and non-penalty estimation and its applications. It is well-suited as a textbook for senior undergraduate and graduate courses surveying penalty and non-penalty estimation strategies, and can also be used as a reference book for a host of related subjects, including courses on meta-analysis. Professional statisticians will find this book to be a valuable reference work, since nearly all chapters are self-contained.

[1]  Mee Young Park,et al.  L1‐regularization path algorithm for generalized linear models , 2007 .

[2]  R. Tibshirani,et al.  A bias correction for the minimum error rate in cross-validation , 2009, 0908.2904.

[3]  M. Neves,et al.  A local maximum likelihood estimator for Poisson regression , 2008 .

[4]  A. A. Weiss,et al.  Semiparametric estimates of the relation between weather and electricity sales , 1986 .

[5]  K. Doksum,et al.  SHRINKAGE, PRETEST AND ABSOLUTE PENALTY ESTIMATORS IN PARTIALLY LINEAR MODELS , 2007 .

[6]  S. Ejaz Ahmed,et al.  Positive shrinkage, improved pretest and absolute penalty estimators in partially linear models , 2009 .

[7]  S. Ejaz Ahmed,et al.  Shrinkage and absolute penalty estimation in linear regression models , 2012 .

[8]  Simultaneous Estimation of Several Intraclass Correlation Coefficients , 2001 .

[9]  A. Bowman,et al.  Applied smoothing techniques for data analysis : the kernel approach with S-plus illustrations , 1999 .

[10]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[11]  S. Ejaz Ahmed,et al.  Optimal Shrinkage Estimation , 2011, International Encyclopedia of Statistical Science.

[12]  S. E. Ahmed,et al.  Nonparametric inference strategies for the quantile functions under left truncation and right censoring , 2007 .

[13]  Jianqing Fan,et al.  Comments on «Wavelets in statistics: A review» by A. Antoniadis , 1997 .

[14]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[15]  B. Efron,et al.  Empirical Bayes on vector observations: An extension of Stein's method , 1972 .

[16]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[17]  S. Ejaz Ahmed,et al.  Absolute Penalty Estimation , 2011, International Encyclopedia of Statistical Science.

[18]  Hua Liang,et al.  Large sample theory of the estimation of the error distribution for a semiparametric model , 1997 .

[19]  P. Chaudhuri,et al.  On a Hybrid Approach to Parametric and Nonparametric Regression , 2011 .

[20]  Florentina Bunea.  Consistent covariate selection and post model selection inference in semiparametric regression , 2004 .

[21]  S. Ejaz Ahmed,et al.  Robust inference strategy in the presence of measurement error , 2010 .

[22]  S. E. Ahmed.  Shrinkage Estimation of Regression Coefficients From Censored Data With Multiple Observations , 2001 .

[23]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[24]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[25]  M. H. Omar,et al.  Stabilizing the Performance of Kurtosis Estimator of Multivariate Data , 2012, Commun. Stat. Simul. Comput..

[26]  S. E. Ahmed,et al.  ESTIMATION OF REGRESSION COEFFICIENTS IN AN EXPONENTIAL REGRESSION MODEL WITH CENSORED OBSERVATIONS , 1999 .

[27]  Kjell A. Doksum,et al.  LASSO and shrinkage estimation in Weibull censored regression models , 2012 .

[28]  David Madigan,et al.  Bayesian Variable and Transformation Selection in Linear Regression , 2002 .

[29]  Asymptotic theory of simultaneous estimation of Poisson means , 2009 .

[30]  ARROLL,et al.  Estimation in Partially Linear Models With Missing Covariates , 2004 .

[31]  Jiti Gao,et al.  The laws of the iterated logarithm of some estimates in partly linear models , 1995 .

[32]  Oliver Linton,et al.  Semiparametric Regression Analysis With Missing Response at Random , 2003 .

[33]  Jian Shi,et al.  Empirical Likelihood for Partially Linear Models , 2000 .

[34]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[35]  S. Sapra.  Pre-test estimation in Poisson regression model , 2003 .

[36]  A. Basu,et al.  Least squares, preliminary test and Stein‐type estimation in general vector AR(p) models , 2000 .

[37]  Hassen A. Muttlak,et al.  Shrinkage estimation in replicated median ranked set sampling , 2010 .

[38]  Hung Chen,et al.  Convergence Rates for Parametric Components in a Partly Linear Model , 1988 .

[39]  Qingfeng Liu.  部分線形モデルの差分推定量の漸近理論 = Asymptotic Theory for Difference-based Estimator of Partially Linear Models , 2009 .

[40]  Pranab Kumar Sen,et al.  Risk comparison of some shrinkage M-estimators in linear models , 2006 .

[41]  N. Hjort,et al.  The Focused Information Criterion , 2003 .

[42]  S. Ahmed.  Improved estimation of the coefficient of variation , 1994 .

[43]  Jiti Gao.  Adaptive parametric test in a semiparametric regression model , 1997 .

[44]  G. Judge,et al.  A Semiparametric Basis for Combining Estimation Problems Under Quadratic Loss , 2004 .

[45]  S. Ahmed,et al.  Improved Estimation of Coefficient Vector in a Regression Model , 2003 .

[46]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[47]  P. Speckman.  Kernel smoothing in partial linear models , 1988 .

[48]  Cun-Hui Zhang.  Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[49]  J. Neyman,et al.  INADMISSIBILITY OF THE USUAL ESTIMATOR FOR THE MEAN OF A MULTIVARIATE NORMAL DISTRIBUTION , 2005 .

[50]  E. AhmedS..  Construction of improved estimators of multinomial proportions , 2000 .

[51]  C. Morris,et al.  Non-Optimality of Preliminary-Test Estimators for the Mean of a Multivariate Normal Distribution , 1972 .

[52]  S. James Press,et al.  International Encyclopedia of Statistics , 1978 .

[53]  Hassen A. Muttlak,et al.  Stein-type estimation using ranked set sampling , 2012 .

[54]  S. E. Ahmed.  Stein–type shrinkage quantile estimation , 2000 .

[55]  S. Ejaz Ahmed,et al.  An application of shrinkage estimation to the nonlinear regression model , 2012, Comput. Stat. Data Anal..

[56]  I. Johnstone,et al.  Minimax estimation via wavelet shrinkage , 1998 .

[57]  S. E. Ahmed,et al.  Estimation strategies for the intercept vector in a simple linear multivariate normal regression model , 1990 .

[58]  Shakhawat Hossain,et al.  Shrinkage and Penalty Estimators of a Poisson Regression Model , 2012 .

[59]  S. E. Ahmed,et al.  Large‐Sample Pooling Procedure for Correlation , 1992 .

[60]  Pravin K. Trivedi,et al.  Regression Analysis of Count Data: Preface , 1998 .

[61]  S. Khan,et al.  Using Several Data to Structure Efficient Estimation of Intraclass Correlation Coefficients , 2002 .

[62]  P. Bickel.  Parametric Robustness: Small Biases can be Worthwhile , 1984 .

[63]  David R. Anderson,et al.  Model Selection and Multimodel Inference , 2003 .

[64]  S. E. Ahmed,et al.  Large-sample estimation strategies for eigenvalues of a Wishart matrix , 1998 .

[65]  Wolfgang Härdle,et al.  Partially Linear Models , 2000 .

[67]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .