Frequentist model averaging estimation: a review

In applications, the traditional estimation procedure generally begins with model selection. Once a specific model is selected, subsequent estimation is conducted under the selected model without consideration of the uncertainty from the selection process. This often leads to the underreporting of variability and too optimistic confidence sets. Model averaging estimation is an alternative to this procedure, which incorporates model uncertainty into the estimation process. In recent years, there has been a rising interest in model averaging from the frequentist perspective, and some important progresses have been made. In this paper, the theory and methods on frequentist model averaging estimation are surveyed. Some future research topics are also discussed.

[1]  Andrew R. Barron,et al.  Information Theory and Mixing Least-Squares Regressions , 2006, IEEE Transactions on Information Theory.

[2]  J. M. Bates,et al.  The Combination of Forecasts , 1969 .

[3]  David R. Anderson,et al.  Multimodel Inference , 2004 .

[4]  Emma Sutton.  Rejoinder , 2010 .

[5]  R. L. Winkler,et al.  Combining Economic Forecasts , 1986 .

[6]  Bruce E. Hansen,et al.  Averaging estimators for autoregressions with a near unit root , 2010 .

[7]  Paul Kabaila,et al.  ON VARIABLE SELECTION IN LINEAR REGRESSION , 2002, Econometric Theory.

[8]  Colin L. Mallows,et al.  Some Comments on Cp , 2000, Technometrics.

[9]  F. Turkheimer,et al.  On the Undecidability among Kinetic Models: From Model Selection to Model Averaging , 2003, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[10]  Yuhong Yang.  Adaptive Regression by Mixing , 2001 .

[11]  Michael Schomaker,et al.  Frequentist Model Averaging with missing observations , 2010, Comput. Stat. Data Anal..

[12]  B. Hansen.  Least Squares Model Averaging , 2007 .

[13]  David Draper,et al.  Assessment and Propagation of Model Uncertainty , 2011 .

[14]  R. Fildes.  Journal of the American Statistical Association : William S. Cleveland, Marylyn E. McGill and Robert McGill, The shape parameter for a two variable graph 83 (1988) 289-300 , 1989 .

[15]  Jan R. Magnus,et al.  On the harm that ignoring pretesting can cause , 2004 .

[16]  Ua,et al.  On Optimal Weight Choice in a Frequentist Model Average Estimator1 , 2009 .

[17]  Bruce E. Hansen,et al.  Least-squares forecast averaging , 2008 .

[18]  Xinyu Zhang,et al.  On the Use of Model Averaging in Tourism Research , 2009 .

[19]  Christophe Croux,et al.  Variable Selection for Logistic Regression Using a Prediction‐Focused Information Criterion , 2006, Biometrics.

[20]  Nils Lid Hjort,et al.  Discussion: Rejoinder to 'The focussed information criterion' and 'Frequentist model averaging' , 2003 .

[21]  Tae-Hwan Kim,et al.  James-Stein-Type Estimators in Large Samples With Application to the Least Absolute Deviations Estimator , 2000 .

[22]  Dl Dmitry Danilov,et al.  Forecast accuracy after pretesting with an application to the stock market , 2004 .

[23]  E. Wagenmakers,et al.  AIC model selection using Akaike weights , 2004, Psychonomic bulletin & review.

[24]  George Kapetanios,et al.  Forecasting Using Bayesian and Information Theoretic Model Averaging: An Application to UK Inflation , 2005 .

[25]  M. Clyde,et al.  Model Uncertainty , 2003 .

[26]  Yuhong Yang,et al.  Combining Linear Regression Models , 2005, Journal of the American Statistical Association.

[27]  C. Granger,et al.  Experience with Forecasting Univariate Time Series and the Combination of Forecasts , 1974 .

[28]  R. Carroll,et al.  An Asymptotic Theory for Model Selection Inference in General Semiparametric Problems , 2007 .

[29]  Yuhong Yang.  REGRESSION WITH MULTIPLE CANDIDATE MODELS: SELECTING OR MIXING? , 1999 .

[30]  Robert F. Phillips.  Composite Forecasting: An Integrated Approach and Optimality Reconsidered , 1987 .

[31]  K. Burnham,et al.  Model selection: An integral part of inference , 1997 .

[32]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[33]  Rand R. Wilcox,et al.  The statistical implications of pre-test and Stein-rule estimators in econometrics , 1978 .

[34]  Guohua Zou,et al.  Least Squares Model Combining by Mallows Criterion , 2008 .

[35]  Guohua Zou,et al.  Estimation of regression coefficients of interest when other regression coefficients are of no interest: The case of non-normal errors , 2007 .

[36]  Nils Lid Hjort,et al.  Model Selection and Model Averaging , 2001 .

[37]  Nils Lid Hjort,et al.  Focused Information Criteria and Model Averaging for the Cox Hazard Regression Model , 2006 .

[38]  James Durbin,et al.  Estimation of Regression Coefficients of Interest when Other Regression Coefficients are of no Interest , 1999 .

[39]  D. Madigan,et al.  Bayesian Model Averaging for Linear Regression Models , 1997 .

[40]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[41]  Jan R. Magnus,et al.  A comparison of two model averaging techniques with an application to growth empirics , 2010 .

[42]  David A. Bessler,et al.  Forecasting livestock prices with individual and composite methods , 1981 .

[43]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[44]  Bruce E. Hansen,et al.  CHALLENGES FOR ECONOMETRIC MODEL SELECTION , 2005, Econometric Theory.

[45]  G. Kapetanios,et al.  Forecasting Using Bayesian and Information-Theoretic Model Averaging , 2008 .

[46]  C. Mallows.  More comments on C p , 1995 .

[47]  Paolo Zaffaroni,et al.  Model Averaging in Risk Management with an Application to Futures Markets , 2008, SSRN Electronic Journal.

[48]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[49]  C. L. Mallows.  Some comments on C_p , 1973 .

[50]  David R. Anderson,et al.  Model Selection and Multimodel Inference , 2003 .

[51]  R. W. Farebrother,et al.  The statistical implications of pre-test and Stein-rule estimators in econometrics , 1978 .

[52]  H. Akaike.  Maximum likelihood identification of Gaussian autoregressive moving average models , 1973 .

[53]  D. Steinberg,et al.  Technometrics , 2008 .

[54]  David R. Anderson,et al.  Understanding AIC and BIC in Model Selection , 2004 .

[55]  Nils Lid Hjort,et al.  Model Selection and Model Averaging: Contents , 2008 .

[56]  N. Hjort,et al.  Frequentist Model Average Estimators , 2003 .

[57]  Owen Powell,et al.  A Comparison of Two Averaging Techniques with an Application to Growth Empirics , 2008 .