Bayesian predictive simultaneous variable and transformation selection in the linear model

Variable selection and transformation selection are two commonly encountered problems in the linear model. It is often of interest to combine these two procedures in an analysis. Due to recent developments in computing technology, such a procedure is now feasible. In this paper, we propose two variable and transformation selection procedures on the predictor variables in the linear model. The first procedure is a simultaneous variable and transformation selection procedure. For data sets with many predictors, a backward elimination procedure for variables and transformations is also presented. The procedures are based on Bayesian model selection criteria introduced by Ibrahim and Laud (1994) and Laud and Ibrahim (1995). Several examples are given to illustrate the methodology.

[1]  Rick L. Edgeman,et al.  LISP-STAT: An Object-Oriented Environment for Statistical Computing and Dynamic Graphics , 1992 .

[2]  H. Akaike.  INFORMATION THEORY AS AN EXTENSION OF THE MAXIMUM LIKELIHOOD , 1973 .

[3]  Adrian E. Raftery,et al.  Simultaneous Variable and Transformation Selection in Linear Regression , 1995 .

[4]  Purushottam W. Laud,et al.  A Predictive Approach to the Analysis of Designed Experiments , 1994 .

[5]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[6]  K. Gaver,et al.  Posterior probabilities of alternative linear models , 1971 .

[7]  G. Box,et al.  Transformation of the Independent Variables , 1962 .

[8]  A. McQuarrie,et al.  Regression and Time Series Model Selection , 1998 .

[9]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[10]  D. Madigan,et al.  Bayesian Model Averaging for Linear Regression Models , 1997 .

[11]  S. Geisser,et al.  A Predictive Approach to Model Selection , 1979 .

[12]  Anders Hald,et al.  Statistical Theory with Engineering Applications , 1952 .

[13]  Purushottam W. Laud,et al.  Predictive Model Selection , 1995 .

[14]  G. Box.  An analysis of transformations (with discussion) , 1964 .

[15]  D. Lindley.  The Choice of Variables in Multiple Regression , 1968 .

[16]  P. Jagers,et al.  Studies in Bayesian econometrics and statistics: S.E. Fienberg and A. Zellner, eds., In honor of Leonard J. Savage (North-Holland, Amsterdam, 1975) , 1977 .

[17]  Matthias Nagel,et al.  LISP-Stat: an object-oriented environment for statistical computing and dynamic graphics: Luke Tierney (1990) New York: John Wiley & Sons, 397 pp, ISBN 0-471-50916-7, 32.15 [pound sign] sterling , 1992 .

[18]  Edgar F. Borgatta,et al.  Sociological Methodology, 1969. , 1969 .

[19]  A. Raftery.  Bayesian Model Selection in Social Research , 1995 .

[20]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[21]  R. R. Hocking.  The analysis and selection of variables in linear regression , 1976 .

[22]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[23]  T. Hassard,et al.  Applied Linear Regression , 2005 .

[24]  S. Geisser,et al.  A Predictive View of the Detection and Characterization of Influential Observations in Regression Analysis , 1983 .

[25]  T. J. Mitchell,et al.  Bayesian Variable Selection in Linear Regression , 1988 .

[26]  Norman R. Draper,et al.  Applied regression analysis (2. ed.) , 1981, Wiley series in probability and mathematical statistics.

[27]  C. L. Mallows.  Some comments on C_p , 1973 .

[28]  Bayesian Variable Selection in Linear Regression: Comment , 1988 .

[29]  Clifford M. Hurvich,et al.  Regression and time series model selection in small samples , 1989 .

[30]  D. A. Sprott,et al.  Foundations of Statistical Inference. , 1972 .

[31]  D. Ruppert,et al.  Transformation and Weighting in Regression , 1988 .

[32]  S. Weisberg,et al.  Residuals and Influence in Regression , 1982 .

[33]  Joseph G. Ibrahim,et al.  On properties of predictive priors in linear models , 1997 .

[34]  W. Andrew.  LO, . Finance: Survey.. Journal of the American Statistical Association, , . , 2000 .

[35]  R. Kohn,et al.  Nonparametric regression using Bayesian variable selection , 1996 .