Exploratory regression analysis: A tool for selecting models and determining predictor importance

Linear regression analysis is one of the most important tools in a researcher’s toolbox for creating and testing predictive models. Although linear regression analysis indicates how strongly a set of predictor variables, taken together, will predict a relevant criterion (i.e., the multiple R), the analysis cannot indicate which predictors are the most important. Although there is no definitive or unambiguous method for establishing predictor variable importance, there are several accepted methods. This article reviews those methods for establishing predictor importance and provides a program (in Excel) for implementing them (available for direct download at http://dl.dropbox.com/u/2480715/ERA.xlsm?dl=1). The program investigates all 2p – 1 submodels and produces several indices of predictor importance. This exploratory approach to linear regression, similar to other exploratory data analysis techniques, has the potential to yield both theoretical and practical benefits.

[1]  Razia Azen,et al.  Using Dominance Analysis to Determine Predictor Importance in Logistic Regression , 2009 .

[2]  R. M. Johnson.  The minimal transformation to orthonormality , 1966 .

[3]  Kim Nimon,et al.  An R package to compute commonality coefficients in the multiple regression case: An introduction to the package and a practical example , 2008, Behavior research methods.

[4]  David V. Budescu,et al.  Comparing Predictors in Multivariate Regression Models: An Extension of Dominance Analysis , 2006 .

[5]  D. Budescu.  Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression. , 1993 .

[6]  Robert E. Ployhart,et al.  A Monte Carlo Comparison of Relative Importance Methodologies , 2004 .

[7]  P. Meehl,et al.  Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical–statistical controversy. , 1996 .

[8]  P. Hoffman.  The paramorphic representation of clinical judgment. , 1960, Psychological bulletin.

[9]  Jeff W. Johnson.  Factors Affecting Relative Weights: The Influence of Sampling and Measurement Error , 2004 .

[10]  J. Carroll,et al.  A New Measure of Predictor Variable Importance in Multiple Regression , 1978 .

[11]  E. Pedhazur.  Multiple Regression in Behavioral Research: Explanation and Prediction , 1982 .

[12]  Pere J. Ferrando,et al.  Two SPSS programs for interpreting multiple regression results , 2010, Behavior research methods.

[13]  J. W. Johnson.  A Heuristic Method for Estimating the Relative Weight of Predictor Variables in Multiple Regression , 2000, Multivariate behavioral research.

[14]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[15]  L. Fabbris.  Measures of predictor variable importance in multiple regression: An additional suggestion , 1980 .

[16]  Kim F. Nimon,et al.  Revisiting Interpretation of Canonical Correlation Analysis: A Tutorial and Demonstration of Canonical Commonality Analysis , 2010, Multivariate behavioral research.

[17]  Joseph M. Madden,et al.  Use of an all possible combination solution of certain multiple regression problems. , 1963 .

[18]  Ulrike Groemping,et al.  Relative Importance for Linear Regression in R: The Package relaimpo , 2006 .

[19]  Xu Yu-bo,et al.  Reachability Checking of Finite Precision Timed Automata , 2006 .

[20]  David V. Budescu,et al.  Beyond Global Measures of Relative Importance: Some Insights from Dominance Analysis , 2004 .

[21]  B. Green.  Parameter Sensitivity In Multivariate Methods. , 1977, Multivariate behavioral research.

[22]  Scott Tonidandel,et al.  Multivariate relative importance: extending relative weight analysis to multivariate criterion spaces. , 2008, The Journal of applied psychology.

[23]  B. Thompson,et al.  Use of Structure Coefficients in Published Multiple Regression Articles: β is not Enough , 2001 .

[24]  D. Budescu,et al.  The dominance analysis approach for comparing predictors in multiple regression. , 2003, Psychological methods.

[25]  Scott Tonidandel,et al.  Determining the Relative Importance of Predictors in Logistic Regression: An Extension of Relative Weight Analysis , 2010 .

[26]  B. Haig.  Exploratory Factor Analysis, Theory Generation, and Scientific Method , 2005, Multivariate behavioral research.

[27]  Bruce Thompson,et al.  Consequences of Not Interpreting Structure Coefficients in Published CFA Research: A Reminder , 2003 .