Strategies for detecting moderator variables: A review of conceptual and empirical issues

Moderator variables (moderators) are of considerable interest to researchers in various academic disciplines (e.g. human resources management, industrial and organizational psychology, management, organizational behavior, and organizational theory) who study phenomena involving the interactive effects of two or more independent variables. Several comprehensive reviews of the literature on moderators have appeared in the past three decades (e.g. Stone, 1988; Zedeck, 1971). Since the latest of these reviews, a large number of papers have been published on moderator-related issues. Unfortunately, however, there is no relatively comprehensive, up-to-date review of this literature. Thus, the purpose of this article is to provide such a review. Among the topics considered in it are conceptual definitions of moderators, prior reviews of the literature on moderators, major strategies for detecting moderators, and strategies for increasing statistical power in studies of moderators.

[1]  Fan Yang,et al.  Nonlinear structural equation models: The Kenny-Judd model with Interaction effects , 1996 .

[2]  Herman Aguinis,et al.  Statistical Power Computations for Detecting Dichotomous Moderator Variables with Moderated Multiple Regression , 1998 .

[3]  William L. Hays,et al.  Statistics, 5th ed. , 1994 .

[4]  H. F. Taylor,et al.  On a General Model for Social and Cognitive Consistency , 1979 .

[5]  J. Jaccard.  Interaction Effects in Factorial Analysis of Variance , 1997 .

[6]  Robert L. Linn,et al.  Range restriction problems in the use of self-selected groups for test validation. , 1968 .

[7]  Margaret Bull Kovera,et al.  Double-blind photoarray administration as a safeguard against investigator bias. , 1999 .

[8]  David R. Heise.  Estimating Nonlinear Models , 1986 .

[9]  J. Levin,et al.  Testing for regression homogeneity under variance heterogeneity , 1982 .

[10]  R. Landis,et al.  Moderated Multiple Regression Tests are Criterion Specific , 2000 .

[11]  P. Wright.  The Social Psychology of Organizations, (2nd Ed): Daniel Katz and Robert L. Kahn New York: Wiley, 1978, 838 pp. , 1979 .

[12]  R. Rosenthal.  Meta-analytic procedures for social research , 1984 .

[13]  L. Witt,et al.  Interpreting Moderator Effects: Substitute for the Signed Coefficient Rule , 1995 .

[14]  J. Cotton,et al.  Mentor functions and outcomes: a comparison of men and women in formal and informal mentoring relationships. , 1999 .

[15]  Brian S. O'Leary,et al.  Sample sizes for specified statistical power in testing for differential validity. , 1980 .

[16]  Wayne F. Velicer,et al.  The moderator variable viewed as heterogeneous regression. , 1972 .

[17]  Philip Bobko,et al.  TESTING FOR FAIRNESS WITH A MODERATED MULTIPLE REGRESSION STRATEGY: AN ALTERNATIVE TO DIFFERENTIAL ANALYSIS , 1978 .

[18]  William P. Dunlap,et al.  Meta-analysis and moderator variables: a cautionary note on transportability , 1989 .

[19]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[20]  Robert M. Guion,et al.  Personnel assessment, selection, and placement. , 1991 .

[21]  Meni Koslowsky,et al.  On the efficacy of credibility intervals as indicators of moderator effects in meta-analytic research , 1993 .

[22]  Steven G. Rogelberg,et al.  Handbook of research methods in industrial and organizational psychology , 2004 .

[23]  Judith A. Hall,et al.  Testing for moderator variables in meta‐analysis: Issues and methods , 1991 .

[24]  Herman Aguinis,et al.  Methodological Artifacts in Moderated Multiple Regression and Their Effects on Statistical Power , 1997 .

[25]  R. Stine,et al.  Bootstrapping Goodness-of-Fit Measures in Structural Equation Models , 1992 .

[26]  Herman Aguinis,et al.  Statistical Power with Moderated Multiple Regression in Management Research , 1995 .

[27]  J. Sánchez-Meca,et al.  TESTING CONTINUOUS MODERATORS IN META-ANALYSIS : A COMPARISON OF PROCEDURES , 1998 .

[28]  Herman Aguinis,et al.  Testing Moderator Variable Hypotheses Meta-Analytically: , 1998 .

[29]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[30]  Craig A. Mason,et al.  Assessing Moderator Variables: Two Computer Simulation Studes , 1996 .

[31]  M. Dean,et al.  To Log or Not to Log: Bootstrap as an Alternative to the Parametric Estimation of Moderation Effects in the Presence of Skewed Dependent Variables , 2000 .

[32]  C. Judd,et al.  Statistical difficulties of detecting interactions and moderator effects. , 1993, Psychological bulletin.

[33]  R. Koestner,et al.  To predict some of the people some of the time: A reexamination of the moderator variable approach in personality theory. , 1988 .

[34]  D. R. Saunders.  Moderator Variables in Prediction , 1956 .

[35]  R. Overton,et al.  A comparison of fixed-effects and mixed (random-effects) models for meta-analysis tests of moderator , 1998 .

[36]  Leonard A. Marascuilo,et al.  Statistical methods for behavioral science research , 1971 .

[37]  L. Humphreys.  Statistical definitions of test validity for minority groups. , 1973 .

[38]  Michael A. Hidiroglou,et al.  Regression Estimation after Correcting for Attenuation , 1978 .

[39]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[40]  R. Sitgreaves.  Psychometric theory (2nd ed.). , 1979 .

[41]  E. Ziegel,et al.  Bootstrapping: A Nonparametric Approach to Statistical Inference , 1993 .

[42]  Single-group validity: Fallacy of the facts? , 1977 .

[43]  J. P. Guilford,et al.  Fundamental statistics in psychology and education, 3rd ed. , 1956 .

[44]  H. J. Arnold,et al.  Testing moderator variable hypotheses: A reply to stone and hollenbeck , 1984 .

[45]  Herman Aguinis,et al.  Heterogeneity of Error Variance and the Assessment of Moderating Effects of Categorical Variables: A Conceptual Review , 1998 .

[46]  D. Rubin,et al.  Comparing Effect Sizes of Independent Studies , 1982 .

[47]  Michael J. Burke,et al.  A new meta-analytic approach. , 1991 .

[48]  J. Elashoff,et al.  Multiple Regression in Behavioral Research. , 1975 .

[49]  Herman Aguinis,et al.  Appraisal of the Homogeneity of Error Variance Assumption and Alternatives to Multiple Regression for Estimating Moderating Effects of Categorical Variables , 1999 .

[50]  Wesley C. King,et al.  Gender and Administration Mode Effects when Pencil-And-Paper Personality Tests are Computerized , 1998 .

[51]  Yoav Ganzach,et al.  Interaction in Linear versus Logistic Models: A Substantive Illustration Using the Relationship between Motivation, Ability, and Performance , 2000 .

[52]  Gregg B. Jackson,et al.  Meta-Analysis: Cumulating Research Findings Across Studies , 1982 .

[53]  Sheldon Zedeck,et al.  Comparison of "joint moderators" in three prediction techniques. , 1971 .

[54]  Edward R. Kemery,et al.  Effects of predictor intercorrelations and reliabilities on moderated multiple regression , 1988 .

[55]  Craig J. Russell,et al.  Moderated Regression Analysis and Likert Scales: Too Coarse for Comfort , 1992 .

[56]  L. A. Marascuilo,et al.  Appropriate Post Hoc Comparisons for Interaction and Nested Hypotheses in Analysis of Variance Designs: The Elimination of Type IV Errors1 , 1970 .

[57]  Lawrence R. James,et al.  Validity generalization in the context of situational models , 1992 .

[58]  Donald W. Marquardt,et al.  Comment: You Should Standardize the Predictor Variables in Your Regression Models , 1980 .

[59]  J. Tisak,et al.  A Theoretical Note on the Stochastics of Moderated Multiple Regression , 1994 .

[60]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[61]  J. A. Algera,et al.  Validity generalization: Some critical remarks on the Schmidt-Hunter procedure* , 1984 .

[62]  J. Jaccard.  Interaction effects in logistic regression , 2001 .

[63]  Ralph A. Alexander,et al.  Effect of error variance heterogeneity on the power of tests for regression slope differences. , 1994 .

[64]  J. D. Sherman,et al.  Failures to detect moderating effects with ordinary least squares-moderated multiple regression: Some reasons and a remedy. , 1986 .

[65]  W. Fuller,et al.  An Errors-In-Variables Analysis of Managerial Role Performance , 1974 .

[66]  John E. Hunter,et al.  Methods of Meta-Analysis: Correcting Error and Bias in Research Findings , 1991 .

[67]  M. Evans.  Moderated Regression: Legitimate Disagreement, Confusion and Misunderstanding , 1987 .

[68]  C. Lance.  Residual Centering, Exploratory and Confirmatory Moderator Analysis, and Decomposition of Effects in Path Models Containing Interactions , 1988 .

[69]  Michael H. Kutner.  Applied Linear Statistical Models , 1974 .

[70]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[71]  F. Oswald,et al.  The homogeneity assumption in differential prediction analysis: does it really matter? , 2000, The Journal of applied psychology.

[72]  E. F. Stone-Romero,et al.  A Comparison of Bias and Mean Squared Error in Parameter Estimates of Interaction Effects: Moderated Multiple Regression versus Error-in-Variables Regression. , 1996, Multivariate behavioral research.

[73]  R. Katzell,et al.  Differential validity revived. , 1977 .

[74]  G. Evans,et al.  Moderating and Mediating Processes in Environment-Behavior Research , 1997 .

[75]  S. Parker,et al.  Minimizing strain and maximizing learning: the role of job demands, job control, and proactive personality. , 1999, The Journal of applied psychology.

[76]  Lyman W. Porter,et al.  Managerial attitudes and performance , 1968 .

[77]  Paul R. Sackett,et al.  On seeking moderator variables in the meta-analysis of correlational data: A Monte Carlo investigation of statistical power and resistance to Type I error. , 1986 .

[78]  S. Zedeck.  Problems with the use of "moderator" variables. , 1971 .

[79]  Eugene F. Stone,et al.  Some issues associated with the use of moderated regression , 1984 .

[80]  E. F. Stone-Romero,et al.  Estimating the Power to Detect Dichotomous Moderators with Moderated Multiple Regression , 1994 .

[81]  Philip Bobko,et al.  A solution to some dilemmas when testing hypotheses about ordinal interactions. , 1986 .

[82]  Eugene F. Stone,et al.  Clarifying some controversial issues surrounding statistical procedures for detecting moderator variables: Empirical evidence and related matters. , 1989 .

[83]  J. Jaccard,et al.  LISREL Approaches to Interaction Effects in Multiple Regression , 1998 .

[84]  H. J. Arnold.  Moderator variables: A clarification of conceptual, analytic, and psychometric issues , 1982 .

[85]  R. Katzell,et al.  On differential validity and bias. , 1978 .

[86]  Eugene F. Stone.  Research methods in organizational behavior , 1978 .

[87]  Edward R. Kemery,et al.  The power of the Schmidt and Hunter Additive Model of Validity Generalization. , 1987 .

[88]  Ralph L. Rosnow,et al.  Essentials of Behavioral Research: Methods and Data Analysis , 1984 .

[89]  Juan I. Sanchez,et al.  Moderator Search in Meta-Analysis: A Review and Cautionary Note on Existing Approaches , 1998 .

[90]  E. F. Stone-Romero,et al.  Relative power of moderated multiple regression and the comparison of subgroup correlation coefficients for detecting moderating effects. , 1994 .

[91]  Meni Koslowsky,et al.  DETECTING MODERATORS WITH META‐ANALYSIS: AN EVALUATION AND COMPARISON OF TECHNIQUES , 1993 .

[92]  Daniel J. Svyantek,et al.  Analyzing meta-analysis: Potential problems, an unsuccessful replication, and evaluation criteria. , 1985 .

[93]  V. Vroom.  Work and motivation , 1964 .

[94]  Craig J. Russell,et al.  On Theory, Statistics, and the Search for Interactions in the Organizational Sciences , 1994 .

[95]  Herman Aguinis,et al.  Type II Error Problems in the Use of Moderated Multiple Regression for the Detection of Moderating Effects of Dichotomous Variables , 1994 .

[96]  R. Linn.  Single-group validity, differential validity, and differential prediction. , 1978 .

[97]  R. Campbell,et al.  A Theory of Leadership Effectiveness. , 1968 .

[98]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[99]  M. Evans.  A Monte Carlo study of the effects of correlated method variance in moderated multiple regression analysis , 1985 .

[100]  John E. Hunter,et al.  Validity generalization results for tests used to predict job proficiency and training success in clerical occupations. , 1980 .

[101]  J. Jaccard,et al.  Interaction effects in multiple regression , 1992 .

[102]  L. James,et al.  Mediators, Moderators, and Tests for Mediation. , 1984 .

[103]  Jacob Cohen.  Multiple regression as a general data-analytic system. , 1968 .

[104]  M. B. Jones.  Correlation as a dependent variable. , 1968, Psychological bulletin.

[105]  George W. Bohrnstedt,et al.  The Reliability of Products of Two Random Variables , 1978 .

[106]  Lawrence R. James,et al.  A Note on Validity Generalization Procedures. , 1986 .

[107]  S. West,et al.  Multiple Regression: Testing and Interpreting Interactions. , 1994 .

[108]  Herman Aguinis,et al.  Statistical power problems with moderated multiple regression in management research. , 1995 .

[109]  G. M. Mullet,et al.  Where Have all the Moderators Gone: The Perils of Type II Error. , 1977 .

[110]  Joel R. Levin,et al.  Multivariate statistics in the social sciences : a researcher's guide , 1985 .

[111]  John E. Hunter,et al.  FORTY QUESTIONS ABOUT VALIDITY GENERALIZATION AND META‐ANALYSIS , 1985 .

[112]  L. E. Jones,et al.  Analysis of multiplicative combination rules when the causal variables are measured with error. , 1983 .

[113]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[114]  L. Cronbach.  Statistical tests for moderator variables: flaws in analyses recently proposed , 1987 .

[115]  Richard,et al.  Motivation through the Design of Work: Test of a Theory. , 1976 .

[116]  Edward R. Kemery,et al.  Failure to detect moderating effects: Is multicollinearity the problem? , 1987 .

[117]  D. Jackson,et al.  Type I error rates for moderated multiple regression analysis. , 1988 .

[118]  David Rogosa,et al.  Comparing nonparallel regression lines. , 1980 .