Statistical Power with Moderated Multiple Regression in Management Research

Due to the increasing importance of moderating (i.e., interaction) effects, the use of moderated multiple regression (MMR) has become pervasive in numerous management specialties such as organizational behavior, human resources management, and strategy, to name a few. Despite its popularity, recent research on the MMR approach to moderator variable detection has identified several factors that reduce statistical power below acceptable levels and, consequently, lead researchers to erroneously dismiss theoretical models that include moderated relationships. The present article (1) briefly describes MMR, (2) reviews factors that affect the statistical power of hypothesis tests conducted using this technique, (3) proposes solutions to low power situations, and (4) discusses areas and problems related to MMR that are in need of further investigation.

[1]  L. Cronbach.  Statistical tests for moderator variables: flaws in analyses recently proposed , 1987 .

[2]  L. Hsu.  More on Transformations and Moderated Regression Analysis: Advantages of Additivity and Homoscedasticity Transformations , 1994 .

[3]  D. Jackson,et al.  Type I error rates for moderated multiple regression analysis. , 1988 .

[4]  Cynthia L. Cordes,et al.  A Review and an Integration of Research on Job Burnout , 1993 .

[5]  R. Alexander,et al.  Effect of Error Variance Heterogeneity on the Power of Tests for Regression Slope Differences , 1994 .

[6]  R. Guion,et al.  A note on concurrent and predictive validity designs: A critical reanalysis. , 1982 .

[7]  Edward R. Kemery,et al.  Failure to detect moderating effects: Is multicollinearity the problem? , 1987 .

[8]  J. M. Cortina,et al.  Interaction, Nonlinearity, and Multicollinearity: implications for Multiple Regression: , 1993 .

[9]  John P. Campbell,et al.  Measurement Theory for the Behavioral Sciences. , 1983 .

[10]  Margaret A. White,et al.  Performance of Acquisitions of Distressed Firms , 1994 .

[11]  G. V. Barrett,et al.  Towards a general model of non-random sampling and the impact on population correlation: generalizations of Berkson's Fallacy and restriction of range. , 1986, The British journal of mathematical and statistical psychology.

[12]  R. Landis,et al.  ITEM BIAS INDICES BASED ON TOTAL TEST SCORE AND JOB PERFORMANCE ESTIMATES OF ABILITY , 1993 .

[13]  Craig J. Russell,et al.  On Theory, Statistics, and the Search for Interactions in the Organizational Sciences , 1994 .

[14]  Neville T. Duarte,et al.  Effects of Dyadic Quality and Duration on Performance Appraisal , 1994 .

[15]  A. Bedeian,et al.  Simple Question, Not So Simple Answer: Interpreting Interaction Terms in Moderated Multiple Regression , 1994 .

[16]  Edward R. Kemery,et al.  Effects of predictor intercorrelations and reliabilities on moderated multiple regression , 1988 .

[17]  C. Judd,et al.  Statistical difficulties of detecting interactions and moderator effects. , 1993, Psychological bulletin.

[18]  Judith A. Hall,et al.  Testing for moderator variables in meta‐analysis: Issues and methods , 1991 .

[19]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[20]  Jeffrey K. Pinto,et al.  Appropriate Moderated Regression and Inappropriate Research Strategy: A Demonstration of Information Loss Due to Scale Coarseness , 1991 .

[21]  D. R. Saunders.  Moderator Variables in Prediction , 1956 .

[22]  George W. Bohrnstedt,et al.  The Reliability of Products of Two Random Variables , 1978 .

[23]  Amy L. Pablo.  Determinants of acquisition integration level: A decision-making perspective. , 1994 .

[24]  Richard L. Tate,et al.  Limitations of Centering for Interactive Models , 1984 .

[25]  Kenneth A. Bollen,et al.  Structural Equations with Latent Variables , 1989 .

[26]  Steffanie L. Wilk,et al.  Within-group norming and other forms of score adjustment in preemployment testing. , 1994, The American psychologist.

[27]  R. Alexander,et al.  A Generalization of James'S Second-Order Approximation to the Test for Regression Slope Equality , 1994 .

[28]  L. Humphreys,et al.  Assessing spurious "moderator effects": Illustrated substantively with the hypothesized ("synergistic") relation between spatial and mathematical ability. , 1990, Psychological bulletin.

[29]  Herman Aguinis,et al.  Type II Error Problems in the Use of Moderated Multiple Regression for the Detection of Moderating Effects of Dichotomous Variables , 1994 .

[30]  R. Friedrich.  In Defense of Multiplicative Terms In Multiple Regression Equations , 1982 .

[31]  S. Lyness.  Predictors of differences between Type A and B individuals in heart rate and blood pressure reactivity. , 1993, Psychological bulletin.

[32]  M. Evans.  A Monte Carlo study of the effects of correlated method variance in moderated multiple regression analysis , 1985 .

[33]  S. Maxwell,et al.  Bivariate median splits and spurious statistical significance. , 1993 .

[34]  W. Chaplin,et al.  The next generation of moderator research in personality psychology. , 1991, Journal of personality.

[35]  Larry E. Toothaker,et al.  Multiple Regression: Testing and Interpreting Interactions , 1991 .

[36]  E. F. Stone-Romero,et al.  Estimating the Power to Detect Dichotomous Moderators with Moderated Multiple Regression , 1994 .

[37]  Eugene F. Stone,et al.  Clarifying some controversial issues surrounding statistical procedures for detecting moderator variables: Empirical evidence and related matters. , 1989 .

[38]  Martin G. Evans,et al.  Testing multiplicative models does not require ratio scales , 1979 .

[39]  James Jaccard,et al.  Measurement error in the analysis of interaction effects between continuous predictors using multiple regression: Multiple indicator and structural equation approaches. , 1995 .

[40]  J. Levin,et al.  Testing for regression homogeneity under variance heterogeneity , 1982 .

[41]  Rosedith Sitgreaves Bowker.  Measurement Theory for the Behavioral Sciences , 1983 .

[42]  L. E. Jones,et al.  Analysis of multiplicative combination rules when the causal variables are measured with error. , 1983 .

[43]  Douglas R. Kahl,et al.  A Comparison of Moderated Regression Techniques Considering Strength of Effect , 1982 .

[44]  Charlotte H. Mason,et al.  Collinearity, power, and interpretation of multiple regression analysis. , 1991 .

[45]  F. Schmidt,et al.  Methodological, Statistical, and Ethical Issues in the Study of Bias in Psychological Tests , 1984 .

[46]  K. E. Southwood.  Substantive Theory and Statistical Interaction: Five Models , 1978, American Journal of Sociology.

[47]  S. Snell,et al.  Strategic Compensation for Integrated Manufacturing: The Moderating Effects of Jobs and Organizational Inertia , 1994 .

[48]  J. Jaccard,et al.  Individual Differences in Attitude‐Behavior Consistency: The Prediction of Contraceptive Behavior , 1990 .

[49]  James B. Stiff.  Cognitive processing of persuasive message cues: A meta‐analytic review of the effects of supporting information on attitudes , 1986 .

[50]  K. Grant,et al.  Taxonomy, assessment, and diagnosis of depression during adolescence. , 1993, Psychological bulletin.

[51]  Robert L. Linn,et al.  Range restriction problems in the use of self-selected groups for test validation. , 1968 .

[52]  L. Hsu.  Using Cohen's tables to determine the maximum power attainable in two-sample tests when one sample is limited in size. , 1993 .

[53]  Herman Aguinis,et al.  Improving The Estimation of Moderating Effects by Using Computer-Administered Questionnaires , 1996 .

[54]  C. Coulton,et al.  Interaction Effects in Multiple Regression , 1993 .

[55]  Jacob Cohen.  The Cost of Dichotomization , 1983 .

[56]  E. F. Stone-Romero,et al.  Relative power of moderated multiple regression and the comparison of subgroup correlation coefficients for detecting moderating effects. , 1994 .

[57]  D. A. Kenny,et al.  Estimating the nonlinear and interactive effects of latent variables. , 1984 .

[58]  P. Bobko,et al.  Moderated regression analysis and Likert scales: too coarse for comfort. , 1992, The Journal of applied psychology.

[59]  S. Zedeck.  Problems with the use of "moderator" variables. , 1971 .

[60]  Eugene F. Stone,et al.  Some issues associated with the use of moderated regression , 1984 .

[61]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[62]  J. Tedeschi,et al.  The Effect of Credibility on Perceived Power1 , 1993 .

[63]  Robert M. Guion,et al.  Personnel assessment, selection, and placement. , 1991 .

[64]  Kevin J. Williams,et al.  Role Stressors, Mood Spillover, and Perceptions of Work-Family Conflict in Employed Parents , 1994 .

[65]  N. Schmitt,et al.  PREDICTION OF TRADES APPRENTICES’PERFORMANCE ON JOB SAMPLE CRITERIA , 1990 .

[66]  L. H. Peters,et al.  Identifying Moderator Variables Using Multiple Regression: A Reply to Darrow and Kahl , 1984 .

[67]  T. Cleary.  TEST BIAS: PREDICTION OF GRADES OF NEGRO AND WHITE STUDENTS IN INTEGRATED COLLEGES , 1968 .

[68]  R. Sitgreaves.  Psychometric theory (2nd ed.). , 1979 .

[69]  J. D. Sherman,et al.  Failures to detect moderating effects with ordinary least squares-moderated multiple regression: Some reasons and a remedy. , 1986 .

[70]  Edward R. Kemery,et al.  Need for Clarity as a Moderator of the Role Ambiguity-Job Satisfaction Relationship , 1985 .

[71]  James A. Shepperd,et al.  Cautions in assessing spurious Moderator effects , 1991 .

[72]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[73]  M. Whisman.  Mediators and moderators of change in cognitive therapy of depression. , 1993, Psychological bulletin.