The Relative Trustworthiness of Inferential Tests of the Indirect Effect in Statistical Mediation Analysis

A content analysis of 2 years of Psychological Science articles reveals inconsistencies in how researchers make inferences about indirect effects when conducting a statistical mediation analysis. In this study, we examined the frequency with which popularly used tests disagree, whether the method an investigator uses makes a difference in the conclusion he or she will reach, and whether there is a most trustworthy test that can be recommended to balance practical and performance considerations. We found that tests agree much more frequently than they disagree, but disagreements are more common when an indirect effect exists than when it does not. We recommend the bias-corrected bootstrap confidence interval as the most trustworthy test if power is of utmost concern, although it can be slightly liberal in some circumstances. Investigators concerned about Type I errors should choose the Monte Carlo confidence interval or the distribution-of-the-product approach, which rarely disagree. The percentile bootstrap confidence interval is a good compromise test.

[1]  Nancy Eisenberg,et al.  Prosociality: the contribution of traits, values, and self-efficacy beliefs. , 2012, Journal of personality and social psychology.

[2]  David P Mackinnon,et al.  Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods , 2004, Multivariate behavioral research.

[3]  H. Rindermann,et al.  Cognitive Capitalism: The Effect of Cognitive Ability on Wealth, as Mediated Through Scientific Achievement and Economic Freedom , 2011 .

[4]  M. Sobel.  Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models , 1982 .

[5]  Ken Kelley,et al.  Methods for the Behavioral, Educational, and Social Sciences: An R package , 2007, Behavior research methods.

[6]  M. Zanna,et al.  Establishing a causal chain: why experiments are often more effective than mediational analyses in examining psychological processes. , 2005, Journal of personality and social psychology.

[7]  K. Fujita,et al.  Change We Can Believe In , 2012, Psychological science.

[8]  Kristopher J Preacher,et al.  SPSS and SAS procedures for estimating indirect effects in simple mediation models , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[9]  Kristopher J Preacher,et al.  Addressing Moderated Mediation Hypotheses: Theory, Methods, and Prescriptions , 2007, Multivariate behavioral research.

[10]  David P Mackinnon,et al.  Explanation of Two Anomalous Results in Statistical Mediation Analysis , 2012, Multivariate behavioral research.

[11]  Matthew S. Fritz,et al.  PSYCHOLOGICAL SCIENCE Research Article Required Sample Size to Detect the Mediated Effect , 2022 .

[12]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[13]  P. Shrout,et al.  Mediation in experimental and nonexperimental studies: new procedures and recommendations. , 2002, Psychological methods.

[14]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[15]  Leo A. Aroian,et al.  The probability function of the product of two normally distributed variables. , 1947 .

[16]  S. Kitayama,et al.  Spontaneous Trait Inference Is Culture-Specific , 2011, Psychological science.

[17]  Kate Sweeny,et al.  On Near Misses and Completed Tasks , 2012, Psychological science.

[18]  S. West,et al.  A comparison of methods to test mediation and other intervening variable effects. , 2002, Psychological methods.

[19]  Shigehiro Oishi,et al.  Progressive Taxation and the Subjective Well-Being of Nations , 2012, Psychological science.

[20]  D. A. Kenny,et al.  Data Analysis in Social Psychology: Recent and Recurring Issues , 2010 .

[21]  Lukas Neville.  Do Economic Equality and Generalized Trust Inhibit Academic Dishonesty? Evidence From State-Level Search-Engine Queries , 2012, Psychological science.

[22]  John E. Mathieu,et al.  Clarifying conditions and decision points for mediational type inferences in Organizational Behavior , 2006 .

[23]  Victoria Savalei,et al.  Assessing Mediational Models: Testing and Interval Estimation for Indirect Effects , 2010, Multivariate behavioral research.

[24]  A. Hayes.  Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach , 2013 .

[25]  Kristopher J Preacher,et al.  Advantages of Monte Carlo Confidence Intervals for Indirect Effects , 2012 .

[26]  Kristopher J Preacher,et al.  Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models , 2008, Behavior research methods.

[27]  “He Loves Me, He Loves Me Not . . . ” , 2011, Psychological science.

[28]  J. S. Long,et al.  Using Heteroscedasticity Consistent Standard Errors in the Linear Regression Model , 2000 .

[29]  D. Mackinnon.  Introduction to Statistical Mediation Analysis , 2008 .

[30]  A. Hayes,et al.  Using heteroskedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation , 2007, Behavior research methods.

[31]  Christine M. Wickens,et al.  Understanding driver anger and aggression: attributional theory in the driving environment. , 2011, Journal of experimental psychology. Applied.

[32]  David P Mackinnon,et al.  Resampling and Distribution of the Product Methods for Testing Indirect Effects in Complex Models , 2008, Structural equation modeling : a multidisciplinary journal.

[33]  J. Edwards,et al.  Methods for integrating moderation and mediation: a general analytical framework using moderated path analysis. , 2007, Psychological methods.

[34]  David P MacKinnon,et al.  RMediation: An R package for mediation analysis confidence intervals , 2011, Behavior research methods.

[35]  Matthew S. Fritz,et al.  Distribution of the product confidence limits for the indirect effect: Program PRODCLIN , 2007, Behavior research methods.