Alternative Methods for Assessing Mediation in Multilevel Data: The Advantages of Multilevel SEM

Multilevel modeling (MLM) is a popular way of assessing mediation effects with clustered data. Two important limitations of this approach have been identified in prior research and a theoretical rationale has been provided for why multilevel structural equation modeling (MSEM) should be preferred. However, to date, no empirical evidence of MSEM's advantages relative to MLM approaches for multilevel mediation analysis has been provided. Nor has it been demonstrated that MSEM performs adequately for mediation analysis in an absolute sense. This study addresses these gaps and finds that the MSEM method outperforms 2 MLM-based techniques in 2-level models in terms of bias and confidence interval coverage while displaying adequate efficiency, convergence rates, and power under a variety of conditions. Simulation results support prior theoretical work regarding the advantages of MSEM over MLM for mediation in clustered data.

[1]  Tom A. B. Snijders,et al.  Multilevel Analysis , 2011, International Encyclopedia of Statistical Science.

[2]  Kristopher J Preacher,et al.  A general multilevel SEM framework for assessing multilevel mediation. , 2010, Psychological methods.

[3]  Yongyun Shin,et al.  A Latent Cluster-Mean Approach to the Contextual Effects Model With Missing Data , 2010 .

[4]  Ulrich Trautwein,et al.  Doubly-Latent Models of School Contextual Effects: Integrating Multilevel and Structural Equation Approaches to Control Measurement and Sampling Error , 2009, Multivariate behavioral research.

[5]  Ehri Ryu,et al.  Level-Specific Evaluation of Model Fit in Multilevel Structural Equation Modeling , 2009 .

[6]  Daniel L. Murphy,et al.  Three-Level Models for Indirect Effects in School- and Class-Randomized Experiments in Education , 2009 .

[7]  Ann Yan Zhang,et al.  Explaining employment relationships with social exchange and job embeddedness. , 2009, The Journal of applied psychology.

[8]  Scott L. Thomas,et al.  An introduction to multilevel modeling techniques, 2nd ed. , 2009 .

[9]  B. Muthén,et al.  The multilevel latent covariate model: a new, more reliable approach to group-level effects in contextual studies. , 2008, Psychological methods.

[10]  Harvey Goldstein,et al.  Modelling measurement errors and category misclassifications in multilevel models , 2008 .

[11]  Kristopher J Preacher,et al.  Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models , 2008, Behavior research methods.

[12]  Laura M. Stapleton,et al.  The Performance of Methods to Test Upper-Level Mediation in the Presence of Nonnormal Data , 2008, Multivariate behavioral research.

[13]  D. Mackinnon.  Introduction to Statistical Mediation Analysis , 2008 .

[14]  U. Rudolph,et al.  School Context Variables and Students’ Smoking , 2008, European Addiction Research.

[15]  B. Muthén,et al.  Growth mixture modeling , 2008 .

[16]  Zhen Zhang,et al.  TESTING MULTILEVEL MEDIATION USING HIERARCHICAL LINEAR MODELS: PROBLEMS AND SOLUTIONS. , 2008 .

[17]  Andrew F. Hayes,et al.  The Sage sourcebook of advanced data analysis methods for communication research , 2008 .

[18]  T Asparouhov,et al.  Muthén, B., & Growth mixture analysis: Analysis with non-Gaussian random effects. , 2008 .

[19]  Geert Verbeke,et al.  Handbooks of Modern Statistical Methods Longitudinal Data Analysis , 2008 .

[20]  Andrew F. Hayes,et al.  Contemporary approaches to assessing mediation in communication research. , 2008 .

[21]  Ehri Ryu,et al.  Evaluation of model fit in multilevel structural equation modeling: Level -specific model fit evaluation and the robustness to non -normality , 2008 .

[22]  Guy Roth,et al.  Autonomous motivation for teaching: How self-determined teaching may lead to self-determined learning. , 2007 .

[23]  Ke-Hai Yuan,et al.  3. Multilevel Covariance Structure Analysis by Fitting Multiple Single-Level Models , 2007 .

[24]  Victor L. Willson,et al.  Comparing Empirical Power of Multilevel Structural Equation Models and Hierarchical Linear Models: Understanding Cross-Level Interactions , 2006 .

[25]  Laura M. Stapleton,et al.  A Comparison of Single Sample and Bootstrap Methods to Assess Mediation in Cluster Randomized Trials , 2006, Multivariate behavioral research.

[26]  Daniel J. Bauer,et al.  Conceptualizing and testing random indirect effects and moderated mediation in multilevel models: new procedures and recommendations. , 2006, Psychological methods.

[27]  Donald Hedeker,et al.  Longitudinal Data Analysis , 2006 .

[28]  Laura M. Stapleton,et al.  A Comparison of Methods to Test for Mediation in Multisite Experiments , 2005, Multivariate behavioral research.

[29]  J. Gill.  Hierarchical Linear Models , 2005 .

[30]  J. Hox,et al.  Sufficient Sample Sizes for Multilevel Modeling , 2005 .

[31]  Kristopher J Preacher,et al.  SPSS and SAS procedures for estimating indirect effects in simple mediation models , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[32]  S. Rabe-Hesketh,et al.  Generalized multilevel structural equation modeling , 2004 .

[33]  David P Mackinnon,et al.  Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods , 2004, Multivariate behavioral research.

[34]  Ke-Hai Yuan,et al.  Eight test statistics for multilevel structural equation models , 2003, Comput. Stat. Data Anal..

[35]  D. A. Kenny,et al.  Lower Level Mediation in Multilevel Models , 2022 .

[36]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[37]  S. West,et al.  A comparison of methods to test mediation and other intervening variable effects. , 2002, Psychological methods.

[38]  G. Molenberghs,et al.  Longitudinal data analysis , 2008 .

[39]  M. Julian.  The Consequences of Ignoring Multilevel Data Structures in Nonhierarchical Covariance Modeling , 2001 .

[40]  D. Mackinnon,et al.  Multilevel Modeling of Individual and Group Level Mediated Effects , 2001, Multivariate behavioral research.

[41]  Cora J. M. Maas,et al.  The Accuracy of Multilevel Structural Equation Modeling With Pseudobalanced Groups and Small Samples , 2001 .

[42]  C. Perry,et al.  How did Project Northland reduce alcohol use among young adolescents? Analysis of mediating variables. , 2001, Health education research.

[43]  D. Culler,et al.  Comparison of methods , 2000 .

[44]  S. Raudenbush,et al.  Assessing Direct and Indirect Effects in Multilevel Designs with Latent Variables , 1999 .

[45]  Ronald H. Heck,et al.  An Introduction to Multilevel Modeling Techniques , 1999 .

[46]  Hettie A. Richardson,et al.  The Impact of High Involvement Work Processes on Organizational Effectiveness , 1999 .

[47]  D P MacKinnon,et al.  Multilevel Mediation Modeling in Group-Based Intervention Studies , 1999, Evaluation review.

[48]  Jan de Leeuw,et al.  Introducing Multilevel Modeling , 1998 .

[49]  D. Mackinnon,et al.  A Simulation Study of Mediated Effect Measures. , 1995, Multivariate behavioral research.

[50]  A. Satorra,et al.  Complex Sample Data in Structural Equation Modeling , 1995 .

[51]  B. Muthén,et al.  Multilevel Covariance Structure Analysis , 1994 .

[52]  Stephen W. Raudenbush,et al.  A Multilevel, Multivariate Model for Studying School Climate With Estimation Via the EM Algorithm and Application to U.S. High-School Data , 1991 .

[53]  Bengt Muthén,et al.  Multilevel Factor Analysis of Class and Student Achievement Components , 1991 .

[54]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[55]  J. A. Hartigan,et al.  [Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy]: Comment , 1986 .

[56]  Robert Tibshirani,et al.  Bootstrap Methods for Standard Errors, Confidence Intervals, and Other Measures of Statistical Accuracy , 1986 .

[57]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[58]  B. Muthén.  A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators , 1984 .

[59]  M. Sobel.  Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models , 1982 .