Testing moderator hypotheses in meta-analytic structural equation modeling using subgroup analysis

Meta-analytic structural equation modeling (MASEM) is a statistical technique to pool correlation matrices and test structural equation models on the pooled correlation matrix. In Stage 1 of MASEM, correlation matrices from independent studies are combined to obtain a pooled correlation matrix, using fixed- or random-effects analysis. In Stage 2, a structural model is fitted to the pooled correlation matrix. Researchers applying MASEM may have hypotheses about how certain model parameters will differ across subgroups of studies. These moderator hypotheses are often addressed using suboptimal methods. The aim of the current article is to provide guidance and examples on how to test hypotheses about group differences in specific model parameters in MASEM. We illustrate the procedure using both fixed- and random-effects subgroup analysis with two real datasets. In addition, we present a small simulation study to evaluate the effect of the number of studies per subgroup on convergence problems. All data and the R-scripts for the examples are provided online.

[1]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[2]  R. MacCallum,et al.  Power analysis and determination of sample size for covariance structure modeling. , 1996 .

[3]  Mike W-L Cheung,et al.  Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R , 2013, Behavior Research Methods.

[4]  Nina Rosenbusch,et al.  The Mediating Role of Entrepreneurial Orientation in the Task Environment–Performance Relationship , 2011 .

[5]  Suniya S. Luthar,et al.  Social-Emotional Factors Affecting Achievement Outcomes Among Disadvantaged Students: Closing the Achievement Gap , 2002, Educational psychologist.

[6]  Paul Symonds,et al.  Diagnostic validity of the Hospital Anxiety and Depression Scale (HADS) in cancer and palliative settings: a meta-analysis. , 2010, Journal of affective disorders.

[7]  P. Heugens,et al.  Synthesizing and Extending Resource Dependence Theory , 2012 .

[8]  Mike W. L. Cheung,et al.  Comparison of methods for constructing confidence intervals of standardized indirect effects , 2009, Behavior research methods.

[9]  Suzanne Jak,et al.  Meta-analytic structural equation modelling with missing correlations , 2013 .

[10]  C. Kidd.  Diagnostic Validity. , 1960, The Journal of the College of General Practitioners.

[11]  David G. Allen,et al.  MECHANISMS LINKING REALISTIC JOB PREVIEWS WITH TURNOVER: A META‐ANALYTIC PATH ANALYSIS , 2011 .

[12]  M. Cheung.  Meta-Analytic Structural Equation Modeling , 2015, Oxford Research Encyclopedia of Business and Management.

[13]  When constraints interact: a caution about reference variables, identification constraints, and scale dependencies in structural equation modeling. , 2002 .

[14]  M. Browne.  Asymptotically distribution-free methods for the analysis of covariance structures. , 1984, The British journal of mathematical and statistical psychology.

[15]  A. Boomsma,et al.  Robustness Studies in Covariance Structure Modeling , 1998 .

[16]  John E. Hunter,et al.  The Advantages of Using Standardized Scores in Causal Analysis. , 2002 .

[17]  Jason Bennett Thatcher,et al.  Can we have fun @ work? The role of intrinsic motivation for utilitarian systems , 2013, Eur. J. Inf. Syst..

[18]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[19]  B. Hamre,et al.  Early teacher-child relationships and the trajectory of children's school outcomes through eighth grade. , 2001, Child development.

[20]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[21]  Kaifeng Jiang,et al.  When and how is job embeddedness predictive of turnover? a meta-analytic investigation. , 2012, The Journal of applied psychology.

[22]  John Done,et al.  The Hospital Anxiety and Depression Scale: a meta confirmatory factor analysis , 2012 .

[23]  Ying Zhang,et al.  Meta-Analytic Structural Equation Modeling (MASEM): Comparison of the Multivariate Methods , 2011 .

[24]  Suzanne Jak,et al.  Meta-Analytic Structural Equation Modelling , 2015 .

[25]  Rüdiger Kabst,et al.  Gender effects on entrepreneurial intention: a meta‐analytical structural equation model , 2013 .

[26]  R. Snaith,et al.  The Hospital Anxiety and Depression Scale , 1983 .

[27]  Mike W.-L. Cheung,et al.  A Two-Stage Approach to Synthesizing Covariance Matrices in Meta-Analytic Structural Equation Modeling , 2009 .

[28]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[29]  F. Oort,et al.  The Influence of Affective Teacher–Student Relationships on Students’ School Engagement and Achievement , 2011 .

[30]  John E. Hunter,et al.  Methods of Meta-Analysis: Correcting Error and Bias in Research Findings , 1991 .

[31]  P. F. Jong,et al.  Underlying skills of oral and silent reading , 2014 .

[32]  Mike W.-L. Cheung,et al.  Meta-Analysis: A Structural Equation Modeling Approach , 2015 .

[33]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[34]  A. Hafdahl,et al.  Combining Heterogeneous Correlation Matrices: Simulation Analysis of Fixed-Effects Methods , 2008 .

[35]  Mike W.-L. Cheung,et al.  metaSEM: an R package for meta-analysis using structural equation modeling , 2015, Front. Psychol..

[36]  Victoria Savalei,et al.  Analysis of Correlation Structures: Current Status and Open Problems , 2010 .

[37]  K. Schermelleh-Engel,et al.  Evaluating the Fit of Structural Equation Models: Tests of Significance and Descriptive Goodness-of-Fit Measures. , 2003 .

[38]  Betsy Jane Becker,et al.  Corrections to “Using Results From Replicated Studies to Estimate Linear Models” , 1995 .

[39]  L. Hedges,et al.  The Handbook of Research Synthesis and Meta-Analysis , 2009 .

[40]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[41]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[42]  Betsy Jane Becker,et al.  Using Results From Replicated Studies to Estimate Linear Models , 1992 .

[43]  Mike W-L Cheung,et al.  Random‐effects models for meta‐analytic structural equation modeling: review, issues, and illustrations , 2016, Research synthesis methods.

[44]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[45]  Wai Chan,et al.  Comparing standardized coefficients in structural equation modeling: a model reparameterization approach , 2011, Behavior research methods.

[46]  Mike W-L Cheung,et al.  Meta-analytic structural equation modeling: a two-stage approach. , 2005, Psychological methods.

[47]  Peter M Bentler,et al.  Can scientifically useful hypotheses be tested with correlations? , 2007, The American psychologist.

[48]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[49]  Timothy R. Brick,et al.  OpenMx User Guide Release 2.0.0-4004 , 2014 .

[50]  D. Ones,et al.  Theory testing: Combining psychometric meta-analysis and structural equations modeling , 1995 .

[51]  Mike W.-L. Cheung,et al.  Educational and Psychological Measurement Classifying Correlation Matrices into Relatively Homogeneous Subgroups: a Cluster Analytic Approach , 2022 .