Fixed-Effects Meta-Analyses as Multiple-Group Structural Equation Models

Meta-analysis is the statistical analysis of a collection of analysis results from individual studies, conducted for the purpose of integrating the findings. Structural equation modeling (SEM), on the other hand, is a multivariate technique for testing hypothetical models with latent and observed variables. This article shows that fixed-effects meta-analyses with the following characteristics can be modeled in the SEM framework: (a) using any type of effect size; (b) including categorical and continuous moderators; and (c) including multivariate effect sizes. Empirical examples in LISREL syntax are used to demonstrate the equivalence between the meta-analytic and SEM approaches. Future directions for and extensions to this approach are discussed.

[1]  P. Allison.  Estimation of Linear Models with Incomplete Data , 1987 .

[2]  Jan de Leeuw,et al.  On the relationship between item response theory and factor analysis of discretized variables , 1987 .

[3]  Kit-Tai Hau,et al.  Goodness of fit in structural equation models , 2005 .

[4]  G. Becker.  The meta-analysis of factor analyses: An illustration based on the cumulation of correlation matrices. , 1996 .

[5]  L. Hedges.  Fitting Categorical Models to Effect Sizes from a Series of Experiments , 1982 .

[6]  William R. Shadish,et al.  Meta-analysis and the exploration of causal mediating processes: A primer of examples, methods, and issues. , 1996 .

[7]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[8]  M. Cheung.  Constructing Approximate Confidence Intervals for Parameters With Structural Equation Models , 2009 .

[9]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[10]  M J Rovine,et al.  A Structural Modeling Approach to a Multilevel Random Coefficients Model , 2000, Multivariate behavioral research.

[11]  Peter M Bentler,et al.  Can scientifically useful hypotheses be tested with correlations? , 2007, The American psychologist.

[12]  Mike W.-L. Cheung,et al.  Comparison of Approaches to Constructing Confidence Intervals for Mediating Effects Using Structural Equation Models , 2007 .

[13]  Steven D. Brown,et al.  Handbook of applied multivariate statistics and mathematical modeling , 2000 .

[14]  Jürgen Baumert,et al.  Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples. , 2000 .

[15]  R. Rosenthal.  Parametric measures of effect size. , 1994 .

[16]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[17]  D. Sörbom.  A GENERAL METHOD FOR STUDYING DIFFERENCES IN FACTOR MEANS AND FACTOR STRUCTURE BETWEEN GROUPS , 1974 .

[18]  Joseph L. Fleiss,et al.  Measures of effect size for categorical data. , 1994 .

[19]  D. Kaplan.  Evaluating and Modifying Covariance Structure Models: A Review and Recommendation. , 1990, Multivariate behavioral research.

[20]  Craig K. Enders,et al.  A Primer on Maximum Likelihood Algorithms Available for Use With Missing Data , 2001 .

[21]  S Natasha Beretvas,et al.  Meta-analytic methods of pooling correlation matrices for structural equation modeling under different patterns of missing data. , 2005, Psychological methods.

[22]  Herman Aguinis,et al.  Comparison of Three Meta-Analytic Procedures for Estimating Moderating Effects of Categorical Variables , 2008 .

[23]  D. Ones,et al.  Theory testing: Combining psychometric meta-analysis and structural equations modeling , 1995 .

[24]  Daniel J Bauer,et al.  Building path diagrams for multilevel models. , 2007, Psychological methods.

[25]  K. Jöreskog,et al.  LISREL-PC and PRELIS [Computer Software]@@@EQS/PC [Computer Software] , 1987 .

[26]  Patrick J Curran,et al.  Have Multilevel Models Been Structural Equation Models All Along? , 2003, Multivariate behavioral research.

[27]  Karl G. Jöreskog,et al.  Lisrel 8: User's Reference Guide , 1997 .

[28]  Betsy Jane Becker,et al.  Synthesizing standardized mean‐change measures , 1988 .

[29]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[30]  Stephen W. Raudenbush,et al.  Random effects models. , 1994 .

[31]  M. S. Patel,et al.  An introduction to meta-analysis. , 1989, Health Policy.

[32]  Mike W.-L. Cheung,et al.  A Two-Stage Approach to Synthesizing Covariance Matrices in Meta-Analytic Structural Equation Modeling , 2009 .

[33]  Mike W.-L. Cheung,et al.  Educational and Psychological Measurement Classifying Correlation Matrices into Relatively Homogeneous Subgroups: a Cluster Analytic Approach , 2022 .

[34]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[35]  John B. Willett,et al.  Using covariance structure analysis to detect correlates and predictors of individual change over time , 1994 .

[36]  Zvi Drezner,et al.  Model specification searches in structural equation modeling using tabu search , 1998 .

[37]  M. Browne.  Asymptotically distribution-free methods for the analysis of covariance structures. , 1984, The British journal of mathematical and statistical psychology.

[38]  Betsy Jane Becker,et al.  Corrections to “Using Results From Replicated Studies to Estimate Linear Models” , 1995 .

[39]  R. MacCallum,et al.  Applications of structural equation modeling in psychological research. , 2000, Annual review of psychology.

[40]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[41]  Daniel J. Bauer.  Estimating Multilevel Linear Models as Structural Equation Models , 2003 .

[42]  William R. Shadish,et al.  Using odds ratios as effect sizes for meta-analysis of dichotomous data: A primer on methods and issues. , 1998 .

[43]  Sophia Rabe-Hesketh,et al.  Generalized latent variable models: multilevel, longitudinal, and structural equation models , 2004 .

[44]  W. Spitzer.  The challenge of meta-analysis. , 1995, Journal of clinical epidemiology.

[45]  Betsy Jane Becker,et al.  Modeling multivariate effect sizes. , 1988 .

[46]  G. A. Marcoulides,et al.  Multilevel Analysis Techniques and Applications , 2002 .

[47]  Betsy Jane Becker,et al.  Multivariate meta-analysis. , 2000 .

[48]  L. Hedges,et al.  The power of statistical tests in meta-analysis. , 2001, Psychological methods.

[49]  P. Bentler,et al.  Fit indices in covariance structure modeling : Sensitivity to underparameterized model misspecification , 1998 .

[50]  L. Hedges.  Distribution Theory for Glass's Estimator of Effect size and Related Estimators , 1981 .

[51]  Jacob Cohen.  Multiple regression as a general data-analytic system. , 1968 .

[52]  Tapabrata Maiti,et al.  Principles and Practice of Structural Equation Modeling (2nd ed.) , 2006 .

[53]  Ingram Olkin,et al.  Stochastically dependent effect sizes. , 1994 .

[54]  R. Hanka.  The Handbook of Research Synthesis , 1994 .

[55]  Mike W-L Cheung,et al.  Meta-analytic structural equation modeling: a two-stage approach. , 2005, Psychological methods.

[56]  Robert C. MacCallum,et al.  SPECIFICATION SEARCHES IN COVARIANCE STRUCTURE MODELING , 1986 .

[57]  J. H. Steiger.  Statistically based tests for the number of common factors , 1980 .

[58]  Ingram Olkin,et al.  Clustering estimates of effect magnitude from independent studies , 1983 .

[59]  Wolfgang Viechtbauer,et al.  Publication bias in meta-analysis: Prevention, assessment and adjustments , 2007, Psychometrika.

[60]  S. Raudenbush,et al.  A multivariate mixed linear model for meta-analysis. , 1996 .

[61]  Kerrie Mengersen,et al.  Multivariate meta‐analysis , 2003, Statistics in medicine.

[62]  Peter M. Bentler,et al.  EQS : structural equations program manual , 1989 .

[63]  G. Barrie Wetherill,et al.  Random Effects Models , 1981 .

[64]  K. Jöreskog,et al.  LISREL 8: New Statistical Features , 1999 .

[65]  S. Thompson,et al.  How should meta‐regression analyses be undertaken and interpreted? , 2002, Statistics in medicine.

[66]  Paul D. Bliese,et al.  Book Review: Modeling Longitudinal and Multilevel Data: Practical Issues, Applied Approaches, and Specific Examples, edited by T. D. Little, K. U. Schnabel, and J. Baumert (2000). Mahwah, Nj: Lawrence Erlbaum. , 2003 .

[67]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[68]  P. Bentler,et al.  Covariance structure analysis: statistical practice, theory, and directions. , 1996, Annual review of psychology.

[69]  F. Schmidt.  Meta-Analysis , 2008 .

[70]  Rick H. Hoyle,et al.  Confirmatory Factor Analysis , 1983 .

[71]  Wolfgang Viechtbauer,et al.  Hypothesis tests for population heterogeneity in meta-analysis. , 2007, The British journal of mathematical and statistical psychology.

[72]  S. West,et al.  The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. , 1996 .

[73]  Bengt Muthén,et al.  On structural equation modeling with data that are not missing completely at random , 1987 .

[74]  G A Colditz,et al.  Understanding research synthesis (meta-analysis). , 1996, Annual review of public health.

[75]  Larry V. Hedges,et al.  A random effects model for effect sizes , 1983 .

[76]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[77]  Larry V. Hedges,et al.  Fitting Continuous Models to Effect Size Data , 1982 .

[78]  David Rindskopf,et al.  Parameterizing inequality constraints on unique variances in linear structural models , 1983 .

[79]  James L. Arbuckle,et al.  Full Information Estimation in the Presence of Incomplete Data , 1996 .

[80]  Betsy Jane Becker,et al.  Using Results From Replicated Studies to Estimate Linear Models , 1992 .

[81]  R. Rosenthal.  Meta-analytic procedures for social research , 1984 .

[82]  M. Petticrew,et al.  Systematic Reviews in the Social Sciences: A Practical Guide , 2005 .

[83]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[84]  Xiao-Hua Zhou,et al.  Statistical Methods for Meta‐Analysis , 2008 .

[85]  R. Kline.  Principles and practice of structural equation modeling, 2nd ed. , 2005 .

[86]  R. MacIntosh,et al.  Variance Estimation for Converting MIMIC Model Parameters to IRT Parameters in DIF Analysis , 2003 .

[87]  Michael C Neale,et al.  People are variables too: multilevel structural equations modeling. , 2005, Psychological methods.

[88]  M. Browne,et al.  Single Sample Cross-Validation Indices for Covariance Structures. , 1989, Multivariate behavioral research.

[89]  R. DeShon,et al.  Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. , 2002 .

[90]  H. Marsh,et al.  In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler's (1999) Findings , 2004 .

[91]  D. P. Gaver,et al.  Combining Information: Statistical Issues and Opportunities for Research , 1993 .

[92]  T Raykov,et al.  Estimation of congeneric scale reliability using covariance structure analysis with nonlinear constraints. , 2001, The British journal of mathematical and statistical psychology.

[93]  Alex J. Sutton,et al.  Methods for Meta-Analysis in Medical Research , 2000 .

[94]  Carolyn F. Furlow,et al.  Evaluation of an Approximate Method for Synthesizing Covariance Matrices for Use in Meta-Analytic SEM , 2006 .

[95]  Betsy Jane Becker,et al.  Examining explanatory models through research synthesis. , 1994 .

[96]  T. Raykov.  Estimation of Composite Reliability for Congeneric Measures , 1997 .

[97]  Rachel T. Fouladi,et al.  Performance of Modified Test Statistics in Covariance and Correlation Structure Analysis Under Conditions of Multivariate Nonnormality , 2000 .

[98]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[99]  M. Cheung.  A model for integrating fixed-, random-, and mixed-effects meta-analyses into structural equation modeling. , 2008, Psychological methods.

[100]  Betsy Jane Becker,et al.  Special Issue on Meta-Analysis , 1992 .

[101]  K. Bollen.  Latent variables in psychology and the social sciences. , 2002, Annual review of psychology.

[102]  Robert C. Gardner,et al.  On the growth of structural equation modeling in psychological journals , 1996 .

[103]  Scott L. Hershberger,et al.  The Growth of Structural Equation Modeling: 1994-2001 , 2003 .

[104]  D. Rindskopf.  Using phantom and imaginary latent variables to parameterize constraints in linear structural models , 1984 .

[105]  P. Bentler,et al.  Multiple population covariance structure analysis under arbitrary distribution theory , 1987 .

[106]  C. Stein,et al.  Structural equation modeling. , 2012, Methods in molecular biology.

[107]  Zvi Drezner,et al.  Model Specification Searches Using Ant Colony Optimization Algorithms , 2003 .

[108]  Multivariate meta-analysis for exploratory factor analytic research. , 2002 .