The multivariate multiple-membership random-effect model: An introduction and evaluation

In the present study, we focused on models that handle several data structure complexities simultaneously. We introduced and evaluated the multivariate multiple-membership random-effect model (MV-MMREM) for handling multiple-membership data in scenarios with multiple, related outcomes. Although a recent study introduced the idea of the MV-MMREM, no research has directly assessed its estimation nor demonstrated its use with real data. Therefore, we used real multiple-membership datasets that included multiple, related outcomes to demonstrate interpretation of the MV-MMREM parameters. In addition, a simulation study was conducted to assess estimation of the MV-MMREM under a number of design conditions. Also, the robustness of the results was assessed for multivariate multiple-membership data when they were analyzed using a multivariate hierarchical linear model that ignores the multiple-membership structure (MV-HLM), as well as when using multiple univariate MMREMs. The results showed that the MV-MMREM works well in comparison with both MV-HLM and univariate MMREMs when the data structure had missingness outcomes, multivariate outcomes, and multiple membership clusters. Finally, we discuss limitations of the MV-MMREM and areas for future research.

[1]  Craig K. Enders,et al.  The Performance of the Full Information Maximum Likelihood Estimator in Multiple Regression Models with Missing Data , 2001 .

[2]  Cora J. M. Maas,et al.  The Accuracy of Multilevel Structural Equation Modeling With Pseudobalanced Groups and Small Samples , 2001 .

[3]  S. N. Beretvas,et al.  A Comparison of Techniques for Handling and Assessing the Influence of Mobility on Student Achievement , 2017 .

[4]  David G Steel,et al.  Ignoring a Level in a Multilevel Model: Evidence from UK Census Data , 2001 .

[5]  K. Pituch,et al.  The Performance of Multilevel Models When Outcome Data Are Incomplete , 2019 .

[6]  S. Natasha Beretvas,et al.  Uncovering Processes and Pathways in Family–School Research: Modeling Innovations for Handling Data Complexities , 2015 .

[7]  A. Boomsma,et al.  Robustness Studies in Covariance Structure Modeling , 1998 .

[8]  Patrick Onghena,et al.  The Effects of Ignoring a Level in Multilevel Analysis , 2005 .

[9]  B. G. Dodd,et al.  Comparing the Performance of Multivariate Multilevel Modeling to Traditional Analyses With Complete and Incomplete Data , 2015 .

[10]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[11]  B. Tabachnick,et al.  Using Multivariate Statistics , 1983 .

[12]  S. Natasha Beretvas,et al.  The Impact of Using Incorrect Weights With the Multiple Membership Random Effects Model , 2014 .

[13]  George Leckie,et al.  R2MLwiN: A Package to Run MLwiN from within R , 2016 .

[14]  Wim Van Den Noortgate,et al.  Design efficiency for imbalanced multilevel data , 2009, Behavior research methods.

[15]  William J Browne,et al.  MCMC Estimation in MLwiN (Version 2.13) Centre for Multilevel Modelling, University of Bristol , 2009 .

[16]  William J. Browne,et al.  Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models , 2000, Comput. Stat..

[17]  Tiffany A. Whittaker,et al.  The Comparison of Model Selection Criteria When Selecting Among Competing Hierarchical Linear Models , 2009 .

[18]  S. Konstantopoulos.  Power Analysis in Two-Level Unbalanced Designs , 2010 .

[19]  Philippa Clarke,et al.  Addressing Data Sparseness in Contextual Population Research , 2007 .

[20]  L. Hedges,et al.  Intraclass Correlation Values for Planning Group-Randomized Trials in Education , 2007 .

[21]  J. Hox.  Multilevel analysis: Techniques and applications, 2nd ed. , 2010 .

[22]  James Algina,et al.  The impact of omitting the interaction between crossed factors in cross-classified random effects modelling. , 2010, The British journal of mathematical and statistical psychology.

[23]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[24]  H. Goldstein.  Multilevel Statistical Models , 2006 .

[25]  Mirjam Moerbeek,et al.  The Consequence of Ignoring a Level of Nesting in Multilevel Analysis , 2004, Multivariate behavioral research.

[26]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[27]  Charles A. Scherbaum,et al.  Estimating Statistical Power and Required Sample Sizes for Organizational Research Using Multilevel Modeling , 2009 .

[28]  J. Mathieu,et al.  Understanding and estimating the power to detect cross-level interaction effects in multilevel modeling. , 2012, The Journal of applied psychology.

[29]  Jan Van Damme,et al.  The Consequence of Ignoring a Level of Nesting in Multilevel Analysis: A Comment , 2005 .

[30]  Analyzing multiple outcomes in clinical research using multivariate multilevel models. , 2014, Journal of consulting and clinical psychology.

[31]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[32]  Donald A. Rock,et al.  Early Childhood Longitudinal Study, Kindergarten Class of 1998?99 (ECLS-K). Psychometric Report for the Fifth Grade. NCES 2006?036. , 2005 .

[33]  George Leckie,et al.  The complexity of school and neighbourhood effects and movements of pupils on school differences in models of educational achievement , 2009 .

[34]  S. Raudenbush,et al.  Statistical power and optimal design for multisite randomized trials. , 2000, Psychological methods.

[35]  Tiffany A. Whittaker,et al.  Multivariate Models for Normal and Binary Responses in Intervention Studies , 2016 .

[36]  Naihua Duan,et al.  Multilevel Modeling : Methodological Advances, Issues, and Applications , 2003 .

[37]  Paul Clarke,et al.  Who you live with and where you live: setting the context for health using multiple membership multilevel models , 2005, Journal of Epidemiology and Community Health.

[38]  Charles B. Fleming,et al.  Longitudinal effects of student mobility on three dimensions of elementary school engagement. , 2008, Child development.

[39]  Harvey Goldstein,et al.  Modelling the effect of pupil mobility on school differences in educational achievement , 2007 .

[40]  David Kerbow,et al.  Patterns of Urban Student Mobility and Local School Reform , 1996 .

[41]  Oi-Man Kwok,et al.  Please Scroll down for Article Multivariate Behavioral Research the Impacts of Ignoring a Crossed Factor in Analyzing Cross-classified Data , 2022 .

[42]  Harvey Goldstein,et al.  Multilevel Statistical Models: Goldstein/Multilevel Statistical Models , 2010 .

[43]  S Natasha Beretvas,et al.  The Impact of Inappropriate Modeling of Cross-Classified Data Structures , 2006, Multivariate behavioral research.

[44]  J. Hox,et al.  Sufficient Sample Sizes for Multilevel Modeling , 2005 .

[45]  Daniel L. Murphy,et al.  An Evaluation of Information Criteria Use for Correct Cross-Classified Random Effects Model Selection , 2013 .

[46]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[47]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[48]  Hyewon Chung,et al.  The impact of ignoring multiple membership data structures in multilevel models. , 2009, The British journal of mathematical and statistical psychology.

[49]  Cora J. M. Maas,et al.  Robustness issues in multilevel regression analysis , 2004 .