Cross-Level Invariance in Multilevel Factor Models

When modeling latent variables at multiple levels, it is important to consider the meaning of the latent variables at the different levels. If a higher-level common factor represents the aggregated version of a lower-level factor, the associated factor loadings will be equal across levels. However, many researchers do not consider cross-level invariance constraints in their research. Not applying these constraints when in fact they are appropriate leads to overparameterized models, and associated convergence and estimation problems. This simulation study used a two-level mediation model on common factors to show that when factor loadings are equal in the population, not applying cross-level invariance constraints leads to more estimation problems and smaller true positive rates. Some directions for future research on cross-level invariance in MLSEM are discussed.

[1]  E. S. Kim,et al.  Within-Level Group Factorial Invariance With Multilevel Data: Multilevel Factor Mixture and Multilevel MIMIC Models , 2015 .

[2]  Suzanne Jak,et al.  Using Two-Level Factor Analysis to Test for Cluster Bias in Ordinal Data , 2014, Multivariate behavioral research.

[3]  Sun-Joo Cho,et al.  Detecting Differential Item Discrimination (DID) and the Consequences of Ignoring DID in Multilevel Item Response Models. , 2017 .

[4]  William N. Venables,et al.  Modern Applied Statistics with S , 2010 .

[5]  Gregory R. Hancock,et al.  Construct Meaning in Multilevel Settings , 2016 .

[6]  K. Schmidt.  Conceptual Framework for , 2002 .

[7]  Sarah Depaoli,et al.  A Bayesian Approach to Multilevel Structural Equation Modeling With Continuous and Dichotomous Outcomes , 2015 .

[8]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[9]  Bengt Muthén,et al.  Mean and Covariance Structure Analysis of Hierarchical Data , 2011 .

[10]  Vicente González-Romá,et al.  Multilevel Modeling: Research-Based Lessons for Substantive Researchers , 2017 .

[11]  David P Mackinnon,et al.  Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods , 2004, Multivariate behavioral research.

[12]  B. Muthén,et al.  Multilevel Covariance Structure Analysis , 1994 .

[13]  P. D. de Jong,et al.  Inter- and intra-individual differences in teachers' self-efficacy: A multilevel factor exploration. , 2016, Journal of school psychology.

[14]  Chunhua Cao,et al.  Testing Group Mean Differences of Latent Variables in Multilevel Data Using Multiple-Group Multilevel CFA and Multilevel MIMIC Modeling , 2015, Multivariate behavioral research.

[15]  Suzanne Jak,et al.  A Test for Cluster Bias: Detecting Violations of Measurement Invariance Across Clusters in Multilevel Data , 2013 .

[16]  S. N. Beretvas,et al.  Sample Size Limits for Estimating Upper Level Mediation Models Using Multilevel SEM , 2013 .

[17]  S. Rabe-Hesketh,et al.  Generalized multilevel structural equation modeling , 2004 .

[18]  Michael C Neale,et al.  People are variables too: multilevel structural equations modeling. , 2005, Psychological methods.

[19]  Ulrich Trautwein,et al.  A 2 × 2 taxonomy of multilevel latent contextual models: accuracy-bias trade-offs in full and partial error correction models. , 2011, Psychological methods.

[20]  J. Vermunt,et al.  A Conceptual and Methodological Framework for Psychometric Isomorphism , 2014 .

[21]  D. Bates,et al.  Parsimonious Mixed Models , 2015, 1506.04967.

[22]  Kristopher J Preacher,et al.  A general multilevel SEM framework for assessing multilevel mediation. , 2010, Psychological methods.

[23]  Terrence D. Jorgensen,et al.  Relating Measurement Invariance, Cross-Level Invariance, and Multilevel Reliability , 2017, Front. Psychol..

[24]  S. Jak,et al.  Are boys better off with male and girls with female teachers? A multilevel investigation of measurement invariance and gender match in teacher-student relationship quality. , 2012, Journal of school psychology.

[25]  Ulrich Trautwein,et al.  Doubly-Latent Models of School Contextual Effects: Integrating Multilevel and Structural Equation Approaches to Control Measurement and Sampling Error , 2009, Multivariate behavioral research.

[26]  John M Ferron,et al.  Multilevel Factor Analysis: Reporting Guidelines and a Review of Reporting Practices , 2016, Multivariate behavioral research.

[27]  Jean-Paul Fox,et al.  Relaxing Measurement Invariance in Cross-National Consumer Research Using a Hierarchical IRT Model , 2007 .

[28]  Michael Eid,et al.  A Comparison of ML, WLSMV, and Bayesian Methods for Multilevel Structural Equation Models in Small Samples: A Simulation Study , 2016, Multivariate behavioral research.

[29]  Mirjam Moerbeek,et al.  Multilevel Analysis : Techniques and Applications, Third Edition , 2017 .

[30]  P. J. Huber.  The behavior of maximum likelihood estimates under nonstandard conditions , 1967 .

[31]  H. White.  Maximum Likelihood Estimation of Misspecified Models , 1982 .

[32]  G. A. Marcoulides,et al.  Multilevel Analysis Techniques and Applications , 2002 .

[33]  P. Bliese.  Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. , 2000 .

[34]  Cora J. M. Maas,et al.  The effect of estimation method and sample size in multilevel structural equation modeling , 2010 .

[35]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[36]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .

[37]  Nigel Guenole,et al.  The Importance of Isomorphism for Conclusions about Homology: A Bayesian Multilevel Structural Equation Modeling Approach with Ordinal Indicators , 2016, Front. Psychol..

[38]  S. Jackson,et al.  Diversity in social context: a multi-attribute, multilevel analysis of team diversity and sales performance , 2004 .

[39]  B. Muthén,et al.  Multiple Group Multilevel Analysis , 2012 .

[40]  B. Muthén.  Latent variable modeling in heterogeneous populations , 1989 .