Multiple-Group Invariance with Categorical Outcomes Using Updated Guidelines: An Illustration Using Mplus and the lavaan/semTools Packages

Meaningful comparisons of means or relationships between latent constructs across groups require evidence that measurement is equivalent across the studied groups– a property known as measurement equivalence or invariance (ME/I). Methods typically involve an evaluation of increasingly stringent models via confirmatory factor analysis, a typical assumption of which is continuous observed variables. When that assumption is not met – as is often the case in many surveys – alternative methods that directly model the categorical nature of the data exist. Although well established, categorical ME/I models pose a number of complexities and various recommendations for their evaluation. To that end, we describe the current state of categorical ME/I and demonstrate an up-to-date method for model identification and invariance testing. In the tutorial, we exemplify a common approach to establishing ME/I via multiple-group confirmatory factor analysis using Mplus and the lavaan and semTools packages in R.

[1]  R. Millsap,et al.  Factorial Invariance in Multiple Populations , 2013 .

[2]  T. Raykov,et al.  Measurement Invariance for Latent Constructs in Multiple Populations , 2012 .

[3]  Katerina M. Marcoulides,et al.  New Ways to Evaluate Goodness of Fit: A Note on Using Equivalence Testing to Assess Structural Equation Models , 2017 .

[4]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[5]  George A. Marcoulides,et al.  Assessing Structural Equation Models by Equivalence Testing With Adjusted Fit Indexes , 2016 .

[6]  R. Hambleton,et al.  Detecting potentially biased test items : Comparison of IRT area and Mantel-Haenszel methods , 1989 .

[7]  H. Swaminathan,et al.  Detecting Differential Item Functioning Using Logistic Regression Procedures , 1990 .

[8]  Gregory R. Hancock,et al.  Structural equation modeling methods of hypothesis testing of latent variable means. , 1997 .

[9]  Ke-Hai Yuan,et al.  Measurement invariance via multigroup SEM: Issues and solutions with chi-square-difference tests. , 2016, Psychological methods.

[10]  Richard P. DeShon,et al.  Measures are not invariant across groups without error variance homogeneity , 2004 .

[11]  Leslie Rutkowski,et al.  Measurement Invariance in International Surveys: Categorical Indicators and Fit Measure Performance , 2017 .

[12]  A Simulation Investigation of the Performance of Invariance Assessment Using Equivalence Testing Procedures , 2018 .

[13]  S. Biddle,et al.  Physical Self-Perceptions in Adolescence , 2003 .

[14]  D. Morrow,et al.  Aging, narrative organization, presentation mode, and referent choice strategies. , 1992, Experimental aging research.

[15]  W. Holmes Finch,et al.  Confirmatory Factor Analytic Procedures for the Determination of Measurement Invariance , 2006 .

[16]  Bengt Muthén,et al.  Multiple-Group Factor Analysis Alignment , 2014 .

[17]  Eldad Davidov,et al.  A Monte Carlo Simulation Study to Assess The Appropriateness of Traditional and Newer Approaches to Test for Measurement Invariance , 2019, Structural Equation Modeling: A Multidisciplinary Journal.

[18]  Bengt Muthén,et al.  Simultaneous factor analysis of dichotomous variables in several groups , 1981 .

[19]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[20]  Ahmed M. Megreya,et al.  The Robustness of the Nine-Factor Structure of the Cognitive Emotion Regulation Questionnaire across Four Arabic-Speaking Middle Eastern Countries , 2016 .

[21]  Hao Wu,et al.  Identification of Confirmatory Factor Analysis Models of Different Levels of Invariance for Ordered Categorical Outcomes , 2016, Psychometrika.

[22]  J. H. Steiger Statistically based tests for the number of common factors , 1980 .

[23]  R. Bagozzi Structural Equation Models in Experimental Research , 1977 .

[24]  Leslie Rutkowski,et al.  Multidimensional Measurement Invariance in an International Context: Fit Measure Performance With Many Groups , 2017 .

[25]  Roger E. Millsap,et al.  Assessing Factorial Invariance in Ordered-Categorical Measures , 2004 .

[26]  G. J. Mellenbergh,et al.  Generalized linear item response theory. , 1994 .

[27]  T. Little Mean and Covariance Structures (MACS) Analyses of Cross-Cultural Data: Practical and Theoretical Issues. , 1997, Multivariate behavioral research.

[28]  Phillip W. Braddy,et al.  Power and sensitivity of alternative fit indices in tests of measurement invariance. , 2008, The Journal of applied psychology.

[29]  F. Chen Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance , 2007 .

[30]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[31]  B. Muthén,et al.  Applying Multigroup Confirmatory Factor Models for Continuous Outcomes to Likert Scale Data Complicates Meaningful Group Comparisons , 2004 .

[32]  Leslie Rutkowski,et al.  Collapsing Categorical Variables and Measurement Invariance , 2019, Structural Equation Modeling: A Multidisciplinary Journal.

[33]  Charles B. Corbin,et al.  The Physical Self-Perception Profile: Development and preliminary validation , 1989 .

[34]  G. Lubke,et al.  Can Unequal Residual Variances Across Groups Mask Differences in Residual Means in the Common Factor Model? , 2003 .

[35]  J. Horn,et al.  When is invariance not invarient: A practical scientist's look at the ethereal concept of factor invariance. , 1983 .

[36]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[37]  W. Meredith Measurement invariance, factor analysis and factorial invariance , 1993 .

[38]  Yan Wang,et al.  Measurement Invariance Testing with Many Groups: A Comparison of Five Approaches , 2017 .

[39]  Bengt,et al.  Latent Variable Analysis With Categorical Outcomes : Multiple-Group And Growth Modeling In Mplus , 2002 .

[40]  André Beauducel,et al.  On the Performance of Maximum Likelihood Versus Means and Variance Adjusted Weighted Least Squares Estimation in CFA , 2006 .

[41]  Leslie Rutkowski,et al.  Assessing the Hypothesis of Measurement Invariance in the Context of Large-Scale International Surveys , 2014 .

[42]  F. Lord Applications of Item Response Theory To Practical Testing Problems , 1980 .

[43]  Maria E. Hernández Finch,et al.  Comparison of Methods for Factor Invariance Testing of a 1-Factor Model With Small Samples and Skewed Latent Traits , 2018, Front. Psychol..

[44]  J. Horn,et al.  A practical and theoretical guide to measurement invariance in aging research. , 1992, Experimental aging research.

[45]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .

[46]  K. G. J8reskoC,et al.  Simultaneous Factor Analysis in Several Populations , 2007 .

[47]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[48]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables , 1985 .