A Direct Comparison Approach for Testing Measurement Invariance

Measurement equivalence/invariance (ME/I) is a condition that should be met before meaningful comparisons of survey results across groups can be made. As an alternative to the likelihood ratio test (LRT), the change in comparative fit index (ΔCFI) rules of thumb, and the modification index (MI), this teaching note demonstrates the procedures for establishing bias-corrected (BC) bootstrap confidence intervals for testing ME/I. Unlike the LRT and ΔCFI methods, which need a different model estimation per item, the BC bootstrap confidence intervals approach can examine item-level ME/I tests using a single model. This method greatly simplifies the search for an invariant item as the reference indicator in the factor-ratio test. Also demonstrated here is how the factor-ratio test and the list-and-delete method can be extended from the metric invariance test to the scalar invariance test. Finally, the BC bootstrap confidence interval procedures for comparing uniqueness variances, factor variances, factor covariances, and latent means across groups are shown.

[1]  Jae-On Kim,et al.  Factor Analysis: Statistical Methods and Practical Issues , 1978 .

[2]  James C. Anderson,et al.  The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis , 1984 .

[3]  H. Marsh,et al.  Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. , 1985 .

[4]  Sandra B. Hartog,et al.  Alpha, beta, and gamma change in evaluation research: A structural equation approach. , 1988 .

[5]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[6]  James C. Anderson,et al.  Monte Carlo Evaluations of Goodness of Fit Indices for Structural Equation Models , 1992 .

[7]  J. Horn,et al.  A practical and theoretical guide to measurement invariance in aging research. , 1992, Experimental aging research.

[8]  Keith F Widaman,et al.  Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. , 1993, Psychological bulletin.

[9]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[10]  Howard T. Everson,et al.  Methodology Review: Statistical Approaches for Assessing Measurement Bias , 1993 .

[11]  Michael T. Brannick,et al.  Critical comments on applying covariance structure modeling , 1995 .

[12]  E. Kevin Kelloway,et al.  Structural equation modelling in perspective. , 1995 .

[13]  P. Bentler,et al.  Bootstrapping Techniques in Analysis of Mean and Covariance Structures , 1996 .

[14]  David Chan,et al.  The Conceptualization and Analysis of Change Over Time: An Integrative Approach Incorporating Longitudinal Mean and Covariance Structures Analysis (LMACS) and Multiple Indicator Latent Growth Modeling (MLGM) , 1998 .

[15]  J. Steenkamp,et al.  Assessing Measurement Invariance in Cross-National Consumer Research , 1998 .

[16]  Gordon W. Cheung,et al.  Cross-cultural comparisons using non-invariant measurement items , 1998 .

[17]  Gordon W. Cheung,et al.  Testing Factorial Invariance across Groups: A Reconceptualization and Proposed New Method , 1999 .

[18]  Gordon W. Cheung.  Multifaceted conceptions of self–other ratings disagreement. , 1999 .

[19]  Karl G. Jöreskog,et al.  Robust standard errors and Chi-Squares for the interaction models , 2000 .

[20]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[21]  Gordon W. Cheung,et al.  Assessing Extreme and Acquiescence Response Sets in Cross-Cultural Research Using Structural Equations Modeling , 2000 .

[22]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[23]  R. Vandenberg.  Toward a Further Understanding of and Improvement in Measurement Invariance Methods and Procedures , 2002 .

[24]  Robert E. Ployhart,et al.  Applications of Mean and Covariance Structure Analysis: Integrating Correlational and Experimental Approaches , 2004 .

[25]  David P Mackinnon,et al.  Confidence Limits for the Indirect Effect: Distribution of the Product and Resampling Methods , 2004, Multivariate behavioral research.

[26]  Adam W. Meade,et al.  A Comparison of Item Response Theory and Confirmatory Factor Analytic Methodologies for Establishing Measurement Equivalence/Invariance , 2004 .

[27]  Gordon W. Cheung,et al.  Noninvariant Measurement Versus Traditional Approaches for Studying Cultural Differences: A Case of Service Expectations , 2004 .

[28]  Adam W. Meade,et al.  A Monte-Carlo Study of Confirmatory Factor Analytic Tests of Measurement Equivalence/Invariance , 2004 .

[29]  On Chi-Squares for the Independence Model and Fit Measures in LISREL , 2004 .

[30]  Adam W. Meade,et al.  Problems With Item Parceling for Confirmatory Factor Analytic Tests of Measurement Invariance , 2006 .

[31]  Adam W. Meade,et al.  Power and Precision in Confirmatory Factor Analytic Tests of Measurement Invariance , 2007 .

[32]  Roger E. Millsap,et al.  Invariance in Measurement and Prediction Revisited , 2007 .

[33]  F. Chen.  Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance , 2007 .

[34]  Roger E. Millsap,et al.  Detecting Violations of Factorial Invariance Using Data-Based Specification Searches: A Monte Carlo Study , 2007 .

[35]  Adam W. Meade,et al.  Are Internet and Paper-and-Pencil Personality Tests Truly Comparable? , 2007 .

[36]  Gordon W. Cheung,et al.  Testing Mediation and Suppression Effects of Latent Variables , 2008 .

[37]  David P Mackinnon,et al.  Resampling and Distribution of the Product Methods for Testing Indirect Effects in Complex Models , 2008, Structural equation modeling : a multidisciplinary journal.

[38]  Phillip W. Braddy,et al.  Power and sensitivity of alternative fit indices in tests of measurement invariance. , 2008, The Journal of applied psychology.

[39]  N. Schmitt,et al.  Measurement invariance: Review of practice and implications , 2008 .

[40]  Gordon W. Cheung.  Testing Equivalence in the Structure, Means, and Variances of Higher-Order Constructs With Structural Equation Modeling , 2008 .

[41]  B. French,et al.  Multigroup Confirmatory Factor Analysis: Locating the Invariant Referent Sets , 2008 .

[42]  Emily C. Johnson,et al.  The Role of Referent Indicators in Tests of Measurement Invariance , 2009 .

[43]  Gordon W. Cheung.  A Multiple-Perspective Approach to Data Analysis in Congruence Research , 2009 .

[44]  Gordon W. Cheung,et al.  Estimating and Comparing Specific Mediation Effects in Complex Latent Variable Models , 2012 .

[45]  Robert E. Ployhart,et al.  The Analysis of Mean Differences Using Mean and Covariance Structure Analysis , 2012 .