Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance

Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which examines the change in the goodness-of-fit index (GFI) when cross-group constraints are imposed on a measurement model. Although many studies have examined the properties of GFI as indicators of overall model fit for single-group data, there have been none to date that examine how GFIs change when between-group constraints are added to a measurement model. The lack of a consensus about what constitutes significant GFI differences places limits on measurement invariance testing. We examine 20 GFIs based on the minimum fit function. A simulation under the two-group situation was used to examine changes in the GFIs (ΔGFIs) when invariance constraints were added. Based on the results, we recommend using Δcomparative fit index, ΔGamma hat, and ΔMcDonald's Noncentrality Index to evaluate measurement invariance. These three ΔGFIs are independent of both model complexity and sample size, and are not correlated with the overall fit measures. We propose critical values of these ΔGFIs that indicate measurement invariance.

[1]  S. H. Irvine.  Contributions of ability and attainment testing in Africa to a general theory of intellect , 1969, Journal of Biosocial Science.

[2]  L. Tucker,et al.  A reliability coefficient for maximum likelihood factor analysis , 1973 .

[3]  J. R. Royce,et al.  Detecting cross-cultural commonalities and differences: Intergroup factor analysis. , 1975 .

[4]  Theory and Method in Cross-Cultural Psychology. , 1977 .

[5]  B. Muthén,et al.  Assessing Reliability and Stability in Panel Models , 1977 .

[6]  P. Bentler,et al.  Significance Tests and Goodness of Fit in the Analysis of Covariance Structures , 1980 .

[7]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[8]  C. Schriesheim.  Causal Analysis: Assumptions, Models, and Data , 1982 .

[9]  Subhash Sharma,et al.  Sample Size Effects on Chi Square and Other Statistics Used in Evaluating Causal Models , 1982 .

[10]  M. Browne,et al.  Cross-Validation Of Covariance Structures. , 1983, Multivariate behavioral research.

[11]  James C. Anderson,et al.  The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis , 1984 .

[12]  R. Kanfer,et al.  Equivalence of psychological measurement in heterogeneous populations. , 1985, The Journal of applied psychology.

[13]  H. Marsh,et al.  Application of confirmatory factor analysis to the study of self-concept: First- and higher order factor models and their invariance across groups. , 1985 .

[14]  A. Shapiro,et al.  On the multivariate asymptotic distribution of sequential Chi-square statistics , 1985 .

[15]  K. Bollen.  Sample size and bentler and Bonett's nonnormed fit index , 1986 .

[16]  H. Akaike.  Factor analysis and AIC , 1987 .

[17]  Sandra B. Hartog,et al.  Alpha, beta, and gamma change in evaluation research: A structural equation approach. , 1988 .

[18]  S. Iwawaki,et al.  Cross-Cultural Comparability of Temperament Among Japanese and American Preschool Children , 1988 .

[19]  R. P. McDonald,et al.  Goodness-of-fit indexes in confirmatory factor analysis : The effect of sample size , 1988 .

[20]  M. Browne,et al.  Single Sample Cross-Validation Indices for Covariance Structures. , 1989, Multivariate behavioral research.

[21]  Karl G. Jöreskog,et al.  Lisrel 8: User's Reference Guide , 1997 .

[22]  R. P. McDonald,et al.  An index of goodness-of-fit based on noncentrality , 1989 .

[23]  K. Bollen.  A New Incremental Fit Index for General Structural Equation Models , 1989 .

[24]  Jeffrey S. Tanaka,et al.  Influence of sample size, estimation method, and model specification on goodness-of-fit assessments in structural equation models. , 1989 .

[25]  S. Mulaik,et al.  EVALUATION OF GOODNESS-OF-FIT INDICES FOR STRUCTURAL EQUATION MODELS , 1989 .

[26]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[27]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[28]  R. P. McDonald,et al.  Choosing a multivariate model: Noncentrality and goodness of fit. , 1990 .

[29]  S J Henly,et al.  Model selection in covariance structures analysis and the "problem" of sample size: a clarification. , 1991, Psychological bulletin.

[30]  R E Millsap,et al.  Confirmatory Measurement Model Comparisons Using Latent Means. , 1991, Multivariate behavioral research.

[31]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[32]  James C. Anderson,et al.  Monte Carlo Evaluations of Goodness of Fit Indices for Structural Equation Models , 1992 .

[33]  J. Horn,et al.  A practical and theoretical guide to measurement invariance in aging research. , 1992, Experimental aging research.

[34]  Robert Cudeck,et al.  Constructing a covariance matrix that yields a specified minimizer and a specified minimum discrepancy function value , 1992 .

[35]  Herbert W. Marsh,et al.  The Multidimensional Structure of Academic Self-Concept: Invariance Over Gender and Age , 1993 .

[36]  Keith F Widaman,et al.  Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. , 1993, Psychological bulletin.

[37]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[38]  Paul R. Jackson,et al.  New measures of job control, cognitive demand, and production responsibility. , 1993 .

[39]  Robert J. Vandenberg,et al.  A Central Question in Cross-Cultural Research: Do Employees of Different Cultures Interpret Work-related Measures in an Equivalent Manner?: , 1994 .

[40]  M. Tayeb,et al.  Organizations and National Culture: Methodology Considered , 1994 .

[41]  M A Pentz,et al.  Measurement invariance in longitudinal clinical research assuming change from development and intervention. , 1994, Journal of consulting and clinical psychology.

[42]  B. Byrne,et al.  Testing for the Factorial Validity, Replication, and Invariance of a Measuring Instrument: A Paradigmatic Application Based on the Maslach Burnout Inventory. , 1994, Multivariate behavioral research.

[43]  Andrew S. Rancer,et al.  Argumentativeness and verbal aggressiveness: Testing for conceptual and measurement equivalence across cultures , 1994 .

[44]  Michael R. Mullen.  Diagnosing Measurement Equivalence in Cross-National Research , 1995 .

[45]  Maddy Janssens,et al.  Confirmatory Cross-Cultural Research: Testing the Viability of a Corporation-Wide Safety Policy , 1995 .

[46]  Jagdip Singh,et al.  Measurement Issues in Cross-National Research , 1995 .

[47]  Michael T. Brannick,et al.  Critical comments on applying covariance structure modeling , 1995 .

[48]  E. Kevin Kelloway,et al.  Structural equation modelling in perspective. , 1995 .

[49]  T. Little.  Mean and Covariance Structures (MACS) Analyses of Cross-Cultural Data: Practical and Theoretical Issues. , 1997, Multivariate behavioral research.

[50]  J. Steenkamp,et al.  Assessing Measurement Invariance in Cross-National Consumer Research , 1998 .

[51]  P. Bentler,et al.  Fit indices in covariance structure modeling : Sensitivity to underparameterized model misspecification , 1998 .

[52]  Rosalie J. Hall,et al.  Item Parceling Strategies in SEM: Investigating the Subtle Effects of Unmodeled Secondary Constructs , 1999 .

[53]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[54]  Gordon W. Cheung,et al.  Assessing Extreme and Acquiescence Response Sets in Cross-Cultural Research Using Structural Equations Modeling , 2000 .