Measurement invariance in confirmatory factor analysis: an illustration using IQ test performance of minorities

Measurement invariance with respect to groups is an essential aspect of the fair use of scores of intelligence tests and other psychological measurements. It is widely believed that equal factor loadings are sufficient to establish measurement invariance in confirmatory factor analysis. Here, it is shown why establishing measurement invariance with confirmatory factor analysis requires a statistical test of the equality over groups of measurement intercepts. Without this essential test, measurement bias may be overlooked. A re-analysis of a study by Te Nijenhuis, Tolboom, Resing, and Bleichrodt (2004) on ethnic differences on the RAKIT IQ test illustrates that ignoring intercept differences may lead to the conclusion that bias of IQ tests with respect to minorities is small, while in reality bias is quite severe.

[1]  P. Bentler.  Comparative Fit Indices in Structural Models , 1990 .

[2]  Laura M. Stapleton,et al.  Teacher's Corner: A Review of Syllabi for a Sample of Structural Equation Modeling Courses , 2005 .

[3]  Barbara M Byrne,et al.  Measurement equivalence: a comparison of methods based on confirmatory factor analysis and item response theory. , 2002, The Journal of applied psychology.

[4]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[5]  J. Elshout.  Human cognitive abilities. A survey of factoranalytic studies: John B. Carroll, Cambridge University Press, 1993. Pp. 819. ISBN-0-521-38712-4 paperback , 1995 .

[6]  Joachim Scholderer,et al.  A procedure for eliminating additive bias from cross-cultural survey data , 2005 .

[7]  Conor V Dolan,et al.  Weak measurement invariance with respect to unmeasured variables: an implication of strict factorial invariance. , 2003, The British journal of mathematical and statistical psychology.

[8]  Y. Poortinga,et al.  Cross-cultural differences in cognitive performance and Spearman's hypothesis: g or c? , 2000 .

[9]  William Stout,et al.  A Multidimensionality-Based DIF Analysis Paradigm , 1996 .

[10]  H. Wainer,et al.  Differential Item Functioning. , 1994 .

[11]  Gideon J. Mellenbergh,et al.  Item bias and item response theory , 1989 .

[12]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[13]  F.J.R. van de Vijver,et al.  Methods and Data Analysis for Cross-Cultural Research , 1997 .

[14]  Wilma C. M. Resing,et al.  Does Cultural Background Influence the Intellectual Performance of Children from Immigrant Groups , 2004 .

[15]  Robert E. Ployhart,et al.  Applications of Mean and Covariance Structure Analysis: Integrating Correlational and Experimental Approaches , 2004 .

[16]  G. Lubke,et al.  Can Unequal Residual Variances Across Groups Mask Differences in Residual Means in the Common Factor Model? , 2003 .

[17]  William Meredith,et al.  The role of factorial invariance in modeling growth and change. , 2001 .

[18]  D. Wechsler.  Wechsler Adult Intelligence Scale , 2021, Encyclopedia of Evolutionary Psychological Science.

[19]  Keith F Widaman,et al.  On specifying the null model for incremental fit indices in structural equation modeling. , 2003, Psychological methods.

[20]  G. J. Mellenbergh.  Contingency Table Models for Assessing Item Bias , 1982 .

[21]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[22]  Denny Borsboom,et al.  When does measurement invariance matter? , 2006, Medical care.

[23]  R. P. McDonald,et al.  Structural Equations with Latent Variables , 1989 .

[24]  Conor V Dolan,et al.  Stereotype threat and group differences in test performance: a question of measurement invariance. , 2005, Journal of personality and social psychology.

[25]  Howard T. Everson,et al.  Methodology Review: Statistical Approaches for Assessing Measurement Bias , 1993 .

[26]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[27]  J. Nijenhuis,et al.  Comparability of GATB scores for immigrants and majority group members: Some Dutch findings. , 1997 .

[28]  D. Wechsler.  Wechsler Intelligence Scale for Children , 2020, Definitions.

[29]  J. Carroll.  Human Cognitive Abilities-a sur-vey of factor-analytic studies , 1993 .

[30]  J. Wicherts,et al.  A Cautionary Note on the Use of Information Fit Indexes in Covariance Structure Modeling With Means , 2004 .

[31]  G. J. Mellenbergh,et al.  A Unidimensional Latent Trait Model for Continuous Item Responses. , 1994, Multivariate behavioral research.

[32]  C. Dolan,et al.  Investigating Spearman's Hypothesis by Means of Multi-Group Confirmatory Factor Analysis , 2000, Multivariate behavioral research.

[33]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[34]  J. S. Long,et al.  Testing Structural Equation Models , 1993 .

[35]  R. Lynn,et al.  Sex differences in general knowledge, semantic memory and reasoning ability. , 2002, British journal of psychology.

[36]  Dorret I. Boomsma,et al.  Sex differences on the Dutch WAIS-III , 2006 .

[37]  Gitta H. Lubke,et al.  On the relationship between sources of within- and between-group differences and measurement invariance in the common factor model , 2003 .

[38]  Richard P. DeShon,et al.  Measures are not invariant across groups without error variance homogeneity , 2004 .

[39]  J. Wicherts.  Group differences in intelligence test performance , 2002 .

[40]  T. Little.  Mean and Covariance Structures (MACS) Analyses of Cross-Cultural Data: Practical and Theoretical Issues. , 1997, Multivariate behavioral research.

[41]  J. Nijenhuis,et al.  Validity of the Differential Aptitude Test for the Assessment of Immigrant Children , 2000 .

[42]  David J. Hessen,et al.  Multi-group covariance and mean structure modeling of the relationship between the WAIS-III common factors and sex and educational attainment in Spain , 2006 .

[43]  W. Meredith,et al.  Factorial Invariance: Historical Perspectives and New Problems , 2007 .