UvA-DARE ( Digital Academic Repository ) Testing and Explaining Differences in Common and Residual Factors Across Many Countries

To make valid comparisons across countries, a measurement instrument needs to be measurement invariant across countries. The present article provides a nontechnical exposition of a recently proposed multilevel factor analysis approach to test measurement invariance across countries. It is explained that strong factorial invariance across countries implies equal factor loadings across levels and zero residual variance at the country level in a two-level factor model. Using two-level factor analysis, the decomposition of the variance at each level can be investigated, measurement invariance can be tested, and country-level variables can be added to explain differences in the common or residual factors. The approach is illustrated using two examples. The first example features data about well-being from the European Social Survey and the second example uses data about mathematical ability from the Programme for International Student Assessment (PISA) study. The input-files and annotated output-files for both examples are provided in the supplementary files.

[1]  J. Billiet.  Cross-cultural equivalence with structural equation modeling , 2003 .

[2]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[3]  Barbara M. Byrne,et al.  Testing for Measurement and Structural Equivalence in Large-Scale Cross-Cultural Studies: Addressing the Issue of Nonequivalence , 2010 .

[4]  Frans J. Oort,et al.  Using restricted factor analysis with latent moderated structures to detect uniform and nonuniform measurement bias; a simulation study , 2010 .

[5]  B. Muthén,et al.  Multilevel Covariance Structure Analysis , 1994 .

[6]  G. A. Marcoulides,et al.  Multilevel Analysis Techniques and Applications , 2002 .

[7]  Gideon J. Mellenbergh,et al.  Item bias and item response theory , 1989 .

[8]  L. Radloff.  The CES-D Scale , 1977 .

[9]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[10]  S. Jak.  Testing strong factorial invariance using three-level structural equation modeling , 2014, Front. Psychol..

[11]  Andreas Ritter,et al.  Structural Equations With Latent Variables , 2016 .

[12]  Y. Poortinga.  Equivalence of cross-cultural data: an overview of basic issues. , 1989, International journal of psychology : Journal international de psychologie.

[13]  Bengt Muthén,et al.  New Methods for the Study of Measurement Invariance with Many Groups , 2013 .

[14]  Jelte M. Wicherts,et al.  Modeling differentiation of cognitive abilities within the higher-order factor model using moderated factor analysis , 2010 .

[15]  D. A. van Hemert,et al.  Conceptual issues in multilevel models , 2008 .

[16]  Mike W-L Cheung,et al.  Fixed- and random-effects meta-analytic structural equation modeling: Examples and analyses in R , 2013, Behavior Research Methods.

[17]  Conor Dolan,et al.  On the likelihood ratio test in structural equation modeling when parameters are subject to boundary constraints. , 2006, Psychological methods.

[18]  S. Reise,et al.  Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. , 1997 .

[19]  B. Muthén,et al.  Computationally Efficient Estimation of Multilevel High-Dimensional Latent Variable Models , 2007 .

[20]  Eldad Davidov,et al.  Using a Multilevel Structural Equation Modeling Approach to Explain Cross-Cultural Measurement Noninvariance , 2012 .

[21]  K. Yuan,et al.  5. Three Likelihood-Based Methods for Mean and Covariance Structure Analysis with Nonnormal Missing Data , 2000 .

[22]  P. Schmidt,et al.  Measurement Equivalence in Cross-National Research , 2014 .

[23]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[24]  Jan de Leeuw,et al.  On the relationship between item response theory and factor analysis of discretized variables , 1987 .

[25]  Michael C Neale,et al.  People are variables too: multilevel structural equations modeling. , 2005, Psychological methods.

[26]  J. Vermunt,et al.  A Conceptual and Methodological Framework for Psychometric Isomorphism , 2014 .

[27]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[28]  Noel A. Card,et al.  Latent variable structural equation modeling in cross-cultural research: Multigroup and multilevel approaches , 2008 .

[29]  Carla Rampichini,et al.  Multilevel Factor Models for Ordinal Variables , 2007 .

[30]  Leslie Rutkowski,et al.  Assessing the Hypothesis of Measurement Invariance in the Context of Large-Scale International Surveys , 2014 .

[31]  J. H. Steiger.  Statistically based tests for the number of common factors , 1980 .

[32]  Daniel Stegmueller,et al.  How Many Countries for Multilevel Modeling? A Comparison of Frequentist and Bayesian Approaches , 2013 .

[33]  Martin H. Fischer,et al.  The Cultural Number Line: A Review of Cultural and Linguistic Influences on the Development of Number Processing , 2011 .

[34]  Bengt Muthén,et al.  Mean and Covariance Structure Analysis of Hierarchical Data , 2011 .

[35]  Todd D. Little,et al.  On the Comparability of Constructs in Cross-Cultural Research , 2000 .

[36]  P. Dolan,et al.  Do we really know what makes us happy A review of the economic literature on the factors associated with subjective well-being , 2008 .

[37]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[38]  Terry A. Ackerman.  A Didactic Explanation of Item Bias, Item Impact, and Item Validity from a Multidimensional Perspective , 1992 .

[39]  Peters Robert.  European Social Survey , 2012 .

[40]  S. Schwartz,et al.  Whence Differences in Value Priorities? , 2011 .

[41]  Suzanne Jak,et al.  A Test for Cluster Bias: Detecting Violations of Measurement Invariance Across Clusters in Multilevel Data , 2013 .

[42]  Joar Vittersø,et al.  Measuring Well-being Across Europe: Description of the ESS Well-being Module and Preliminary Findings , 2009 .

[43]  S. Rabe-Hesketh,et al.  Generalized multilevel structural equation modeling , 2004 .

[44]  Nigel Guenole,et al.  The Importance of Isomorphism for Conclusions about Homology: A Bayesian Multilevel Structural Equation Modeling Approach with Ordinal Indicators , 2016, Front. Psychol..

[45]  Suzanne Jak,et al.  Measurement Bias in Multilevel Data , 2014 .

[46]  Mike W.-L. Cheung,et al.  Evaluating Multilevel Models in Cross-Cultural Research , 2006 .

[47]  J. Hox,et al.  Sufficient Sample Sizes for Multilevel Modeling , 2005 .

[48]  D. Sörbom.  A GENERAL METHOD FOR STUDYING DIFFERENCES IN FACTOR MEANS AND FACTOR STRUCTURE BETWEEN GROUPS , 1974 .

[49]  Roger E. Millsap,et al.  Detecting Violations of Factorial Invariance Using Data-Based Specification Searches: A Monte Carlo Study , 2007 .

[50]  P. Bentler,et al.  Comparative fit indexes in structural models. , 1990, Psychological bulletin.

[51]  J. Fox,et al.  Bayesian tests of measurement invariance. , 2012, The British journal of mathematical and statistical psychology.