Using a Multilevel Structural Equation Modeling Approach to Explain Cross-Cultural Measurement Noninvariance

Testing for invariance of measurements across groups (such as countries or time points) is essential before meaningful comparisons may be conducted. However, when tested, invariance is often absent. As a result, comparisons across groups are potentially problematic and may be biased. In the current study, we propose utilizing a multilevel structural equation modeling (SEM) approach to provide a framework to explain item bias. We show how variation in a contextual variable may explain noninvariance. For the illustration of the method, we use data from the second round of the European Social Survey (ESS).

[1]  R. Inglehart.  Culture Shift in Advanced Industrial Society , 2018 .

[2]  Gilbert Swinnen,et al.  Biased latent variable mean comparisons due to measurement noninvariance. A simulation study , 2018 .

[3]  Joop J. Hox,et al.  How few countries will do? Comparative survey analysis from a Bayesian perspective , 2012 .

[4]  E. Davidov,et al.  Can we Rate Public Support for Democracy in a Comparable Way? Cross-National Equivalence of Democratic Attitudes in the World Value Survey , 2011 .

[5]  T. Milfont,et al.  Does Social Context Affect Value Structures? Testing the Within-Country Stability of Value Structures With a Functional Theory of Values , 2011 .

[6]  Edith D. de Leeuw,et al.  Analysis Models for Comparative Surveys , 2010 .

[7]  Barbara M. Byrne,et al.  Testing for Measurement and Structural Equivalence in Large-Scale Cross-Cultural Studies: Addressing the Issue of Nonequivalence , 2010 .

[8]  J. Harkness.  Survey Methods in Multinational, Multiregional, and Multicultural Contexts , 2010 .

[9]  Willem E. Saris,et al.  Testing Structural Equation Models or Detection of Misspecifications? , 2009 .

[10]  J. S. Long,et al.  Measurement Models Matter: Implicit Assumptions and Cross-national Research , 2009 .

[11]  Holger Steinmetz,et al.  Testing measurement invariance using multigroup CFA: differences between educational groups in human values measurement , 2009 .

[12]  E. Davidov,et al.  Changing attitudes toward immigration in Europe, 2002-2007: a dynamic group conflict theory approach. , 2009, Social science research.

[13]  Jaak Billiet,et al.  A Monte Carlo sample size study: How many countries are needed for accurate multilevel SEM? , 2009 .

[14]  Gerard Delanty,et al.  The Foundations of Social Theory , 2009 .

[15]  S. Schwartz,et al.  Bringing Values Back In The Adequacy of the European Social Survey to Measure Values in 20 Countries , 2008 .

[16]  Shalom H. Schwartz,et al.  Structural Equivalence of the Values Domain Across Cultures , 2008 .

[17]  Eldad Davidov,et al.  A cross-country and cross-time comparison of the human values measurements with the second round of the European Social Survey , 2008 .

[18]  Phil Wood.  Confirmatory Factor Analysis for Applied Research , 2008 .

[19]  Y. Poortinga,et al.  Multilevel analysis of individuals and cultures , 2008 .

[20]  Filip Lievens,et al.  Measurement equivalence in the conduct of a global organizational survey across countries in six cultural regions , 2007 .

[21]  F. Chen.  Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance , 2007 .

[22]  Mike W.-L. Cheung,et al.  Evaluating Multilevel Models in Cross-Cultural Research , 2006 .

[23]  Mike W.-L. Cheung,et al.  Applications of Multilevel Structural Equation Modeling to Cross-Cultural Research , 2005 .

[24]  H. Marsh,et al.  In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler's (1999) Findings , 2004 .

[25]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[26]  Timothy J. Robinson,et al.  Multilevel Analysis: Techniques and Applications , 2002 .

[27]  S. Schwartz,et al.  Extending the Cross-Cultural Validity of the Theory of Basic Human Values with a Different Method of Measurement , 2001 .

[28]  B. Byrne.  Structural equation modeling with EQS : basic concepts, applications, and programming , 2000 .

[29]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[30]  J. Steenkamp,et al.  Assessing Measurement Invariance in Cross-National Consumer Research , 1998 .

[31]  B. Muthén,et al.  Multilevel Covariance Structure Analysis , 1994 .

[32]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[33]  Bengt Muthén,et al.  Multilevel Factor Analysis of Class and Student Achievement Components , 1991 .

[34]  Y. Poortinga.  Equivalence of cross-cultural data: an overview of basic issues. , 1989, International journal of psychology : Journal international de psychologie.

[35]  B. Muthén.  Latent variable modeling in heterogeneous populations , 1989 .

[36]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[37]  Bengt Muthén,et al.  A Method for Studying the Homogeneity of Test Items with Respect to Other Relevant Variables , 1985 .

[38]  Dag Sörbom,et al.  An alternative to the methodology for analysis of covariance , 1978 .

[39]  D. Sörbom.  A GENERAL METHOD FOR STUDYING DIFFERENCES IN FACTOR MEANS AND FACTOR STRUCTURE BETWEEN GROUPS , 1974 .

[40]  Eldad Davidov,et al.  Cross cultural analysis: Methods and applications , 2010 .

[41]  W. Saris,et al.  Causes of generalized social trust: An innovative cross-national evaluation , 2009 .

[42]  E. Davidov.  Measurement Equivalence of Nationalism and Constructive Patriotism in the ISSP: 34 Countries in a Comparative Perspective , 2009, Political Analysis.

[43]  Bart Meuleman,et al.  Measurement inequivalence as a source of useful information. Using multi-level SEM to explain why measurements are inequivalent , 2009 .

[44]  Johnny Fontaine,et al.  Traditional and Multilevel Approaches in Cross-Cultural Research: An Integration of Methodological Frameworks. , 2008 .

[45]  Noel A. Card,et al.  Latent variable structural equation modeling in cross-cultural research: Multigroup and multilevel approaches , 2008 .

[46]  G. Loosveldt,et al.  Measuring meaningful data in social research , 2007 .

[47]  K. G. J8reskoC,et al.  Simultaneous Factor Analysis in Several Populations , 2007 .

[48]  Other.  Human Development Report 2006: Beyond scarcity: power, poverty and the global water crisis , 2006 .

[49]  Ricardo Fuentes,et al.  Human Development Report 2006 , 2006 .

[50]  A. D. Beuckelaer,et al.  Measurement Invariance Issues in International Management Research , 2005 .

[51]  J. Billiet.  Cross-cultural equivalence with structural equation modeling , 2003 .

[52]  G. A. Marcoulides,et al.  Multilevel Analysis Techniques and Applications , 2002 .

[53]  F. Vijver,et al.  A comparison of methods for the evaluation of construct equivalence in a multigroup setting , 2001 .

[54]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[55]  Frans J. Oort,et al.  Simulation study of item bias detection with restricted factor analysis , 1998 .

[56]  R. Inglehart.  Modernization and Postmodernization: Cultural, Economic, and Political Change in 43 Societies , 1997 .

[57]  S. Schwartz.  Are There Universal Aspects in the Structure and Contents of Human Values , 1994 .

[58]  S. Schwartz.  Universals in the Content and Structure of Values: Theoretical Advances and Empirical Tests in 20 Countries , 1992 .

[59]  J. Horn,et al.  A practical and theoretical guide to measurement invariance in aging research. , 1992, Experimental aging research.

[60]  F. Oort.  Using restricted factor analysis to detect item bias , 1992 .

[61]  Willem E. Saris,et al.  The Detection and Correction of Specification Errors in Structural Equation Models , 1987 .

[62]  A. Maslow.  Motivation and Personality , 1954 .