The Role of Measurement Quality on Practical Guidelines for Assessing Measurement and Structural Invariance

Although differences in goodness-of-fit indices (ΔGOFs) have been advocated for assessing measurement invariance, studies that advanced recommended differential cutoffs for adjudicating invariance actually utilized a very limited range of values representing the quality of indicator variables (i.e., magnitude of loadings). Because quality of measurement has been found to be relevant in the context of assessing data-model fit in single-group models, this study used simulation and population analysis methods to examine the extent to which quality of measurement affects ΔGOFs for tests of invariance in multiple group models. Results show that ΔMcDonald’s NCI is minimally affected by loading magnitude and sample size when testing invariance in the measurement model, while differences in comparative fit index varies widely when testing both measurement and structural variance as measurement quality changes, making it difficult to pinpoint a common value that suggests reasonable invariance.

[1]  J. Horn,et al.  When is invariance not invarient: A practical scientist's look at the ethereal concept of factor invariance. , 1983 .

[2]  Willem E. Saris,et al.  Testing Structural Equation Models or Detection of Misspecifications? , 2009 .

[3]  J. Bobes,et al.  Measurement Invariance of the Reynolds Depression Adolescent Scale across Gender and Age , 2010 .

[4]  D. Sass,et al.  A multicultural assessment of adolescent connectedness: testing measurement invariance across gender and ethnicity. , 2010, Journal of counseling psychology.

[5]  T. Keith,et al.  Testing for multigroup invariance of second-order WISC-IV structure across China, Hong Kong, Macau, and Taiwan , 2010 .

[6]  A. Boomsma,et al.  Robustness Studies in Covariance Structure Modeling , 1998 .

[7]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[8]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[9]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[10]  G. Hancock.  Effect size, power, and sample size determination for structured means modeling and mimic approaches to between-groups hypothesis testing of means on a single latent construct , 2001 .

[11]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[12]  Amery D. Wu,et al.  Decoding the Meaning of Factorial Invariance and Updating the Practice of Multi-group Confirmatory Factor Analysis: A Demonstration With TIMSS Data , 2007 .

[13]  Timothy Teo,et al.  Assessing the intention to use technology among pre-service teachers in Singapore and Malaysia: A multigroup invariance analysis of the Technology Acceptance Model (TAM) , 2009, Comput. Educ..

[14]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[15]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[16]  F. Chen.  Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance , 2007 .

[17]  T. Brown,et al.  Psychometric evaluation of the behavioral inhibition/behavioral activation scales in a large sample of outpatients with anxiety and mood disorders. , 2004, Psychological assessment.

[18]  Todd D. Little,et al.  Representing contextual effects in multiple-group MACS models. , 2007 .

[19]  Gregory R. Hancock,et al.  The Reliability Paradox in Assessing Structural Relations Within Covariance Structure Models , 2011 .

[20]  P. Cuijpers,et al.  Measurement invariance of the center for epidemiological studies depression scale (CES-D) among chinese and dutch elderly , 2011, BMC medical research methodology.

[21]  B. Byrne.  Structural equation modeling with EQS : basic concepts, applications, and programming , 2000 .

[22]  J. Hox,et al.  A checklist for testing measurement invariance , 2012 .

[23]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[24]  Phillip W. Braddy,et al.  Power and sensitivity of alternative fit indices in tests of measurement invariance. , 2008, The Journal of applied psychology.

[25]  P. Matos,et al.  Children's Perception of Interparental Conflict Scale (CPIC): Factor Structure and Invariance Across Adolescents and Emerging Adults , 2010 .

[26]  H. Toyoda,et al.  Structural Equation Modeling : Present and Future. Festschrift in honor of Karl Joreskog , 2001 .

[27]  Gordon W. Cheung,et al.  Testing Factorial Invariance across Groups: A Reconceptualization and Proposed New Method , 1999 .

[28]  M. Savickas,et al.  Career Adapt-Abilities Scale: Construction, reliability, and measurement equivalence across 13 countries , 2012 .

[29]  Jeanne A. Teresi,et al.  An Essay on Measurement and Factorial Invariance , 2006, Medical care.

[30]  Adam W. Meade,et al.  Power and Precision in Confirmatory Factor Analytic Tests of Measurement Invariance , 2007 .

[31]  Norbert K Tanzer,et al.  Bias and equivalence , 2000 .

[32]  P. Barrett.  Structural equation modelling : Adjudging model fit , 2007 .

[33]  L. Hayduk,et al.  Testing! testing! one, two, three - Testing the theory in structural equation models! , 2007 .

[34]  Karl G. Jöreskog,et al.  Lisrel 8: User's Reference Guide , 1997 .

[35]  Gordon W. Cheung,et al.  Assessing Extreme and Acquiescence Response Sets in Cross-Cultural Research Using Structural Equations Modeling , 2000 .

[36]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[37]  Phil Wood.  Confirmatory Factor Analysis for Applied Research , 2008 .

[38]  Roger E. Millsap,et al.  Factorial Invariance : Historical Trends and New Developments , 2004 .

[39]  J. Horn,et al.  A practical and theoretical guide to measurement invariance in aging research. , 1992, Experimental aging research.

[40]  P. Gagné,et al.  Measurement Model Quality, Sample Size, and Solution Propriety in Confirmatory Factor Models , 2006, Multivariate behavioral research.

[41]  K. G. J8reskoC,et al.  Simultaneous Factor Analysis in Several Populations , 2007 .

[42]  B. Byrne.  Testing for multigroup equivalence of a measuring instrument: a walk through the process. , 2008, Psicothema.

[43]  P. Bentler,et al.  Fit indices in covariance structure modeling : Sensitivity to underparameterized model misspecification , 1998 .