Evaluating Sensitivity of Parameters of Interest to Measurement Invariance in Latent Variable Models

Latent variable models can only be compared across groups when these groups exhibit measurement equivalence or “invariance,” since otherwise substantive differences may be confounded with measurement differences. This article suggests examining directly whether measurement differences present could confound substantive analyses, by examining the expected parameter change (EPC)-interest. The EPC-interest approximates the change in parameters of interest that can be expected when freeing cross-group invariance restrictions. Monte Carlo simulations suggest that the EPC-interest approximates these changes well. Three empirical applications show that the EPC-interest can help avoid two undesirable situations: first, it can prevent unnecessarily concluding that groups are incomparable, and second, it alerts the user when comparisons of interest may still be invalidated even when the invariance model appears to fit the data. R code and data for the examples discussed in this article are provided in the electronic appendix ( http://hdl.handle.net/1902.1/21816 ).

[1]  J. Hausman.  Specification tests in econometrics , 1978 .

[2]  Daniel L. Oberski,et al.  Comparability of Survey Measurements , 2012 .

[3]  Raymond J. Carroll,et al.  Measurement error in nonlinear models: a modern perspective , 2006 .

[4]  B. King.  Unbiased measurement of health-related quality-of-life , 2011 .

[5]  N. Schmitt,et al.  Measurement invariance: Review of practice and implications , 2008 .

[6]  Tiffany A. Whittaker.  Using the Modification Index and Standardized Expected Parameter Change for Model Modification , 2012 .

[7]  Impact of Post Hoc Measurement Model Overspecification on Structural Parameter Integrity , 2006 .

[8]  Bengt Muthén,et al.  Bayesian structural equation modeling: a more flexible representation of substantive theory. , 2012, Psychological methods.

[9]  J. Fox,et al.  Bayesian modification indices for IRT models , 2005 .

[10]  David J. Bartholomew,et al.  Latent Variable Models and Factor Analysis: A Unified Approach , 2011 .

[11]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[12]  Simon Jackman,et al.  Multidimensional Analysis of Roll Call Data via Bayesian Simulation: Identification, Estimation, Inference, and Model Checking , 2001, Political Analysis.

[13]  David A. Armstrong.  Stability and change in the Freedom House political rights and civil liberties measures , 2011 .

[14]  R. MacCallum,et al.  Model modifications in covariance structure analysis: the problem of capitalization on chance. , 1992, Psychological bulletin.

[15]  Peter M. Bentler,et al.  8. Assessing the Effect of Model Misspecifications on Parameter Estimates in Structural Equation Models , 2003 .

[16]  Oi-Man Kwok,et al.  Using Modification Indexes to Detect Turning Points in Longitudinal Data: A Monte Carlo Study , 2010 .

[17]  P. Bentler,et al.  Fit indices in covariance structure modeling : Sensitivity to underparameterized model misspecification , 1998 .

[18]  Gregory R. Hancock.  A sequential Scheffé‐type respecification procedure for controlling type I error in exploratory structural equation model modification , 1999 .

[19]  Roger E. Millsap,et al.  Detecting Violations of Factorial Invariance Using Data-Based Specification Searches: A Monte Carlo Study , 2007 .

[20]  S. Schwartz,et al.  Toward A Universal Psychological Structure of Human Values , 1987 .

[21]  S. Schwartz,et al.  Sex differences in value priorities: cross-cultural and multimethod studies. , 2005, Journal of personality and social psychology.

[22]  Willem E. Saris,et al.  The Detection and Correction of Specification Errors in Structural Equation Models , 1987 .

[23]  Caroline Roberts,et al.  Measuring Attitudes Cross-Nationally , 2007 .

[24]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .

[25]  Andreas Ritter,et al.  Structural Equations With Latent Variables , 2016 .

[26]  W. Holmes Finch,et al.  Confirmatory Factor Analytic Procedures for the Determination of Measurement Invariance , 2006 .

[27]  Joshua D. Clinton,et al.  The Statistical Analysis of Roll Call Data , 2004, American Political Science Review.

[28]  P. Bentler,et al.  Invariant Standardized Estimated Parameter Change for Model Modification in Covariance Structure Analysis. , 1993, Multivariate behavioral research.

[29]  C. Chou,et al.  Some New Covariance Structure Model Improvement Statistics , 1992 .

[30]  B. Byrne,et al.  Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. , 1989 .

[31]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[32]  Willem E. Saris,et al.  Testing Structural Equation Models or Detection of Misspecifications? , 2009 .

[33]  B. Thompson,et al.  EFFECTS OF SAMPLE SIZE, ESTIMATION METHODS, AND MODEL SPECIFICATION ON STRUCTURAL EQUATION MODELING FIT INDEXES , 1999 .

[34]  R. Jowell.  Measuring attitudes cross-nationally : lessons from the European social survey , 2007 .

[35]  R. Millsap,et al.  Evaluating the impact of partial factorial invariance on selection in two populations. , 2004, Psychological methods.

[36]  Jan R. Magnus,et al.  Local Sensitivity and Diagnostic Tests , 2004 .

[37]  R. Vandenberg,et al.  A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research , 2000 .

[38]  Gideon J. Mellenbergh,et al.  Item bias and item response theory , 1989 .

[39]  D Kaplan,et al.  Model Modification in Covariance Structure Analysis: Application of the Expected Parameter Change Statistic. , 1989, Multivariate behavioral research.

[40]  J. Steenkamp,et al.  Assessing Measurement Invariance in Cross-National Consumer Research , 1998 .

[41]  W. Meredith.  Measurement invariance, factor analysis and factorial invariance , 1993 .

[42]  P. Schmidt,et al.  Values and Support for Immigration : A Cross-Country Comparison , 2008 .

[43]  Bengt Muthén,et al.  Bayesian SEM : A more flexible representation of substantive theory , 2010 .

[44]  Howard T. Everson,et al.  Methodology Review: Statistical Approaches for Assessing Measurement Bias , 1993 .

[45]  E. Davidov.  Measurement Equivalence of Nationalism and Constructive Patriotism in the ISSP: 34 Countries in a Comparative Perspective , 2009, Political Analysis.

[46]  Roger E. Millsap,et al.  Invariance in Measurement and Prediction Revisited , 2007 .

[47]  Roger E. Millsap,et al.  Invariance in measurement and prediction: Their relationship in the single-factor case. , 1997 .

[48]  Albert Satorra,et al.  Linear structural relations: Gradient and Hessian of the fitting function , 1991 .

[49]  F. Chen.  Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance , 2007 .

[50]  Stanislav Kolenikov,et al.  Biases of Parameter Estimates in Misspecified Structural Equation Models , 2011 .

[51]  A. Satorra.  Alternative test criteria in covariance structure analysis: A unified approach , 1989 .

[52]  Simon Jackman,et al.  Democracy as a Latent Variable , 2008 .

[53]  Teppei Yamamoto,et al.  Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis , 2010 .

[54]  Bart Meuleman.  When are item intercept differences substantively relevant in measurement invariance testing , 2012 .