Sensitivity of SEM Fit Indexes With Respect to Violations of Uncorrelated Errors

This simulation study investigated the sensitivity of commonly used cutoff values for global-model-fit indexes, with regard to different degrees of violations of the assumption of uncorrelated errors in confirmatory factor analysis. It is shown that the global-model-fit indexes fell short in identifying weak to strong model misspecifications under both different degrees of correlated error terms, and various simulation conditions. On the basis of an example misspecification search, it is argued that global model testing must be supplemented by this procedure. Implications for the use of structural equation modeling are discussed.

[1]  D. W. Zimmerman,et al.  Correction for Attenuation With Biased Reliability Estimates and Correlated Errors in Populations and Samples , 2007 .

[2]  Expected Values of Correlated Measurements and Correction for Attenuation , 1970 .

[3]  P. Schönemann.  Power as a function of communality in factor analysis , 1981 .

[4]  Scott B. MacKenzie,et al.  Common method biases in behavioral research: a critical review of the literature and recommended remedies. , 2003, The Journal of applied psychology.

[5]  Error and Reliability in Stochastic Processes and Psychological Measurement , 1972 .

[6]  R. MacCallum,et al.  Model modifications in covariance structure analysis: the problem of capitalization on chance. , 1992, Psychological bulletin.

[7]  Moritz Heene,et al.  Masking misfit in confirmatory factor analysis by increasing unique variances: a cautionary note on the usefulness of cutoff values of fit indices. , 2011, Psychological methods.

[8]  D. Kaplan.  Evaluating and Modifying Covariance Structure Models: A Review and Recommendation. , 1990, Multivariate behavioral research.

[9]  P. Costa,et al.  A contemplated revision of the NEO Five-Factor Inventory , 2004 .

[10]  J. Miles,et al.  A time and a place for incremental fit indices , 2007 .

[11]  K. Bollen.  Latent variables in psychology and the social sciences. , 2002, Annual review of psychology.

[12]  Cameron N. McIntosh,et al.  Rethinking fit assessment in structural equation modelling: A commentary and elaboration on Barrett (2007) , 2007 .

[13]  R. P. McDonald,et al.  Principles and practice in reporting structural equation analyses. , 2002, Psychological methods.

[14]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[15]  Victoria Savalei,et al.  Is the ML Chi-Square Ever Robust to Nonnormality? A Cautionary Note With Missing Data , 2008 .

[16]  Dale N. Glaser,et al.  Jiving the Four-Step, Waltzing Around Factor Analysis, and Other Serious Fun , 2000 .

[17]  R. Goffin.  Assessing the adequacy of structural equation models: Golden rules and editorial policies , 2007 .

[18]  Peter M. Bentler,et al.  On tests and indices for evaluating structural models , 2007 .

[19]  Bruno D. Zumbo,et al.  Coefficient Alpha as an Estimate of Test Reliability Under Violation of Two Assumptions , 1993 .

[20]  P. Meehl.  Why Summaries of Research on Psychological Theories are Often Uninterpretable , 1990 .

[21]  André Beauducel,et al.  Simulation Study on Fit Indexes in CFA Based on Data With Slightly Distorted Simple Structure , 2005 .

[22]  R. Peterson.  A Meta-Analysis of Variance Accounted for and Factor Loadings in Exploratory Factor Analysis , 2000 .

[23]  H. G. Osburn,et al.  Coefficient alpha and related internal consistency reliability coefficients. , 2000, Psychological methods.

[24]  Willem E. Saris,et al.  Testing Structural Equation Models or Detection of Misspecifications? , 2009 .

[25]  Mark N. O. Davies,et al.  Coefficient alpha: A useful indicator of reliability? , 2000 .

[26]  D. W. Zimmerman,et al.  Validity Coefficients and Correlated Errors in Test Theory. , 1977 .

[27]  Xitao Fan,et al.  Sensitivity of Fit Indexes to Misspecified Structural or Measurement Model Components: Rationale of Two-Index Strategy Revisited , 2005 .

[28]  R. Lomax,et al.  The Effect of Varying Degrees of Nonnormality in Structural Equation Modeling , 2005 .

[29]  H. Marsh,et al.  In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler's (1999) Findings , 2004 .

[30]  S. Hershberger,et al.  Correlated Errors in True Score Models and Their Effect on Coefficient Alpha , 2000 .

[31]  Dag Sörbom,et al.  DETECTION OF CORRELATED ERRORS IN LONGITUDINAL DATA , 1975 .

[32]  P. Costa,et al.  Validation of the five-factor model of personality across instruments and observers. , 1987, Journal of personality and social psychology.

[33]  Stanley A. Mulaik,et al.  There is a place for approximate fit in structural equation modelling. , 2007 .

[34]  Daniel Oberski.  Jrule for Mplus: A program for post-hoc power evaluation of structural equation models estimated by Mplus , 2009 .

[35]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[36]  Marieke E. Timmerman,et al.  Comparison of Methods for Adjusting Incorrect Assignments of Items to Subtests , 2009 .

[37]  Roger E. Millsap,et al.  Structural equation modeling made difficult , 2007 .

[38]  K. Yuan.  Fit Indices Versus Test Statistics , 2005, Multivariate behavioral research.

[39]  R. Stine,et al.  Bootstrapping Goodness-of-Fit Measures in Structural Equation Models , 1992 .

[40]  James H. Steiger,et al.  Understanding the limitations of global fit assessment in structural equation modeling , 2007 .

[41]  D. W. Zimmerman,et al.  Reconsideration of the "Attenuation Paradox"--and Some New Paradoxes in Test Validity. , 1982 .

[42]  T. Little,et al.  To Parcel or Not to Parcel: Exploring the Question, Weighing the Merits , 2002 .

[43]  B. Thompson,et al.  EFFECTS OF SAMPLE SIZE, ESTIMATION METHODS, AND MODEL SPECIFICATION ON STRUCTURAL EQUATION MODELING FIT INDEXES , 1999 .

[44]  The theory of test validity and correlated errors of measurement , 1977 .

[45]  Xitao Fan,et al.  Sensitivity of Fit Indices to Model Misspecification and Model Types , 2007 .

[46]  Tenko Raykov,et al.  Bias of Coefficient afor Fixed Congeneric Measures with Correlated Errors , 2001 .

[47]  P. Barrett.  Structural equation modelling : Adjudging model fit , 2007 .

[48]  L. Hayduk,et al.  Testing! testing! one, two, three - Testing the theory in structural equation models! , 2007 .

[49]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.