The Model Size Effect in SEM: Inflated Goodness-of-Fit Statistics Are Due to the Size of the Covariance Matrix

The size of a model has been shown to critically affect the goodness of approximation of the model fit statistic T to the asymptotic chi-square distribution in finite samples. It is not clear, however, whether this “model size effect” is a function of the number of manifest variables, the number of free parameters, or both. It is demonstrated by means of 2 Monte Carlo computer simulation studies that neither the number of free parameters to be estimated nor the model degrees of freedom systematically affect the T statistic when the number of manifest variables is held constant. Increasing the number of manifest variables, however, is associated with a severe bias. These results imply that model fit drastically depends on the size of the covariance matrix and that future studies involving goodness-of-fit statistics should always consider the number of manifest variables, but can safely neglect the influence of particular model specifications.

[1]  P. Barrett.  Structural equation modelling : Adjudging model fit , 2007 .

[2]  A. Boomsma,et al.  Robustness Studies in Covariance Structure Modeling , 1998 .

[3]  L. Harlow,et al.  Effects of estimation methods, number of indicators per factor, and improper solutions on structural equation modeling fit indices , 1995 .

[4]  A Skrondal,et al.  Design and Analysis of Monte Carlo Experiments: Attacking the Conventional Wisdom , 2000, Multivariate Behavioral Research.

[5]  B. Muthén.  A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators , 1984 .

[6]  H W Marsh,et al.  Is More Ever Too Much? The Number of Indicators per Factor in Confirmatory Factor Analysis. , 1998, Multivariate behavioral research.

[7]  Anne Boomsma,et al.  Small-Sample Robust Estimators of Noncentrality-Based and Incremental Model Fit , 2009 .

[8]  K. Bollen,et al.  The Noncentral Chi-square Distribution in Misspecified Structural Equation Models: Finite Sample Results from a Monte Carlo Simulation , 2002, Multivariate behavioral research.

[9]  S. West,et al.  The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. , 1996 .

[10]  Rachel T. Fouladi,et al.  Performance of Modified Test Statistics in Covariance and Correlation Structure Analysis Under Conditions of Multivariate Nonnormality , 2000 .

[11]  Albert Satorra,et al.  A scaled difference chi-square test statistic for moment structure analysis , 1999 .

[12]  C. D. Vale,et al.  Simulating multivariate nonnormal distributions , 1983 .

[13]  Dennis L. Jackson.  Sample Size and Number of Parameter Estimates in Maximum Likelihood Confirmatory Factor Analysis: A Monte Carlo Investigation , 2001 .

[14]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[15]  M. Browne.  Asymptotically distribution-free methods for the analysis of covariance structures. , 1984, The British journal of mathematical and statistical psychology.

[16]  Dennis L. Jackson.  Revisiting Sample Size and Number of Parameter Estimates: Some Support for the N:q Hypothesis , 2003 .

[17]  Ke-Hai Yuan,et al.  Normal theory likelihood ratio statistic for mean and covariance structure analysis under alternative hypotheses , 2007 .

[18]  H. Marsh,et al.  In Search of Golden Rules: Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for Fit Indexes and Dangers in Overgeneralizing Hu and Bentler's (1999) Findings , 2004 .

[19]  P. Gagné,et al.  Measurement Model Quality, Sample Size, and Solution Propriety in Confirmatory Factor Models , 2006, Multivariate behavioral research.

[20]  Dennis L. Jackson.  The Effect of the Number of Observations per Parameter in Misspecified Confirmatory Factor Analytic Models , 2007 .

[21]  James C. Anderson,et al.  The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis , 1984 .

[22]  A. Boomsma.  Nonconvergence, improper solutions, and starting values in lisrel maximum likelihood estimation , 1985 .

[23]  D. A. Kenny,et al.  Effect of the Number of Variables on Measures of Fit in Structural Equation Modeling , 2003 .

[24]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables: A note on the size of the model , 1992 .

[25]  K. Bollen,et al.  An Empirical Evaluation of the Use of Fixed Cutoff Points in RMSEA Test Statistic in Structural Equation Models , 2008, Sociological methods & research.

[26]  Sven Reinecke,et al.  The Model-Size Effect on Traditional and Modified Tests of Covariance Structures , 2007 .