Can test statistics in covariance structure analysis be trusted?

Covariance structure analysis uses chi 2 goodness-of-fit test statistics whose adequacy is not known. Scientific conclusions based on models may be distorted when researchers violate sample size, variate independence, and distributional assumptions. The behavior of 6 test statistics is evaluated with a Monte Carlo confirmatory factor analysis study. The tests performed dramatically differently under 7 distributional conditions at 6 sample sizes. Two normal-theory tests worked well under some conditions but completely broke down under other conditions. A test that permits homogeneous nonzero kurtoses performed variably. A test that permits heterogeneous marginal kurtoses performed better. A distribution-free test performed spectacularly badly in all conditions at all but the largest sample sizes. The Satorra-Bentler scaled test statistic performed best overall.

[1]  A. Basilevsky,et al.  Factor Analysis as a Statistical Method. , 1964 .

[2]  K. Jöreskog.  A general approach to confirmatory maximum likelihood factor analysis , 1969 .

[3]  M. Browne.  Generalized Least Squares Estimators in the Analysis of Covariance Structures. , 1973 .

[4]  A. E. Maxwell,et al.  Factor Analysis as a Statistical Method. , 1964 .

[5]  G. Chamberlain.  Multivariate regression models for panel data , 1982 .

[6]  P. Bentler.  Some contributions to efficient statistics in structural models: Specification and estimation of moment structures , 1983 .

[7]  A. Boomsma.  On the robustness of LISREL (maximum likelihood estimation) against small sample size and non-normality. , 1984 .

[8]  T. W. Anderson.  Estimating Linear Statistical Relationships , 1984 .

[9]  M. Browne.  Asymptotically distribution-free methods for the analysis of covariance structures. , 1984, The British journal of mathematical and statistical psychology.

[10]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables , 1985 .

[11]  E. Ziegel.  Multivariate Analysis VI , 1986 .

[12]  P. Bentler,et al.  Greatest lower bound to the elliptical theory kurtosis parameter , 1986 .

[13]  A. Shapiro,et al.  Analysis of Covariance Structures under Elliptical Distributions , 1987 .

[14]  P. Bentler,et al.  Multiple population covariance structure analysis under arbitrary distribution theory , 1987 .

[15]  M. Browne.  Robustness of statistical inference in factor analysis and related models , 1987 .

[16]  Peter M. Bentler,et al.  Scaling Corrections for Statistics in Covariance Structure Analysis , 1988 .

[17]  T. W. Anderson,et al.  The asymptotic normal distribution of estimators in factor analysis under general conditions , 1988 .

[18]  A. Shapiro,et al.  Robustness of normal theory methods in the analysis of linear latent variate models. , 1988 .

[19]  T. Micceri.  The unicorn, the normal curve, and other improbable creatures. , 1989 .

[20]  Peter M. Bentler,et al.  EQS : structural equations program manual , 1989 .

[21]  A. Satorra.  Alternative test criteria in covariance structure analysis: A unified approach , 1989 .

[22]  T. W. Anderson,et al.  Asymptotic Chi-Square Tests for a Large Class of Factor Analysis Models , 1990 .

[23]  H. Wakaki.  A class of tests for a general covariance structure , 1990 .

[24]  Albert Satorra,et al.  Model Conditions for Asymptotic Robustness in the Analysis of Linear Relations , 1990 .

[25]  R. P. McDonald,et al.  Structural Equations with Latent Variables , 1989 .

[26]  A. Satorra,et al.  Scaled test statistics and robust standard errors for non-normal data in covariance structure analysis: a Monte Carlo study. , 1991, The British journal of mathematical and statistical psychology.

[27]  P. Bentler,et al.  Robustness of normal theory statistics in structural equation models , 1991 .

[28]  P. Bentler,et al.  Covariance Structure Analysis Under a Simple Kurtosis Model , 1992 .

[29]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables: A note on the size of the model , 1992 .