Structural equation modeling with near singular covariance matrices

Conventional structural equation modeling involves fitting a structural model to the sample covariance matrix S. Due to collinearity or small samples with practical data, nonconvergences often occur in the estimation process. For a small constant a, this paper proposes to fit the structural model to the covariance matrix S"a=S+aI. When treating S"a as the sample covariance matrix in the maximum likelihood (ML) procedure, consistent parameter estimates are still obtained. The asymptotic distributions of the parameter estimates and the corresponding likelihood ratio statistic are studied and compared to those by the conventional ML. Two rescaled statistics for the overall model evaluation with modeling S"a are constructed. Empirical results imply that the estimates from modeling S"a are more efficient than those of fitting the structural model to S even when data are normally distributed. Simulations and real data examples indicate that modeling S"a allows us to evaluate the overall model structure even when S is literally singular. Implications of modeling S"a in a broader context are discussed.

[1]  Olivier Ledoit,et al.  A well-conditioned estimator for large-dimensional covariance matrices , 2004 .

[2]  K. Bollen.  Latent variables in psychology and the social sciences. , 2002, Annual review of psychology.

[3]  Peter M. Bentler,et al.  8. Assessing the Effect of Model Misspecifications on Parameter Estimates in Structural Equation Models , 2003 .

[4]  O. Nelles,et al.  An Introduction to Optimization , 1996, IEEE Antennas and Propagation Magazine.

[5]  T. W. Anderson.  An Introduction to Multivariate Statistical Analysis , 1959 .

[6]  R. Beran,et al.  Bootstrap Tests and Confidence Regions for Functions of a Covariance Matrix , 1985 .

[7]  P. K. Sen,et al.  Multivariate Analysis V. , 1982 .

[8]  Sik-Yum Lee.  Handbook of latent variable and related models , 2007 .

[9]  T. Louis,et al.  Empirical Bayes Confidence Intervals Based on Bootstrap Samples , 1987 .

[10]  Effects of employing ridge regression in structural equation models , 1997 .

[11]  K. Yuan,et al.  A unified approach to exploratory factor analysis with missing data, nonnormal data, and in the presence of outliers , 2002 .

[12]  G. Hancock,et al.  Evaluating Small Sample Approaches for Model Test Statistics in Structural Equation Modeling , 2004 .

[13]  Kenneth Lange,et al.  Numerical analysis for statisticians , 1999 .

[14]  Peter M. Bentler,et al.  Improving parameter tests in covariance structure analysis , 1997 .

[15]  Mitchell H. Gail,et al.  A Reminder of the Fallibility of the Wald Statistic , 1996 .

[16]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics (Revised Edition) , 1999 .

[17]  C. Kelley.  Iterative Methods for Linear and Nonlinear Equations , 1987 .

[18]  Tron Foss,et al.  Two Equivalent Discrepancy Functions for Maximum Likelihood Estimation: Do Their Test Statistics Follow a Non-Central Chi-Square Distribution under Model Misspecification? , 2004 .

[19]  Herman Wold,et al.  Systems under indirect observation : causality, structure, prediction , 1982 .

[20]  J. S. Long,et al.  Testing Structural Equation Models , 1993 .

[21]  Xin-Yuan Song,et al.  Evaluation of the Bayesian and Maximum Likelihood Approaches in Analyzing Structural Equation Models with Small Sample Sizes , 2004, Multivariate behavioral research.

[22]  Sik-Yum Lee,et al.  Structural equation modelling: A Bayesian approach. , 2007 .

[23]  J. S. Tanaka.  "How Big Is Big Enough?": Sample Size and Goodness of Fit in Structural Equation Models with Latent Variables. , 1987 .

[24]  P. Bentler,et al.  Bootstrapping Techniques in Analysis of Mean and Covariance Structures , 1996 .

[25]  B. Efron.  Jackknife‐After‐Bootstrap Standard Errors and Influence Functions , 1992 .

[26]  R. Beran.  Prepivoting Test Statistics: A Bootstrap View of Asymptotic Refinements , 1988 .

[27]  L. R. Haff.  Empirical Bayes Estimation of the Multivariate Normal Covariance Matrix , 1980 .

[28]  R. Hoyle.  Statistical Strategies for Small Sample Research , 1999 .

[29]  Ke-Hai Yuan,et al.  Normal theory likelihood ratio statistic for mean and covariance structure analysis under alternative hypotheses , 2007 .

[30]  Y Kano,et al.  Can test statistics in covariance structure analysis be trusted? , 1992, Psychological bulletin.

[31]  Alexander Shapiro,et al.  Asymptotic distribution theory in the analysis of covariance structures , 1983 .

[32]  Ke-Hai Yuan,et al.  Fitting structural equation models using estimating equations: a model segregation approach. , 2002, The British journal of mathematical and statistical psychology.

[33]  K. Yuan,et al.  Structural Equation Modeling with Small Samples: Test Statistics. , 1999, Multivariate behavioral research.

[34]  Christine DiStefano,et al.  The Impact of Categorization With Confirmatory Factor Analysis , 2002 .

[35]  R. MacCallum,et al.  Applications of structural equation modeling in psychological research. , 2000, Annual review of psychology.

[36]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[37]  Robert I. Jennrich,et al.  A study of algorithms for covariance structure analysis with specific comparisons using factor analysis , 1979 .

[38]  James C. Anderson,et al.  The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis , 1984 .

[39]  A. J. Swain.  A class of factor analysis estimation procedures with common asymptotic sampling properties , 1975 .

[40]  R. Stine,et al.  Bootstrapping Goodness-of-Fit Measures in Structural Equation Models , 1992 .

[41]  K. Yuan,et al.  Standard errors in covariance structure models: asymptotics versus bootstrap. , 2006, The British journal of mathematical and statistical psychology.

[42]  C. Kelley.  Solving Nonlinear Equations with Newton's Method , 1987 .

[43]  K. Holzinger,et al.  A study in factor analysis : the stability of a bi-factor solution , 1939 .

[44]  Harsharanjeet S. Jagpal.  Multicollinearity in Structural Equation Models with Unobservable Variables , 1982 .

[45]  Hrishikesh D. Vinod,et al.  Maximum entropy measurement error estimates of singular covariance matrices in undersized samples , 1982 .

[46]  G. Box.  Some Theorems on Quadratic Forms Applied in the Study of Analysis of Variance Problems, I. Effect of Inequality of Variance in the One-Way Classification , 1954 .

[47]  Ke-Hai Yuan,et al.  ON NORMAL THEORY AND ASSOCIATED TEST STATISTICS IN COVARIANCE STRUCTURE ANALYSIS UNDER TWO CLASSES OF NONNORMAL DISTRIBUTIONS , 1999 .

[48]  F. E. Satterthwaite.  Synthesis of variance , 1941 .

[49]  M. Browne.  Asymptotically distribution-free methods for the analysis of covariance structures. , 1984, The British journal of mathematical and statistical psychology.

[50]  Karl G. Jöreskog,et al.  Lisrel 8: User's Reference Guide , 1997 .

[51]  Sadanori Konishi,et al.  Application of the bootstrap methods in factor analysis , 1995 .

[52]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[53]  Herman Wold,et al.  Soft modelling: The Basic Design and Some Extensions , 1982 .

[54]  Anne Boomsma,et al.  Structural equation models: Present and Future. A Festschrift in honor of Karl Jöreskog. , 2001 .

[55]  A. Boomsma.  Nonconvergence, improper solutions, and starting values in lisrel maximum likelihood estimation , 1985 .

[56]  Ke-Hai Yuan,et al.  Mean and Covariance Structure Analysis: Theoretical and Practical Improvements , 1997 .

[57]  K. Mardia.  Assessment of multinormality and the robustness of Hotelling's T^2 test , 1975 .

[58]  K. Yuan,et al.  Bootstrap approach to inference and power analysis based on three test statistics for covariance structure models. , 2003, The British journal of mathematical and statistical psychology.

[59]  Alan M. Polansky,et al.  Stabilizing bootstrap‐t confidence intervals for small samples , 2000 .

[60]  K. Yuan,et al.  Robust Procedures in Structural Equation Modeling , 2007 .