Construct Validity of Likert Scales through Confirmatory Factor Analysis: A Simulation Study Comparing Different Methods of Estimation Based on Pearson and Polychoric Correlations

The widespread use of Pearson correlations and, by extension, the Maximum Likelihood estimation method, does not take into account the measurement properties of Likert scales observed variables when carrying out a construct validity process through Confirmatory Factor Analysis (CFA). This simulation study compares four estimation methods (Maximum Likelihood ¨CML-, Robust Maximum Likelihood ¨CRML-, Robust Unweighted Least Squares ¨C, RULS) according to two of the assumptions CFA is supposed to fulfil: multivariate normality and, especially, the continuous measurement nature of both latent and observed variables. Goodness of fit is diagnosed by ¦O2 Likelihood Ratio Test and RMSEA indices. Results suggest ULS and RULS are preferable as polychoric correlations help to overcome grouping and transformation errors produced when using Pearson correlations for ordinal observed variables. Data measurement scale consideration enhances the ability of hypothesized models to reproduce accurately construct variables relationships.

[1]  Robert M. O’Brien.  The relationship between ordinal measures and their underlying values: Why all the disagreement? , 1985 .

[2]  Vangie A. Foshee,et al.  Higher Order Factor Structure Of A Self-Control Test: Evidence From Confirmatory Factor Analysis With Polychoric Correlations , 2003 .

[3]  Christine DiStefano,et al.  The Impact of Categorization With Confirmatory Factor Analysis , 2002 .

[4]  Conor V. Dolan,et al.  Factor analysis of variables with 2, 3, 5, and 7 response categories: A comparison of categorical variable estimators using simulated data , 1994 .

[5]  Stephen G. West,et al.  Structural equation models with non-normal variables: Problems and remedies , 1995 .

[6]  S. Asthana,et al.  Cognitive benefits of hormone therapy: cardiovascular factors and healthy-user bias. , 2009, Maturitas.

[7]  Annette Scherpenzeel,et al.  Validity and Reliability of Subjective Social Indicators: The effect of different measures of association , 1998 .

[8]  Emin Babakus,et al.  The Sensitivity of Confirmatory Maximum Likelihood Factor Analysis to Violations of Measurement Scale and Distributional Assumptions , 1987 .

[9]  C. Zamora,et al.  Modelos de ecuaciones estructurales ¿Qué es eso? , 2008 .

[10]  Howard B. Lee,et al.  A first course in factor analysis , 1973 .

[11]  On Chi-Squares for the Independence Model and Fit Measures in LISREL , 2004 .

[12]  F. Holgado-Tello,et al.  Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables , 2008 .

[13]  Theresa M. Akey,et al.  Effect of the number of scale points on chi‐square fit indices in confirmatory factor analysis , 1997 .

[14]  B. Byrne.  Structural Equation Modeling with LISREL, PRELIS, and SIMPLIS: Basic Concepts, Applications, and Programming , 1998 .

[15]  A. Satorra,et al.  Corrections to test statistics and standard errors in covariance structure analysis. , 1994 .

[16]  R. P. McDonald,et al.  Structural Equations with Latent Variables , 1989 .

[17]  Howard B. Lee,et al.  A First Course in Factor Analysis 2nd Ed , 1973 .

[18]  S. Finney.  Nonnormal and categorical data in structural equation modeling , 2013 .

[19]  R. Bagby,et al.  Validation of a Greek adaptation of the 20-item Toronto Alexithymia Scale. , 2010, Comprehensive psychiatry.

[20]  María Angeles Cea D'Ancona.  Análisis multivariable: teoría y práctica en la investigación social , 2002 .

[21]  R. Martín,et al.  MODELOS DE ECUACIONES ESTRUCTURALES , 2010 .

[22]  Richard G. Lomax,et al.  A Beginner's Guide to Structural Equation Modeling , 2022 .

[23]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables , 1985 .

[24]  D. R. Johnson,et al.  Ordinal measures in multiple indicator models: A simulation study of categorization error. , 1983 .

[25]  N. Cliff,et al.  Ordinal measurement in the behavioral sciences , 2002 .

[26]  B. Muthén,et al.  A comparison of some methodologies for the factor analysis of non‐normal Likert variables: A note on the size of the model , 1992 .

[27]  W. Saris,et al.  Categorization and measurement quality. The choice between Pearson and Polychoric correlations , 1995 .

[28]  Susan R. Hutchinson,et al.  Behavior of descriptive fit indexes in confirmatory factor analysis using ordered categorical data , 1998 .

[29]  K. Jöreskog,et al.  Confirmatory Factor Analysis of Ordinal Variables With Misspecified Models , 2010 .

[30]  Karl G. Jöreskog,et al.  Structural equation modeling with ordinal variables , 1994 .

[31]  M. F. Ramalhoto,et al.  Ordinal Methodology in the Analysis of Likert Scales , 2007 .

[32]  R. B. Alzina,et al.  Introducción conceptual al análisis multivariable: un enfoque informático con los paquetes SPSS-X, BMDP, LISREL y SPAD , 1989 .

[33]  Karl G. Jöreskog,et al.  LISREL 7: A guide to the program and applications , 1988 .

[34]  Jarl K. Kampen,et al.  The Ordinal Controversy Revisited , 2000 .

[35]  W. Gilley,et al.  Factor Analysis and Ordinal Data , 1993 .

[36]  Karl G. Jöreskog,et al.  Lisrel 8: User's Reference Guide , 1997 .

[37]  S. West,et al.  Effects of sample size and nonnormality on the estimation of mediated effects in latent variable models. , 1997 .

[38]  Vicente Ponsoda,et al.  Performance of Velicer’s Minimum Average Partial Factor Retention Method With Categorical Variables , 2011 .

[39]  Francisco Pablo Holgado Tello,et al.  Polychoric versus Pearson correlations in exploratory and confirmatory factor analysis of ordinal variables , 2010 .

[40]  Studies of the polychoric correlation and other correlation measures for ordinal variables , 1992 .